Q: Is it good to use dependency injection in a java library?
This is totally up to you. However, imposing a specific framework or tool to make the library to work could dissuade developers from adopting the library. Those concerned by design best practices could argue that such a library is locking the application to a specific transitive dependency. In this case, Guice.
Transitive dependencies are not dependencies of our application. These are implementation details of those libraries we directly depend on. Ideally, we don't want to couple our application to the implementation details of 3rd party libs.
Adopting a new library should be possible without explicitly accessing or using its implementation details. Instead, it should provide us with abstractions that keep its "details" away from us. If you could hide Guice from the consumer then it's ok. But if you force the consumer to know about Guice (and use its elements explicitly) then I would suggest dropping Guice in favour of a cleaner design so developers can choose the DI implementation they want. By "cleaner design" I mean declaring constructors and setters to inject dependencies and do a proper use of the accessor modifiers.
Summarising, it's strongly advisable to keep the implementation details of our libraries away from the consumers. In your case, provide an abstraction simple to reason about but don't go beyond the library responsibility. Leave some gaps unfilled so consumers can fill them up with the glue of their choice. Even if you are the only consumer, at some point you might decide to move from Guice to Spring or vice-versa. Or, why not, to stop using both and go back to the healthy practice of instantiating dependencies yourself.