Should constants of my class only be
public if I know that they will be used from their users or doesn't it hurt to make all of my constants public per se?
Is not exposing all constants cleaner?
Not exposing things is fundamentally preferable because it avoids commitment to an implementation.
Once you've added something to a public interface, you cannot easily remove it ever again without possibly breaking clients. Someone, somewhere might be using it and complain if you later have to rename or remove it.
You should view publishing things as a cost. It may be necessary to achieve the purpose you're going for, but if it's not necessary, making such a commitment is a pure cost with no benefit. In general, systems engineering should avoid incurring costs without need.