If your class is a data structure, then obviously it's fine to do so.
Java Map has a put method, which stores a reference. If you modify that reference, you would expect to get the modified object back when calling Map.get().
If your class is a Car, and you are setting the Tires, then well, the problem is there: what happens if I set the Tire and then in another part of the program I modify it? In this case (when there are side-effects), if the result is inconsistency, then no, the setter shouldn't be there.
I think what you ask is a bit broad, as there are cases where you should be cautious and others where it's perfectly valid. Setters of that type exist and are all over the place in Java.
Perhaps a better solution is to pass in a class that is immutable (all it's fields are final) if you want to avoid side effects. If I were using an API, I wouldn't expect all it's setters cloning my objects!
you have a
Person class. He can have a
JobPlace or not. Do you know the job a person is going to have when you create it? Most likely no. Also, can the person change jobs? Sure.
This justifies a
person.setJob(job). Now the person has a reference to his job, but what happens if the job itself changes? The company could change name for example.
Person only needs to know the interface of
JobPlace to function, so the job itself can change and if several
Person's hold the same reference to
JobPlace, then you need to change it only in one place.