Depending on your language, this isn't the only way to separate out that sort of code. In .NET there is the concept of partial classes which can achieve the same. There are also code regions but these are falling out of favour for a number of reasons.
From a personal POV, I tend to wait until the set up object needs to be reused in another set of tests (think: rule of three) before putting them in a separate class - YMMV.
Be aware that a class without logic is a known code smell. Make sure the helper class is actually doing something and isn't just a series of properties.
A good rule of thumb to stop test classes getting unwieldy is to make sure they mirror the underlying classes. A benefit of this is that you can see at a glance which classes still need testing. Of course a huge class could yield a large test class but these should be very much the exception.