It is a lot more flexible and less confusing to be adding and removing event listeners as your application starts to grow. Polling is easier to conceptualise when you are first imagining everything clocking in to a central location, but as your application gets more complex that is liable to get increasingly difficult to do, whereas events can come from anywhere and go to anywhere so they are much less tightly coupled.
Also when an event is triggered and there is nobody to hear it, nothing happens. If you have a whole lot of timeout based polling loops, you may find that things are happening after the main system crashed as timeouts trigger, which makes debugging the problem or following up any logs more confusing.
Practically if you are animating things, you will probably want to use some degree of polling for your animation loop and events to communicate when to trigger other animations or when interactions occur, so you're likely to want to use both.