Testing async code: are long sleeps acceptable?

Question

I am testing some AngularJS code using Selenium. Angular updates the DOM in unpredictable order, sometimes completely async (via setTimeout), so it is difficult to know when the DOM has been completely updated after an AJAX request (or if the AJAX request has finished or not). (Because of this, all solutions that purely hooks the AJAX requests but not Angular itself are unstable.)

I was fully prepared to find all the places in Angular I need to hook to detect the end of a rendering cycle and inject loads of JS to hook them, then it was suggested to me that I could add sleeps until the result is stable.

Personally, I really hate sleeping in tests. It feels hacky as hell and unnecessarily slows down the already slow Selenium tests. But in this case, sleeping is much simpler and more maintainable.

What are your opinions on this?

Answer 1

Do not do this

Sleeps are unpredictable. You never know when your test environment happens to be downloading Windows Updates when the virusscanner runs and the timeout happens to be a second too short. Your tests will become brittle and useless because you'll have to run every failed scenario multiple times to make sure it was an actual failure. And sleeps force you to wait for the entire time of the sleep, so you are forever torn between big sleeps (which seldom fail) and small sleeps (which terminate and therefore run your tests quicker)

Instead, Selenium offers builtin Waits to handle this. This differs from sleeps because a Selenium wait will continue as soon as the element you are waiting for becomes available. Because they are part of your test framework, you don't have to maintain this code as well.

Basically, what you do is have your AJAX call change the DOM in an observable way: a remote call to fetch a list would put the list element with id "myRemoteList" on the page when loading is finished and Selenium would wait for that element to become visible. The point is that even asynchronous code has an observable outcome and you should wait for the result to become available. Just like you would if you were testing manually, or the user would when working with the application.

I set timeouts in my tests for 60 seconds: if it hasn't loaded then it will never load because something bad happened and I'm also guaranteed it won't timeout because my laptop is doing some background thing.