In the end the runtime [seconds] of the implementation of an algorithm is the deciding factor but the coefficients and the big-O notation help you in inventing efficient algorithms.
The big-O notation tells you how the algorithm behaves if the number of inputs/.. n changes dramatically (orders of magnitude). You mostly use it in a relative sense, i.e. increases polynomial, linear, nlog(n), ...
If you compare two algorithms which have similar big-O terms you should include the coefficients. It could be that the worse order algorithm but with a better coefficient is actually faster for your typical data sets and you don't want to miss that.
And finally in practise people often just take some implementations and let them run against each other. Gives you a practical view close to reality (used hardware and typical data sets) but dependent on the implementation, hardware and used data sets.
All together is the best.