-1

Given:

  • You have a list of opportunities* that may improve the speed metrics of your website. For example Total Blocking Time (TBT) is one metric that heavily impacts the experience of a visitor.

  • You've run lighthouse (or any other speed testing tool), which tells you "Estimated Savings" for each of these opportunities.

  • You do not have enough time to fix every opportunity, so you need to narrow it down to some.

*A few examples of opportunities are preload images, reduce unused javascript, serve images in WebP instead of png and jpg, reduce unused css, avoid legacy javascript. Lighthouse tells you specific scripts, files, etc, for each of the opportunities.

In Chrome browser, how would you measure the real life savings of a specific opportunity, in a production build?

For example, if I remove script x, how do I measure how much that impacted the Total Blocking Time, with a before and after metric?

Basically, so I can say "I took this action, and this is the direct impact of that action on site speed".

Note: I asked this same question on stack overflow, but I think it might be a little too broad and methodology oriented for there, which is why I've come here.

2
  • Did you do any performance testing to record metrics before optimizing? Or is your question basically "how do I record these metrics before optimizing?" Jul 5, 2022 at 19:03
  • I have used lighthouse which gives you values for the metrics, but yes, my question is "how do I record these metrics" before/after optimizing so I can accurately say "this optimization improved my site speed". A lot of the opportunities affect the lighthouse score by a fraction of a point, and there is variability in the readings, so I need a way to more accurately measure what's happening. For example with other performance testing you might test 200 times to get an average, but lighthouse is too slow for that. Jul 5, 2022 at 19:15

1 Answer 1

2

There is no magic solution.

  • You pick a metric and a corresponding measurement approach.
  • You gather baseline metrics.
  • You implement the optimization.
  • You gather updated metrics.
  • You compare the metrics and revert the change if isn't worth it.

Google Lighthouse is a perfectly fine tool for gathering certain metrics.

You are correctly concerned that a lot of changes might be too small to be significant on their own. For example, removing a single unused script will likely fail to have any measurable effect. You can either respond to this problem with statistics (taking tons of measurements and then performing statistical tests), or by performing a larger optimization campaign so that there is an overall larger effect to observe. Instead of removing one script, you might measure before/after performing a class of optimizations.

It's generally impossible to know in advance which optimizations will be best. You will have to use your professional judgement here. However, an optimization can save at most as much time as eliminating the task entirely, especially if this task is part of a critical path. Lighthouse does use such measurements to estimate the maximum possible savings from some savings. You can use this to prioritize optimization ideas.

It is also OK to not perform some suggested optimizations. Your time is finite and valuable. Chasing the last millisecond of latency is probably not a wise use of your time. In contrast, a change that can save 100ms will be noticeable and will not be masked by random noise – and many sites do have ample opportunity for improvement.

For example, Lighthouse scores the frontpage of this Software Engineering site with a 6.2 second Time To Interactive on mobile and estimates that about 1 second could be saved by avoiding render-blocking resources like jquery.min.js from being requested. Turns out that the previous best practice of loading resources from CDNs is no longer a best practice in the age of HTTP/2 and cache partitioning.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.