SpeedCurve Blog

Demystifying Cumulative Layout Shift with CLS Windows

Tue, 01 Aug 2023 00:00:00 +1200

As we all know, naming things is hard.

Google's Core Web Vitals are an attempt to help folks new to web performance focus on three key metrics. Not all of these metrics are easy to understand based on their names alone:

Largest Contentful Paint (LCP) – When the largest visual element on the page renders
First Input Delay (FID) – How quickly a page responds to a user interaction (FID will be replaced by Interaction to Next Paint in March 2024)
Cumulative Layout Shift (CLS) – How visually stable a page is

Any time a new metric is introduced, it puts the burden on the rest of us to first unpack all the acronyms, and then explore and digest what concepts the words might refer to. This gets even trickier if the acronym stays the same, but the logic and algorithm behind the acronym changes.

In this post, we will dive deeper into Cumulative Layout Shift (CLS) and how it has quietly evolved over the years. Because CLS has been around for a while, you may already have some idea of what it represents. Before we go any further, I have a simple question for you:

How do you think Cumulative Layout Shift is measured?

Hold your answer in your head as we explore the depths of CLS. I'm interested if your assumptions were correct, and there's a poll at the bottom of this post I'd love you to answer.

Old-school CLS

In May 2020, Google introduced CLS as part of Core Web Vitals. To help improve user experience, CLS tracked any elements that moved around on the page as it renders. You know... that annoying moment when the ad pops in late in the page render and pushes the content you were already reading further down the page. That yucky moment is called a layout shift.

With the original version of CLS, all those layout shifts went through some funky maths, which compared the size of the layout shift to the size of the viewport and how far the element moved, and then converted it into a numeric score. The scores of all the individual layout shifts were then added together to give you the cumulative layout shift.

I've always found it odd that Google left the representation of CLS as a decimal score. I'm a simpleton and find parsing all those decimals takes extra mental effort. It feels more complicated than it needs to be. Since it's an unintuitive metric anyway and takes effort to understand, why not just turn it into a score out of 100? That's why people love the Lighthouse scores so much.

The problem

After the launch of CLS, a major issue quickly surfaced. Old-school CLS measured all the layout shifts over the entire lifespan of the page. This collided head-on with developers moving to Single Page Apps (SPAs). Since a SPA is by definition a full app shoved inside a single page, the CLS for a SPA would continue to grow for the lifetime of the app/page. This led to massive CLS scores for SPAs and a rethink of how CLS should be measured.

Introducing CLS windows

In 2021, Google relaunched CLS with a session window. The idea was to measure a burst of layout shifts within a 5s session window. A new session starts if there is a gap of more than 1s between layout shifts. At the end of the page lifespan, the session window with the highest total layout shifts becomes CLS.

Example of CLS sessions windows from web.dev

By introducing a session window, Google reduced the impact of the SPA problem. The CLS for any long-lived page was now limited to a 5s window containing the largest layout shifts.

(Measuring the performance of SPAs is still an ongoing problem for Core Web Vitals, however, and there's more work being done on measuring metrics relative to the soft navigations of a SPA.)

Same name, different algorithm... yikes

This was a major change to CLS that I think a lot of people missed. Google did a great job of consulting the web performance community on the change, and those changes were well thought out and welcomed. But by keeping the naming of CLS the same, many people we speak to still fundamentally misunderstand what CLS represents. As it still says on the tin, many people still believe that CLS is the accumulation of all the layout shifts on a page. That's what it used to be, but it's not anymore. Maybe Google should have renamed CLS to indicate how different it is now. I don't think anyone could face a four letter acronym. CLSW? WCLS?

To help people get their heads around CLS windows, we've introduced a new visualization to SpeedCurve that shows you each session window, as well as which session is the largest and therefore used as the CLS score.

A synthetic test showing three session windows with the largest window of 0.71 used as the CLS score.

Any layout shift is a bad shift

Your pages should really be fully rendered in under 5s, but if you have slow loading pages and dodgy third parties that take longer than 5s to stop jiggling your elements about, then there's a chance your CLS score doesn't accurately let you know just how bad your page is. Only the worst layout shifts within the 5s session window will contribute to CLS.

In the example above, there are large shifts in the second session window, but these are not included in the final CLS score because the total of the layout shifts in the third session window is larger. This masks the fact that overall this page has a total layout shift score nearly twice as bad as what is reported by CLS, and the user suffered through many large layout shifts as the page rendered.

All metrics need validation before you rely on them

The scenario described above is something to watch out for and a good reminder that you have to validate what a metric represents on your pages. In the same way that you need to check that the element selected for Largest Contentful Paint is an element you actually care about, you should also check that windowed CLS is not hiding any large layout shift(s) from you.

From a user's perspective, any large layout shift is a bad shift. So on our Test dashboard, we continue to show you all the layout shifts that happened as the page rendered and which specific elements moved. You really do need to get rid of all of them for your page to feel as stable and solid as possible as it renders.

Focus on improving your user experience first and making your pages feel rock solid as they render. That will naturally lead to a lower CLS score and improved ranking on Google.

Don't forget: Core Web Vitals are just a means to an end

Core Web Vitals are meant to help you measure and improve user experience. You can either try and improve your Core Web Vitals metrics, or just work on improving your user experience and the metrics will follow. At SpeedCurve, we're here to help you craft a fast and joyful user experience. Metrics are just a means to that end goal.

Learn more:

Super-quick poll

Before you depart, I'd love to know if the windowed CLS changes were already something you understood, or if you thought all layout shifts were still included in the Cumulative Layout Shift score.

Exploring performance and conversion rates just got easier

Tue, 11 Jul 2023 00:00:00 +1200

Demonstrating the impact of performance on your users – and on your business – is one of the best ways to get your company to care about the speed of your site.

Tracking goal-based metrics like conversion rate alongside performance data can give you richer and more compelling insights into how the performance of your site affects your users. This concept is not new by any means. In 2010, the Performance and Reliability team I was fortunate enough to lead at Walmartlabs shared our findings around the impact of front-end times on conversion rates. (This study and a number of other case studies tracked over the years can be found at WPOstats.)

Setting up conversion tracking in SpeedCurve RUM is fairly simple and definitely worthwhile. This post covers:

What is a conversion?
How to track conversions in SpeedCurve
Using conversion data with performance data for maximum benefit
Conversion tracking and user privacy

What is a conversion?

The performance of your website can make or break engagement with your users. Lighthouse scores and audits and Core Web Vitals thresholds (which are both available in SpeedCurve) offer guidance on thresholds and optimizations, but they can't show you how performance affects user behavior and outcomes.

It's not enough to improve your Lighthouse score or Core Web Vitals results. You need to validate that improving your performance metrics actually helps improve user experience, and ultimately your business metrics. This is where conversion tracking comes in.

Simply put, a conversion is a measurable outcome for a user session. Those outcomes can take many forms, such as:

Checkout events, such as adding an item to a shopping cart or paying for an order
Signing up for a newsletter
Downloading an ebook
Requesting a quote for a service
Booking a hotel reservation

Those are all examples of conversions.

How do you track conversions in SpeedCurve?

One of the biggest factors I've seen that keeps people from tracking conversions alongside performance data is the assumption that it's difficult to configure and requires code modifications. This is no longer the case!

Conversions are one example of custom data you can collect in SpeedCurve. There are two supported methods for tracking a conversion in SpeedCurve:

Using the JavaScript API, you can instrument your code to include:
LUX.addData('order_confirmed', 1)
Or you can elect to flag conversions based on the traffic to a known URL pattern
domain.com/checkout/orderconfirmation*

This second option has become the preferred method for users due to the frictionless 'no code' approach.

Learn more: Setting up conversion tracking in SpeedCurve

Methods and use cases for using conversion data

After you've added conversion data to SpeedCurve, there are a few different ways you can use it.

Correlation charts

Correlation charts demonstrate how sensitive users are to different metrics. Correlation charts give you a histogram view of all your user traffic, broken out into cohorts based on performance metrics such as Start Render and Largest Contentful Paint. The chart also includes an overlay that shows you a user engagement metric or business metric – such the bounce rate or conversion rate – that correlates to each of these cohorts. This lets you see at a glance the relationship between performance, user engagement, and your business.

In the example below, Largest Contentful Paint has a demonstrated negative correlation with the conversion rate. In other words, as sessions get slower, the conversion rate gets worse.

Some metrics <cough, First Input Delay, cough, cough> end up having little to no correlation with goal-based metrics like conversion.

Sometimes results vary by site. For example, conversion rates for luxury goods tend to be much lower than for commodity goods. So for some brands, not only are conversions low, but users seem to have much more tolerance due to higher switching costs. This is why it's so important to look at your own data when understanding which metrics matter for your site.

Whether you attempt to shoot holes in it or not, it's pretty easy to get behind the idea that the data in correlation charts is at least directionally accurate.

Illustrating this relationship can help justify resourcing for page speed and improving the user experience. Stakeholders who aren't in the weeds on performance can at least understand the potential impact on revenue.

On the other hand, this chart is not super helpful as an operational tool on a daily basis. Typically it is used to prove a point to stakeholders, or validate which metrics you should be focused on. It's not something you really need in your daily performance toolbox.

Learn more: Creating correlation charts

Time series charts and trend analysis

Trending metrics over time is nothing new and pretty commonplace for any metrics you collect consistently. Conversion is no different. It's important to see how user behavior changes during time of day, time of week, during a major campaign, etc.

It's also helpful to see if performance is having a sustained direct impact. In this example, there was a regression in LCP, which appears to have had a sustained impact on conversion rate.

Learn more: Trend metrics & compare time periods

A/B testing and cohort analysis

Segmenting user sessions gives you the ability to compare cohorts to each other and see how each group trends over time.

In the example below, segmentation is done using custom dimensions to compare an A/B test where front-end optimizations were made in the variant. This allows you to see if improvements to conversion are consistent or more variable and not as definitive.

After looking at this data, there was much more confidence in the positive impact the changes had on conversion rates.

Learn more: A/B testing in RUM

Comparing converted sessions and pages

Once you've started tracking conversions, you can really open up the possibilities in some of our default dashboards.

We recently released the RUM Compare dashboard. This unique view allows you to compare different cohorts to each other and understand the differences in metrics, often driven by demographics or particular clusters of users exhibiting a distinct behavior. Sometimes this is a time-based cohort, while other times it's an A/B test. However, one of the more interesting questions this can help to answer is "What is different between converted sessions and non-converted sessions?"

For example:

On the surface, in the chart below things look to be in conflict with the principle that faster pages = $$. As you can see, the converted sessions have a much slower LCP overall than non-converted sessions. Say it isn't so!

However, after more inspection you see that the makeup of page views for converted sessions contains a lot more checkout pages, which in this case have more backend time, driving up all the front-end focused metrics, such as LCP, for the overall session numbers.

If you break this down further to look at individual page performance where there were common pages, such as product pages, now you can see a different story. Illustrated here, the pages for converted sessions appear have a much faster LCP:

In this case, there is a strong argument for making the product pages faster, even though LCP scores may have been flagged as 'passing' the Core Web Vitals threshold. This is just one example of how the RUM Compare dashboard can be used to explore your converted user base.

User privacy

Data privacy is serious business and always has been at SpeedCurve. We sometimes get customers who are concerned about tracking conversion, fearing that the information collected from end users contains PII (personally identifiable information).

SpeedCurve RUM does not gather any personally identifiable information such as IP addresses, localStorage, and cookies. By tracking conversion, you are simply providing another segment to look at your data. No additional user information is collected. (Click here to learn more about SpeedCurve and data privacy.)

Summary

As a SpeedCurve RUM customer, it's easier than ever to start tracking conversion alongside your performance metrics. Doing so may open up a lot of opportunity to make business-driven decisions about performance.

Not currently a SpeedCurve customer? Start a free trial today!

What is page bloat? And how is it hurting your business, your search rank, and your users?

Tue, 04 Jul 2023 00:00:00 +1200

For more than ten years, I've been writing about page bloat, its impact on site speed, and ultimately how it affects your users and your business. You might think that this topic would be played out by now, but every year I learn new things – beyond the overarching fact that pages keep getting bigger and more complex, as you can see in this chart, using data from the HTTP Archive.

In this post, we'll cover:

How much pages have grown over the past year
How page bloat hurts your business and – at the heart of everything – your users
How page bloat affects Google's Core Web Vitals (and therefore SEO)
If it's possible to have large pages that still deliver a good user experience
Page size targets
How to track page size and complexity
How to fight regressions

What do we mean when we talk about page size?

When we talk about page size, we're referring to overall page weight and complexity. This includes:

Size – Total page weight in bytes. Size matters, especially to mobile users who have limited and/or metered data.
Resources – Total number of resources (HTML, images, fonts, etc.) on the page. The more resources, the greater the complexity and the increased likelihood of rendering delays and blockages.
HTML – Typically the smallest resource on the page, HTML's performance risk is usually negligible. Having said that, I recently did some digging into a page where the total HTML size jumped dramatically because of a bunch of inline JavaScript, which led to rendering delays, so keeping an eye on HTML size is still a good idea.
Images – Often the greatest contributor to page bloat. Looking at the 90th percentile of the distribution of page weight, images account for a whopping 5.7 MB of a roughly 8.2 MB page. In other words, images comprised almost 75% of the total page weight. And if that already wasn’t enough, the number of images on a page has been linked to lower conversion rates on retail sites. (More on that later.)
Video – For the first time in the many years that I've been covering the topic of page growth, I'm going to focus on the volume of video that pages contain. That's because video seems to have had a surge in growth, which should be a cause of concern for anyone who cares about metrics like Largest Contentful Paint, which measures the largest visual element on a page – including videos. (More on that below.)
JavaScript – A page can have a relatively low JS weight but still suffer from JS-inflicted performance problems. Even a single 100 KB third-party script can wreak havoc with your page. The more scripts on your page, the greater the risk. It’s not enough to focus solely on blocking JavaScript. It’s possible for your pages to contain zero blocking resources and still have less-than-optimal performance because of how your JavaScript is rendered. That’s why it’s so important to understand CPU usage on your pages, because JavaScript consumes more CPU than all other browser activities combined. While JavaScript blocks the CPU, the browser can’t respond to user input. This creates what’s commonly called “jank” – that annoying feeling of jittery, unstable page rendering.
CSS – Like JavaScript, CSS doesn’t have to be bulky to cause problems. Poorly executed stylesheets can create a host of performance problems, ranging from stylesheets taking too long to download and parse, to improperly placed stylesheets that block the rest of the page from rendering. And, similar to JavaScript, more CSS files equals more potential trouble.

How does page bloat hurt Core Web Vitals?

Core Web Vitals are a Google search ranking factor. Given that Google continues to dominate search usage, you should care about Vitals alongside the other metrics you should be tracking.

Page bloat can have a huge impact on your Vitals:

Cumulative Layout Shift – Excessive resources can contribute to a poorer CLS score, as more elements shift on the page.
Largest Contentful Paint – LCP measures when the largest visual element (image or video) in the viewport finishes rendering. Heavier visual elements can take much longer to render, especially videos. If you're serving huge videos that take several seconds to fully render, it could be hurting your LCP times.
Interaction to Next Paint & Total Blocking Time – Excessive and/or non-performant JavaScript can hurt interactivity metrics, like INP, FID, and TBT. The more scripts on your pages, the greater the risk.

How does page bloat hurt your business?

A Google machine-learning study I helped with a few years ago found a few strong predictors of whether or not a page resulted in a conversion, ranked in the following order:

The total number of page elements was the single greatest predictor of conversions.
The number of images on the page was the second greatest predictor.
The more scripts there were in a series of pages in a session, the less likely that session was to convert.

Image size is another issue, as excessive image weight hurts your SEO ranking in Google Image Search. Given that Image Search comprises upwards of 26% of Google searches, this is something you should care about. (You can dive deeper into image optimization and SEO tips in this article in Google Search Central.)

How does page bloat hurt your visitors?

In his excellent series of blog posts, The Performance Inequality Gap, Alex Russell makes a compelling case that serving huge pages is an "ethical crisis for front end":

Meanwhile, sites continue to send more script than is reasonable for 80+% of the world's users, widening the gap between the haves and the have-nots...

Essential public services are now delivered primarily through digital channels in many countries. This means what the frontend community celebrates and promotes has a stochastic effect on the provision of those services — which leads to an uncomfortable conversation because, taken as a whole, it isn't working.

Pervasively poor results are part of why responsible public sector organisations are forced to develop HTML-first, progressive enhancement guidance in stark opposition to the "frontend consensus".

This is an indictment: modern frontend's fascination with towering piles of JavasScript complexity is not delivering better experiences for most users.

HTTP Archive research: Background and caveats

Before we get into the analysis, some context:

The numbers cited below all come from the HTTP Archive. I looked at the top 1M sites and top 1K sites, focusing on median and 90th percentile numbers. This is to try to understand how a "typical" page might perform, as well as pages in the "longtail". (It's super important to understand longtail performance. If 10% of your visitors are suffering, that's a lot of users!)
These numbers should not be taken as a benchmark for your own site. You haven't necessarily achieved anything great if your pages are smaller than this, nor have you failed by default if your pages are bigger.

Desktop: The median page has grown 6% in one year

If you look at the year-over-year increase from 2087.2 to 2214.3 KB, you're forgiven if your initial reaction is "that's not too bad". But 6% growth is significant. At that rate, the median page will be close to 3 MB within five years. That's not trivial.

Even if you consider this growth not too worrisome, it's important to keep in mind that it's just an aggregated number, which masks the more dramatic increases in specific resource types, such as JavaScript and video.

Desktop: Median JavaScript weight increased by 14%

Almost 627 KB of JavaScript – spread across the 24 JS requests contained on the median page – is a lot of JavaScript.

Each script on your page represents a potential single point of failure, especially if the script is render-blocking or has an excessive Long Tasks time. The more scripts on your page, the greater the performance risk. Even if the script doesn't block the page from rendering, excessive and unoptimized JavaScript can hurt interactivity metrics like Interaction to Next Paint and Total Blocking Time.

Desktop: Median video weight increased by 32%

In other words, the amount of video being served has increased by almost one-third. That's huge.

As mentioned earlier in this post, Largest Contentful Paint (one of Google's Core Web Vitals) measures when the largest visual element (image or video) in the viewport *finishes* rendering. Heavier visual elements can take much longer to render, especially videos. If you're serving huge videos to your users, it could be hurting your LCP times, which could then be hurting your Google search ranking.

(If you're adding up the numbers and wondering how the video weight makes sense given the overall page weight, it's important to keep in mind that these medians are calculated based on pages that contain these resources. Of the million URLs tracked by the HTTP Archive, not all contain video.)

Mobile: The median page is smaller (but there's a catch)

It's genuinely heartening to see that, overall, pages served to mobile haven't gotten bigger.

However, when we look at the breakdown across resource types, we can see growth in video and JavaScript, as we did with desktop:

Mobile: Median video weight increased by 62%

It's hard to decide which is more concerning: the fact that video weight has increased so dramatically in just one year, or the fact that the median page that serves video to mobile devices now serves more than 2.5 MB of video.

It's easy to forget that not everyone has unlimited data. If you're serving unasked-for video to your users, you could be eating up their data plans.

It's also important to keep in mind that Google's recommended 2.5s threshold for "good" LCP applies to both mobile and desktop experiences – it doesn't differentiate. If your videos are the hero element on your pages, this could be causing your LCP times for mobile to be dangerously slower than what Google demands.

(If you're skimming this post and missed the earlier note about video weight, I'll mention again that these medians are calculated based on pages that contain these resources. Of the million URLs tracked by the HTTP Archive, not all contain video. That's why the numbers for median total size don't "match" the median video weight.)

Mobile: Median JavaScript weight increased by 11%

Almost 580 KB of JavaScript – spread across the 24 JS requests contained on the median page – is a lot of JavaScript. Given that mobile devices tend to have slower CPUs than desktop, parsing all that JavaScript comes with a performance cost.

Median vs 90th percentile: Total size and resources

Things get really interesting when we compare numbers at the median versus 90th percentile. As said earlier, this is to try to understand how a "typical" page might perform, as well as pages in the "longtail". Why is it important to understand longtail performance? Think of it this way: if 10% of your visitors are suffering, that's a lot of visitors!

90p page is almost 4X larger than median page

The 90th percentile page served to desktop is close to 9MB, making it almost four times larger than the median page.

The 90th percentile page served to mobile is similarly huge, coming in at almost 8 MB.

90p page contains almost 3X the number of requests

The median page served to desktop contained a total of 82 resources (HTML, JS, images, videos, etc.) compared to the 90th percentile with a total of 233 resources. In other words, the 90p page contains almost three times the number of requests.

You can see the breakdown of requests across different resource types:

Not surprisingly, images and JS account for the majority of requests. I was surprised to see how many requests come from CSS and HTML. And I was very surprised to see that the 90p page serves a total of 12 videos!

Desktop: 90p JavaScript weight increased by almost 20%

20% growth in a single resource is alarming on its own. Also alarming is the fact that pages with almost 2 MB of JavaScript – across 77 scripts –are being served to users. It bears repeating: excessive JS can hurt all your user-oriented performance metrics. If the scripts are blocking the main thread, then they're affecting the page looks to your users. And if those scripts are slow to respond to user interaction, then they're also affecting how the page feels to your users.

Mobile: 90p JavaScript weight increased by 17%

As stated above, this growth is concerning, alongside the fact that such a massive amount of CPU-intensive JavaScript is being served to mobile devices. Phone processors have gotten better, but they're still no match for a desktop. Next time you're visiting a site and your phone starts to heat up in your hand, remember this chart.

Desktop: 90p video weight increased by almost 28%

Mobile: 90p video weight increased by 77%

Yes, mobile video weight at the 90th percentile is actually greater than desktop video weight – not completely surprising when you consider that 12 or more video resources are being served to mobile devices.

Top 1K vs Top 1M

The top thousand pages tend to belong to leading sites. Those businesses perhaps have more awareness of the importance of performance optimization, along with the resources to monitor and tackle performance issues. This might be why the top 1K pages – across desktop and mobile – tend to be smaller and leaner.

Desktop image weight

Desktop video weight

Mobile image weight

Mobile video weight

The important takeaway here isn't that if your site isn't in the top 1K you should just assume that poor performance comes with the territory. Instead, this is a great opportunity to audit your key pages and see what you can delete, defer, or optimize, so that you can emulate what bigger players are doing.

Is it possible to have large pages that deliver a good user experience?

Yes. While page size can be a red flag for real performance issues, if you care about user experience, you need to take a closer look at how your pages are built to see if the size and complexity of your pages actually affect how fast your site feels to your users.

It's not enough to look at crude metrics like total requests and size. You need to know:

How many of your requests are blocking requests?
If your page contains blocking requests, how many of them occur in the critical rendering path? That is, how many blocking requests are there before key page metrics like Start Render and Largest Contentful Paint?
How many of your potentially problematic requests come from third parties, and how do you maintain visibility into how they're performing?
Are the most important images on your page the first images to render? How quickly do they show up?

Amazon is a good example of a site that serves large, fast pages, as you can see in this recent look at our Retail Page Speed Benchmarks:

In the waterfall chart for this test run, you can see why the Amazon home page ranks fastest for Start Render. There's only a small number of resources on the critical rendering path, and they're all highly optimized.

So, despite the fact that the page contains 459 requests and weighs in at 4,412 KB – far beyond the median sizes shared above...

...the page has a Start Render time of 0.4 seconds, a Largest Contentful Paint time of 0.63 seconds, and a Last Painted Hero time of 3.4 seconds.

To recap: yes, you can have pages that are large and fast, but you need to be mindful of the critical rendering path.

How much content should you be serving?

Having said all that, making your pages as small as possible is in the best interest of your users who don't have access to fast networks and devices. Alex Russell suggests these per-page content targets for 2023:

150 KB of HTML, CSS, images, and render-blocking font resources
No more than 300-350 KB of JavaScript

Are these targets aggressive? Yes. Will you be able to meet them? Possibly not. But as the saying goes, don't let perfect be the enemy of good.

Takeaways

I meet with so many people who build and optimize websites. When we investigate how their pages are built, I routinely witness surprise at spotting things like ghost scripts, huge unoptimized images, and blocking resources they weren't aware of. These are smart people. The problem isn't them – it's the scale of their sites, the speed of their release cycles, and the number of people who touch each page.

We're never going to get our lean, pre-1999, under-1MB web pages back. But we can regain control over the pages we have today.

1. Understand the critical rendering path for each page

Your pages probably have a some dead weight on them, and some of that weight is unoptimized. Too much stuff means you can't see the forest for the trees. The key to a good user experience is quickly delivering the most important content first. Here are some great resources for analyzing and optimizing the critical rendering path.

2. Make sure everyone who touches a page understands the performance impact of what they do

All the fancy performance monitoring tools in the world can't help you if you don't have a strong performance culture at your organization. Here are some tips and best practices to help on that journey.

3. Use performance budgets to fight regression

Page bloat happens when people stop paying attention. Pages need to be monitored consistently over time. Integrating performance testing into your CI/CD process is a great way to fight regression, especially if you combine this with creating performance budgets. By creating performance budgets for key metrics – such as Start Render, Largest Contentful Paint, and various page size and weight metrics – you can get alerted when they go out of bounds.

4. Don't assume hardware and networks will mitigate page bloat

Increased page size and complexity is not fully mitigated by faster devices and networks, or by our hard-working browsers. Clearly we need to keep talking about it. We need to understand how ever-growing pages work against us. And we need to have strategies in place to understand and manage our pages.

5. Don't forget to monitor longtail performance

While some of your users may have newer devices and speedy networks, not all are this lucky. If you're using a real user monitoring tool, keep an eye on your performance metrics at the 75th and 95th percentiles so you have an understanding of your site's less-than-optimal performance.

Questions or feedback?

I'd love to hear your thoughts and insights. If you're interested in tracking page size and complexity for your own site, we'd love to have you try SpeedCurve for free!

SpeedCurve Recipes: Multi-step test scripting made easy!

Mon, 05 Jun 2023 00:00:00 +1200

You may already know that SpeedCurve lets you test multiple URLs for a site across a wide selection of browsers, devices, connection types, and geolocations.

You may also know that SpeedCurve lets you add synthetic scripts to your test settings, which lets you easily do things like simulate a repeat view or block a third party.

But did you also know that we've made it easy to simulate more complex use cases? These include:

User journeys through your site
Checkout processes
Submit a login form
SPA and AJAX navigations
Set cookies

Keep reading to learn how Recipes make it easy for you to test a variety of scenarios that can help you understand how your visitors are experiencing your site – and how to improve their experience!

What is a Recipe?

SpeedCurve Recipes are detailed walkthroughs – including sample scripts – of multi-step tasks.

You still have the ability to write your own custom scripts from scratch and add them to your SpeedCurve test settings. With Recipes, we want to make your testing easier, so you have more time to do the important work of making your pages faster for your users.

Here are some of the most-used Recipes:

Simulate a user journey – Use data logging to complete a multi-step script that measures an individual page.
Simulate a checkout process – Simulate a user journey through the checkout process.
Simulate a user journey with a SPA or AJAX navigation – Step through a user journey that includes a hard navigation as well as a SPA or AJAX navigation in the flow.
Submit a login form – Example scripts that authenticate the browser session in different ways.
Block a request (such as a third party) – Block a specific domain, or block everything *but* a specific domain.
Override DNS in a script – This script creates a fake DNS entry, forces a resolution to the localhost, sets a cookie on the domain, and navigates to the URL.

We're always adding more, so let us know if you'd like to have one that you can't find in our Recipe book.

How to add Recipes to your tests

To use a Recipe in SpeedCurve, you need to:

1. Copy the Recipe template you want to use

Browse the Recipe book to find the script you need. Here's the script template for the 'Simulate a User Journey' recipe:

2. Customize the script template

For example, in the sample above, you would add your own URLs for the first and last pages in the user flow.

3. Insert the script into your synthetic test settings

You can add a script to any URL in your synthetic settings by opening up the site, and then clicking the tag icon to the right of the URL. That will expand the test parameters for that image, including a window for you to insert your script.

IMPORTANT: You need to add separate versions of the same URL if you want to test it in multiple ways. For example, if you want to test the same URL as a regular first view as well as a repeat view, you'll need to create a separate test URL for each view, and then apply the 'Repeat View' script to the second URL.

RUM Recipes

We've also recently started to create Recipes for real user monitoring. For these Recipes, you can add custom JavaScript to your pages:

Track size for a single resource – Capture a number of size metrics (total bytes, compressed/uncompressed size, compression ratio, HTTP headers) for a specific resource.
Track size for a group of images – Find all <img> resources that come from a specific domain, and then capture a number of metrics, such as largest downloaded image size, total bytes for all images, largest non-cached image size, and more.

We'll be adding more RUM Recipes in the future. Let us know what you'd like to see!

Questions or feedback?

We hope that Recipes help make your testing workflow easier, so you have more time to focus on making your pages faster. If you have any questions about how to add a Recipe to your testing, send us a note at support@speedcurve.com.

Farewell FID... and hello Interaction to Next Paint!

Wed, 10 May 2023 00:00:00 +1200

Today at Google I/O 2023, it was announced that Interaction to Next Paint (INP) is no longer an experimental metric. INP will replace First Input Delay (FID) as a Core Web Vital in March of 2024.

It's been three years since the Core Web Vitals initiative was kicked off in May 2020. In that time, we've seen people's interest in performance dramatically increase, especially in the world of SEO. It's been hugely helpful to have a simple set of three metrics – focused on loading, interactivity, and responsiveness – that everyone can understand and focus on.

During this time, SpeedCurve has stayed objective when looking at the CWV metrics. When it comes to new performance metrics, it's easy to jump on hype-fuelled bandwagons. While we definitely get excited about emerging metrics, we also approach each new metric with an analytical eye. For example, back in November 2020, we took a closer look at one of the Core Web Vitals, First Input Delay, and found that it was sort of 'meh' overall when it came to meaningfully correlating with actual user behavior.

Now that INP has arrived to dethrone FID as the responsiveness metric for Core Web Vitals, we've turned our eye to scrutinizing its effectiveness.

In this post, we'll take a closer look and attempt to answer:

What is Interaction to next Paint?
How does INP compare to FID?
What is a 'good' INP result?
Will there be differences between INP collected in RUM vs. Chrome User Experience Report (CrUX)?
What correlation does INP have with real user behavior?
When should you start caring about INP?
How can you see INP for your own site in SpeedCurve?

Onward!

What is Interaction to Next Paint?

Interaction to Next Paint (INP) is intended to measure how responsive a page is to user interaction(s). This is measured based on how quickly the page responds visually after a user interaction (i.e. when the page paints something in the browser's next frame after you interact with it). Because INP measures actual user interactions, it can only be monitored using a real user monitoring (RUM) tool.

For the purposes of INP, an interaction is considered any of the following:

Mouse click
Touchscreen tap
Key press

INP is measured using the Event Timing API, which as of today is supported in Chrome, Edge, Firefox and Opera. Alas, there is no support for Safari.

For a more detailed explainer of INP – including a breakdown of what's in an interaction, how INP is calculated and more – see this post by Jeremy Wagner from the Google Chrome team.

What is a 'good' INP number?

INP is a new metric, so the suggested thresholds from Google are subject to change. For now, those thresholds are as follows:

Good – Under 200ms
Needs improvement – Between 200-500ms
Poor – More than 500ms

Note that these thresholds are all based on RUM data at the 75th percentile.

What can affect INP?

Most user interactions will take place after the page is initially loaded, so it's important to understand that the measurement is typically captured after most of your other metrics (i.e. FCP, LCP, Load, etc.) have occurred. Oftentimes the page seems dormant at this point, but this is not always the case.

Things that can affect INP include:

Long-running JavaScript event handlers
Input delay due to Long Tasks blocking the main thread
Poorly performing JavaScript frameworks
Page complexity leading to presentation delay

For some ideas around optimizing INP, I've included some great resources at the bottom of this article.

How do INP numbers compare to FID?

There are a few important things to understand about FID versus INP. Most importantly, INP includes more than just input delay. Processing time and presentation delay are also included. This is inclusive of time spent running event handlers, as well as rendering/painting delay. The following is a comparison of FID and INP at the 75th percentile for Mobile and Desktop for RUM data collected from SpeedCurve.

It's good to see INP values so much higher than FID, which was expected but still nice to see. While desktop numbers look encouraging, mobile is clearly an opportunity for focus. This isn't a big surprise considering the impact JavaScript has on lower-powered mobile devices.

What can I expect when comparing CrUX data to RUM?

There are some important caveats to understand when comparing INP measured from CrUX (Chrome user experience report) to INP from RUM. While RUM and CrUX do their best to align, there are other times where you may see differences between the two. Barry Pollard wrote a comprehensive post on this found here.

For INP specifically, there a a few gotchas to consider:

For CrUX, INP can change during the lifecycle of the page. Today, SpeedCurve is reporting INP for the first interaction. Other RUM tooling may also be limited by when the beacon is being sent, likely only collecting interactions that occur before the page is loaded.
RUM, which uses JavaScript APIs, will not be able to collect INP from within iframes for the page they are measuring, while CrUX will.
INP is collected for all supported browsers in SpeedCurve RUM. CrUX is a subset of Chrome users only.

How does INP correlate with user behavior?

While understanding INP is important, does it really correlate with how users interact with your site? It's easy to focus so much on improving metrics such as Core Web Vitals that we lose site of the bigger picture. Creating a delightful experience for end users is at the heart of web performance.

One way of understanding how metrics such as INP align with users is to correlate with outcomes, such as conversion rates. Here is an example of how INP correlated with conversion rates for four different sites.

Not surprisingly, we see the impact is different based on the slope of the conversion line, as well as the distribution of INP across user sessions. However, it's notable that there is an overall negative correlation between INP and conversion. This tells us that yes, INP seems to be a meaningful metric when it comes to user-perceived performance.

Measuring INP in SpeedCurve

We've added INP everywhere that your Vitals are showcased throughout SpeedCurve – including the Home, Vitals, Performance, and JavaScript dashboards.

FID hasn't disappeared from the dashboards yet, but likely will as we get closer to March 2024. If you're still interested in tracking FID at that point, you can do that by creating custom charts in your Favorites dashboards.

Summary

We hope that focusing on Interaction to Next Paint will drive more attention to the issues that were being glossed over by FID. There has been a lot of back and forth over whether or not improving your Core Web Vitals has a significant impact on search rankings. I don't have any data to make an informed statement on that question.

However, I will say that if you're solely optimizing your site for SEO purposes, perhaps you're missing the bigger picture. Doing the right thing for your users – including creating a responsive and delightfully fast experience – will have a lasting impact on your brand and yes, your revenue.

Resources

A Complete Guide to Performance Budgets

Tue, 02 May 2023 00:00:00 +1200

It's easier to make a fast website than it is to keep a website fast. If you've invested countless hours in speeding up your pages, but you're not using performance budgets to prevent regressions, you could be at risk of wasting all your efforts.

In this post we'll cover how to:

Use performance budgets to fight regressions
Understand the difference between performance budgets and performance goals
Identify which metrics to track
Validate your metrics to make sure they're measuring what you think they are – and to see how they correlate with your user experience and business metrics
Determine what your budget thresholds should be
Focus on the pages that matter most
Get buy-in from different stakeholders in your organization
Integrate with your CI/CD process
Synthesize your synthetic and real user monitoring data
Maintain your budgets

This bottom of this post also contains a collection of case studies from companies that are using performance budgets to stay fast.

Let's get started!

Background: How performance budgets work

1. What is a performance budget?

A performance budget is a threshold that you apply to the metrics you care about the most. You can then configure your monitoring tools to send you alerts – or even break the build, if you're testing in your staging environment – when your budgets are violated.

2. What should a performance budget look like?

A good performance budget chart, such as the one above, should show you:

The metric you're tracking
The threshold you've created for that metric
When you exceed that threshold
How long you stayed out of bounds
When you returned to below the threshold

3. How do you determine budget thresholds?

A good practice is to:

Look at your last 2-4 weeks of data for a given metric
Identify the worst number
Set your performance budget for that number

In the example above, you can see a time series chart that shows Largest Contentful Paint times over a one-month period. The slowest LCP time is 8.64 seconds, therefore the performance budget – represented by the red line – is set for 8.64 seconds. As the person responsible for the performance of this page, you don't want to see it get worse than this while you work to make things faster.

4. Performance budgets vs. performance goals

Your performance budgets are NOT the same as your performance goals.

Performance goals are aspirational. They answer the question "How fast do I want to be eventually?"

Performance budgets are practical. They answer the question "How can I keep my site from getting slower while I work toward my performance goals?"

Let's continue with the example above, where the worst LCP time was 8.64 seconds, making that the most pragmatic performance budget. At the same time, the person responsible for this page might know that they would like the LCP time to eventually be 2.5 seconds or less, which is Google's recommendation for page experience and SEO. That's the goal, but it's not the budget.

It's important to have your 2.5 second performance goal in mind, but it's not helpful to make that number your performance budget. If you do, you'll end up with a chart that looks like this:

Why this chart isn't helpful:

It's demoralizing. It looks like a history of failure that's intended to shame you, not help you.
It's not actionable. Because you've already violated your budget, you won't get alerts if performance degrades even further.
It's ignorable. Because it's demoralizing and not actionable, you'll quickly learn to ignore it or rationalize it away.

5. Why do you need performance budgets?

Performance budgets mitigate two of the biggest challenges you probably face in your daily life: not enough time, plus too many people touching the site.

You're busy. You don't have time to check your charts daily to make sure your latest deploy hasn't introduced any regressions. After you've set up performance budgets, you can relax knowing that everything is fine until you get an alert in your inbox or Slack channel.

Your site has a number of people contributing content, such as unoptimized images and third-party tags that have the potential to seriously hurt the speed of your pages. You might not know that a new 1MB hero image is suddenly slowing down an important landing page, but the performance budget you created for tracking image weight violations knows.

Now that you understand the "what" and "why", let's talk about getting started with your own budgets.

Get started with performance budgets

6. Which metrics should you start with?

With hundreds of potential performance metrics to track, this is a huge question. While you can (and arguably should) track many metrics across your site, you don't need to set budgets for all of them. In fact, I strongly urge you not to do that.

Instead, I recommend starting with what I call Minimum Viable Budgets. Even if you create performance budgets for just one or two metrics, that's a great start. That approach lets you:

Get comfortable with the mechanics of budgets
Confirm that your budgets are working (i.e., you're catching regressions in a timely manner)
Teach other people in your organization why these metrics matter
Avoid alert fatigue

Here are a few metrics to consider, and why:

> Backend (Synthetic and RUM)

Also known as Time to First Byte (TTFB), this is the time from the start of the initial navigation until the first byte is received by the browser (after following redirects). Even if you're not responsible for backend time (e.g., you're a front-end developer), it's a good idea to track it because it can delay all your other metrics.

> Start Render (Synthetic and RUM)

Start Render time is measured as the time from the start of the initial navigation until the first non-white content is painted to the browser display. Even if that first visible paint isn't a meaningful amount of content, it's still a useful signal that the page is working, and it can help stop users from bouncing.

People don't talk much about Start Render these days, perhaps because newer, shinier metrics have emerged. But I've participated in many usability studies that have found a strong, consistent correlation between Start Render time, user engagement, and business metrics.

> Largest Contentful Paint (Synthetic and RUM)

Largest Contentful Paint (LCP) is one of Google's Core Web Vitals. LCP is the time at which the largest visible element in the viewport is rendered. It's only tracked on certain elements, e.g., IMG and VIDEO.

LCP is only available in Chromium-based browsers, so if you have a significant number of users that come in via other browsers, you should also consider tracking Last Painted Hero, below.

> Last Painted Hero (Synthetic)

Last Painted Hero (LPH) is a synthetic metric that's measurable in any browser. (Fun fact: Largest Contentful Paint was partially inspired by Last Painted Hero.) LPH shows you when the last piece of critical content is painted in the browser. It's a handy metric for knowing when all your important content has rendered.

> Cumulative Layout Shift (Synthetic and RUM)

Cumulative Layout Shift (CLS) is another one of Google's Core Web Vitals. CLS is a score that captures how often a user experiences unexpected layout shifts as the page loads. Elements like ads and custom fonts can push important content around while a user is already reading it. A poor CLS score could be a sign that page feels janky to your users.

> Interaction to Next Paint (RUM)

While Interaction to Next Paint (INP) is not a Core Web Vital, it's currently in the experimental stage as a potential lead-up to being added, so it bears mentioning here.

INP measures a page's responsiveness to individual user interactions. According to the Chrome dev team:

"INP is a metric that aims to represent a page's overall interaction latency by selecting one of the single longest interactions that occur when a user visits a page. INP logs the latency of all interactions throughout the entire page lifecycle. The highest value of those interactions – or close to the highest for pages with many interactions – is recorded as the page's INP. A low INP ensures that the page will be reliably responsive at all times."

> Long Tasks (Synthetic and RUM)

Long Tasks time is the total time of all your JavaScript tasks over 50ms, from navigation start until the page is fully loaded. Tracking Long Tasks will give you a better understanding of the impact that Long Tasks have on the entire page load and your users. This can be especially handy if you have a lot of third-party tags on your pages, as third parties can be a heavy contributor to excessive Long Tasks time.

> Lighthouse Scores (Synthetic)

Google Lighthouse is an open-source tool that checks your page against rules for Performance, PWA, Accessibility, Best Practice, and SEO. For each of those categories, you get a score out of 100 and recommendations on what to fix. It can be a good idea to track your Performance score to make sure you're not regressing, and then if you do experience a regression, drill down into your audits to identify the cause.

> Page Size and Weight (Synthetic)

If you're concerned about serving huge pages to your mobile users, or if you're worried about site contributors accidentally publishing huge unoptimized image and video files, then you should consider tracking metrics like page size and weight.

In an ideal world, pages served to mobile devices should be under 1 MB – and definitely not more than 2 MB – but I often see pages in excess of 5 MB. Media sites are particularly prone to this issue.

Above is a content breakdown for the home page of a mainstream news site. The page contains 480 requests and is over 13 MB in size. Looking at the resource breakdown, you might want to set specific performance budgets on JavaScript and image size.

> Element Timing (Synthetic and RUM)

Similar to LCP, Element Timing measures when a DOM element is rendered. Unlike LCP, Element Timing allows you (rather than Chrome) to decide which elements you want to measure. And unlike LCP, Element Timing allows you to measure more than one element on a page. (Learn more here.)

Element Timing is a somewhat more advanced metric, so if you're just getting started, you might want to make note of it now and come back to it later, when you're ready.

> User Timing (Synthetic and RUM)

You can create custom metrics to track everything from headlines to call-to-action buttons. Twitter has used custom timers to create a Time to First Tweet metric. Pinterest has created a Pinner Wait Time metric. Using the W3C User Timing spec, you can add timestamps around the specific page elements that matter most to you. (Here's how to add custom timers in SpeedCurve.)

Like Element Timing, User Timing requires some expertise to identify what you want to track and then add the timestamps to your pages, as well as ongoing maintenance. Still, they're worth investigating (if not now, then later) if you have the resources and the need.

7. Focus on key pages

You don't need to apply performance budgets to all your pages. When it comes to the impact of page speed on metrics like conversion rate and bounce rate, some pages are more critical than others.

These are some of the key pages for retail, ranked in order of impact:

Product detail
Product category
Shopping cart
Home

And these are some key pages for media and other sites:

Articles
Search
Home

Keep in mind that your budgets will most likely vary from page to page, because the performance of your pages may differ widely due to how each page is built.

8. Get buy-in from different stakeholders

Everyone who touches a page should understand the performance impact of changes they introduce to that page. They should also collaborate on setting performance budgets and remediating budget violations.

For example, if your marketing team is responsible for adding and maintaining third-party tags, they should:

Have a basic understanding of the metrics – such as Long Tasks time – that measure the performance of each tag.
Collaborate on setting the performance budget – again, based on the worst result over the previous 2-4 weeks – for each metric.
Receive alerts (typically via email, Slack, or whatever webhook you use) when the performance budget is violated.
Participate in identifying and fixing the issue (or at least be cc'ed) and get alerted when the budget has been resolved.

In SpeedCurve, you can set up separate dashboards for each stakeholder group in your organization. You can create charts and performance performance budgets within each dashboard, and then configure alerts to be sent only to specific stakeholders.

Below is an example of a dashboard created for an SEO team. It focuses on the Lighthouse SEO score, as well as Largest Contentful Paint and Cumulative Layout Shift, as those are both Core Web Vitals and therefore important search ranking factors.

Note that in the Lighthouse chart at the top of this dashboard, you can click on any data point to drill down into your Lighthouse test details, where among other things, you can get detailed audits that recommend what you can fix on the page.

9. Use synthetic testing to visually validate your metrics

The metrics mentioned above are not hard-and-fast suggestions. That's because a metric that is relevant and helpful for one page may not be helpful for another. Before you invest the time and energy in setting up performance budgets for a metric, first take a good look at how that metric aligns with your own data.

The easiest way to validate your metrics is to look at rendering filmstrips in your synthetic test data, like this:

In the example above (taken from our Industry Benchmarks) you can see:

Start Render does correlate to content appearing in the viewport.
Largest Contentful Paint doesn't quite align with the appearance of the largest image.
Last Painted Hero, on the other hand, does align with the largest image.
Visually Complete comes in much later and arguably isn't helpful for this page.

Based on these observations, you might choose to focus on Start Render and Last Painted Hero.

If you need to validate more metrics, you can look at your waterfall chart and see how the various metrics line up with the rendering filmstrip, like this:

Using this view, it's relatively fast and easy to see which metrics work or don't work for a given page. It's important to keep in mind that just because a metric isn't relevant for one page, that doesn't necessarily mean it's a bad metric. Often, any variability you might see is due to how the page is built.

10. Use real user monitoring to validate user and business impact

This step might not be essential, but if you can do it, it's a good way to give yourself the confidence that you're tracking the right metrics. Ultimately, you want to know that changes you make to your site – for better or for worse – will directly affect user behaviour and business outcomes.

This is where real user monitoring (RUM) really shines. RUM can track data about bounce rate and conversion rate (along with other user experience and business KPIs). Using this data alongside your performance data, you can create correlation charts that demonstrate the relationship between performance and business results.

In the correlation chart above, you can clearly see that as LCP times get slower, bounce rate gets worse. This chart demonstrates that, for this particular site, LCP time is a good metric to set a performance budget for.

11. Synthesize your synthetic and real user monitoring data

In an ideal world, you're using both synthetic and real user monitoring (RUM). Several metrics are available in both tools, so you can create charts in which you track the same metric in both synthetic and RUM.

(It's important to know that your synthetic and RUM metrics most likely will not match, for reasons explained here. This is nothing to be concerned about. The important thing to track is consistency and changes within a single tool and settings.)

For a metric that's available in synthetic and RUM, such as Start Render or Largest Contentful Paint, you might want to consider this type of setup:

Track the metric for synthetic and RUM within the same chart.

Create the performance budget for the RUM metric, so you get an alert when the budget is violated. This lets you know that real users are experiencing this issue.

Because you're tracking synthetic data in the same chart, you can easily drill down and get detailed test results and diagnostics.

Add a note to the chart, stating when you implemented the necessary fixes. After your fixes go live, you can see (and get an alert) when your metric returns to normal.

This is just one potential configuration. If you're using your RUM and synthetic data in other ways, I'd love to learn more about it! Leave a comment below.

12. Set up alerting (but not too much!)

Avoiding alert fatigue is crucial to the success of your performance budget strategy. If you're just starting out, it's absolutely fine to focus on just a handful of metrics. You can create performance budgets for all of them, but if you're setting up alerting, focus on just setting up alerts for critical metrics such as Backend time, Start Render, and Largest Contentful Paint.

13. Integrate with your CI/CD process

You can integrate your performance budgets and alerts with your CI/CD process. This gives you the ability to do a couple of things:

Run synthetic tests in your staging/development environment and get alerts if any changes you've introduced have caused budget violations before the page goes live. You can even opt to break the build if any of your budgets are violated.
Run tests each time you do a deploy, catching issues immediately after they go live.

Maintain your budgets

Your budgets will ideally change over time, as your various metrics (hopefully) improve. After you've taken the time to create your performance budgets, you want them to stay relevant and helpful.

14. Update your budgets

If you're taking the practical, iterative approach recommended above, then you should revisit your budgets every 2-4 weeks and adjust them (hopefully downward) accordingly.

You should also periodically revisit your metrics – going through the validation steps described in steps 9 and 10 above – to make sure you're still tracking the right things go through the validation Are you still tracking the right metrics?

Case studies

Here's how SpeedCurve users have used performance budgets in a number of different ways to stay fast:

More resources

If you're a SpeedCurve user, these resources will help you get started with performance budgets. If you're not using SpeedCurve yet, signing up for a free trial is easy!

NEW! Lighthouse 10, Core Web Vitals updates, and Interaction to Next Paint

Mon, 10 Apr 2023 00:00:00 +1200

There is a lot of excitement in the world of web performance these days, and April has been no exception! At SpeedCurve, we've been focused on staying on top of the items that affect you the most.

Here is a look at what's new in SpeedCurve:

Support for Lighthouse 10, including metric scoring changes as well as audits
Updated RUM Core Web Vitals, including the much-anticipated addition of Interaction to Next Paint (INP)

All of this work driven by the community is having a big impact in our collective goal to make performance accessible for everyone.

Read on to learn more about these exciting changes!

What changed in Lighthouse 10?

Here's how the update to Lighthouse 10 may affect your metrics.

Scoring

Lighthouse scoring has been updated for the first time since version 8. Metrics and scoring weights include the deprecation of Time to Interactive (TTI) and the increase in weight for Cumulative Layout Shift (CLS).

Weighting adjustments are highlighted in bold below:

First Contentful Paint (FCP): 15
Speed Index: 10
Largest Contentful Paint (LCP): 25
Time To Interactive (TTI): 15
Total Blocking Time (TBT): 30
Cumulative Layout Shift (CLS): 15 -> 25

What impact will this have on my score?

According to Google:

“We expect this to improve most pages' performance scores, since most pages tend to score better on CLS than TTI. In an analysis of 13 million page loads in the latest HTTP Archive run, 90% of those pages would see an improvement in their Lighthouse performance score, with 50% of them seeing a performance improvement of more than 5 points.“

According to us:

"It depends."

Basically, we see a trend similar to the trend observed by Google. Overall, sites saw some improvement in their Lighthouse scores. However, sites that have not-so-great CLS scores might see their Lighthouse Performance score degrade.

If you are delivering a consistent experience AND already have super CLS, you're probably going to see an improvement in your Lighthouse score. It also helps that, in this case, a poor TTI is now being omitted from Lighthouse scoring.

Audits

SpeedCurve leverages the performance audits from Lighthouse throughout your dashboards. With the release of Lighthouse 10, one new performance audit was added and another best practices audit was modified:

Page prevented back/forward (BF) cache navigation – When a site is able to leverage BF cache navigations, users can see a huge benefit and a perceived 'instant' loading of the page. When a BF cache navigation is prevented, Lighthouse lists the causes for the failure.

(Important: Changes to performance for BF cache navigations will likely not show up in your synthetic monitoring tests – with the exception of repeat views, which may see an improvement. However, your RUM data should reflect these improvements. Look for an update to in a future release that will allow closer inspection of BF cache navigations)

Allows users to paste into input fields – This best practices audit has been updated for Lighthouse 10. Previously, the audit checked for password inputs only. Now the audit includes all input fields that are not read-only.

Core Web Vitals updates

Cumulative Layout Shift (CLS)

There was an update to CLS a while ago that didn't quite make sense for us to adopt until now. We've updated our RUM library to include the latest CLS implementation, which includes session windows. This may have a positive impact on CLS scores, especially for longer lived pages. We've updated this for synthetic too!

Interaction to Next Paint (INP)

There has been a lot of buzz about this potential replacement for First Input Delay (FID). Our fingers are crossed that we will say good riddance to FID and that this metric will do a better job of identifying responsiveness issues for applications. Look for an upcoming INP explainer and analysis after we've had a chance to observe it in the wild for a bit.

For now, you can track INP in your Favorites dashboards, but it has yet to be added to the Vitals dashboard or other RUM dashboards due to its experimental status. We quietly added INP on March 20, so if you start tracking it in your Favorites, you should already have a few weeks worth of data to explore.

The recommended thresholds at the 75th percentile are as follows:

Good: < 200ms

Needs Improvement: 200ms - 500ms

Poor: > 500ms

So far, we are seeing that on average most sites are around or above the 200ms threshold. It's important to note that the recommended thresholds may still be adjusted, as this is still an experimental metric.

Interested in tracking INP for your site? Here is a quick explainer video that shows you how to add INP to your custom Favorites dashboard.

Here are some useful links where you can learn more!

NEW! Home, Site, Page and Tests dashboards

Mon, 13 Mar 2023 00:00:00 +1300

Things have been busy over here at SpeedCurve HQ! Coming off of the back of our latest RUM Compare dashboard release, we are super excited to launch four new dashboards to make your life better, your work easier, and your websites faster.

Home - Well, now you have one. We needed a place for folks to land when they were jumping into all that SpeedCurve has to offer. The newly released Home page is a great starting place, pulling together views of both RUM and Synthetic data and directing you on where to dig in next.
Sites - We've rebuilt our Sites dashboard to give you a holistic look across all of the URLs you are monitoring. Start here to understand which areas of your site need the most attention and identify areas you can have the most impact.
Pages - This detailed view gives you trended information for all the metrics you know and love. Easily identify areas of opportunity and how things are performing over time for specific pages you are monitoring.
Tests - We've had a lot of requests for this, so we're happy to deliver it. You can now see a list of your latest synthetic tests and spot any failed tests.

Let's take a look!

Home

This is the new default dashboard you'll see when you jump into SpeedCurve for the first time. If you are using both RUM and Synthetic, you'll see both datasets reflected in this view. Otherwise, the metrics reflected will be from your synthetic data.

Beginning with a snapshot of your Core Web Vitals, this page highlights key areas of interest and a high-level snapshot for the selected site. Links from each section take you to a deeper look within other areas of your dashboards.

If you have integrated SpeedCurve into your CI/CD environment (leveraging the deploy API), or if you are manually running synthetic tests using the 'Test Now' option, you'll see the state of your last deployment compared to the previous. And finally, a 7-day trend is displayed for your most important metrics.

Site Dashboard

This view is intentionally focused on the latest data you've collected for your Site which gives you a great starting place when managing performance across various URLs. What pages have the slowest LCP? Home, Order and Print look like a good place to start digging.

How are users perceiving the visual rendering of all the pages in my site? Which pages should I dive into first?

From the LightHouse audits run against your site, you can quickly see which pages need the most attention across the 5 areas of focus.

An aggregate view of the most critical areas for improvement identifies which recommendations will have the most collective impact across your pages.

Page Dashboard

From the Site dashboard, you can narrow your focus and dive into the new Page dashboard. Here you'll see key metrics for the page trended over time. You can use the filters at the top of the page to further refine your data by browser, region, and date range.

A look at the key rendering moments and a full filmstrip for the page focuses on user-perceived performance.

Now that you understand how the page is performing, you can take a look at how the construction for the page has changed over time and where optimization opportunities exist, such as reducing network requests and their file size.

Finally, a list of prioritized improvements for the page is displayed. Pay special attention to critical recommendations that are flagged with the Core Web Vitals metrics. If you implement those recommendations, you should see an improvement in your Vitals.

Tests Dashboard

We love simplicity, and this dashboard is a great example of why. We have received a lot of feature requests such as "Just show me a list of the tests I've run recently" as well as "How can I quickly see which tests are failing?" This dashboard achieves both!

If you want to refine this view, you can apply filters to show you tests for a specific site, page, region or browser. You can also filter on test status ('succeeded' or 'failed') and test type ('scheduled test' or 'deploy').

Finally, if you have any failed tests we'll show those at the top of the previously mentioned Home page in order to quickly alert you to the failures. You can then click through to the test details to determine why the test failed.

We hope this dashboard will help to quickly inform you so you can take action and protect your synthetic checks budget.

We hope you find these latest additions to SpeedCurve useful. As always, let us know what you think!

Why you need to know your site's performance plateau (and how to find it)

Mon, 06 Mar 2023 00:00:00 +1300

"I made my pages faster, but my business and user engagement metrics didn't change. WHY???"

"How do I know how fast my pages should be?"

"How can I demonstrate the business value of performance to people in my organization?"

If you've ever asked yourself any of these questions, then you could find the answers in identifying and understanding the performance plateau for your site.

What is the "performance plateau"?

The performance plateau is the point at which changes to your website’s rendering metrics (such as Start Render and Largest Contentful Paint) cease to matter because you’ve bottomed out in terms of business and user engagement metrics.

In other words, if your performance metrics are on the performance plateau, making them a couple of seconds faster probably won't help your business.

The concept of the performance plateau isn't new. I first encountered it more than ten years ago, when I was looking at data for a number of sites and noticed that – not only was there a correlation between performance metrics and business/engagement metrics – there was also a noticeable plateau in almost every correlation chart I looked at.

A few months ago someone asked me if I've done any recent investigation into the performance plateau, to see if the concept still holds true. When I realized how much time has passed since my initial research, I thought it would be fun to take a fresh look.

In this post, I'll show how to use your own data to find the plateau for your site, and then what to do with your new insights.

Background

For this new investigation, I selected four sites that experience a significant amount of user traffic. For each site, I used a month's worth of RUM (real user monitoring) data to generate correlation charts.

Correlation charts show the relationship between performance metrics – in these instances, Start Render and Largest Contentful Paint (LCP) – and user engagement (measured as bounce rate). They're a great tool for showing non-technical folks how performance affects the business. (You can also create correlation charts that show the relationship between performance metrics and business metrics, such as conversion rate and cart size, but bounce rate is easier to measure right out of the box with most RUM tools.)

Results

The correlation charts below show the distribution of all visits, with each yellow bar representing a cohort of visits that experienced a given Start Render or LCP time. The blue bar represents the change in bounce rate across all cohorts.

In each of the correlation charts below, I've highlighted:

Optimal speed – The cohort of sessions that correlated with the lowest (aka best) bounce rate for that site
Beginning of the performance plateau – The cohort of sessions where the bounce rate begins to plateau
Median measurement for all visits represented in the chart

Keep reading for observations and takeaways.

Site A

Site B

Site C

Site D

Observations

1. A clear performance plateau emerged for each site

Each site experienced a plateau at which business metrics remained more or less the same as performance continued to degrade.

2. Plateaus emerged for both Start Render and Largest Contentful Paint

I expected the results for Start Render, as it's been around as a page speed metric for many years, and has been proven to correlate to business metrics. LCP is relatively new, and has received a great deal of attention since it was introduced a couple of years ago as part of Google's Core Web Vitals. It's good to see evidence that it's a meaningful metric to track in terms of business goals.

3. The plateau emerges surprisingly quickly in some cases

For example, Site C's performance plateau starts at 400 milliseconds. That's early!

4. There's a lot of variability in the distance between the optimal bounce rate and the plateau

For some sites, you can see a much steeper incline in the curve from optimal to plateau. For some sites (such as Site C), the difference was as little as 300 milliseconds. For others (such as Site A), the gap was as long as 9 seconds.

5. The plateau sometimes started later when looking at LCP

Creating correlation charts in both Start Render and LCP generated interesting results. In two of the four sites I looked at, the charts were roughly comparable. For the other two sites, the plateau started later for LCP than it did for Start Render. This could be attributed to the fact that LCP measures when the largest visual element has completely finished rendering, so it can occur much later than Start Render.

6. For some sites the performance plateau starts well before the median

Predictably, the optimal bounce rate generally correlated to the cohort of sessions that is much faster than the median. A bit more surprisingly, for some sites the performance plateau started well before the median. This could come as a scary revelation for some site owners, because it means that the bulk of your user sessions are occurring on the plateau.

How to measure the performance plateau for your own site

I can't emphasize enough that the examples I've shared are illustrative, not prescriptive. The performance plateau for your site will be different from the plateau for another site. You need to look at your own real user data. (If you're new to performance, you might be interested in this synthetic and real user monitoring explainer.)

Fortunately, the process for identifying the low end of your site’s performance threshold is fairly straightforward. All you need is access to a statistically significant amount of your RUM data, plus whatever analytics tool you use for tracking business or user engagement metrics.

Step 1: Identify the metrics you want to measure

As mentioned above, bounce rate is a good metric to start with, because it's already gathered automatically by most real user monitoring tools.

If you have access to other data sources, then you can create a variety of correlation charts, If run an ecommerce site, then you can measure revenue, cart size, and conversion rate. If you work on a media site, then page views and bounce rate matter.

Step 2: Gather lots of real user data

To ensure that you get statistically relevant results, the more data you can gather, the better. If your dataset is too small, you could get wonky results. When I conducted my investigation, I aggregated millions of transactions that took place over a single month. (If you're interested in trying real user monitoring, you can start a free RUM trial in speedCurve.))

Step 3: Create correlation charts

I've demonstrated how I like to show bounce rate (or whatever business/engagement metric you're plotting) across the distribution of sessions. (If you're a SpeedCurve user, here's how to create correlation charts.)

What to do with your findings

After you've finished your own investigation, you can do a few things with the results:

1. Share your findings within your organization

Correlation charts are a powerful tool for showing stakeholders the impact that site speed has on the business. Even if your results aren't what you hoped they would be, you can use this data to prove the value of continuing to invest in performance.

2. Understand why your business metrics are not improving despite your efforts

This might seem a bit demoralizing, but when you think about it, it's actually helpful to know. When you know where your performance pleateau begins, you can answer the question "Why don't my business or user engagement metrics improve when I make my site faster?" If you improve Start Render from 5 seconds to 3 seconds, but the performance plateau for your site starts at 2 seconds, you haven't yet made Start Render fast enough.

3. Change your performance targets

Set targets for moving more of your users into the cohorts that experience faster Start Render or LCP times. Ideally, improving key site speed metrics for more of your users should improve bounce rate (or whatever user engagement or business metric you're tracking) for more of your users. Ultimately, this is good for your business.

You can use your performance plateau to set goals. Continuing with the example in point 1, above, if you know that the plateau starts at 2 seconds, you can create a Start Render target of 1.5 seconds to work toward.

4. Or don't change your performance targets

In the Site C example, the optimal bounce rate occurs for the 100-millisecond LCP cohort, and the plateau starts just 300 milliseconds later. With a huge amount of work, you might succeed in delivering faster LCP times to more sessions, but would the effort be worth it?

As the close-up view of the chart below shows, the bulk of sessions have speedy LCP times that are at the beginning of the performance plateau. In this case, the chart shows that perhaps you can be satisfied with your efforts, and your focus should be on fighting regressions and staying fast.

5. Create a baseline to measure against

Repeat this exercise periodically – perhaps monthly, or semi-annually, or after a deploy where you've made a number of performance improvements – and compare the correlation charts over time. Ideally, you'll see more of your sessions fall into the faster section of the distribution, before the performance plateau.

Questions? Feedback?

If you experiment with creating correlation charts and plotting the performance plateau for your site, I'd love to hear about your results!

2022 in review: New dashboards, Core Web Vitals enhancements, third-party tracking & more!

Thu, 22 Dec 2022 00:00:00 +1300

Every year feels like a big year, and 2022 has been no exception. Not only did we celebrate our ninth birthday (!!!) we also:

Launched a series of new and improved dashboards to help you identify and fix your performance issues more quickly and efficiently (More on those below.)
Expanded our test agent locations to bring you even more places to test from worldwide
Jumped into in-person events, and were so happy to catch up with some of you at SmashingConf and performance.now() (You can watch videos of my talk about performance budgets and Andy Davies' talk about third parties.)

Keep reading for a full recap of the past year...

Home dashboard

It only took nine years, but we now have a home page! ;-)

Your Home dashboard brings together your Synthetic and RUM data to provide an easy-to-digest – and easy-to-share – snapshot of how your site is performing. You can see at a glance:

Key metrics like Core Web Vitals and your User Happiness score
The current status of your performance budgets
Performance in different geolocations around the world
Your current competitive benchmarks status
Top Lighthouse recommendations for performance improvements

(Hint: If you're considering installing a big-screen performance monitor at your organization, this dashboard is a good candidate for that.)

Synthetic Site dashboard

You may have noticed that your Site dashboard has been given a complete overhaul. It now focuses on helping you spot differences between the performance of your pages, making it easier to identify which pages in your site have performance issues.

When you identify a page that needs further investigation, you can quickly navigate through to your new Synthetic Page dashboard...

Synthetic Page dashboard

You'll find your new Page dashboard to be similar to your old Synthetic Site dashboard. Here you can still find time series charts that show how performance is trending over time across a number of key metrics, including Core Web Vitals.

Contextual Web Vitals recommendations

As you may already know, we run full Lighthouse audits on every page you test in synthetic. The results – including detailed performance optimization recommendations – are available in your test details.

In addition to that, we now badge all performance recommendations with the Core Web Vitals metric (LCP, CLS, FID, and TBT) they affect. Fix those performance issues and you should see an improvement in your Core Web Vitals and Lighthouse scores.

Here are the dashboards where you can find your Vitals-badged performance recommendations:

Home
Vitals
Improve
Synthetic Site
Synthetic Page

RUM Compare dashboard

With your RUM Compare dashboard, you can easily generate side-by-side comparisons for any two cohorts of real user data.

Some of the many reasons you might want to compare RUM cohorts:

Improve Core Web Vitals by identifying the tradeoffs between pages that have different layout and construction
Triage a performance regression related to the latest change or deployment to your site by looking at a before/after comparison
Explore and compare different out-of-the-box cohorts, such as device types, geographies, page labels, and more
Analyze A/B tests or experiments to understand which had the most impact on user behaviour, as well as performance
Optimize your funnel by understanding differences between users that convert or bounce from your site and users who don't
Evaluate CDN performance by exploring the impact of time-of-day traffic patterns

RUM update: Page labels

Page labels are an integral part of getting the most out of dashboards like your RUM Compare dashboard (above). We've added new capabilities for defining page labels with an easy, no-code, server-side (SpeedCurve) option using URL patterns. (This means you now have four options for labeling pages in RUM, depending on your use case.)

RUM update: Performance heat maps

We've added heat maps to your RUM Performance dashboard. These let you quickly spot which page labels or paths are outliers. You can then prioritize and focus on these areas to improve important metrics, such as Largest Contentful Paint. Fixing high-traffic but poor-performing areas of your site will help lift your overall metrics.

Synthetic update: Compare third parties

When comparing any two synthetic tests, we now show you a comparison of common third parties and identify which third parties are unique to each test. This lets you quickly spot changes in common third parties, as well as new third parties that might have introduced performance issues.

Expanded Industry Speed Benchmarks

Page Speed Benchmarks is an interactive dashboard that lets you explore and compare web performance data for leading websites across several industries – from retail to media – over the past year. The dashboard allows you to easily filter by region, industry, mobile/desktop, fast/slow, and key web performance metrics, including Core Web Vitals.

The Industry Benchmarks dashboard is publicly available (meaning you don't need a SpeedCurve account to explore it) and is a treasure trove of meaningful data that you can use for your own research.

Lots of new videos!

We've added a plethora of new videos to give you everything from dashboard tours to best practices for fighting third-party regressions and tracking Core Web Vitals. You can find them in our Support Hub.

Here's to the year that was and the year ahead. :-)

As we enter our tenth year of bringing you the very best performance monitoring tools we can muster – and helping you deliver fast, joyous experiences to your users – we welcome your feedback and suggestions. Let us know your thoughts by leaving a comment or sending us a note at support@speedcurve.com.