SpeedCurve Blog

Five ways cookie consent managers hurt web performance (and how to fix them)

Mon, 15 Apr 2024 00:00:00 +1200

I've been spending a lot of time looking at the performance of European sites lately, and have found that the consent management platforms (CMPs) consistently create a false reality for folks when trying to understand performance using synthetic monitoring. Admittedly, this is not a new topic, but I feel it's important enough that it warrants another PSA.

In this post, I will cover some of the issues related to measuring performance with CMPs in place and provide some resources for scripting around consent popups in SpeedCurve.

What are CMPs and how do they work?

A consent management platform (CMP) is a tool that helps organizations handle user consent for data collection and processing, following privacy rules like GDPR or CCPA. On websites, CMPs handle cookie popups. They tell users about cookies and ask for permission before saving them. Unless you've been abstaining from the internet for the past several years, you know consent managers well.

What challenges do CMPs create for performance?

These are some of the most common performance issues caused by content management platforms.

1. Single Point of Failure (SPOF)

As a third party, CMPs can have performance issues of their own that can affect the user experience. It's common to see the consent scripts blocking by design. This can have an impact on most metrics – such as Start Render and Largest Contentful Paint – downstream.

In this example, the synchronous requests for the CMP timed out, causing an extremely long delay in Start Render due to the SPOF. Consent scripts should be loaded async whenever possible.

2. Identifying the wrong LCP element

Most people assume the LCP element on a page is their main hero or product image, but they're frequently wrong. Oftentimes the text or element in the consent popup is identified as the LCP element. While this may be technically accurate, it's probably not what you want to measure.

In this case, LCP is slower after opt-in, when the hero image has taken 2.5s longer to load.

3. Consent banners can hurt Cumulative Layout Shift scores

Some sites choose to use a consent banner instead of a popup. As Andy Davies discovered, this can sometimes cause CLS issues, depending on how the banner is implemented. In this example, the consent banner causes a large layout shift for first-time viewers, pushing the CLS score well beyond recommended Core Web Vitals thresholds.

Sia Karamalegos from Shopify provided another great example of how cookie notices that are animating position properties vs. using the preferred method of CSS transform can cause massive CLS scores. (Side note: This post is a great read for anyone looking to optimize for CLS their Shopify site)

4. Masking third-party performance

Assuming that you have visibility into your third-party performance is another pitfall when testing the first view of a site synthetically. For some site owners, the difference between the initial experience and an an opted-in experience can be extreme due to the presence of third-party JavaScript.

If you were just looking at metrics, the event timeline comparison below may be a bit of a head-scratcher. LCP is deceptively slower in the first view, due to the late render of the consent popup, whose text block is identified as the LCP element. Meanwhile, Time to Interactive (TTI) is much faster. If you look at the highlighted activity timeline, it's apparent that the third-party JavaScript for the opted-in experience creates a lot of additional CPU activity and Long Tasks.

The opted-in experience also loads an additional 73 third-party requests!

5. Interaction to Next Paint (INP) and cookie consent

In his latest blog post on debugging INP, Andy Davies provides an example (#3) of how the interaction of accepting the consent dialog causes high processing time. This is due to the functions called from the Event Handler. Given the recent announcement that INP has replaced First Input Delay as a Core Web Vital, this is certainly something to look out for.

How do I bypass the CMP?

Testing an opted-in experience is highly recommended. This is possible by setting the correct cookie values or in some cases localStorage entries.

If you're a SpeedCurve user, we've created a Scripting Cookie Consents Guide with scripting examples for some of the more common CMPs. This includes LiveRamp, OneTrust, Quantcast Choice, TrustArc and Usercentrics.

If possible, you should test both experiences – pre- and post-opt-in – and treat them as the unique experiences they are.

It's been said before...

Katie Hempenius, Simon Hearne, Sia Karamalegos, Andy Davies and others have done deep dives into this topic. I've learned a ton from them, and frankly repeated a lot of what they said. Again, this is a PSA that warrants over-communication! ;)

Here are some must-reads by these fine folks:

A Complete Guide to Web Performance Budgets

Wed, 27 Mar 2024 00:00:00 +1300

It's easier to make a fast website than it is to keep a website fast. If you've invested countless hours in speeding up your site, but you're not using performance budgets to prevent regressions, you could be at risk of wasting all your efforts.

In this post we'll cover how to:

Use performance budgets to fight regressions
Understand the difference between performance budgets and performance goals
Identify which metrics to track
Validate your metrics to make sure they're measuring what you think they are – and to see how they correlate with your user experience and business metrics
Determine what your budget thresholds should be
Focus on the pages that matter most
Get buy-in from different stakeholders in your organization
Integrate with your CI/CD process
Synthesize your synthetic and real user monitoring data
Maintain your budgets

This bottom of this post also contains a collection of case studies from companies that are using performance budgets to stay fast.

Let's get started!

Why prioritize fighting regressions?

There's a great quote from Michelle Vu (an engineering lead at Pinterest) from her talk at PerfNow a few years ago:

In other words, why work on continuously filling the bathtub if you're not going to plug the drain?

Background: How performance budgets work

1. What is a performance budget?

A performance budget is a threshold that you apply to the metrics you care about the most. You can then configure your monitoring tools to send you alerts – or even break the build, if you're testing in your staging environment – when your budgets are violated.

2. What should a performance budget look like?

A good performance budget chart, such as the one above, should show you:

The metric you're tracking
The threshold you've created for that metric
When you exceed that threshold
How long you stayed out of bounds
When you returned to below the threshold

3. How do you determine budget thresholds?

A good practice is to:

Look at your last 2-4 weeks of data for a given metric
Identify the worst number
Set your performance budget for that number

In the example above, you can see a time series chart that shows Largest Contentful Paint times over a one-month period. The slowest LCP time is 3.55 seconds, therefore the performance budget – represented by the red line – is set for 3.55 seconds. As the person responsible for the performance of this page, you don't want to see it get worse than this while you work to make things faster.

4. Performance budgets vs. performance goals

Your performance budgets are NOT the same as your performance goals.

Performance goals are aspirational. They answer the question "How fast do I want to be eventually?"

Performance budgets are practical. They answer the question "How can I keep my site from getting slower while I work toward my performance goals?"

Let's continue with the example above, where the worst LCP time was 3.55 seconds, making that the most pragmatic performance budget. At the same time, the person responsible for this page might know that they would like the LCP time to be blazing fast at just 1 second or less. That's the goal, but it's not the budget.

It's important to have your 1-second performance goal in mind, but it's not helpful to make that number your performance budget. If you do, you'll end up with a chart that looks like this:

There are a few reasons why this chart isn't helpful:

It's demoralizing. It looks like a history of failure that's intended to shame you, not help you.
It's not actionable. Because you've already violated your budget, you won't get alerts if performance degrades even further.
It's ignorable. Because it's demoralizing and not actionable, you'll quickly learn to ignore it or rationalize it away.

5. Why do you need performance budgets?

Performance budgets mitigate two of the biggest challenges you probably face in your daily life: not enough time, plus too many people touching the site.

You're busy. You don't have time to check your charts daily to make sure your latest deploy hasn't introduced any regressions. After you've set up performance budgets, you can relax knowing that everything is fine until you get an alert in your inbox or Slack channel.

Your site has a number of people contributing content, such as unoptimized images and third-party tags that have the potential to seriously hurt the speed of your pages. You might not know that a new 1MB hero image is suddenly slowing down an important landing page, but the performance budget you created for tracking image weight violations knows.

Now that you understand the "what" and "why", let's talk about getting started with your own budgets.

Get started with performance budgets

6. Which metrics should you start with?

With hundreds of potential performance metrics to track, this is a huge question. While you can (and arguably should) track many metrics across your site, you don't need to set budgets for all of them. In fact, I strongly urge you not to do that.

Instead, I recommend starting with what I call Minimum Viable Budgets.

Even if you create performance budgets for just one or two metrics, that's a great start. That approach lets you:

Get comfortable with the mechanics of budgets
Confirm that your budgets are working (i.e., you're catching regressions in a timely manner)
Teach other people in your organization why these metrics matter
Avoid alert fatigue

Here are a few metrics to consider, and why:

> Backend (Synthetic and RUM)

Also known as Time to First Byte (TTFB), this is the time from the start of the initial navigation until the first byte is received by the browser (after following redirects). Even if you're not responsible for backend time (e.g., you're a front-end developer), it's a good idea to track it because it can delay all your other metrics.

> Start Render (Synthetic and RUM)

Start Render time is measured as the time from the start of the initial navigation until the first non-white content is painted to the browser display. Even if that first visible paint isn't a meaningful amount of content, it's still a useful signal that the page is working, and it can help stop users from bouncing.

People don't talk much about Start Render these days, perhaps because newer, shinier metrics have emerged. But I've participated in many usability studies that have found a strong, consistent correlation between Start Render and business and user engagement metrics such as conversions and bounce rate.

Other things that make Start Render a must-watch: It's available in both synthetic and real user monitoring tools, and it's broadly supported across browsers. This is hugely important if you care about understanding site speed for all your users, not just certain cohorts.

> Largest Contentful Paint (Synthetic and RUM)

Largest Contentful Paint (LCP) is one of Google's Core Web Vitals. LCP is the time at which the largest visible element in the viewport is rendered. It's only tracked on certain elements, e.g., IMG and VIDEO.

However, there are a number of caveats to using LCP, such as the fact that different elements can be measured for first-time versus repeat views, or for desktop versus mobile views.

Also, LCP is only available in Chromium-based browsers. If you have a significant number of users that come in via other browsers, you should consider tracking Last Painted Hero, below.

> Last Painted Hero (Synthetic)

Last Painted Hero (LPH) is a synthetic metric that's measurable in any browser. (Fun fact: Largest Contentful Paint was partially inspired by Last Painted Hero.) LPH shows you when the last piece of critical content is painted in the browser. It's a handy metric for knowing when all your important content has rendered.

> Cumulative Layout Shift (Synthetic and RUM)

Cumulative Layout Shift (CLS) is another one of Google's Core Web Vitals. CLS is a score that captures how often a user experiences unexpected layout shifts as the page loads. Elements like ads and custom fonts can push important content around while a user is already reading it. A poor CLS score could be a sign that page feels janky to your users.

> Interaction to Next Paint (RUM)

Interaction to Next Paint (INP) is another Core Web Vital. INP measures a page's responsiveness to individual user interactions.

Like LCP, INP is only available in Chromium-based browsers, so if you have a significant number of users that come in via other browsers, you should also consider tracking other responsiveness metrics, such as Total Blocking Time (TBT).

> Total Blocking Time (Synthetic)

Total Blocking Time (TBT) lets you know how much time the various scripts on your page are blocking rendering. Because slow JavaScript is a major cause of delayed responsiveness, TBT is a good proxy for responsiveness metrics like INP.

As a bonus, TBT shows you all the Long Tasks on a page. (More on this below.)

> Long Tasks (Synthetic and RUM)

Long Tasks time is the total time of all your JavaScript tasks over 50ms, from navigation start until the page is fully loaded. Tracking Long Tasks will give you a better understanding of the impact that Long Tasks have on the entire page load and your users.

This can be especially handy if you have a lot of third-party tags on your pages, as third parties can be a heavy contributor to excessive Long Tasks time. Because you're measuring in synthetic, you can also get a detailed list of all the scripts on a page, along with their Long Task times.

And if you're wondering, yes, Long Tasks correlate to business metrics like conversion rate.

> Lighthouse Scores (Synthetic)

Google Lighthouse is an open-source tool that checks your page against rules for Performance, PWA, Accessibility, Best Practice, and SEO. For each of those categories, you get a score out of 100 and recommendations on what to fix. It can be a good idea to track your Performance score to make sure you're not regressing, and then if you do experience a regression, drill down into your audits to identify the cause.

> Page Size (Synthetic)

If you're concerned about serving huge pages to your mobile users, or if you're worried about site contributors accidentally publishing huge unoptimized image and video files, then you should consider tracking metrics like page size and weight.

In an ideal world, pages served to mobile devices would be under 1 MB – and definitely not more than 2 MB – but I often see pages in excess of 10 MB. Media sites are particularly prone to this issue.

Above is a content breakdown for the home page of a mainstream news site. The page contains 725 requests and is over 17 MB in size. Looking at the resource breakdown, I would want to set performance budgets on JavaScript weight (8,680 KB) and image size (6,295 KB). Wow!

> Image Size (Synthetic)

As said, if your pages contain a number of images – and if you have multiple content contributors touching your site – a specific performance budget for image size is a highly recommended guardrail.

> Element Timing (Synthetic and RUM)

Similar to LCP, Element Timing measures when a DOM element is rendered. Unlike LCP, Element Timing allows you (rather than Chrome) to decide which elements you want to measure. And unlike LCP, Element Timing allows you to measure more than one element on a page. (Learn more here.)

Element Timing is a somewhat more advanced metric, so if you're just getting started, you might want to make note of it now and come back to it later, when you're ready.

> User Timing (Synthetic and RUM)

You can create custom metrics to track everything from headlines to call-to-action buttons. Twitter has used custom timers to create a Time to First Tweet metric. Pinterest has created a Pinner Wait Time metric. Using the W3C User Timing spec, you can add timestamps around the specific page elements that matter most to you. (Here's how to add custom timers in SpeedCurve.)

Like Element Timing, User Timing requires some expertise to identify what you want to track and then add the timestamps to your pages, as well as ongoing maintenance. Still, they're worth investigating (if not now, then later) if you have the resources and the need.

7. Focus on key pages

You don't need to apply performance budgets to all your pages. When it comes to the impact of page speed on metrics like conversion rate and bounce rate, some pages are more critical than others.

These are some of the key pages for retail, ranked in order of impact:

Product detail
Product category
Shopping cart
Home

And these are some key pages for media and other sites:

Articles
Search
Home

Keep in mind that your budgets will most likely vary from page to page, because the performance of your pages may differ widely due to how each page is built.

8. Get buy-in from different stakeholders

Everyone who touches a page should understand the performance impact of changes they introduce to that page. They should also collaborate on setting performance budgets and remediating budget violations.

For example, if your marketing team is responsible for adding and maintaining third-party tags, they should:

Have a basic understanding of the metrics – such as Long Tasks time – that measure the performance of each tag.
Collaborate on setting the performance budget – again, based on the worst result over the previous 2-4 weeks – for each metric.
Receive alerts (typically via email, Slack, or whatever webhook you use) when the performance budget is violated.
Participate in identifying and fixing the issue (or at least be cc'ed) and get alerted when the budget has been resolved.

In SpeedCurve, you can set up separate dashboards for each stakeholder group in your organization. You can create charts and performance performance budgets within each dashboard, and then configure alerts to be sent only to specific stakeholders.

Below is an example of a dashboard created for an SEO team. It focuses on the Lighthouse SEO score, as well as Largest Contentful Paint, Interaction to Next Paint, and Cumulative Layout Shift, as those are both Core Web Vitals and therefore important search ranking factors.

A couple of things to note:

For any metrics that are measurable in synthetic and RUM, it's helpful to track them in the same chart. Set the performance budget on your RUM metric so you get an alert when the budget is violated. Then drill down into the synthetic test data to identify and fix the issue. (More on this further down in this post.)
In any of the charts where synthetic test data is collected, you can click on any data point to drill down into your test details where, among other things, you can get detailed audits that recommend what you can fix on the page.

9. Use synthetic testing to visually validate your metrics

The metrics mentioned above are not hard-and-fast suggestions. That's because a metric that is relevant and helpful for one page may not be helpful for another. Before you invest the time and energy in setting up performance budgets for a metric, first take a good look at how that metric aligns with your own data.

The easiest way to validate your metrics is to look at rendering filmstrips in your synthetic test data, like this:

In the example above (taken from our Industry Benchmarks) you can see:

Start Render does correlate to content appearing in the viewport.
Largest Contentful Paint doesn't quite align with the appearance of the largest image.
Last Painted Hero, on the other hand, does align with the largest image.
Visually Complete comes in much later and arguably isn't helpful for this page.

Based on these observations, you might choose to focus on Start Render and Last Painted Hero.

If you need to validate more metrics, you can look at your waterfall chart and see how the various metrics line up with the rendering filmstrip, like this:

Using this view, it's relatively fast and easy to see which metrics work or don't work for a given page. It's important to keep in mind that just because a metric isn't relevant for one page, that doesn't necessarily mean it's a bad metric. Often, any variability you might see is due to how the page is built.

10. Use real user monitoring to validate user engagement and business impact

This is a good way to give yourself the confidence that you're tracking the right metrics. Ultimately, you want to know that changes you make to your site – for better or for worse – will directly affect user behaviour and business outcomes.

This is where real user monitoring (RUM) really shines. RUM can track data about bounce rate and conversion rate (along with other user experience and business KPIs). Using this data alongside your performance data, you can create correlation charts that demonstrate the relationship between performance and business outcomes.

In the correlation chart above, you can clearly see that as LCP times get slower, bounce rate gets worse. This chart demonstrates that, for this particular site, LCP time is a good metric to set a performance budget for.

11. Synthesize your synthetic and real user monitoring data

In an ideal world, you're using both synthetic and real user monitoring (RUM). Several metrics are available in both tools, so you can create charts in which you track the same metric in both synthetic and RUM.

(It's important to know that your synthetic and RUM metrics most likely will not match, for reasons explained here. This is nothing to be concerned about. The important thing to track is consistency and changes within a single tool and settings.)

For a metric that's available in synthetic and RUM, such as Start Render or Largest Contentful Paint, you might want to consider this type of setup:

> Track the metric for synthetic and RUM within the same chart.

> Create the performance budget for the RUM metric, so you get an alert when the budget is violated. This lets you know that real users are experiencing this issue.

> Because you're tracking synthetic data in the same chart, you can easily drill down and get detailed test results and diagnostics.

> Add a note to the chart, stating when you implemented the necessary fixes. After your fixes go live, you can see (and get an alert) when your metric returns to normal.

This is just one potential configuration. If you're using your RUM and synthetic data in other ways, I'd love to learn more about it!

12. Set up alerting (but not too much!)

Avoiding alert fatigue is crucial to the success of your performance budget strategy. If you're just starting out, it's absolutely fine to focus on just a handful of metrics. You can create performance budgets for all of them, but if you're setting up alerting, focus on just setting up alerts for critical metrics such as:

Backend Time
Start Render
Largest Contentful Paint
Image Size

13. Integrate with your CI/CD process

You can integrate your performance budgets and alerts with your CI/CD process. This gives you the ability to do a few of things:

Run synthetic tests in your staging/development environment and get alerts if any changes you've introduced have caused budget violations before the page goes live. You can even opt to break the build if any of your budgets are violated.
Run tests each time you do a deploy, catching issues immediately after they go live.
Run tests against GitHub pull requests, so you can test the performance of your PRs before they're merged.

Keep your budgets relevant

Your budgets will ideally change over time, as your various metrics (hopefully) improve. After you've taken the time to create your performance budgets, you want them to stay relevant and helpful.

14. Update your budgets

If you're taking the practical, iterative approach recommended above, then you should revisit your budgets every 2-4 weeks and adjust them (hopefully downward) accordingly.

You should also periodically revisit your metrics – going through the validation steps described in steps 9 and 10 above – to make sure you're still tracking the right things go through the validation Are you still tracking the right metrics?

15. Celebrate wins!

If you consistently improve a metric and have just updated your budget, share your charts and let your teams (and your boss!) know. It's important to celebrate even small wins, because big wins are rare. Performance improvement is cumulative. Getting faster should always be celebrated!

Case studies

Here's how some of our customers have used performance budgets to stay fast:

More resources

If you're a SpeedCurve user, these resources will help you get started with performance budgets. If you're not using SpeedCurve yet, signing up for a free trial is easy!

(This post has been updated from an earlier version published in May 2023.)

Navigate your way to better performance with prerendering and the bfcache

Mon, 25 Mar 2024 00:00:00 +1300

I was inspired by Tim Vereecke's excellent talk on noise-cancelling RUM at PerfNow this past November. In this talk, he highlighted a lot of the 'noise' that comes along with capturing RUM data. Tim's approach was to filter out the noise introduced by really fast response times that can be caused by leveraging the browser cache, prerendering, and other performance optimization techniques.

I thought Tim's focus on 'human viewable navigations' was a great approach to use when looking at how to improve user experience. But there also may be times when you want to understand and embrace the noise. Sometimes there are opportunities in the signals that we often forget are there.

In this post, I'll demonstrate how you can use SpeedCurve RUM to identify all types of navigations, their performance impact, and potential opportunities for delivering lightning-fast page speed to your users.

We'll cover things like:

Understanding SPA navigations and performance
Whether or not to track hidden pages (such as pages opened in background tabs)
How to take advantage of prerendering and the back-forward cache (aka bfcache)

Understanding navigation types

We've recently released a new filter in RUM that allows you to explore navigation types. You can find navigation types in the filters of your RUM and Favorites dashboards:

The different navigation types we track are:

Navigation – Full-page navigation

Reload – Page is reloaded from the browser history

Back-Forward Navigation – Page navigation using back/forward navigation (also known as bfcache navigation) controls

Other – All other navigations

Not all navigations are created equal.

For example, full-page navigations have very different characteristics than bfcache navigations. It's helpful to see this in a histogram, where we can see the distribution for each navigation type for a metric such as Largest Contentful Paint (LCP):

Understanding slow page reloads

In doing research for this post, I have to admit that I had an 'uh-oh' moment. Looking at the reloads from the previous chart on the surface was a bit of a head-scratcher:

How could LCP be so much slower on a reload versus a full-page navigation?

After a closer look at the histogram, I noticed there were a lot of reloads that happen to be slower in the long tail of the histogram:

When comparing the fast versus slow reloads, I found that these reloads were for our RUM Live dashboard. This dashboard forces a reload automatically. This was for a specific user who was loading the page on a very slow connection.

Lesson learned: Segmenting your pages in to cohorts is extremely helpful when exploring anomalies like this one.

Introducing Page Attributes

Another useful filter that we recently added is for something we refer to as Page Attributes. This goes a step further in explaining the different types of navigations – both visible and hidden from the end user.

This is extremely useful when looking at pages that have unique performance characteristics including:

Page was a soft navigation – When implementing RUM for a SPA, you can use this attribute to compare initial page loads/hard navigations to SPA/soft navigations.

Page visibility was hidden* – For pages that are loaded in a hidden state, such as when you open a link in a background tab for viewing later, the performance can vary greatly given the browsers ability to mitigate resource consumption in an effort to preserve the user experience.

Page was prerendered – Prerendering can happen automagically in the browser or by using the Speculation Rules API . When this occurs, pages that are activated appear to load instantaneously and have unique characteristics compared to other types of navigations. For example, in SpeedCurve, prerendered pages will have a value of '0' for most metrics.

Page was restored from back-forward cache* – The bfcache essentially stores the full page in memory when navigating away from the page. This browser optimization has the effect of instantaneous page loads when a user is navigating back (or forward) to a previously viewed page.

*Important: Currently you need to opt-in for tracking of hidden pages and bfcache restores. This is an option in your advanced RUM settings.

Understanding your SPA performance

Single-page application (SPA) performance can be hard to get your head around. The benefits of a SPA can sometimes be hindrances when you are trying to understand the user experience. In a SPA soft navigation (versus a full-page navigation), you don't always get the metrics you are looking for, such as render metrics like First Contentful Paint (FCP) and Largest Contentful Paint (LCP). Often you'll have to revert back to traditional timing metrics or those that are triggered by using custom metrics.

This custom metric comparison illustrates that performance characteristics can be drastically different between soft navigations and full-page navigations:

The differences between the full and soft SPA navigations can have a big impact on other derived metrics as well, such as user happiness:

Hidden pages: To track or not to track?

If a tree falls in the forest, does anybody hear? By default, SpeedCurve does NOT track pages that are hidden from the user, such as a page opened in a background tab. However, you may want to understand if those pages are having issues.

Be warned that rendering of hidden pages is deprioritized by the browser when trying to conserve resources, which may or may not have something to do with your site. The charts below illustrate that, all things equal, hidden pages are more than a second slower to load than visible navigations.

Take advantage of prerendering

Prerendering pages based on where you expect users to click next may sound a little creepy, but it sure does make your user experience a lot faster. In this example of product pages for a major online retailer, prerendered pages were 200% faster than full navigations:

What is this mysterious prerendering magic we speak of, and how do you take advantage of it? Prerendering has been around in Chrome for a little while, but just recently the Speculation Rules API was introduced, which supersedes the now deprecated <link rel="prerender">.

Using prerender json instructions within <script type="speculationrules">, browsers will prefetch and load the page into memory cache, making subsequent navigations appear instantaneous.

Chrome also uses prerendering for addresses typed into the omnibox for pages with a high confidence. The search bar may also leverage prerendering, depending on the provider.

Barry Pollard provides a comprehensive overview of the ins and outs of prerendering here.

Putting the BF (blazing fast) in BFCache

Another notable cheat code for web performance is the use of the browser's bfcache, which essentially stores the fully rendered page in memory. Used in the aforementioned back/forward navigation, if the page is in the bfcache, you're in for another 'instant' page load.

Bfcache is available in all modern browsers. As shown here, the performance difference between a full-page navigation and a bfcache restore – 2.59 seconds versus 0.07 seconds – is pretty impressive!

If you are a Lighthouse user or familiar with the audits provided in SpeedCurve, you may have noticed the following audit as of Lighthouse 10:

For Chrome, there are a number of reasons why a bfcache restore may fail. Some of them are within your control, while others are not. Here is a great resource for understanding how to optimize your pages for bfcache.

Important: The criteria for a page entering the bfcache is different between Chrome and other browsers. Safari has been leveraging bfcache for quite some time and appears to be far less restrictive.

For the same site listed above – which failed the bfcache audit – Safari (desktop and mobile) has a very large number of bfcache restores, while Chrome and Firefox are missing all together!

Get started

Low-hanging fruit tastes the best. While everything else you're doing to speed up your pages is by no means a wasted effort, understanding how to leverage the browser in a more effective way for repeated navigations is totally worth it!

If you're already using SpeedCurve RUM, then you can take advantage of all the insights described in this post. If you're not using our RUM yet, we'd love to have you try it! Sign up for your free trial and follow the steps in our welcome guide to get started.

Hello INP! Here's everything you need to know about the newest Core Web Vital

Tue, 12 Mar 2024 00:00:00 +1300

After years of development and testing, Google has added Interaction to Next Paint (INP) to its trifecta of Core Web Vitals – the performance metrics that are a key ingredient in its search ranking algorithm. INP replaces First Input Delay (FID) as the Vitals responsiveness metric.

Not sure what INP means or why it matters? No worries – that's what this post is for. :)

What is INP?
Why has it replaced First Input Delay?
How does INP correlate with user behaviour metrics, such as conversion rate?
What you need to know about INP on mobile devices
How to debug and optimize INP

And at the bottom of this post, we'll wrap thing up with some inspiring case studies from companies that have found that improving INP has improved sales, pageviews, and bounce rate.

Let's dive in!

What is Interaction to Next Paint?

"Chrome usage data shows that 90% of a user's time on a page is spent after it loads. Thus, careful measurement of responsiveness throughout the page lifecycle is important. This is what the INP metric assesses."

~Jeremy Wagner, Google

In other words, we need a reliable metric that helps us understand how a page's responsiveness (or lack thereof) during the entire time a user is on the page helps (or hurts) their experience.

When you interact with a page, you want the page to respond seamlessly. The longer the wait, the worse the user experience, as you start wondering if the page is broken and start anticipating a frustrating, laggy experience. (This frustration is hardwired, as you can learn in this post about the psychology of site speed and human happiness.)

This is where measuring Interaction to Next Paint can help. INP measures the time from when a user starts an interaction – such as a mouse click, touchscreen tap, or physical or onscreen key press – until the next frame is painted in the browser. The faster the INP time, the more seamless the interaction feels.

According to Google, an INP of 200 milliseconds or less is ideal. Having said that, you may wish to aim for a faster INP for your own pages, for reasons I'll go into further down in this post.

In this excellent post, Jeremy Wagner explains how Interaction to Next Paint is calculated and how Google's INP thresholds (pictured above) are determined.

Farewell, First Input Delay!

Before we go any further, you might find it interesting to learn a bit more about First Input Delay (FID), the interactivity metric that preceded INP in Core Web Vitals, and why it was deprecated.

Here at SpeedCurve, we definitely get excited about emerging metrics. But we also approach each new metric with an analytical eye. Way back in 2020, Cliff Crocker took a closer look at First Input Delay and found that FID did not meaningfully correlate with actual user behaviour.

Does INP correlate to user behaviour?

The point of measuring responsiveness is because unresponsiveness hurts the user experience – which ultimately hurts your business metrics, such as bounce rate and conversions. If this is true, then we should be able to draw a direct line between Interaction to Next Paint and business metrics.

This is where correlation charts come in super handy.

Correlation charts give you a histogram view of all your user traffic, broken out into cohorts based on performance metrics such as INP. The chart also includes an overlay that shows you a user engagement metric or business metric – such as bounce rate or conversion rate – that correlates to each of these cohorts. This lets you see at a glance the relationship between performance, user engagement, and your business.

A few months ago, Cliff Crocker turned his attention to analyzing INP. In his exploration of INP's correlation to conversion rate for a handful of different sites, Cliff found that yes, there typically is a correlation: when INP gets worse, conversions suffer.

However, Cliff also made the following observations:

Results vary across sites – Not surprisingly, the impact is different based on the slope of the conversion line, as well as the distribution of INP across user sessions.
There is no consistent correlation with Google's thresholds for 'Good', 'Needs Improvement', and 'Poor'. This is a hugely important observation. As Cliff states: "For one site, conversions suffer when INP is 100ms – well within Google's 'good' parameter of 200ms. This doesn't mean it's a bad idea to have a general set of thresholds. It just means those thresholds might not apply to your site. You must look at your own data."

Mobile INP is really important!

After further analysis, Cliff discovered still more important INP insights, including:

1. Only two-thirds of mobile sites have 'good' INP

Looking at data from the HTTP Archive, which tracks performance metrics for the top million sites on the web, the percentage of sites that have good INP is 96.8% for desktop, but only 64.9% for mobile. One of the biggest culprits: latency, which is typically worse on mobile.

2. Mobile INP = Android INP

This is due to the lack of Safari support for INP (among other performance metrics). If a significant number of your users are coming to you from Safari, you need to track a different responsiveness metric.

3. Mobile INP has an even stronger correlation with bounce rate and conversions than desktop INP

You can see this in the charts below. Ignoring mobile responsiveness isn't something most of us can afford to do.

One of the many important takeaways from this post is that responsiveness on mobile is absolutely crucial to business and UX metrics. But if you're focusing on just INP to measure responsiveness, you're only getting insights into mobile performance for just one cohort of your users.

How to debug and optimize INP

Now that you understand how slow INP hurts your users and your business, let's talk about solutions. In this in-depth post, Andy Davies walks through what INP is, how to identify and debug slow interactions, and some approaches he's used to improve them.

"Many sites tend to be in the 'Needs Improvement' or 'Poor' category. My experience over the last few months is that getting to 'Good' (under 200ms) is achievable, but it's not always easy."

In his very detailed and comprehensive post, Andy walks through:

How he helps people identify the causes of poor INP times
Examples of some of the most common issues
Approaches he's used to help sites improve their INP

INP case studies

The best way to understand the importance of optimizing INP is to look at your own RUM data for your own site. The second-best way is to look at case studies from other companies that have had success. Here's a handful for you to check out:

Trendyol reduced INP by 50%, resulting in a 1% increase in click-through rate
redBus improved INP and increased sales by 7%
The Economic Times reduced INP from 1s to 257ms, leading to a 50% decrease in bounce rate and 43% increase in pageviews

Takeaways

1. You definitely should be monitoring Interaction to Next Paint for your site, ideally using real user monitoring as the best source of truth.

2. You may find it helpful to create correlation charts to validate INP as a meaningful business-related metric.

3. When you create correlation charts, take careful note of when your business metrics start to suffer. Even though Google's recommended 'Good' INP threshold is 200 milliseconds, good INP for your own site may be higher or lower than that.

4. Look at INP separately for mobile and desktop. Your numbers could be quite different. You may also find that INP correlated to business metrics differently in each environment.

5. Think beyond INP. This is crucial if you have a lot of users coming to your site from different browsers. Remember that INP is only supported in Chrome-based browsers.

How to monitor INP in SpeedCurve

If you're not already using our RUM to monitor INP alongside your other important metrics, we'd love to have you give us a try!

Start your free 30-day trial
Follow our handy guide to enabling RUM
Create your Core Web Vitals dashboard (it's easy!) so you can get alerts when any of your metrics start to suffer
Contact us at support@speedcurve.com with questions or feedback

Continuous performance with guardrails and breadcrumbs

Mon, 04 Mar 2024 00:00:00 +1300

The hardest part about web performance isn’t making your site faster – it’s keeping it that fast. Hearing about a company that devoted significant effort into optimizing their site, only to find their performance right back where it started a few months later, is all too familiar.

The reality is that, as critical as site speed is, it’s also very easy to overlook. It doesn’t jump out like a blurry image or a layout issue. And the majority of modern tools and frameworks that are used to build sites today make it all too easy to compound the issue.

Making performance more visible throughout the development process is one of the most critical things a company can do.

I like to think of it as setting up guardrails and breadcrumbs.

We need guardrails to help protect us from shipping code that will result in unexpected regressions.
We need breadcrumbs that we can follow back to the source to help us identify why a metric may have changed.

Guardrails and breadcrumbs need to work together. Setting up guardrails without the ability to dive in more will lead to frustration. Having proper breadcrumbs without guardrails in place all but assures we will constantly be fighting regressions after they’ve already wreaked havoc on our sites and our users.

Let’s take a look at both of these concepts and how they should work together.

Guardrails

The default stance of most tooling today is to make it easier to add code to your site or application at a moment’s notice. (Hey there, npm install.) What they don’t prioritize nearly as much is making you aware that you’re doing something that’s going to negatively impact your performance.

To combat that, we need to find ways to put guardrails into our workflows to help us notice when we're at risk of causing a performance regression. Luckily we have a few tools at our disposal.

Performance budgets

Tammy has written quite a bit about performance budgets, and it’s something worth repeating. While a budget alone doesn’t ensure good performance, I have yet to see a company sustain performance over the long haul without using some sort of performance budget (whether they use that term or not).

I like to define a performance budget as a clearly defined limit on one or more performance metrics that the team agrees not to exceed. That budget then is used to alert on potential regressions, break deployments when a regression is detected, and guide design and development decisions.

SpeedCurve comes with some default performance budgets, but you’ll want to reassess for yourself, probably on a 2-4 week basis to make sure the budgets you have set align with your reality. We’re looking for a balance of "alerts if there's a regression" and "doesn’t alert so much that we start ignoring the alerts".

Here's an example from a recent KickStart engagement. (KickStarts are consulting projects where we help companies get set up and running with their monitoring.) The budget for JS Scripting was set to 2000ms (that’s the red line), but the site was well below budget. The budget was useless. They would have to have a regression of over 500% to trigger any sort of alert!

We ended up setting the budget at 375ms based on the last two weeks of data at the time. That enables us to get alerted on regressions without getting alert fatigue.

Looking at the chart, if they continue on the current trend, the next time they reassess their budget they may want to bring it down a little bit further, as some recent changes seem to have resulted in improvements.

On the other hand, their Start Render budget was too aggressive. They were constantly over, resulting in notification fatigue – it was always over, so why pay attention?

Adjusting the budget to 3 seconds makes it much more actionable.

Now they will only get emailed when the budget is triggered by a regression, so they know it's something they should take action on.

Automated testing of code changes

Another important guardrail is to have automated performance testing when code changes occur. Just as important, those test results should show up somewhere in the development process – where the developers are. Requiring the team to go looking for results in another tool every time adds unnecessary friction.

A great way to do this is to have an automated test triggered with every pull request, and then have the results put right back into the details of the pull request for easy review.

I’m a big fan of the Github integration for this reason. With zero code changes (just a few button clicks to get GitHub and SpeedCurve talking to each other), you can get automated performance testing set up on every pull request. (If you’re not using GitHub, you can also use the deploy API to trigger automated tests.)

Now when a team makes a code change, they get immediate performance feedback right in the pull request review process, helping performance stay front of mind.

Breadcrumbs

Guardrails are awesome and important for helping catch regressions before they occur, but they’re only half the battle. If we can see that we’re suffering from – or about to suffer from – a regression, but we can’t figure out why, we’ll quickly become frustrated... and may even lose faith in the tools and processes we're using.

We need to leave ourselves breadcrumbs – a path back to identifying the actual source of a regression so we can fix things.

Deployment tracking

If you’re using the GitHub integration, you get a breadcrumb by default. When the integration comments on the pull request, you’re given a link back to the deployment report. (You also get the report when you use the Deploy API, minus the Github specific information.)

The deployment report lets you see:

The associated code repository and pull request (when using the GitHub integration)
The code branch
The commit ID
The test results
The status of any budgets for that test and whether or not the test failed or passed
Full, comprehensive test results, so you can dig in further

Don’t overlook the importance of that last one! Having automated testing that provides a basic overview report with no way to really dig in is a fast track to frustration.

Notes

Deployment tracking automatically adds notes to your charts, so you can quickly spot what changed, and when.

You can also manually add notes for situations where a deployment wasn’t the cause of the change. For example, if you’ve made changes to your CDN configuration, or a third-party provider had an outage.

Notes don’t clutter the charts (they show up as little icons below each chart and only bring up more detail when hovered over), so you can – and probably should – use them liberally to help you remember why you may be seeing a shift in metrics at a particular point in time. Again, think of these as little breadcrumbs you’re leaving for you and your teammates to be able to quickly identify the source of changes in the future.

For deployments, you’ll also be able to click on the notes to be taken back to the deployment details page and dive right back in.

Guardrails and breadcrumbs

Performance isn’t something we fix once and then walk away from. As the content and composition of our pages change, we need to make performance a continuous focus.

Guardrails and breadcrumbs work together to help us get a comprehensive picture of what's happening.

Performance budgets let us set thresholds, with alerting, on key metrics so we know right away if there's a problem
Automated code testing puts performance information right in front of the developers, using those same performance budgets to give a clear indication of the impact of every code change
Pairing automated testing with detailed test results ensure that developers are able to dive in and quickly identify the source of the regression
Having annotations appear in historical charting that connects to those deployments – as well as other changes – ensures the entire team has full context in why performance may have shifted

Putting up the appropriate guardrails to protect ourselves from regressions – then pairing that with a trail of breadcrumbs so that we can dive in and quickly identify what the source is when a regression occurs – are essential steps to ensure that when we make our sites fast, they stay that way.

NEW: On-demand testing in SpeedCurve!

Thu, 29 Feb 2024 00:00:00 +1300

^{Image by Freepik}

On-demand testing has sparked a lot of discussion here at SpeedCurve over the past year. You've always had the ability to manually trigger a round of tests – based on the scheduled tests in your settings – using the 'Test Now' button. But there hasn't been a lot of flexibility to support nuanced use cases, such as...

"I just deployed changes to my site and want to check for any regressions."

"I saw a change to my RUM data and I want to see if I can replicate it with synthetic for further diagnostics."

"I have a paused site that I don't want to test regularly, but would like to test from time to time."

"Please just let me test any URL I want without setting up a site and scheduling testing."

"I need to quickly debug this script without kicking off tests for my entire site."

"I would like to get a first look at a page in order to troubleshoot regressions I saw in RUM."

Based on your feedback, we've just launched new capabilities for on-demand testing. We're pretty excited about these, and we hope you will be, too!

In this post, we'll:

Highlight the differences between on-demand and scheduled testing
Cover the various types of on-demand testing, including some of the more common use cases we've heard from SpeedCurve users
Step you through running an on-demand test

Let's goooooooo!

What are the two types of tests within SpeedCurve?

Synthetic performance testing comes in two forms:

Scheduled testing

Baselining. Benchmarking. Continuous performance testing. Tried-and-true performance monitoring goes by many names. This is a necessary offering that keeps folks honest, supports the use of performance budgets, and gives you a lot of consistency when identifying what has changed over time or between deploys.

On-demand testing

This is something our new release fully embraced. Based on feedback from our customers – as well as the industry at large – we now give you the ability to run tests when debugging, benchmarking, or just because you feel like it. This is a highly sought after arrow in your web performance quiver.

On-demand testing in SpeedCurve

You now have two options for testing on demand:

Site testing

For each site you have configured in your SpeedCurve test settings, you have the ability to test on demand. Using this option, you'll test the existing URLs you have configured for a Site using the pre-defined configuration in your settings.

Custom URL testing

You now have the ability to test any URL using selected browsers and locations. You can also script a test if you are testing something using any of the scripting options, such as blocking third parties or testing a page within a multi-step transaction.

How do I run tests on demand?

You can trigger on-demand tests either automatically or manually, using the options described below.

Automatic: API, CLI, GitHub integration

On-demand testing of a site can be triggered using these options:

SpeedCurve REST API
SpeedCurve CLI
SpeedCurve Github integration – NEW! This is a great option for CI/CD or other means of triggering non-scheduled tests.

Manual: 'Test Now'

We've added new options to the 'Test Now' feature in SpeedCurve. To run an on-demand test, simply click on 'Test Now' from the side menu.

Support article: Trigger deployment tests

On-demand site testing

Here are some common use cases that may sound familiar to you:

"I just deployed changes to my site and want to check for any regressions."

"I saw a change to my RUM data and I want to see if I can replicate it with synthetic for further diagnostics."

"I have a paused site that I don't want to test regularly, but would like to test from time to time."

To test an existing site, select the site (or sites) to be tested. You can optionally add a note here, or elect to group the tests as a deployment. (I'll discuss that later in this post.)

Support article: Test a site on demand

Custom URL (adhoc) testing

Some of these use cases may be familiar to you as well:

"Please just let me test any URL I want without setting up a site and scheduling testing."

"I need to quickly debug this script without kicking off tests for my entire site."

"I would like to get a first look at a page in order to troubleshoot regressions I saw in RUM."

There are many reasons you may want to execute an adhoc test in SpeedCurve. The custom URL option when selecting 'Test Now' allows you to do just that.

To test a custom URL, click the 'Test Now' button and select the custom URL in the dialog. You have the option to add one or more URLs for testing, as well as the ability to select different regions, browser types, and the number of times to test the URL.

You also have the option to add basic authentication, as well as a scripted test. This can be very useful if you are trying to debug a script before adding it to a site, or if you simply need advanced options for the site or user flow you are testing.

Support article: Adhoc (custom URL) testing

Viewing on-demand tests

Once your test(s) are kicked off, you'll be directed to the Synthetic Tests dashboard. From here, you can see the status and history of all of your tests – and even filter by test type.

For each test, you have the option to:

View the results
Compare the results with another test
Retest with the same on-demand settings

Test details

Once your tests have completed, the Test Details dashboard is ready for viewing.

If you are new to this dashboard, here is a summary of what's captured and displayed for every test.

Test overview and Lighthouse scores

See what was tested, including your test settings
Lighthouse results with details are provided for every synthetic test

Render times

Key rendering moments of the page lifecycle are visualized with user focused milestone metrics. Both the LCP and Last Painted Hero elements are highlighted.

Compressed and expanded waterfall

The best of both worlds. Seeing a simplified event timeline makes it easy to understand how key metrics line up with the actual filmstrip through the use of a scrubber.

Expanding the waterfall exposes a great amount of detail for every request. If you are looking for more detail, click on 'Detailed Results' to see the full test results for every test run.

CPU and content breakdowns

Understanding the impact that JavaScript has on CPU usage can be seen in these visuals for three points in the page lifecycle: Start Render, Page Load, and Fully Loaded.

Page construction – both HTTP request count and size – is a helpful indicator of good/bad performance. This breakdown shows you how the content you choose affects the overall size, weight, and complexity of the page.

CLS diagnostics

Cumulative Layout Shift is broken down by each layout shift and displayed within session windows. Start with the largest layout shifts in the highlighted window first!

Grouping tests as a deployment

For all on-demand tests, you have the option to group them as a deployment. You'll want to add a name and any details you wish to include.

Once the deployment test(s) are triggered, you'll be taken to the Deployments dashboard in lieu of the synthetic tests dashboard to view your deploy.

Once completed, you'll be able to see the high-level pass/fail results for the deploy.

Clicking through to the Deployment Details gives you details around the deployment, including performance budget status, filmstrips, and more for each test in the deployment.

Read this article to learn more about deployments.

Summary

You asked, we listened. We hope you continue to get value from your testing in SpeedCurve with the on-demand testing capabilities we've introduced. As always, your feedback is welcome!

Not a SpeedCurve customer? Start a free trial today!

Debugging Interaction to Next Paint (INP)

Mon, 26 Feb 2024 00:00:00 +1300

Not surprisingly, most of the conversations I've had with SpeedCurve users over the last few months have focused on improving INP.

INP measures how responsive a page is to visitor interactions. It measures the elapsed time between a tap, a click, or a keypress and the browser next painting to the screen.

INP breaks down into three sub-parts

Input Delay – How long the interaction handler has to wait before executing
Processing Time – How long the interaction handler takes to execute
Presentation Delay – How long it takes the browser to execute any work it needs to paint updates triggered by the interaction handler

Pages can have multiple interactions, so the INP time you'll see reported by RUM products and other tools, such as Google Search Console and Chrome's UX Report (CrUX), will generally be the worst/highest INP time at the 75th percentile.

Like all Core Web Vitals, INP has a set of thresholds:

INP thresholds for Good, Needs Improvement, and Poor

Many sites tend to be in the Needs Improvement or Poor categories. My experience over the last few months is that getting to Good is achievable, but it's not always easy.

In this post I'm going to walk through:

How I help people identify the causes of poor INP times
Examples of some of the most common issues
Approaches I've used to help sites improve their INP

Identifying Interactions

The Chrome UX Report (CrUX) can provide a high-level view of INP. Individual pages can be spot checked via the CrUX API or tools such as Page Speed Insights

But as Cliff's already covered in How to find (and fix!) INP interactions on your pages, CrUX is no substitute for having your own RUM data that you can group and filter by dimensions such as the different page and device types.

My favourite place to start is with the Web Vitals heatmap on SpeedCurve's RUM > Performance dashboard. It gives a high-level summary that can be filtered by page label to check if the behavior is consistent across all paths in the group.

Heatmap of the most popular pages and their Web Vitals metrics

I then typically switch to the RUM > Design dashboard and use the list of popular interaction elements to determine which ones I want to investigate further.

Most popular interactions

Ideally we'd have a view that shows which interactions are responsible for high INP times. (We're currently working on adding full attribution for INP to SpeedCurve.) In the meantime, we've discovered that, in practice, just knowing which page types have high INP times and the interactions visitors are using on those pages is really effective at identifying interactions to investigate further.

Some companies I work with don't have RUM. In those cases, we think about the common interactions visitors are likely to use – dismissing cookie dialogs, opening menus, zooming on product images, etc. – and investigate those further. The caveat here is that it's not as effective as having RUM data to work from and can lead improvements that don't seem to influence INP much.

Profiling Interactions

Once we know which pages have high INP times, and what are the popular interactions on those pages, I switch to Chrome DevTools, profile the interactions, and identify ways to improve them.

The Performance panel can be overwhelming – even for experienced engineers – as it exposes how much work the browser is doing to load pages or handle interactions.

Here is the approach I use when debugging interactions. Although I'm using Chrome Canary in these examples, the same approach works in stable Chrome and other Chromium-based browsers.

1. Switch to a guest user profile

As guest user profiles don't load extensions, they help minimise some of the noise that extensions and other factors can have on performance analysis.

Guest profiles also start with empty caches, empty cookie stores, empty browser storage, etc. These may get populated during testing, but we can clear them at any time via Application > Storage > Clear Site Data in DevTools.

Opening a guest profile in Chrome

2. Open DevTools and switch to the Performance panel

As mobile visitors tend to be the majority of visitors for most sites, I also switch to mobile emulation.

Switching to the DevTools Performance panel

3. Load a page

Load the page you want to investigate. Wait until has finished load before profiling it.

4. Hit 'Record' and interact with the page

After the page has loaded, press the record icon in the DevTools toolbar, wait for the profile to start recording, and then interact with the page.

The profiler starting up often creates a Long Task right at the start of the profile, so I tend to wait a second or so before actually interacting.

Recording a profile

5. Stop recording

After you've recorded data on the interactions you're interested in, stop recording. After a few moments you should be greeted with a view something like the one below.

In this view I've opened the tracks for Frames, Interactions, and Main Thread so I can see what's visible on the page when I interacted with it as well as the activity that happened on the main thread.

Example performance profile in Chrome DevTools

In the Main Thread track you can see the Profiling Overhead task right at the start of the profile and then a second call stack in response to the interaction.

A quick guide to interpreting this panel:

The width of each cell in the call stack represents the elapsed time it (and its children) executed.
The dark yellow, magenta, and green cells represent internal browser work.
The pastel cells represent the scripts included in the page. (Each script gets its own color.)

Clicking on an individual cell will show more detail in the summary panel at the bottom of the tab (not shown in screenshot). You can zoom in/out and scroll using either the mouse or the W A S D keys.

Analyzing Interactions

After we've captured profiles, we can start analyzing them to understand why we're seeing long INP times, and perhaps more importantly, what we can to do reduce them.

While writing this post, I tested interactions on a few sites and chose three examples that illustrate the common issues I see.

The examples were captured in Chrome Canary on a 2020 i5 MacBook Pro without CPU throttling enabled. If CPU throttling was enabled or the tests were carried out on an actual Android device, then I'd expect the INP times to be higher.

If you want to explore the traces in more detail I've uploaded them to Paul Irish's trace.cafe

Example 1 – Opening the menu on H&M

In this first example, I opened the menu on the mobile version of H&M by clicking on the icon in the top right.

Even though I only clicked on the page once, multiple event handlers were invoked. The one for the menu was the longest and had INP time of 350ms – in other words, 150ms longer than Google's 200ms threshold for 'Good'.

Long interaction when opening the menu on H&M

In this case, most of the time is spent in the actual event handler (Processing Time) for the menu, but there is a slight delay before the event handler can execute.

Examining the flame chart reveals four main groups of processing that happen in response to the interaction:

Akamai's bot manager is the source of the first two event handlers. These event handlers execute before the one for the menu, so creating the Input Delay for the menu interaction.
Within the longest event handler, the first group of processing creates an analytics event to record the visitor opening the menu.
The second group is a JS component that constructs the menu and then adds it to the DOM, triggering style recalculations and layout.
The last group adds a style element to the DOM, again triggering style recalculations.

Main thread activity when opening the menu on H&M

Here I'd start by focusing on what's the source of the long Processing Time, asking questions such as:

Can the menu be rendered without using React?
Does the stylesheet need to be injected?
Could the design of the interaction be changed to avoid it?

> Explore the trace

Example 2 – Opening the menu on John Lewis

For the second example, I've also chosen to open the menu. The trace for John Lewis shows similar patterns to the one for H&M.

Again there's another event handler that fires before the handler for the menu, but the execution of the both handlers is also delayed by a separate task on the Main Thread. These tasks create a 170ms Input Delay before the interaction handler for the menu executes.

Long interaction when opening the menu on John Lewis

Breaking down the main thread activity shows eight groups of activities that delay the response to the interaction:

Initial interaction occurs while a Long Task caused by a customer feedback widget is executing.
Even though feedback widget isn't visible, it triggers a style recalculation and layout.
A pointerdown event handler within the site's own code executes.
Akamai's bot manager is listening to pointerdown and touchstart events, and handlers for these events execute.
Handlers within the site's own code for gotpointercpature, lostpointercapture, pointerup, pointerout and touchend events execute.
A style recalculation (which I believe is triggered by the bot manager) and a mousedown handler registered by Akamai bot manager execute.
The menu hander finally executes and generates the DOM for the menu.
Lastly, a style recalculation triggered by the menu hander executes.

John Lewis uses New Relic. New Relic wraps many of the script calls, and this has some impact on the duration of the tasks. If I were investigating this further, I'd profile with New Relic disabled to measure what impact it's having (if any).

Main thread activity when opening the menu on John Lewis

The main question I'm asking when I see this kind of profile is this:

What can be done to reduce the Input Delay – the Long Task at the start, then focusing in on the intermediate event handlers, and lastly the style and layout calculations?

(In the chart above, the whisker for the Presentation Delay extends into a GTM task, but I believe this is a Chrome issue. You might also notice Chrome Canary doubles up some Long Tasks in the Profile, too.)

> Explore the trace

Example 3 – Accepting the consent dialog on Wales Online

For the last example, I'm closing the consent dialog that all sites in Europe are required to display before they inject third-parties such as ads and analytics into the page.

Here the main issue is the amount of work the event handler is trying to complete in a single task. The Processing Time for the interaction is 382ms

Long interaction when clicking 'accept' on Wales Online

Examining the flame chart reveals six main groups of processing that happen in response to the interaction:

Closing the dialog is actually pretty quick as it just needs to be removed from the DOM.
The consent manager starts communicating consent to the ad providers, so they can begin to load.
Amazon Ads executes.
Prebid executes.
A second Prebid component executes .
A 'Bad Ads reporting tool' adds a stylesheet. After the styles are parsed and recalculated, something forces a layout task.

Main thread activity after clicking 'accept' on Wales Online

One thing that's noticeable with the Wales Online example is that the Processing Time is entirely due to third-party scripts. That can limit the options to reduce it, but even then it should be possible to divide the task up.

> Explore the trace

Fixing Interactions

After we've identified why an interaction has a high INP time, our next goal is to reduce it. I find that separating how I think about Input Delay versus Processing Time and Presentation Delay can help.

Input Delay is due to other tasks blocking the main thread and so delay when the interaction handler can execute. As such, Input Delay is outside an interaction handler's control.
Processing Time and Presentation Delay are the time the interaction handler takes to execute, and then the time it takes the browser to complete layout, styling, paint, and other tasks created by the event handler.

Split between Input Delay, and combined Processing Time and Presentation Delay

As Processing Time and Presentation Delay are easier to identify and fix, I'm going to cover them first before moving on to Input Delay.

How to improve Processing Time and Presentation Delay

When it comes to reducing Processing Time and Presentation Delay, many articles focus on breaking up Long Tasks up or 'yielding to the main thread' with setTimeout, scheduler.yield or requestIdleCallback, etc.

While that is one place to start, it's not the only approach. My view is that reducing the time Long Tasks take to execute is just as important as splitting tasks up.

My other guiding view is to focus on what's most important from a user perspective and optimize that. For example, if the user is opening a menu, then showing them the menu is the most important activity. Anything else that might be triggered by the same action should be secondary.

1. Defer less important activities

Both the H&M and John Lewis menus record an analytics event when someone opens the menu. These calls happen before the menu is actually displayed. (I've seen this pattern on many other sites, too.)

While analytic events are useful to help us understand our visitors' behavior, they're secondary and shouldn't delay the visitors primary goal, in this case opening the menu.

Scheduling these events into a new task via setTimeout (e.g. setTimeout(analytics_fn, 0) or scheduler.postTask for browsers that support it) moves the work out of the interaction handler to be executed later and allows the interaction handler to complete sooner.

The same method can be used with Wales Online's consent manager. After the visitor has clicked 'accept' or 'reject' they want to get on and read the news rather than wait for multiple ad providers to be given permission to load (or not). Scheduling setConsentInfo into a separate task enables the browser to paint the next frame sooner, while the ad providers can carry on loading in the background.

Here's an example of what this looks like in a trace:

Breaking up a click event handler by scheduling work into a separate task

The Long Task was originally part of the click event handler. Using setTimeout to schedule it into its own separate task allows the browser to paint before the Long Task executes. The Long Task might still be a problematic if someone interacts while it's executing, but it's no longer part of the click handler's INP.

I've seen many examples where publishers have improved INP by scheduling the setting of consent into a separate task. I expect consent managers to adopt this approach as a default.

2. Do less work

Ryan Townsend spoke about The Unbearable Weight of Massive JavaScript at performance.now() in November 2023. He shared a case study where they replaced over 50,000 lines of JavaScript with native HTML and CSS features. The result was a faster user experience with more maintainable codebase.

H&M relies on JavaScript components to create the menu elements, add them to the DOM, and apply styles. The result: a processing time of 303ms.

Long interaction when opening the menu on H&M

Let's compare this to another fashion retailer, French Connection. French Connection creates the menu elements when they render the page server-side, and then just changes the elements styles to display the menu. This illustrates the dramatic difference in processing time between the two approaches:

H&M processing time: 303ms
French Connection processing time: 22ms (of which nearly half is sending an analytics event to Google Tag Manager!)

Interaction when opening the menu on French Connection

Of course the French Connection page is going to have more DOM elements. When tools like Lighthouse warn you to "avoid an excessive DOM size", it's tempting to choose other approaches without perhaps fully considering the tradeoffs. Menus often contain large numbers of DOM elements. The choice is whether they're created when the page is initially generated, or at runtime using JavaScript when the menu is requested.

Lighthouse warns about DOM size because it "will increase memory usage, cause longer style calculations, and produce costly layout reflows." But when used carefully, CSS properties like content-visibility, isolation, and will-change can help reduce the cost of a large number of DOM nodes.

Talking of CSS, you might have noticed that in some of the examples above, INP was affected by some long style and layout calculations. These were caused by interaction handlers injecting styles that affected the whole DOM, or by interaction handlers querying style or size properties via methods like getComputedStyle or getBoundingClientRect, so forcing recalculations. (Paul Irish keeps a list of JavaScript methods that typically trigger style and layout calculations.)

Think about the work you're asking the browser to do and whether there are more efficient ways of achieving the end result.

3. Yield to the main thread

Sometimes there isn't work that can be deferred, or the efficiency of interaction handlers can't be improved. In those cases, we just need give control back to the main thread so that it can get on with painting the next frame.

One approach is to use setTimeout wrapped in a Promise:

function yieldToMain() {

return new Promise(resolve => {

setTimeout(resolve,0);

});

}

And then at suitable points in the code insert:

// Yield to the main thread:

await yieldToMain();

setTimeout creates a new task, so enabling the browser's scheduler to take over and process other tasks like input before resuming.

Jeremy Wagner discusses this approach in more detail in his web.dev posts on Optimizing INP and Optimizing Long Tasks. I'd suggest you read those for a deeper view.

The other option to consider for tasks that are hard to optimize is whether the work they're doing can be moved off the main thread via a Web Worker. I've not had the need to use this approach with clients yet.

How to improve Input Delay

Diagnosing the causes of slow Input Delay isn't always easy with just Synthetic monitoring and DevTools. That's because the length of the delay depends on when the visitor interacted with the page and what tasks were executing when the visitor interacted.

The Long Animations Frame (LoAF) API, which is set to ship in Chrome 123, will help RUM tools identify the tasks that caused Input Delay. Once identified, we will be able to profile the tasks in DevTools.

Until LoAF is widely supported, there are profiling approaches in DevTools that can help identify some of the problematic scripts.

1. Investigate other interaction handlers

As the H&M and John Lewis examples demonstrated, other touch, mouse, and keyboard event handlers are also triggered by interactions and can execute before our event handler for the main interaction.

Fortunately these event handlers are captured in the DevTools profile. We can also inspect which event handlers are active using the Event Listeners panel in the right sidebar of the Elements panel in DevTools.

Viewing active event listeners in Chrome DevTools

Some monitoring products, such as New Relic, wrap other scripts calls. This can make identifying the code that's actually going to execute a bit harder. To identify the actual event handler, you can either:

Block the external instrumentation script
Use DevTools Content Overrides to create a copy of the page with New Relic, etc., removed

Once we've identified event listeners that are active for click, key, mouse, and tap events, we can review and remove any that aren't really necessary.

If third-party tags are adding their own event handlers, then it's a case of:

Evaluating the tradeoff between the features the third party provides and its impact on the visitors experience
Raising the issue with the provider
Expecting the provider to fix the issue

Some third-party providers are serious about addressing the impact they have on INP. If you're using one that's not, then I'd advocate switching to an alternative.

2. Investigate other Long Tasks

It's harder to identify any other Long Tasks that contribute to Input Delay. It depends on what task is executing when the visitor interacts (this is where LoAF will really help) and how long the task continued executing after the initial interaction.

But we do know there is a relationship between total Long Task Time and INP:

Relationship between Long Tasks and Interaction to Next Paint (INP)

And we also know that visitors start to interact shortly after they start to see content:

Relationship between First Contentful Paint (FCP) and First Click Interaction

The exact relationships will vary from site to site, but the overall pattern was pretty consistent across the many sites I checked.

Knowing this, we can make an informed guess that any Long Tasks that occur after useful content starts to appear are in danger of contributing to Input Delay.

In John Lewis' case, profiling the page while it loads shows there are a bunch of Long Tasks that happen after 2.2s – notice the gap in the filmstrip – and these are likely to lead to higher INP times if the visitor tries to interact at this point.

Profile showing Long Tasks during John Lewis home page loading

Some other things you might want to experiment with:

Profiling a page while it loads and then trying to interact as soon as there's visible content.
Profiling the acceptance of a cookie consent dialog and then interacting immediately after while third parties might be still starting up.
If you have tasks that repeat at a regular frequency (e.g. ad refreshes), you can profile what happens if a visitor interacts during those.

As far as optimising these Long Tasks goes, my advice is very similar to optimising slow interaction handlers:

Defer secondary work into new tasks
Optimize code so it uses the browser efficiently
Yield to the main thread appropriately

One thing to watch out for:

There's often a Long Task just before the DOM Content Loaded event. This is because any deferred and module scripts execute as a single – potentially long – task just before DOM Content Loaded. Until browser makers change this behaviour, there's always a potential for this Long Task to create Input Delay if someone interacts at this point.

Wrapping up

Getting to the root of high INP times and fixing them can be quite complex and sometimes overwhelming. It's important to remember that even small incremental changes add up to larger overall improvements.

Some other things to keep in mind...

Use RUM when you can

RUM is great for quickly identifying pages and interactions with high INP times. But even without RUM it's possible to start improving INP. There's just a danger that you might not be profiling the most influential interactions.

When it comes to actually understanding and optimizing INP:

Profile in a guest window to remove noise caused by extension or previous state
Identify secondary activities and defer them into separate tasks
Optimize the primary tasks – lean on the native power of browsers but watch activities that create unnecessary style and layout calculations
Break remaining Long Tasks up by yielding to the main thread
Review other interaction handlers to check they're necessary and aren't contributing to Input Delay

It's easy to write a list of actions, but not always easy to implement them

This is especially true if a site is heavily reliant on a JavaScript framework.

Measurement of INP isn't a perfect science

You may come across things that don't quite make sense. I've seen requestAnimationFrame and setTimeout loops delay input handlers. I've also seen JavaScript dialogs really affect INP. As with all Core Web Vitals, it's a work in progress. I expect the Chrome team will address some of these edge cases eventually.

Progress may not be linear

The reported INP measurement represents the worst interaction on the page ,and pages can have many interactions. If the slowest interaction has an INP time of 500ms, and the second slowest has an INP time of 450ms, then fixing the worst interaction will only reduce INP by 50ms!

To borrow the words of Pokémon, you gotta catch 'em all!

If you need some inspiration...

I've been using the techniques from this post to help companies improve their INP time. For one company, we reduced INP by more than 50% in the space of just a couple of weeks.

Reduction in INP due to Long Tasks improvements

Need help measuring and fixing INP?

If you'd like to start measuring INP effectively, we offer a free 30-day trial that includes both real user and synthetic monitoring.

We also have some of the most experienced web performance consultants in the world. If you're not sure where to start with INP, or are stuck with what to do next, feel free to get in touch via support@speedcurve.com.

How to use Server Timing to get backend transparency from your CDN

Tue, 06 Feb 2024 00:00:00 +1300

80% of end-user response time is spent on the front end.

That performance golden rule still holds true today. However, that pesky 20% on the back end can have a big impact on downstream metrics like First Contentful Paint (FCP), Largest Contentful Paint (LCP), and any other 'loading' metric you can think of.

Server-timing headers are a key tool in understanding what's happening within that black box of Time to First Byte (TTFB).

In this post we'll explore a few areas:

Look at industry benchmarks to get an idea of how a slow backend influences key metrics, including Core Web Vitals
Demonstrate how you can use server-timing headers to break down where that time is being spent
Provide examples of how you can use server-timing headers to get more visibility into your content delivery network (CDN)
Show how you can capture server-timing headers in SpeedCurve

How slow backend times influence key metrics

First, we need to understand what 'slow' means. For more years than I care to mention, I've been advising folks that their TTFB should be under 500ms. Google recommends that TTFB be 800ms at the 75th percentile. For the purpose of this post, let's say that 500ms backend time is 'good enough'.

Looking at the industry benchmarks for US retailers, four well-known sites have backend times that are approaching – or well beyond – that threshold.

Pagespeed Benchmarks - US Retail - Backend

Those same sites, with the exception of Lowe's, were also in the slower cohorts for First Contentful Paint (FCP), Largest Contentful Paint (LCP), and other loading metrics.

Pagespeed Benchmarks - US Retail - LCP

When you examine a waterfall, it's pretty obvious that TTFB is the long pole in the tent, pushing out render times for the page. Given that TTFB is synchronous by nature, we can expect to see this pattern for any site that has opportunities.

Cue server-timing headers

Historically, when looking at page speed, we've had the tendency to ignore TTFB when trying to optimize the user experience. I mean, why wouldn't we? Steve told us to focus on the front-end! ;)

Well, a lot has changed over the years, including the ability to get more detailed information from the server on the user agent through the use of Server Timing.

Server Timing is a specification that allows communication of data from the server to the client through the use of a server-timing header. This is a special header that is accessible through a JavaScript interface. SpeedCurve's RUM collection library – lux.js – accesses this interface to collect custom data from the user agent.

The specification requires that header values are passed through a duration field (dur) and/or a description field (desc). In practice, it looks something like this:

server-timing: processing_time; dur=123.4; desc="Time to process request at origin"

NOTE: This is not a new API. Charlie Vazac introduced server timing in a Performance Calendar post circa 2018. However, wider adoption has only just started to take off.

It's not just about timing

While the intention of server-timing headers may have originally been to get more insight into how long things are taking, another great use case involves sending across dimension data or meta data.

For example, you might use the headers to pass across identifiers such as a datacenter location or other geographic details:

server-Timing: datacenter; desc="denDRP01"

For debugging purposes, maybe send along a link to a session trace to provide a 'front-end' APM solution:

server-Timing: sessiontrace; desc="https://your.logfiles.com/transId=T1234-5678-9012-3456

Or, did the request use early hints (or some other form of web performance magic):

server-timing: earlyhints

Server timing and your CDN

For a large majority of sites, content delivery networks (CDNs) serve a critical role in delivering consistent user experiences. Caching the base page/HTML is common, and it should have a positive impact on backend times. But what happens when it doesn't?

CDNs have traditionally been a bit of a black box when it comes to finding out where time is being spent. With as much as we are moving compute and other capabilities (e.g., bot management, WAF) to the 'edge', there are more and more checkpoints that often go unreported.

The use of server-timing headers by content delivery networks closes a big gap. Today, it's possible to add these headers from your CDN with ease, if they aren't already set up out of the box.

Key things to understand from your CDN

Cache Hit/Cache Miss – Was the resource served from the edge, or did the request have to go to origin?
Latency – How much time does it take to deliver a packet from A to B. Also measured by round trip time (RTT).
Origin Time – How much time did the request spend from your origin? (In the case of a cache miss, this should be zero.)
Edge Time – How much time was spent at the CDN? This can include a lot of different service layers, not just serving from cache. For example, processing of web application firewall (WAF) rules, detecting bots or other malicious traffic though security services, and growing in popularity, edge compute.

Below are some examples from the major CDN providers that you can leverage.

Akamai

Akamai was the first to start emitting server-timing headers and set the tone for CDN transparency. This data is available by enabling the mPulse behavior in property manager. With the behavior enabled, you will start seeing the following server-timing headers:

Cache HIT/MISS

Server-Timing: cdn-cache; desc=<MISS|HIT>

Edge Time

Server-Timing: edge; dur=<# ms>

Origin Time

Server-Timing: origin; dur=<# ms>

Note that if you don't use the mPulse product, you can still enable the headers without the snippet by modifying property settings. Another option would be using EdgeWorkers to add the headers to the request, similar to the example shown next.

Amazon Cloudfront

You can add server-timing headers via opt-in via the AWS Console as mentioned in this post.

CDN Layer (edge, edge-cache, origin shield)

DNS Time

Connect Time

Upstream/downstream first byte latency

Cache status

POP

Here is an example of server-timing headers taken from https://www.perfwork.com/:

server-timing: cdn-upstream-layer;desc="EDGE",cdn-upstream-dns;dur=0,cdn-upstream-connect;dur=69,cdn-upstream-fbl;dur=562,cdn-cache-miss,cdn-pop;desc="DEN52-P3",cdn-rid;desc="5McHcGf1pCMEZKUtTuHH-UI7Co2qq-817CJu_cD7oVUo9BmxBtpIHQ==",cdn-downstream-fbl;dur=563

Cloudflare

Using Cloudflare Workers, you can add values from existing headers such as CF-Cache-Status into server-timing headers.

Here is an example returning the cache status (hit, miss, revalidate, etc.):

/**

* @param {Response} response

* @returns {Response}

*/

function addServerTimingHeaders(response, startTime) {

const serverTiming = [];

const cfCache = response.headers.get('cf-cache-status');

if (cfCache) {

serverTiming.push(`cf_cache;desc=${cfCache}`);

}

serverTiming.push(`worker;dur=${Date.now() - startTime}`);

response.headers.set('Server-Timing', serverTiming.join(', '));

}

Alternatively, you can add/modify response headers using transform rules via the dashboard or API as described here.

Fastly

While you could likely use Compute@Edge to add server-timing headers via Fastly, using VCL is pretty straightforward as discussed in this post.

TLDR:

Request start (from edge)

Elapsed time (edge time)

POP (edge location)

Cache status (hit, miss)

To get the following:

Server-Timing: time-start-msec;dur=1544705663920,time-elapsed;dur=0,fastly-pop;desc=LCY,hit-state;desc=HIT

Use the following VCL:

set resp.http.Server-Timing = "time-start-msec;dur=" time.start.msec ",time-elapsed;dur=" time.elapsed.msec ",fastly-pop;desc=" server.datacenter ",hit-state;desc=" fastly_info.state;

There are a lot of VCL variables available that might be useful server-timing headers.

Shopify

Shopify provides the following server-timing headers for all Shopify sites. It's important to note that these are not considered public, so use at your own risk.

server-timing: processing;dur=15, db;dur=5, asn;desc="7922", edge;desc="DFW", country;desc="US", theme;desc="Prestige", pageType;desc="index", servedBy;desc="8jlx", requestID;desc="4ab33c3d-21e6-425a-9754-a6f42a27d36f"

server-timing: cfRequestDuration;dur=48.999786, earlyhints

As of the writing of this article, our understanding of each of the headers is as follows:

cfRequestDuration = Duration from the time the request hits Cloudflare (CDN) until it is finished processing.

processing = Duration from the time the request reaches the Storefront until processing of the request is complete.

db = Duration of the request processing spent querying the database. (Subset of processing time)

asn = Autonomous System Number

edge = Location of CDN edge server

country = Country of CDN edge server

theme = Shopify theme used

pageType = page identifier

earlyhints = Were early hints used for the request (if present, assume yes)

Collecting server-timing headers with SpeedCurve

If you use SpeedCurve RUM, server timing is one of the preferred methods for capturing custom data. See this guide to learn how you can define custom dimension data, metadata, or metric (timing, size, numeric).

Here are a few example charts created by leveraging server-timing headers provided by CDNs:

TTFB by cache state

TTFB and CDN Edge time

If you're not using SpeedCurve RUM and want to experiment with capturing server-timing headers, start a free trial today!

Recent server timing case studies

It's great to see server timing starting to get more use in the wild. Here are a couple of great blog posts from last year's Web Performance Calendar. Definitely worth a read!

The psychology of site speed and human happiness

Tue, 30 Jan 2024 00:00:00 +1300

In the fourteen years that I've been working in the web performance industry, I've done a LOT of research, writing, and speaking about the psychology of page speed – in other words, why we crave fast, seamless online experiences. In fact, the entire first chapter of my book, Time Is Money (reprinted here courtesy of the good folks at O'Reilly), is dedicated to the subject.

I recently shared some of my favourite research at Beyond Tellerrand (video here) and thought it would be fun to round it up in a post. Here we're going to cover:

Why time is a crucial (and often neglected) usability factor
How we perceive wait times
Why our memory is unreliable
How the end of an experience has a disproportionate effect on our perception
How fast we expect pages to be (and why)
"Flow" and what it means in terms of how we use the web
How delays hurt our productivity
What we can learn from measuring "web stress"
How slowness affects our entire perception of a brand

There's a lot of fascinating material to cover, so let's get started!

Time is a crucial usability factor

If you don't consider time a crucial usability factor, you're missing a fundamental aspect of the user experience.

I'm embarrassed to admit that, in my previous career as a usability tester, I spent years testing websites in lab conditions. It never crossed my mind to take rendering time into consideration.

In fairness, that was in the early 2000s, and site speed was barely on anyone's radar. A lot has changed since then, thankfully. There's been a wealth of research into why waiting is hard – which is why site speed matters – not just from a business perspective, but from a hard-wired neurological perspective.

Let's start with a wide-angle look at how we humans handle waiting, in all its forms.

How do we perceive wait times?

Short answer: Poorly.

Queueing theory is the mathematical study of waiting lines, both real and virtual. If you jump down the research rabbit hole, you can find some fascinating stories. My favourite is one that takes place at a Houston airport.

The airport's customer relations department was fielding a huge number of complaints about how long they were forced to wait for their luggage at the baggage carousels. Airport executives tried to fix the problem by hiring more baggage handlers, which cut average wait times down to seven minutes. But the number of complaints remained the same.

The solution? Rather than hire more baggage handlers, they simply made sure that the arrival gates for each flight were located as far as possible from the assigned baggage carousel for that flight. As a result, passengers had to walk six times longer to get to their bags, while the average wait time at the carousel dropped to a minute.

Passenger complaints dropped to almost zero.

The takeaways from this experiment:

Waiting is hard
Passive waiting is even harder
Perceived speed is more important than reality

These principles apply to waiting in almost any context, including waiting for pages to load.

Our memory is unreliable

Our perception of time varies according to many factors, including (but certainly not limited to) our age, our location, our emotions, and assorted external stimuli. Not surprisingly, this inconsistency applies to our online experiences as well:

The average web user perceives load times as being 15% slower than they actually are. Later, when recalling the experience, they remember load times as being 35% slower.

The average person believes they spend 9 minutes per day waiting for slow websites. This translates to two full days every year. (This statistic is an interesting gauge of how people feel about the web, even if it is not entirely accurate.)

Adding indicators like spinners and progress bars can trick us into believing that pages are up to 10% faster than they actually are. Not only do we feel wait times to be slower than they actually are, we later remember them as being even slower.

The end of an experience has a disproportionate effect on perception

The "colonoscopy effect" was identified in a study in which two patients tracked their perceived levels of pain throughout a colonoscopy procedure.

In the chart below, you can see the data for Patient A and Patient B. Even though Patient A's experience was shorter and had similar-sized pain peaks, they believed their experience was longer and more painful. The conclusion was that, because patient A's experience ended at a painful peak, that pain coloured their overall perception.

What does this finding mean in web performance terms? If your site delivers a relatively fast experience throughout most of a user's journey, but then is slow and janky during the last stage – for example, checkout – then your users may take away a disproportionate sense of your site's overall slowness.

Is that fair? Perhaps not, but it's how our brains work.

How fast do we expect web pages to be?

While what we say we expect from online experiences is inaccurate and highly variable, how we actually respond to different page speeds is much more consistent – and has been so for several decades.

In 1968, Robert Miller published a study called Response Time in Man-Computer Conversational Transactions. In the study, Miller shared that a wait of longer than 2 seconds breaks concentration and hurts productivity.

In 1993 and again in 2010, usability expert Jakob Nielsen found that:

0.1 seconds gives us the illusion of instantaneous response
1 second keeps our flow of thought seamless
10 seconds is enough to keep our attention – barely.
After 10 seconds, our minds wander, making it harder to get back on task once a page finally loads.

The internet may change, and web pages may grow and evolve, but user expectations are constant. The numbers about human perception and response times have been consistent for more than 45 years. These numbers are hard-wired. We have zero control over them. They are consistent regardless of the type of device, application, or connection we are using at any given moment.

But why? This is where things get really interesting.

Nielsen has stated that human responses to poor load times are based on two aspects of how our brains function:

Our poor short-term memory – Information stored in short-term memory decays quickly.
Our need to feel in control – Being forced to wait makes us feel powerless and frustrated.

Impatience: It's in our heads

Our impatience is an indelible part of our incredible human circuitry. At any given moment, there are three types of memory processing at work in your brain:

Sensory memory
Short-term memory
Working memory

(There's also long-term memory, but it doesn't really come into play here.)

Sensory memory

Every time you see something, this visual information is taken in by photoreceptor cells in your eyes and sent to the occipital lobe in your brain. This is your iconic memory. It's just one of your three types of sensory memories. (The other two govern sound and touch.)

People have been studying how iconic memory works for almost 300 years. In one of the earliest studies, back in the early 1800s, a glowing coal was attached to the wheel of a cart. The wheel was spun faster and faster until observers perceived an unbroken circle of light. The study concluded that the glowing coal had to perform a complete cycle in 100 milliseconds or less in order to create the illusion of a fiery circle.

That early study identified the phenomenon we now call "persistence of vision", which is predicated on the fact that our iconic memory holds on to visual information for about 100 milliseconds. After that, the "memory store" runs out and the iconic memory needs to be refreshed with new visual information. This number has remained consistent throughout the centuries.

Interestingly, and perhaps not coincidentally, 100 milliseconds is Google's stated goal when it comes to page load times.

Iconic memory, along with the other two types of sensory memory, is primitive. We can't consciously choose what information is stored in it, and we can't will it to last longer. (If we could, we'd probably go insane or accidentally walk in front of a bus.)

Some sensory memory does stick, of course... provided it's used quickly and eventually consolidated into your long-term memory.

Short-term memory and working memory

If our sensory memory's role is to provide comprehensive information on our entire sensory experience, it's our short-term memory's job to extract the relevant bits and throw them into the hopper of our working memory. Your short-term memory can store information for 10-15 seconds – at most – just enough time for your working memory to process, manipulate, and control it.

So the goal in getting perceived page speed rendering down to 100 milliseconds is to:

Keep information from falling through the cracks in our iconic memory, while we
Give our short-term and working memory ample time to do all the parsing they need to do before they start losing information.

What is "flow" and what does it mean in terms of how we use the web?

For hundreds of thousands of years, human beings have evolved to perform actions in beautiful, sequential flows. Our day-to-day tasks – building a fire, hunting antelope, baking bread, milking a cow – have been comprised of a series of minute actions that flow more or less seamlessly into the next.

In his book Finding Flow: The Psychology of Engagement with Everyday Life, noted psychology researcher Mihaly Csikszentmihalyi observes that people who perform seamless, sequence-based activities on a regular basis are happier than people who do not. He coined the term "flow" to describe this state of being.

It's only in the past 40 years, with the advent of computers in our homes and workplaces – and even our pockets! – that we have imposed a new set of demands on our brains. As most of us are painfully aware, instead of offering a series of smoothly sequential actions, computer use is characterized by lag, downtime, and restarts.

Simply put, our flow-oriented brains are not wired to deal with the fits and starts of human-computer interaction.

There are some people who are skeptical about the impact of lag, downtime, and restarts on productivity and other performance indicators. A common argument is that most people do, in fact, adjust to poor performance. As it turns out, those people may be somewhat correct in their assumption, but they may also be focusing on the wrong part of the picture.

Do delays really hurt productivity?

In a 1999 study of workplace interruptions, groups of workers were subjected to various disruptions in the course of their day-to-day responsibilities. They were then measured in terms of:

their productivity, and
their self-reported state of mind.

While that study focused on general workplace interruptions, with only some attention given to human-computer interaction, there were some fascinating findings that are quite arguably relevant to web performance:

Finding 1: Participants developed strategies that let them deal effectively with interruptions and maintain their productivity

The research suggested that, at least for some workers in some environments, not only did they learn how to cope with interruptions, they may even have striven to overcompensate for their potential performance decline.

Finding 2: However, this coping mechanism is achieved at the expense of higher psychological costs

Cumulatively, interruptions had a negative impact on emotions and well-being. In addition, participants ultimately needed to increase the amount of effort required to perform the same tasks.

Finding 3: Over time, interruptions affected participants' ability and willingness to resume work and take on new tasks

Interruptions seemed to have a cumulative effect. When the number of interruptions grew, the resumption time (i.e., the time needed to restart the task) became disproportionately longer. The participants seemed to lose motivation and develop mental fatigue.

What do these findings mean in web performance terms?

When dealing with application delays, it is possible that people can develop coping strategies that allow them to maintain productivity in the short term. But the missing ingredient here is flow. And without flow, eventually our sense of motivation and well-being suffers.

It's important to remind ourselves that application performance is just one part of the greater world. Our everyday lives are filled with events – from sitting in traffic to standing in line at the grocery store – that challenge our need for flow.

Slow websites are just one problem, but for those of us who spend much of our work and personal time online, slow sites creates extra friction in an already friction-filled world. The effects are cumulative, as most of us are not capable of compartmentalizing our stress.

"Web stress" is measurable

When websites perform poorly, we react badly. (There is even some research that suggests using slow websites increases our blood pressure!) This is not surprising given what we now know about our deep craving for flow.

In 2011, CA Technologies commissioned Foviance, a customer experience consultancy, to conduct a series of lab experiments at Glasgow Caledonian University. The participants wore an EEG (electroencephalography) cap to monitor their brainwave activity while they performed routine online transactions. Participants completed tasks using either a 5 MB web connection or a connection that had been artificially slowed down to 2 MB.

Brainwave analysis from the experiment revealed that participants had to concentrate up to 50% more when using websites via the slower connection. When asked what they liked most and least about the websites they used during the study, participants frequently cited speed as a top concern:

"The website was very slow, so it took a really long time to load the book preview."

"What I liked least about the site is its speed."

The study also found that people were most likely to experience the greatest levels of stress during these points in the transaction process:

Search
Finding and selecting products
Checkout
Entering personal information and concluding the sale

Intuitively, this makes sense. Online shopping already comes with an inherent amount of stress, as most of us are concerned with finding the right item at the best possible price. And the checkout process – when we hand over our personal and credit card information – is fraught with a certain amount of stress as well. Add page slowdowns to the mix and it is easy to understand why the online shopping experience can become unpleasant.

Mobile users feel "web stress" too

Based on the desktop neuroscientific research conducted by CA Technologies, Radware conducted a similar study in 2013, this time focusing on users of mobile devices.

(Disclosure: I worked at Radware at the time and directed the study. In order to ensure that there were no biases, the research and analysis was outsourced to a third-party neuroscientific research company called Neurostrata.)

The mobile stress study involved using a groundbreaking combination of eyetracking and electroencephalography (EEG) technologies to monitor neural activity in a group of mobile users who were asked to perform a series of online transactions via mobile devices. (Below is a photo of one of the study participants.)

In the study, participants were asked to complete standardized shopping tasks on four ecommerce sites while using a smartphone. Some participants were served pages at normal speeds over WiFi, while others were served pages at a consistently slowed-down speed (using software that created an artificial 500-millisecond network delay).

The participants did not know that speed was a factor in the tests; rather, they believed they were participating in a generic usability/brand perception study.

Some highlights of the study's findings:

Users experienced frustration peaks of up to 26% at critical points.
Like the CA Technologies study, frustration peaks were most common during the browsing and checkout phases.
Faster pages correlated with increased user engagement. (That's a good thing!)
All users experienced some level of "web stress" even under ideal mobile browsing conditions.

Slowness affects our entire perception of a brand

Yes, that includes non-performance aspects of the site, such as content, design, and navigation.

After the mobile stress study above, we conducted exit interviews with participants, in which we asked them about their impressions of the site and the company. We then poured all the adjectives from the interviews into a word cloud generator and generated clouds for each version (normal and slow) of each site.

Reminder: The only differentiator was site speed. And because this was a blind study, the testers were not consciously aware of the speed difference. The results indicate that slower page speed affects the brand on a global level.

What we found: Slow pages undermine overall brand health.

This is the word cloud that was generated by test participants after using the site at normal speed:

And this is the word cloud that was generated by participants after experiencing the same site with a 500ms network delay:

While it’s true that both word clouds contain positive and negative descriptors, it is important to note that the word cloud for the slower site contains almost three times more negative adjectives than the faster site. The adjectives shift from mainly easy-to-use (in the first word cloud) to a range of negative associations (in the second word cloud) — solely because of the page delays.

While some participants clearly picked up on the slight deterioration in performance (“slow” and “sluggish”), participants also developed negative perceptions of areas that are unrelated to speed. They reported that the site seemed "boring", "inelegant", "clunky", "tacky", and “hard to navigate".

In other words, slower page loads affected people’s perception of three important aspects of the site that are completely unrelated to load time:

Content (“boring”)
Visual design (“tacky” and “confusing”)
Ease of navigation (“frustrating” and “hard-to-navigate”)

Takeaway

There is a fascinating disconnect between what we say we want and what – deep down – we really need from our online experiences.

Over the past dozen or so years, user surveys have revealed that what we claim to want changes over time – from 8-second load times back in 1999 to 4 seconds in 2006 to around 2 seconds today. If we were to believe these surveys, then we would conclude that we are an increasingly hasty, impatient species. We might be tempted to judge (or pity) ourselves as victims of our frantic modern lives.

But neuroscientific research – which studies how we actually take in and respond to visual information – tells a very different story. Over the decades, researchers have reproduced the same results: that, by and large, we function at our happiest best when our websites and apps (and technology in general) respond in fractions of a second. We may learn how to adapt to slower response times, but this adaptation will always – or at least for the foreseeable future – be awkward and uneasy.

Yes, there's a business case for making your site faster. But caring about web performance is about more than business.

As technologists – and as empathetic human beings – we need to do more than just deliver adequate online experiences. Our goal should be to deliver online experiences that are frictionless and delightful, so our visitors leave our sites and apps happier than when they arrived.

Building the future of web performance with SpeedCurve

Tue, 16 Jan 2024 00:00:00 +1300

I’m beyond excited to announce that I’m joining the SpeedCurve team this year! I’ll still be doing some consulting work, but I’ll be taking on a few less clients this year so I can focus on helping to make an already amazing performance tool even better, working alongside some of my favorite people in the performance community.

A long time coming

I started working on performance back in 2010 (oh no, I’m old). I remember reading Steve Souders's book and immediately becoming hooked—living in the middle of nowhere with terrible connectivity and devices that tended to lag behind mean the pain of bad performance was something I couldn’t ignore.

I got to know Steve a bit and when I attended my first Velocity event in 2012, Tammy, Andy and Cliff were some of the first people I met. So much of what I know about performance, I learned from this group of folks. They were patient with advice and feedback and endless questions over the years, and I continue to learn from them daily.

I met Mark shortly after.

I can remember seeing very very early versions of SpeedCurve before it was launched and getting incredibly excited about the tool and what Mark was doing with it.

I got even more excited when, later, Mark and Steve pulled me aside at a conference to show me the new RUM capabilities they were going to start adding. (At the time, I think it was just a simple log of all the beacons coming back to them.) I’m a firm believer that the best possible thing a company can do with monitoring is pair their RUM and synthetic data together, and SpeedCurve was the first tool I’d ever seen put them both under the same umbrella.

It’s been fun to watch the tool develop over the years from a small newcomer to an influential tool that has quite literally paved the way to better metrics. (SpeedCurve’s fingerprints are all over the development of critical paint metrics like Largest Contentful Paint, for example.)

Long story short, I’ve been a fan of the tool for a long time, and the team is composed of people who I’ve been learning from since literally day one of my performance career. A chance to work alongside longtime friends on an industry critical tool—hard to beat that!

"What would you say you do here?"

I’ve been lucky enough to work on performance from a variety of angles over the past 14 years. I’ve been a consultant. I’ve worked on tools. I’ve been a product and engineering leader. I’m a bit like a performance squirrel, I guess: there’s a lot about performance that interests me and I’ve worked on it from pretty much every angle at this point.

The one common element through it all is that I love doing whatever I can to help make the web faster.

So… I’m going to continue doing a little bit of everything with SpeedCurve.

In addition to my own consulting, I’ll be doing some consulting with SpeedCurve customers. I’m also excited to work along this absurdly talented team to help make SpeedCurve an even better product than it already is. There are some big plans, and a massive opportunity to make it easier for everyone to build a faster web.

I’m over the moon about this—it’s been a long time coming and I feel honored and humbled to be able to join the team.