SpeedCurve Blog

NEW! Home, Site, Page and Tests dashboards

Mon, 13 Mar 2023 00:00:00 +1300

Things have been busy over here at SpeedCurve HQ! Coming off of the back of our latest RUM Compare dashboard release, we are super excited to launch four new dashboards to make your life better, your work easier, and your websites faster.

Home - Well, now you have one. We needed a place for folks to land when they were jumping into all that SpeedCurve has to offer. The newly released Home page is a great starting place, pulling together views of both RUM and Synthetic data and directing you on where to dig in next.
Sites - We've rebuilt our Sites dashboard to give you a holistic look across all of the URLs you are monitoring. Start here to understand which areas of your site need the most attention and identify areas you can have the most impact.
Pages - This detailed view gives you trended information for all the metrics you know and love. Easily identify areas of opportunity and how things are performing over time for specific pages you are monitoring.
Tests - We've had a lot of requests for this, so we're happy to deliver it. You can now see a list of your latest synthetic tests and spot any failed tests.

Let's take a look!

Home

This is the new default dashboard you'll see when you jump into SpeedCurve for the first time. If you are using both RUM and Synthetic, you'll see both datasets reflected in this view. Otherwise, the metrics reflected will be from your synthetic data.

Beginning with a snapshot of your Core Web Vitals, this page highlights key areas of interest and a high-level snapshot for the selected site. Links from each section take you to a deeper look within other areas of your dashboards.

If you have integrated SpeedCurve into your CI/CD environment (leveraging the deploy API), or if you are manually running synthetic tests using the 'Test Now' option, you'll see the state of your last deployment compared to the previous. And finally, a 7-day trend is displayed for your most important metrics.

Site Dashboard

This view is intentionally focused on the latest data you've collected for your Site which gives you a great starting place when managing performance across various URLs. What pages have the slowest LCP? Home, Order and Print look like a good place to start digging.

How are users perceiving the visual rendering of all the pages in my site? Which pages should I dive into first?

From the LightHouse audits run against your site, you can quickly see which pages need the most attention across the 5 areas of focus.

An aggregate view of the most critical areas for improvement identifies which recommendations will have the most collective impact across your pages.

Page Dashboard

From the Site dashboard, you can narrow your focus and dive into the new Page dashboard. Here you'll see key metrics for the page trended over time. You can use the filters at the top of the page to further refine your data by browser, region, and date range.

A look at the key rendering moments and a full filmstrip for the page focuses on user-perceived performance.

Now that you understand how the page is performing, you can take a look at how the construction for the page has changed over time and where optimization opportunities exist, such as reducing network requests and their file size.

Finally, a list of prioritized improvements for the page is displayed. Pay special attention to critical recommendations that are flagged with the Core Web Vitals metrics. If you implement those recommendations, you should see an improvement in your Vitals.

Tests Dashboard

We love simplicity, and this dashboard is a great example of why. We have received a lot of feature requests such as "Just show me a list of the tests I've run recently" as well as "How can I quickly see which tests are failing?" This dashboard achieves both!

If you want to refine this view, you can apply filters to show you tests for a specific site, page, region or browser. You can also filter on test status ('succeeded' or 'failed') and test type ('scheduled test' or 'deploy').

Finally, if you have any failed tests we'll show those at the top of the previously mentioned Home page in order to quickly alert you to the failures. You can then click through to the test details to determine why the test failed.

We hope this dashboard will help to quickly inform you so you can take action and protect your synthetic checks budget.

We hope you find these latest additions to SpeedCurve useful. As always, let us know what you think!

Why you need to know your site's performance plateau (and how to find it)

Mon, 06 Mar 2023 00:00:00 +1300

"I made my pages faster, but my business and user engagement metrics didn't change. WHY???"

"How do I know how fast my pages should be?"

"How can I demonstrate the business value of performance to people in my organization?"

If you've ever asked yourself any of these questions, then you could find the answers in identifying and understanding the performance plateau for your site.

What is the "performance plateau"?

The performance plateau is the point at which changes to your website’s rendering metrics (such as Start Render and Largest Contentful Paint) cease to matter because you’ve bottomed out in terms of business and user engagement metrics.

In other words, if your performance metrics are on the performance plateau, making them a couple of seconds faster probably won't help your business.

The concept of the performance plateau isn't new. I first encountered it more than ten years ago, when I was looking at data for a number of sites and noticed that – not only was there a correlation between performance metrics and business/engagement metrics – there was also a noticeable plateau in almost every correlation chart I looked at.

A few months ago someone asked me if I've done any recent investigation into the performance plateau, to see if the concept still holds true. When I realized how much time has passed since my initial research, I thought it would be fun to take a fresh look.

In this post, I'll show how to use your own data to find the plateau for your site, and then what to do with your new insights.

Background

For this new investigation, I selected four sites that experience a significant amount of user traffic. For each site, I used a month's worth of RUM (real user monitoring) data to generate correlation charts.

Correlation charts show the relationship between performance metrics – in these instances, Start Render and Largest Contentful Paint (LCP) – and user engagement (measured as bounce rate). They're a great tool for showing non-technical folks how performance affects the business. (You can also create correlation charts that show the relationship between performance metrics and business metrics, such as conversion rate and cart size, but bounce rate is easier to measure right out of the box with most RUM tools.)

Results

The correlation charts below show the distribution of all visits, with each yellow bar representing a cohort of visits that experienced a given Start Render or LCP time. The blue bar represents the change in bounce rate across all cohorts.

In each of the correlation charts below, I've highlighted:

Optimal speed – The cohort of sessions that correlated with the lowest (aka best) bounce rate for that site
Beginning of the performance plateau – The cohort of sessions where the bounce rate begins to plateau
Median measurement for all visits represented in the chart

Keep reading for observations and takeaways.

Site A

Site B

Site C

Site D

Observations

1. A clear performance plateau emerged for each site

Each site experienced a plateau at which business metrics remained more or less the same as performance continued to degrade.

2. Plateaus emerged for both Start Render and Largest Contentful Paint

I expected the results for Start Render, as it's been around as a page speed metric for many years, and has been proven to correlate to business metrics. LCP is relatively new, and has received a great deal of attention since it was introduced a couple of years ago as part of Google's Core Web Vitals. It's good to see evidence that it's a meaningful metric to track in terms of business goals.

3. The plateau emerges surprisingly quickly in some cases

For example, Site C's performance plateau starts at 400 milliseconds. That's early!

4. There's a lot of variability in the distance between the optimal bounce rate and the plateau

For some sites, you can see a much steeper incline in the curve from optimal to plateau. For some sites (such as Site C), the difference was as little as 300 milliseconds. For others (such as Site A), the gap was as long as 9 seconds.

5. The plateau sometimes started later when looking at LCP

Creating correlation charts in both Start Render and LCP generated interesting results. In two of the four sites I looked at, the charts were roughly comparable. For the other two sites, the plateau started later for LCP than it did for Start Render. This could be attributed to the fact that LCP measures when the largest visual element has completely finished rendering, so it can occur much later than Start Render.

6. For some sites the performance plateau starts well before the median

Predictably, the optimal bounce rate generally correlated to the cohort of sessions that is much faster than the median. A bit more surprisingly, for some sites the performance plateau started well before the median. This could come as a scary revelation for some site owners, because it means that the bulk of your user sessions are occurring on the plateau.

How to measure the performance plateau for your own site

I can't emphasize enough that the examples I've shared are illustrative, not prescriptive. The performance plateau for your site will be different from the plateau for another site. You need to look at your own real user data. (If you're new to performance, you might be interested in this synthetic and real user monitoring explainer.)

Fortunately, the process for identifying the low end of your site’s performance threshold is fairly straightforward. All you need is access to a statistically significant amount of your RUM data, plus whatever analytics tool you use for tracking business or user engagement metrics.

Step 1: Identify the metrics you want to measure

As mentioned above, bounce rate is a good metric to start with, because it's already gathered automatically by most real user monitoring tools.

If you have access to other data sources, then you can create a variety of correlation charts, If run an ecommerce site, then you can measure revenue, cart size, and conversion rate. If you work on a media site, then page views and bounce rate matter.

Step 2: Gather lots of real user data

To ensure that you get statistically relevant results, the more data you can gather, the better. If your dataset is too small, you could get wonky results. When I conducted my investigation, I aggregated millions of transactions that took place over a single month. (If you're interested in trying real user monitoring, you can start a free RUM trial in speedCurve.))

Step 3: Create correlation charts

I've demonstrated how I like to show bounce rate (or whatever business/engagement metric you're plotting) across the distribution of sessions. (If you're a SpeedCurve user, here's how to create correlation charts.)

What to do with your findings

After you've finished your own investigation, you can do a few things with the results:

1. Share your findings within your organization

Correlation charts are a powerful tool for showing stakeholders the impact that site speed has on the business. Even if your results aren't what you hoped they would be, you can use this data to prove the value of continuing to invest in performance.

2. Understand why your business metrics are not improving despite your efforts

This might seem a bit demoralizing, but when you think about it, it's actually helpful to know. When you know where your performance pleateau begins, you can answer the question "Why don't my business or user engagement metrics improve when I make my site faster?" If you improve Start Render from 5 seconds to 3 seconds, but the performance plateau for your site starts at 2 seconds, you haven't yet made Start Render fast enough.

3. Change your performance targets

Set targets for moving more of your users into the cohorts that experience faster Start Render or LCP times. Ideally, improving key site speed metrics for more of your users should improve bounce rate (or whatever user engagement or business metric you're tracking) for more of your users. Ultimately, this is good for your business.

You can use your performance plateau to set goals. Continuing with the example in point 1, above, if you know that the plateau starts at 2 seconds, you can create a Start Render target of 1.5 seconds to work toward.

4. Or don't change your performance targets

In the Site C example, the optimal bounce rate occurs for the 100-millisecond LCP cohort, and the plateau starts just 300 milliseconds later. With a huge amount of work, you might succeed in delivering faster LCP times to more sessions, but would the effort be worth it?

As the close-up view of the chart below shows, the bulk of sessions have speedy LCP times that are at the beginning of the performance plateau. In this case, the chart shows that perhaps you can be satisfied with your efforts, and your focus should be on fighting regressions and staying fast.

5. Create a baseline to measure against

Repeat this exercise periodically – perhaps monthly, or semi-annually, or after a deploy where you've made a number of performance improvements – and compare the correlation charts over time. Ideally, you'll see more of your sessions fall into the faster section of the distribution, before the performance plateau.

Questions? Feedback?

If you experiment with creating correlation charts and plotting the performance plateau for your site, I'd love to hear about your results!

2022 in review: New dashboards, Core Web Vitals enhancements, third-party tracking & more!

Thu, 22 Dec 2022 00:00:00 +1300

Every year feels like a big year, and 2022 has been no exception. Not only did we celebrate our ninth birthday (!!!) we also:

Launched a series of new and improved dashboards to help you identify and fix your performance issues more quickly and efficiently (More on those below.)
Expanded our test agent locations to bring you even more places to test from worldwide
Jumped into in-person events, and were so happy to catch up with some of you at SmashingConf and performance.now() (You can watch videos of my talk about performance budgets and Andy Davies' talk about third parties.)

Keep reading for a full recap of the past year...

Home dashboard

It only took nine years, but we now have a home page! ;-)

Your Home dashboard brings together your Synthetic and RUM data to provide an easy-to-digest – and easy-to-share – snapshot of how your site is performing. You can see at a glance:

Key metrics like Core Web Vitals and your User Happiness score
The current status of your performance budgets
Performance in different geolocations around the world
Your current competitive benchmarks status
Top Lighthouse recommendations for performance improvements

(Hint: If you're considering installing a big-screen performance monitor at your organization, this dashboard is a good candidate for that.)

Synthetic Site dashboard

You may have noticed that your Site dashboard has been given a complete overhaul. It now focuses on helping you spot differences between the performance of your pages, making it easier to identify which pages in your site have performance issues.

When you identify a page that needs further investigation, you can quickly navigate through to your new Synthetic Page dashboard...

Synthetic Page dashboard

You'll find your new Page dashboard to be similar to your old Synthetic Site dashboard. Here you can still find time series charts that show how performance is trending over time across a number of key metrics, including Core Web Vitals.

Contextual Web Vitals recommendations

As you may already know, we run full Lighthouse audits on every page you test in synthetic. The results – including detailed performance optimization recommendations – are available in your test details.

In addition to that, we now badge all performance recommendations with the Core Web Vitals metric (LCP, CLS, FID, and TBT) they affect. Fix those performance issues and you should see an improvement in your Core Web Vitals and Lighthouse scores.

Here are the dashboards where you can find your Vitals-badged performance recommendations:

Home
Vitals
Improve
Synthetic Site
Synthetic Page

RUM Compare dashboard

With your RUM Compare dashboard, you can easily generate side-by-side comparisons for any two cohorts of real user data.

Some of the many reasons you might want to compare RUM cohorts:

Improve Core Web Vitals by identifying the tradeoffs between pages that have different layout and construction
Triage a performance regression related to the latest change or deployment to your site by looking at a before/after comparison
Explore and compare different out-of-the-box cohorts, such as device types, geographies, page labels, and more
Analyze A/B tests or experiments to understand which had the most impact on user behaviour, as well as performance
Optimize your funnel by understanding differences between users that convert or bounce from your site and users who don't
Evaluate CDN performance by exploring the impact of time-of-day traffic patterns

RUM update: Page labels

Page labels are an integral part of getting the most out of dashboards like your RUM Compare dashboard (above). We've added new capabilities for defining page labels with an easy, no-code, server-side (SpeedCurve) option using URL patterns. (This means you now have four options for labeling pages in RUM, depending on your use case.)

RUM update: Performance heat maps

We've added heat maps to your RUM Performance dashboard. These let you quickly spot which page labels or paths are outliers. You can then prioritize and focus on these areas to improve important metrics, such as Largest Contentful Paint. Fixing high-traffic but poor-performing areas of your site will help lift your overall metrics.

Synthetic update: Compare third parties

When comparing any two synthetic tests, we now show you a comparison of common third parties and identify which third parties are unique to each test. This lets you quickly spot changes in common third parties, as well as new third parties that might have introduced performance issues.

Expanded Industry Speed Benchmarks

Page Speed Benchmarks is an interactive dashboard that lets you explore and compare web performance data for leading websites across several industries – from retail to media – over the past year. The dashboard allows you to easily filter by region, industry, mobile/desktop, fast/slow, and key web performance metrics, including Core Web Vitals.

The Industry Benchmarks dashboard is publicly available (meaning you don't need a SpeedCurve account to explore it) and is a treasure trove of meaningful data that you can use for your own research.

Lots of new videos!

We've added a plethora of new videos to give you everything from dashboard tours to best practices for fighting third-party regressions and tracking Core Web Vitals. You can find them in our Support Hub.

Here's to the year that was and the year ahead. :-)

As we enter our tenth year of bringing you the very best performance monitoring tools we can muster – and helping you deliver fast, joyous experiences to your users – we welcome your feedback and suggestions. Let us know your thoughts by leaving a comment or sending us a note at support@speedcurve.com.

NEW! RUM Compare dashboard

Thu, 01 Dec 2022 00:00:00 +1300

Exploring real user (RUM) data can be a hugely enlightening process. It uncovers things about your users and their behavior that you never might have suspected. That said, it's not uncommon to spend precious time peeling back the layers of the onion, only to find false positives or uncertainty in all that data.

At SpeedCurve, we believe a big part of our job is making your job easier. This was a major driver behind the Synthetic Compare dashboard we released last year, which so many of you given us great feedback on.

As you may have guessed, since then we've been hard at work coming up with the right way to explore and compare your RUM datasets using a similar design pattern. Today, we are thrilled to announce your new RUM Compare dashboard!

With your RUM Compare dashboard, you can easily generate side-by-side comparisons for any two cohorts of data. Some of the many reasons you might want to do this include:

Improve Core Web Vitals by identifying the tradeoffs between pages that have different layout and construction
Triage a performance regression related to the latest change or deployment to your site by looking at a before/after comparison
Explore and compare different out-of-the-box cohorts, such as device types, geographies, page labels, and more
Analyze A/B tests or experiments to understand which had the most impact on user behavior, as well as performance
Optimize your funnel by understanding differences between users that convert or bounce from your site and users who don't
Evaluate CDN performance by exploring the impact of time-of-day traffic patterns

Let's take a tour...

How to compare RUM sessions

In September of last year, we released the RUM Sessions dashboard. Along with the Sessions dashboard, we provided you with the ability to drill into specific datapoints on a time series to explore those sessions, along with the ability to bookmark them. You'll now find the ability to compare sessions from a time series with those existing options.

To compare two different data points from any RUM time series chart, simply click 'Compare Sessions' on two separate data points:

After you've selected two sessions to compare, you'll see a dashboard with those sessions in a side-by-side comparison:

Filtering your data

Navigating directly to the dashboard is another option. From the RUM menu in the side navigation, click on 'Compare'. By default you are presented with a view of Mobile vs. Desktop device types:

You can alter each of the cohorts by adding and removing filters from the menu for the cohort:

If you're already experienced with our other RUM dashboards, the different options available in your filter will look familiar. The 'Metric Threshold' filter has also been introduced as a way for you to explore 'fast' versus 'slow' sessions for a metric. To use, simply select the metric along with the upper and lower bounds for each cohort.

To demonstrate, let's look at a comparison of fast sessions (0-2 seconds) to slow sessions (2-15 seconds) for Largest Contentful Paint:

Dashboard components

Overview

This comparison highlights the comparison of key performance metrics between the cohorts. Here we can see that the faster cohort has a median LCP of 1.25 seconds, while the slower is at 3 seconds. More importantly, we see the bounce rate is 27% higher for the slower subset of users. Speed matters!

Sample sizes

If you've spent any amount of time exploring your RUM data, you've probably learned that the size of your population matters a lot. This chart lets you quickly discern whether or not you're comparing statistically viable sets of data. While the example below is not a 50/50 split, there seems to be enough data in the slower cohort compared to the faster one to make some valid observations.

Location

Understanding where your users are coming from and how their experience may vary is a key part of RUM analysis, specifically as you make investment decisions on CDNs or CSPs. While not largely pronounced, in the visualization below you can see there is a larger subset of 'fast' users coming from Norway and the United States. Note that you have the ability in this graph to toggle between an absolute value and a percentage. This is key when you are comparing datasets that are not the same size.

User Happiness

Somewhat self-explanatory, the comparison of your 'Happy Page Views' shown here illustrates a lot more of your slower users in the 'Unhappy' bucket. More can be found on our Happiness Index and how it's calculated here.

Looking at cohorts by other filters

This table breakdown defaults to a breakdown across cohorts by Browser, but there are several other dimensions to explore. In this example, we don't see much of a browser-driven impact to LCP. (If you're wondering where iPhone is, keep in mind that, unfortunately, not all metrics are supported across platforms.)

However, here we can see that a breakdown by connection speed highlights that while most of the users are on fast connections, there are a higher percentage of users coming from slower connection speeds in the slow cohort.

Event Timeline

Your event timeline allows you to compare key metrics between your cohorts. This can be useful in identifying other metrics that may be more interesting or potentially responsible for downstream metrics such as LCP. Backend time here is 500ms slower, which affects ALL metrics.

Metrics

The last, and potentially most compelling, component in the dashboard is focused on metrics. This component contains a LOT of information you can dig into.

When you expand each metric, you have the ability to look at a comparison distribution, a time series comparison or a breakdown across all of our key dimensions. In this distribution chart, you can see that fast users often have network times of 0 seconds, most likely due to connection reuse.

Stacked bubble charts can help you quickly identify the presence of outliers. In the graph below, a lot of the slow LCP traffic is driven by older versions of Chrome. Oftentimes this can be automated traffic or other non-user traffic.

We welcome your feedback!

Phew! That's a lot of information packed into a single dashboard. We're excited to hear how you put this new RUM capability to work for your site. Drop us a line and let us know what you think!

And if you're not yet using our real user monitoring to learn how actual users are experiencing your site, try our RUM for free!

Page labels: Why they are important and how to use them

Wed, 02 Nov 2022 00:00:00 +1300

Labeling your pages in your synthetic and real user monitoring (RUM) tools is a crucial step in your performance monitoring setup. We recently released some exciting new capabilities for labeling your RUM pages that we want to share with you. This is also a great opportunity to reiterate why page labels are important, and to show you how easy it is to apply labels to your pages.

Why should I care about page labels?

There are so many benefits to labeling your pages in both your synthetic and real user monitoring (RUM) tools. Page labels give you the ability to:

Compare and benchmark similar pages across different sites
Create more meaningful performance budgets in RUM
Correlate RUM data with synthetic diagnostics in the same charts (e.g., for Core Web Vitals that are measurable in synthetic and RUM)
Prioritize performance optimizations according to their projected impact on real users

Ready to learn more? Let's get to it!

How do page labels help me?

Page labels give you the opportunity to gain richer insights from your data, and then to make that data actionable. Here are a few examples.

Compare and benchmark similar pages across different sites

For synthetic testing, page labels give you the ability to add URLs to a defined segment. When you configure a synthetic test, you provide the URL as well as a page label. There are a few common use cases for comparing similar pages:

Compare different URLs of the same type across different sites you manage.
Compare a pre-production URL to a page on your production site.
Benchmark key pages against those of your competitors.

Benchmark Dashboard from SpeedCurve

URL paths can be too granular

While using URL paths to filter your RUM data can be really useful, by default when looking at datasets for a specific page template it is also important to have the ability to look at them as groups or segments.

To illustrate, let's compare two distribution charts: the first for a set of grouped product pages (PDP), and the second for a unique URL path for a specific product page.

This distribution for pages labeled as 'PDP' (product details page) is a good example of a fully-formed set of data representing a population:

Next, this distribution illustrates the issue with cardinality that occurs when attempting to analyze a specific URL. Even the p75 is quite different between the two distributions:

It's difficult to attempt to draw firm conclusions or use the information in the second chart to inform your performance budgets for such a small dataset.

For more information on interpreting histograms, see this support article.

Combine synthetic and RUM datasets

One of the things SpeedCurve is known for (and extremely passionate about) is using the complementary datasets from RUM and synthetic monitoring. When set up correctly, you have the ability to match synthetic page labels with RUM page labels. This opens up a lot of opportunity to do things like:

Present both datasets on a single chart

This example shows Largest Contentful Paint (LCP) from RUM for a page labeled 'Home' along with Image Size and the Lighthouse Performance score from a synthetic test with the same label:

Correlate RUM data with synthetic diagnostics for Core Web Vitals

When using both RUM and synthetic, your SpeedCurve Vitals dashboard provides the most informed metrics (Largest Contentful Paint, First Input Delay, and Cumulative Layout Shift) from RUM, alongside synthetic diagnostics such as LCP element identification and performance optimization recommendations.

Prioritize performance opportunities by RUM impact

Your Improve dashboard aggregates performance recommendations from synthetic testing and identifies the percentage of your RUM traffic that would benefit. This is extremely helpful when trying to prioritize performance work for the most impact on your actual users.

How do I set up page labels in SpeedCurve?

Synthetic page labels

This is really straightforward. From your Site Settings, simply add a label next to the URL you want to test.

As a best practice, page labels should be unique for each URL if you are planning on comparing them in your benchmarks or correlating them with your RUM data.

As a rule of thumb, you should have page labels for everything you label in RUM. For an overview of Site Settings, see this support article.

RUM page labels

Product update! We've recently rolled out the ability for customers to define page labels with a no-code, server-side (SpeedCurve) option using URL patterns (option 2 below). There are now four options for labeling pages in RUM depending on your use case. The following are in priority order. If you use more than one option to define a unique page label, we will default to option with the highest priority.

Option 1: Defining page labels using the RUM API

To programmatically assign labels, simply call the API from your pages using the following JavaScript placed after your SpeedCurve snippet:

LUX = window.LUX || {};

LUX.label = "Home";

For more information on the RUM JS API, see this support article.

Option 2: Defining page labels using URL rules (NEW!)

This no-code approach is by far the most convenient. From your RUM Settings, choose the Page Label URL Rules tab in the top navigation and select 'Add Label'.

Next, choose a name for your label and add one or more URL rules. Full paths are supported (including domain), as well as paths relative to the root domain. Wildcard characters may also be used as demonstrated below.

Now that you've added a rule (or rules), click on 'URLs with no associated Page Label' to show a list of URLs that are unmatched. You can then build a rule that matches unmatched URLs and assign it to an existing page label.

Option 3: Use an existing JavaScript variable

This is a good option if you're already using labels that you'd like to repurpose. From your RUM Settings, choose the 'Page Label JS Variable' tab. Simply fill in the field provided with the JavaScript variable, array, or property. (Note that functions are not supported.)

Option 4: Default to `document.title`

If no other options are used to define a page label, the document.title of the page will be used. This is typically not the most desirable option due to the uniqueness of the title. The titles are sometimes extremely long and separated by pipes and spaces (' | ') which can make it hard to navigate dashboard filters, chart titles, etc. Additionally, in some cases titles are so unique that you'll run into the same issues discussed earlier around URL cardinality.

For more information on setting up your RUM page labels, see this support article.

Element Timing: One true metric to rule them all?

Wed, 21 Sep 2022 00:00:00 +1200

One of the great things about Google's Core Web Vitals is that they provide a standard way to measure our visitors’ experience. Core Web Vitals can answer questions like:

When was the largest element displayed? Largest Contentful Paint (LCP) measures when the largest visual element (image or video) finishes rendering.
How much did the content move around as it loads? Cumulative Layout Shift (CLS) measures the visual stability of a page.
How responsive was the page to visitors' actions? First Input Delay (FID) measures how quickly a page responds to a user interaction, such as a click/tap or keypress.

Sensible defaults, such as Core Web Vitals, are a good start, but one pitfall of standard measures is that they can miss what’s actually most important.

The (potential) problems with Largest Contentful Paint

Largest Contentful Paint (LCP) makes the assumption that the largest visible element is the most important content from the visitors’ perspective; however, we don’t have a choice about which element it measures. LCP may not be measuring the most appropriate – or even the same – element for each page view.

The LCP element can vary for first-time vs repeat visitors

In the case of a first-time visitor, the largest element might be a consent banner. On subsequent visits to the same page, the largest element might be an image for a product or a photo that illustrates a news story.

The screenshots from What Hifi (a UK audio-visual magazine) illustrate this problem. When the consent banner is shown, then one of its paragraphs is the LCP element. When the consent banner is not shown, an article title becomes the LCP element. In other words, the LCP timestamp varies depending on which of these two experiences the visitor had!

What Hi Fi with and without the consent banner visible

Even when it’s visible, a cookie banner may not always be the LCP element

In this example from my local newspaper, the story image is displayed and then the consent dialog covers it. But because the text elements in the dialog are smaller than the image, the image is measured for LCP.

Stroud News with and without the consent banner visible

The largest element can vary by viewport size

On some devices What HiFi has a story image hidden behind the ad at the bottom of the page, which is gets measured as LCP. But more commonly we see variations between desktop and mobile viewport sizes, where different elements are measured for LCP.

Joules clothing has this challenge on its product listing pages where the message about the impact of UK postal services going on strike is the LCP element at mobile viewport sizes, but one of the product images becomes LCP element at larger viewports.

Joules on mobile and desktop viewports

LCP uses the rendering timestamp to prioritize same-size elements

Joules’ also highlights another other challenge with LCP – if I asked you to guess which element would be measured for LCP you’d probably guess the first product image, but as all four images are the same size, and the third image was actually rendered first, it’s measured for the LCP timestamp. This might be fine in some situations, but the LCP element might not be the most important image from your – or your visitors' – perspective.

Other caveats...

Those are just some examples of the issues we come across when measuring visitors’ experiences day-to-day. There are other examples, such as:

image elements that are animated in, and
'soft-navigations' in single-page apps (SPAs).

How to understand which elements are chosen for LCP

If you want to understand which elements are being chosen for LCP, you can either use synthetic monitoring or the webvital.js library, which includes support to help determine which element is the LCP element.

With LCP, Chrome is making a best guess on which element is most important, but it may not be consistent between different pages, or different viewports, and it can be affected by other UI elements too.

There's no guarantee that LCP is measuring the moments that matter to your site and your visitors. That’s where Element Timing helps.

What is Element Timing?

Similar to LCP, Element Timing measures when a DOM element is rendered. Unlike LCP, Element Timing allows you (rather than Chrome) to decide which elements you want to measure. And unlike LCP, Element Timing allows you to measure more than one element on a page.

Element Timing is implemented by adding the elementtiming attribute to the element you want to measure, with its value set to the identifier the timestamp will be given.

For example, if we wanted to measure when an image is displayed, we could use this markup:

<img src=”/images/logo.png” elementtiming="logo-shown" />

Then, when the logo is displayed, an Element Timing entry with the identifier ‘logo-shown’ will be created, containing a timestamp and other data about the element, e.g., intersection with the viewport.

Timing entries can be retrieved using a Performance Observer, and the data can be forwarded to a RUM or analytics product:

<script>
    const observer = new PerformanceObserver((list) => {
        let entries = list.getEntries().forEach((entry) => {

            // process entry here;

        });
    });

    observer.observe({ entryTypes: ["element"] });
</script>

How Element Timing can fill the gaps that Largest Contentful Paint leaves

Revisiting the Joules’ example from earlier, we could use Element Timing to measure when landmarks such as the logo or category title are displayed. We could also measure when the first product image is rendered.

Example landmarks on Joules' Product Listing Page (PLP)

Similarly, a publisher might want to measure when their logo, the story headline, and the accompanying image are shown.

It should also be possible to use Element Timing to measure some pop-ups, such as consent banners; however, for now this probably depends on a site serving its own consent banner rather than relying on a third-party service.

A replacement for LCP in single-page apps?

A common question we get from customers with single-page applications (SPAs) is: "How do I measure LCP after a route change?"

The short answer is: "You can’t."

LCP starts measuring when a navigation starts and stops on user input. SPAs often only have a single navigation as the initial page loads, with responses to user actions updating the contents of the page rather than navigating to a new one.

Element Timing doesn't have these restrictions. It can measure new elements as they're inserted into the DOM and rendered, even after user interaction.

In the past we've typically relied on User Timing's marks and measures to measure how long a route change or other action takes to complete. With Element Timing we can also measure when the visual changes are displayed and get a more accurate picture of the users' experience.

How to collect Element Timing data in SpeedCurve

Element Timings can be collected in both Synthetic and RUM by configuring them in the Custom Metrics section of your settings:

Adding a custom metric in SpeedCurve

Configuring Element Timing in SpeedCurve

After you’ve started collecting Element Timings, you can add them to custom charts on your Favorites dashboards and start tracking the moments that matter for you and your visitors. You can also create performance budgets and get alerts when they exceed their thresholds.

In this example, a retailer is tracking when their logo and the first product image is displayed:

Visualising when a retailer's logo and first product image is displayed to a visitor

Limitations of Element Timing

Currently, Element Timing has a few limitations.

Only available in Chromium-based browsers

Neither WebKit's nor Firefox’s status trackers mention it, but as it's implemented as an attribute in markup, it will degrade gracefully in those browsers:

Can I Use Element Timing?

Supported on a limited subset of HTML elements

For privacy and security reasons Element Timing is only supported on a limited subset of HTML elements:

img elements
image elements within an SVG
video element's poster image
elements with a background-image
text nodes

But even with just this subset we can still measure many of the loading milestones we care about, such as key images and headings.

Can't track elements within iframes

As Element Timing only measure elements in the current document, we can’t yet use it to track when say an embedded YouTube video renders or when ads are displayed, as these are typically contained within an iframe. (There’s a proposal to allow iframes to expose some of their performance data to their parent that might enable this, but it’s at an early stage.)

Elements in the ShadowDOM are currently excluded from Element Timing

Watch the discussion about this at the most recent Web Performance Working Group TPAC meeting.

Summary

Largest Contentful Paint is a useful default, but the largest element isn't necessarily the most important from a visitor's perspective.

Element Timing allows us to choose the elements that are most important on our own pages, and therefore measure moments that are most essential to our visitors' experience.

You can use SpeedCurve to collect and track Element Timings so you can see how the performance of key elements changes over time or varies between different pages.

If you've been using Element Timings to measure your visitors' experiences, or if you've implemented them in a SPA, we'd love to hear about it.

References

July 2022 product update: Web Vitals support, more synthetic test agents & performance heat maps

Wed, 27 Jul 2022 00:00:00 +1200

It's been another busy month here at SpeedCurve! Check out our latest product updates below.

RUM: Performance heat maps

Fixing high-traffic but poorly performing areas of your site will improve your overall metrics – and create a faster, better experience for more of your users. To help you do this, we've added heat maps to your RUM Performance dashboard that let you quickly spot which page labels or paths are experiencing performance issues. You can then prioritize and focus on these areas to improve important metrics like Largest Contentful Paint.

Synthetic: New test regions in Italy, Bahrain, South Africa, and Indonesia

Based on your feedback and requests, we've added four new regions from which you can run your synthetic tests:

Italy (Milan)
South Africa (Cape Town)
Bahrain (Manama)
Indonesia (Jakarta)

If none of our current test regions gives you the testing locations you need you can also run your own private agent.

Change summary metrics on your charts

We've added a new display option that lets you change the summary metrics shown at the top of each chart. You can now choose between Aggregate, Minimum, Maximum or Latest. Hovering over any metric will give you a tooltip with more context.

In case you missed it...

Here are some other features we've released in recent months:

Core Web Vitals badges in performance recommendations – Identify exactly which performance issues you need to fix to improve your Lighthouse and CWV metrics. All performance recommendations are badged, so you can see which Core Web Vitals you're improving when you fix performance issues. (You can find recommendations in your Vitals and Improve dashboards, as well as test details.)

Compare third parties – When comparing two synthetic tests, see a comparison of common third parties and identify which third parties are unique to each test.

Bookmarks dashboard for both Synthetic and RUM – Compare any two synthetic tests or RUM sessions.

RUM path filters – Monitor popular or custom user paths in your data.

Most-watched help videos

We've created two video series – dashboard tours and how-to videos – to help you find and fix performance issues on your site. Here are the current top five most-watched videos:

Most-read support articles

We take a lot of pride in maintaining a comprehensive Support Hub. These are the most popular articles users looked up in the past month:

Questions? Feedback? Suggestions?

We'd love to hear from you! Leave a comment below or email us at support@speedcurve.com.

June 2022 product update: Performance recommendations on Vitals dashboard, RUM path filters & more

Mon, 13 Jun 2022 00:00:00 +1200

We've been busy here at SpeedCurve HQ! Here's a roundup of our recent product updates.

Performance recommendations on your Vitals dashboard

On your Vitals dashboard, you now get performance recommendations that are specific to each of the Vitals you're tracking – Largest Contentful Paint, First Input Delay, Total Blocking Time, and Cumulative Layout Shift. This makes your Vitals dashboard a powerful tool for not only seeing how your metrics perform relative to Google's thresholds, but also diagnose your biggest pain points and get prioritized solutions.

Vitals badges on Lighthouse performance audits

Elsewhere in SpeedCurve, all the performance recommendations you see in your Vitals and Improve dashboards – as well as in your synthetic test details – are now badged so you can see which Web Vitals they affect. Fix those issues and you should see improvements in your Vitals and Lighthouse scores.

RUM update: Path filters

RUM update: lux.js v301

New features:

The synthetic onload time for SPAs can be marked with LUX.markLoadTime(), allowing LUX.send() to be called later in the page lifecycle.?
Added the SpeedCurve RUM Debug Parser to help interpret the debug messages.?
LUX.getDebug() now includes events that help to debug some metrics including LCP, CLS, element timing, and long tasks.?
Source maps are now available for lux.js.?

Bug fixes:

Fixed a bug where JavaScript errors were only tracked on the first SPA page view.?

Synthetic update: Compare third parties

When comparing two tests, we now give you a comparison of common third parties. We also identify which third parties are unique to each test. Using this feature, you can quickly identify new and problematic third parties. This short video explains how to diagnose third-party regressions in SpeedCurve.

New in the Support Hub

Questions? Feedback? Suggestions?

We'd love to hear from you! Send us a note at support@speedcurve.com

Sampling RUM: A closer look

Wed, 01 Jun 2022 00:00:00 +1200

Being able to set a sample rate in your real user monitoring (RUM) tool allows you to monitor your pages while managing your spending. It's a great option if staying within a budget is important to you. With the ability to sample real user data, comes this question...

"What should my RUM sample rate be?"

This frequently asked question doesn't have a simple answer. Refining your sample rate can be hit or miss if you aren’t careful. In a previous post, I discussed a few considerations when determining how much RUM data you really need to make informed decisions. If you sample too much, you may be collecting a lot of data you may never use. On the other hand, if you sample too little, you risk creating variability in your data that is hard to trust.

In this post, we are going to do a bit of research and let the data speak for itself. I took a look at the impact of sampling at various levels for three t-shirt sized companies (Small, Medium, Large) with the hope of providing some guidance for those of you considering sampling your RUM data.

In this post, I'll cover:

Methodology
Key findings
Considerations
Recommendations

Methodology

Traffic size

I tried to keep this research as simple as possible. We see a large variety of sites at SpeedCurve, representing an assortment of countries, industry segments, traffic levels and more. For the purposes of this study, I'll use example sites from three cohorts:

Large: >1M daily page views
Medium: 250K-500K daily page views
Small: 10K-100K daily page views

It's important to note that the sites I looked at collect 100% of their RUM data.

Time frame

24 hours. Traffic fluctuates based on the hour of the day, day of the week, and due to seasonality. I looked at the same date, mid-week, for each of the sites, which represented a consistent pattern of daily traffic.

Metric

This was a little tough. Not all metrics are created equal and I try to avoid picking favorites. At the time of this writing, Largest Contentful Paint (LCP) is not supported by all browsers, so it brings with it a bit of bias. This is true of many of the metrics we collect at SpeedCurve. We'll discuss this and other considerations a bit later. In the end, I settled on loadEventEnd due to the fact that it has widespread support across browser platforms.

Sampling method

At SpeedCurve, we have the ability to sample based on sessions versus randomly sampling page views. We feel it's more important to maintain the integrity of the session than to specify precisely how many page views you want to look at. Because we track and identify user sessions, it made things a lot easier for me to sample the data after the fact.

Interpreting the data

There are a lot of ways to compare the data. I'm not a data scientist and I wanted to demonstrate the impact of sampling using views of the data that are familiar to those who have at least seen performance data before.

Aggregates: We will look at the percentage change between the 50th, 75th, and 95th percentiles. I considered anything under 5% acceptable.

Histograms: You can learn a lot if you just look at your data. Histograms are great for showing the performance characteristics of an entire population. (Learn more about understanding and interpreting histograms.) For this experiment, we are comparing the overall shape and distribution of our sampled versus unsampled populations. In some cases, the aggregates may have been under 5%, but the the histogram was very sparse and didn't resemble the original distribution. For example, the differences between these two histograms is obvious despite their medians being within reason. When looking at the 95th percentile, you observe that the long tail is essentially 'missing' from the sampled data. While somewhat unscientific, I used the eyeball test along with the aggregates to decide if the rate was appropriate.

Time series: Intraday variability is important if you're using the RUM data operationally. A simple time series will be used to illustrate how sampling impacts the 'noise' factor.

Key findings

TL;DR

For the most part, I found that if the sampled population of users was greater than 3,000, the aggregate stats were pretty close to your upsampled population (1-2% difference in the median). However, you should read on to understand some of the trade-offs dependent on your use case for RUM. Or, if you'd rather, go ahead and jump to the results.

RUM for reporting

If you're simply looking to RUM as a reporting tool that can represent your daily performance, you're in luck. You can get away with a relatively small sample of your overall population depending on your size.

To determine the smallest sample rate for each group, we looked at a combination of the aggregate numbers and a comparison of the histograms. Note the consistency in the 95th percentile illustrated in these comparison charts.

Small (10K-100K daily page views sampled at 50%)

Medium (250K-500K daily page views sampled at 10%)

Large (>1M daily page views sampled at 1%)

Intraday performance monitoring

You might be one of those sites deploying code multiple times a day. Maybe you're susceptible to variability from things such as third parties, varying traffic patterns, or other unknowns. (Aren't we all?) If this is the case, you may have more operational need for RUM. Your sampling rate can have a bit of impact on whether or not your data appears noisy or unpredictable.

Looking at the recommended rates from the previous use case, the examples below show you how much you'll need to dial that up to get a reliable picture of hourly performance, and even more if you are looking at real-time monitoring (by minute).

Hourly monitoring:

Small – increased from 50% -> 75%

While increasing the rate helped remove some of the large deviations seen, the data is naturally much more variable for small-traffic sites.

Medium – increased from 10% -> 25%

While the peak hours were somewhat consistent at 10%, increasing the rate to 25% removed the larger off-peak deviations.

Large – increased from 1% -> 10%

Increasing the rate by 10% greatly improved consistency for the large-traffic site.

Real-time monitoring:

Small – increased from 75% -> 95%

For some of the larger spikes in the data, increasing the sample to 95% was effective. However, given how variable the data is, it's hard to say if real-time monitoring of smaller sites like this is really very effective.

Medium – increased from 25% -> 75%

For the medium-traffic site, there was benefit when increasing the rate to 75%.

Large – increased from 10% -> 40%

For this particular large-traffic site, getting real-time data consistent with the whole population required a much larger increase in the sample rate than anticipated.

Considerations

Data segmentation

Here comes the kicker. One of the great things about RUM is the ability to slice and dice your data. The distribution of your user population is made up of all types of user experiences. This has a pretty big impact on your sample rate, as when you filter/segment/slice/dice your data, you're effectively reducing the size of your population.

When determining how sampling will be affected by the segments you care about, get an idea of the percentage of traffic that is represented by the segment and factor that percentage into your overall rate. Some of the common segments include country, page types, device types and browsers. After applying a lot of segmentation to the experiments above, a good rule of thumb is to increase your sample rate by 50% (or collect 100% of the data for small sites).

Metrics

As mentioned earlier, there are some metrics (okay, many metrics) that aren't supported across browsers. Just as you would increase your sample rate for the segments, you should consider increasing the sample rate for metrics such as FCP, LCP and Total Blocking Time, which don't have broad browser support. This is also true of some network-related metrics that don't occur on every page load (DNS, Connect, SSL, Redirect).

Increasing time windows

It's sometimes recommended that you need to capture 100% of your data if you are comparing RUM data for different experiments, or capturing conversion data in order to understand the business impact of performance. This is not always the case. As an alternative, you can look at a much larger time window with a LOT more sampled data. This is also true of sites with low traffic numbers. Simply expand your time window until you have a healthy distribution.

Recommendations

The intent of this post was to help provide some direction around sampling RUM data. The recommended levels are not intended to be precise, as there are too many factors that could influence things one way or the other. Use this table as a guide in addition to the knowledge you have about your users:

Learn more about RUM sampling

As you may have guessed, SpeedCurve supports data sampling in RUM. This article goes into detail about how our RUM sampling works and explains the different ways you can implement it. If you have any questions or feedback, we'd love to hear from you. Leave a comment below or send us a note at support@speedcurve.com.

Understanding the performance impact of anti-flicker snippets

Thu, 28 Apr 2022 00:00:00 +1200

Experimentation tools that use asynchronous scripts – such as Google Optimize, Adobe Target, and Visual Web Optimizer – recommend using an anti-flicker snippet to hide the page until they've finished executing. But this practice comes with some performance measurement pitfalls:

Hiding the contents of the page can have a dramatic effect on the Web Vitals that measure visual experience, such as First Contentful Paint (FCP) and Largest Contentful Paint (LCP)
Anti-flicker snippets can also affect Cumulative Layout Shift (CLS) and the synthetic equivalent of First Input Delay (FID), Total Blocking Time (TBT).

In this post we'll look at how anti-flicker snippets work, their impact on Web Vitals, and how to measure the delay they add to visitors' experience.

Hiding the page contents

Normally web pages are rendered progressively. As browsers start to receive content, they can layout and render page elements and display the content bit by bit.

Anti-flicker snippets hide the contents of a page until the the experimentation tool (e.g. Google Optimize) has finished applying its experiments. The hypothesis is that if a visitor sees the page changing, it may influence how they behave – either because they had an unpleasant experience or simply because they became aware that they're in an experiment.

But as you can see in these filmstrips of Wiggle, a UK cycling retailer, hiding the page can have a dramatic impact on a visitor's experience:

Comparison of a site loading with (top) and without (bottom) the anti-flicker snippet enabled

The test in top row has the anti-flicker snippet in place. The content is displayed 'all at once' when the anti-flicker snippet reveals the page.

The test in the bottom row has the anti-flicker snippet removed. The page loads progressively, with the menu and logo appearing about 1.5s before the hero image. The anti-flicker snippet delayed showing the content for two seconds.

These tests were over a fast network connection, so First Contentful Paint and Last Contentful Paint happened simultaneously. On slower connections, however, there is a gap between FCP and LCP – and some progressive rendering – but FCP still won't start until the anti-flicker snippet finishes.

(I chose Wiggle semi-randomly after searching the HTTP Archive for sites that use Google Optimize.)

Effects on other Web Vitals

Anti-flicker snippets can affect other Web Vitals, too:

Decrease in Total Blocking Time – TBT is a synthetic monitoring metric that measures how long JavaScript and others tasks prevent the page from handling user interaction. It starts measuring at First Contentful Paint (FCP) and stops at Time to Interactive (TTI). As anti-flicker snippets delay FCP, then the window for measuring the Long Tasks gets smaller, which means you may see a decrease in TBT.
Lower Cumulative Layout Shift score – CLS measures how much content moves around on the page. If some of this movement happens while the page is hidden, then the CLS score will be lower.

To summarize: On one hand, anti-flicker snippets make metrics such as FCP (and possibly LCP) worse. On the other hand, they can appear to improve TBT and CLS.

My instinct is that showing content to the visitor sooner may be a higher priority than the incidental TBT or CLS boost. But rather than trust my instincts, we should measure the impact of hiding the page. To do that, we need to understand a little about how the snippets work.

How anti-flicker snippets work

Anti-flicker snippets typically add a style with opacity: 0 to the elements to be hidden. In Google Optimize's default case is the whole document.

The (un-minified) Optimize snippet below declares the .async-hide class in a style block, and then applies it to the document using a script. It also defines a function to remove the class and sets a timer to call this function after four seconds.

The .async-hide class will either be removed when Google Optimize finishes applying its variants or when the timeout value is reached. In the example below, FCP can be delayed by up to four seconds (the default in the snippet example).

<!-- anti-flicker snippet for Google Optimize (recommended) -->

<style>
    .async-hide {
        opacity: 0 !important
    }
</style>

<script>
    (function(a, s, y, n, c, h, I, d, e) {
        s.className += ' ' + y;
        h.start = 1 * new Date;
        h.end = I = function() {
            s.className = s.className.replace(RegExp(' ?' + y), '')
        };
        (a[n] = a[n] || []).hide = h;
        setTimeout(function() {
            I();
            h.end = null
        }, c);
        h.timeout = c;
    }
    )(window, document.documentElement, 'async-hide', 'dataLayer', 4000, {
        'GTM-XXXXXX': true
    });
</script>?

If you'd like to understand the snippet in more detail, there's an annotated version in this Optimize support article: Using the Optimize anti-flicker snippet | Google Developers

As a fallback, four seconds is a long time. Based on the Chrome UX Report thresholds, a page needs to display the Largest Contentful Paint element within 2.5s for it to be considered good.

Not every visitor may reach that timeout. For some visitors the experiments may complete soon enough to avoid it.

How often the snippet reaches the timeout will depend on factors like:

the number of experiments,
how long the experiments take to execute,
what device the visitor is using, and
the speed of the network the device is connected to.

If we measure how long the page is hidden, we can start to understand how Optimize affects our visitors experiences, the range of delays it adds, and how it influences visitor behaviour.

Measuring how long the page is hidden

Unfortunately, Google Optimize – like most third-party tags – doesn't expose any timing information for its key milestones (page hidden, page shown), but there are still ways we can measure them.

1. Update the Optimize anti-flicker snippet to include performance marks and measures

A start mark is recorded just before the hide class is added, and then when the class is remove an end mark, and duration measure are recorded.

<!-- anti-flicker snippet for Google Optimize (recommended) -->
<style>
    .async-hide {
        opacity: 0 !important
    }
</style>

<script>
    (function(a, s, y, n, c, h, I, d, e) {
        performance.mark('anti-flicker-start');

        s.className += ' ' + y;
        h.start = 1 * new Date;
        h.end = I = function() {
            s.className = s.className.replace(RegExp(' ?' + y), '')
            performance.mark('anti-flicker-end');
            performance.measure('anti-flicker-duration', 'anti-flicker-start', 'anti-flicker-end');
        };
        (a[n] = a[n] || []).hide = h;
        setTimeout(function() {
            I();
            h.end = null
        }, c);
        h.timeout = c;
    }
    )(window, document.documentElement, 'async-hide', 'dataLayer', 4000, {
        'GTM-XXXXXX': true
    });
</script>

Editing the predefined snippet might be a bit fragile, as in the future someone might not notice it's been customised and overwrite it with the default version.

2. Create a second snippet that uses a MutationObserver to detect when the `async-hide` class is removed from the document

This is probably more sustainable as it's less prone to being overwritten.

(function (node, selector, name) {

    performance.mark(name + '-start');

    const callback = function (mutationsList, observer) {

        // Use traditional 'for loops' for IE 11 support
        for (const mutation of mutationsList) {

            if (mutation.attributeName === 'class' &&
                !mutation.target.classList.contains(selector) &&
                mutation.oldValue.includes(selector)) {

                    performance.mark(name + '-end');
                    performance.measure(name + '-duration', name + '-start', name + '-end');

                    observer.disconnect();
                    break;
            }
        }
    }

    const observer = new MutationObserver(callback);
    observer.observe(node, { attributes: true, attributeOldValue: true });

})(document.documentElement, 'async-hide', 'anti-flicker');

This measurement snippet should be placed immediately after Google Optimize's anti-flicker snippet. It creates a mark when it runs, and then another when the class is removed from the document. It also creates a measure to record how long the page was hidden for.

The snippet takes three parameters:

the element that's being hidden,
the name of the class used to hide it, and
a prefix for the name of the marks and measures.

The first two must match their equivalents in the anti-flicker snippet.

A similar measuring approach can be used for Adobe Target and VisualWebOptimizer. There are example snippets for these in our support docs: Custom metrics for anti-flicker snippets

Using the data

Once the snippet is installed on the page and the User Timing metrics configured in SpeedCurve, they can be included in dashboard charts. For example, you can create a correlation chart in RUM to plot how bounce rate is affected by the length of time page is hidden for.

Correlation chart showing relationship between bounce rate and how long the page is hidden

In this example, the chart shows the bounce rate increasing when the page is hidden for longer durations. It also shows that some visitors are waiting four seconds before the see any content!

Measuring how long the page is hidden opens up opportunities to get a better understanding of how this wait time affects our visitors' experience. We can use the RUM Sessions dashboard to identify and explore which visitors are being affected by slow Optimize experiments. We can experiment with reducing the timeout so that visitors won't see a blank screen for as long. Or if we recorded which variant the visitor was seeing via RUM's custom data API, we could see which experiments took the longest to execute.

(While testing the snippet, I discovered a common third-party reviews service was corrupting the duration measure, so in some cases you may need to switch to using the anti-flicker end mark instead)

Summary

Managing the performance of third-party tags is a key aspect of delivering great web experiences. Unfortunately, the performance of third-party tags can be pretty opaque. In an ideal world, tag vendors would use marks and measures to help us understand how their tags behave in the field, but until they do, browser APIs such as MutationObserver and User Timing can help us to measure some aspects of them.

If you've got other third-party tags that you'd like help with measuring, or if you try out one of the snippets for measuring anti-flicker snippets, we'd love to hear from you.

SpeedCurve Blog

NEW! Home, Site, Page and Tests dashboards

Home

Site Dashboard

Page Dashboard

Tests Dashboard

Related:

Why you need to know your site's performance plateau (and how to find it)

What is the "performance plateau"?

Background

Site A

Site B

Site C

Site D

Observations

1. A clear performance plateau emerged for each site

2. Plateaus emerged for both Start Render and Largest Contentful Paint

3. The plateau emerges surprisingly quickly in some cases

4. There's a lot of variability in the distance between the optimal bounce rate and the plateau

5. The plateau sometimes started later when looking at LCP

6. For some sites the performance plateau starts well before the median

How to measure the performance plateau for your own site

Step 1: Identify the metrics you want to measure

Step 2: Gather lots of real user data

Step 3: Create correlation charts

1. Share your findings within your organization

2. Understand why your business metrics are not improving despite your efforts

3. Change your performance targets

4. Or don't change your performance targets

5. Create a baseline to measure against

Questions? Feedback?

Related:

2022 in review: New dashboards, Core Web Vitals enhancements, third-party tracking & more!

Home dashboard

Synthetic Site dashboard

Synthetic Page dashboard

Contextual Web Vitals recommendations

RUM Compare dashboard

RUM update: Page labels

RUM update: Performance heat maps

Synthetic update: Compare third parties

Expanded Industry Speed Benchmarks

Lots of new videos!

Here's to the year that was and the year ahead. :-)

NEW! RUM Compare dashboard

How to compare RUM sessions

Filtering your data

Dashboard components

Overview

Sample sizes

Location

User Happiness

Looking at cohorts by other filters

Event Timeline

Metrics

We welcome your feedback!

Page labels: Why they are important and how to use them

Why should I care about page labels?

How do page labels help me?

Compare and benchmark similar pages across different sites

URL paths can be too granular

Combine synthetic and RUM datasets

Present both datasets on a single chart

Correlate RUM data with synthetic diagnostics for Core Web Vitals

Prioritize performance opportunities by RUM impact

How do I set up page labels in SpeedCurve?

Synthetic page labels

RUM page labels

Option 1: Defining page labels using the RUM API

Option 2: Defining page labels using URL rules (NEW!)

Option 3: Use an existing JavaScript variable

Option 4: Default to document.title

Element Timing: One true metric to rule them all?

The (potential) problems with Largest Contentful Paint

The LCP element can vary for first-time vs repeat visitors

Even when it’s visible, a cookie banner may not always be the LCP element

The largest element can vary by viewport size

LCP uses the rendering timestamp to prioritize same-size elements

Other caveats...

How to understand which elements are chosen for LCP

Option 4: Default to `document.title`

2. Create a second snippet that uses a MutationObserver to detect when the `async-hide` class is removed from the document