Skip to content

A/B Testing JSONP and the Preloader with RUM

SpeedCurve is a SPA (Single Page App) so we construct the charts dynamically using JSONP. It works great, but we're always looking for ways to make the dashboards faster. One downside to making requests dynamically is that the browser preloader isn't used. This isn't a factor for later SPA requests, but on the first page view the preloader might still bring some benefits. Or maybe not. We weren't sure, so we ran an A/B test. Long story short, doing the first JSONP request via markup caused charts to render 300 milliseconds faster. 

A/B Test Results

It's easy to run this A/B test using our RUM product. The first step is to add a User Timing metric that marks when the JSONP response is received. We call this metric OnDataReceived. Some popular browsers don't support User Timing, so we use RUM's shim for mark() and measure() since it works in all browsers:

    function OnDataReceived(jsondata) {

The code above measures the time from NavigationStart to when the JSONP response is processed. All User Timing marks and measures are automatically captured by RUM and displayed in customers' dashboards.

The next step is to split users across two buckets. In the control bucket we continue to fetch the JSONP dynamically. In the test bucket we use markup to make the JSONP request. We use RUM's addData API to segment the data across these buckets. The control bucket has JsonpMarkup=no, while the test bucket has JsonpMarkup=yes. Here's the backend PHP snippet:

    if ( rand(0,1) ) {
        echo "<script>onGo();</script>\n"; // request JSONP dynamically
        echo "<script>LUX.addData('JsonpMarkup', 'no');</script>\n";
    else {
        echo "<script src='/site-data?jsonp=OnDataReceived'></script>\n";
        echo "<script>LUX.addData('JsonpMarkup', 'yes');</script>\n";

Within a few minutes the data shows that OnDataReceived is ~300 milliseconds faster for the JsonpMarkup=yes bucket. We let the A/B test run a few days to confirm that using markup for the initial JSONP request is faster thanks to the browser preloader.

If you make requests dynamically using JavaScript, you might be missing a performance opportunity by letting the preloader do its work. This isn't just true for JSONP requests, it also applies to images. If you're curious about this optimization, run an A/B test to see if making initial requests using markup makes your site faster and your users happier. We'd be happy to set you up with a free trial to make it easy peasy (lemon squeezy).

Read Next

Customer Stories

See how customers are using SpeedCurve to make their sites faster.

Industry Benchmarks

See industry-leading sites ranked on how fast their pages load from a user's perspective.