To begin, a huge apology for the disruption in the schedule to Lazy Load.
As we mentioned in our email, COVID continues to have an impact on speakers and with our speakers located all over the world-and we get them equipment to record these fantastic presentations-we can attest that the supply chain impact of COVID is definitely not entirely resolved everywhere.
But we do have something special for you today.
And then week two of Lazy Load will be next Friday.
Tim Kadlec is a long-term Web Performance expert, indeed guru, author of high performance images and Web Performance Day book amongst other books, he's also the chair of the fantastic Performance.now conference, which will return to Amsterdam in October.
So for more performance goodness, in-person, do keep an eye on that fantastic event.
And as a bonus, we have access to some past presentations from Performance.now, right here on Conffab.
Scott Jehl will be, and indeed should be very well known to many of us here.
He's the author of Responsible Responsive Design for A Book Apart, amongst other books.
He's a frequent speaker on Web Performance, as well as the author of a great online course, Lightning Fast Web Performance, which we highly recommend.
Now, WebPageTest has come up a lot at this conference and all our different speakers, because it's so foundational as a Web Performance tool.
What began life at AOL, yes, AOL developed there by Patrick Meenan for their internal use was open-sourced in 2008.
And then for the next dozen years or so was run from Patrick's basement.
His literal basement.
In 2020 WebPageTest found a new home at Catchpoint and Tim and Scott, along with a bunch of other great folks there at Catchpoint are now responsible for its continued development.
So as a bonus today, Tim and Scott will share their many years of hard earned expertise and audit a real world site for its Web Performance issues, and explore some fixes that you can make quickly as well.
WebPageTest remains open source, but there's also a hosted professional version from Catchpoint that takes care of the hassle of hosting your own version.
And now here's Tim Kadlec and Scott Jehl.
Hey everyone.
My name is Tim Kadlec and I am Scott Jael and we are going to be we both work on WebPageTest and we are going to be talking about experimenting with performance at the edge.
So a little bit about performance today and how that's changed, I think since, you know, we first kind of, started looking at performance years ago when it first became a thing a little bit about, you know, some of the things that we're seeing and how we would potentially analyze a site.
And then how we'd take that through to figuring out whether an optimization is meaningful or not.
Sounds good.
Thanks for joining us.
So, yeah.
Again, just to reiterate on that agenda a little bit about what matters today we'll talk a little bit, some of the terms, make sure that we're all using kind of, you know, everybody's on the same page for that, how we look for those bottlenecks, minding the gap, which is kind of, I think an approach that both of us, as well as a few other folks that we know like to do for helping them zero in on where the most problematic areas are.
And then that final stages that experimentation that...
...yeah.
That'll get into some new tooling.
So let's dive in.
Performance today.
It's a little bit different.
It is, things are changing.
I think you know, there are some tried and true things that still apply that have been performance concerns for quite a while, right?
Like what, like download size maybe file optimization caching, right?
Whether whether you're making sure that your your pages are cached locally in the browser so that they don't need to request files again, when you revisit something for a second time what else?
Compression, right?
That was classic concern.
So what Gzip, making sure that we send files over the wire as small as possible.
Yeah.
All those things still matter today.
Like, I think there's been one of the things that has happened in the last decade or so is we've moved towards metrics that are very, very user centric or like, it used to be metrics like total load time, download time and stuff like that have a questionable relationship with actually that user experience.
Right.
And...
...yeah, that's a good point.
Yeah.
And we've moved towards metrics that prioritize that user experience a bit.
But these things, these sort of fundamentals and traditional concerns absolutely still apply there.
So I think to, to sort of get into a little more of what wasn't, so you know observational, I guess, with old metrics would be how long a page and all of its little invisible requests took to load, even though that wasn't necessarily perceivable, right, to the user.
And why are we measuring that in the first place when we could be measuring the things that are very visible causing very visible delays for the user experience, right?
Yeah.
And I think so the metrics have matured, so we've started to focus on those things that are a little bit more visible for that user perspective.
You know, the other thing is I think, yeah, sites have gotten a lot more complex.
Like we throw more stuff at the browser than ever.
The things that we have to be aware of from performance perspective has also gotten a lot more, intense.
Like, yeah.
Yeah.
Like different, different browsers and devices for one thing that we have to care about you know, I guess that was always sort of the case, but but definitely more so today than it was 20 years ago when we had maybe two or three browsers that we were really focused on and they tended to be just on the desktop.
Now we have mobile, tablet, watches desktop browsers...
...and each of those different engines, like in an attempt to deal with the complexity of pages today, they all take slightly different approaches to how to load things up on the screen, right?
Like they all have this concept of priorities around resources, which is a huge part of performance, nowadays.
It's understanding how, you know, how to prioritize resources appropriately.
You know, in Chrome has a very, Chromium and Edge, you know, I think based on Chromium, it has a very intense prioritization scheme.
Firefox and Safari have much more simplified approaches to priorities because their goals are a little bit different, but like that's one of those things that we kind of have to be aware of now, because it has a dramatic impact on those front end performance metrics.
Yeah, I agree.
And then, you know, for all of those browsers and device types, we have different connection speeds as well to care about you know, wired connections at home wifi in public spaces with all of its spottiness and you know, a variety of of cell network style connections which are more and more kind of the, the most common use case for users of our sites today, right?
Yeah, absolutely.
Yeah.
I mean, it's hard to find any company or a site nowadays that doesn't have the majority of the traffic coming from mobile or at least that it's not trending that way.
There are exceptions.
I mean, WebPageTest tends to get a primarily desktop thing so far.
But that's exactly, but those are trending in the opposite direction for the most part.
So yeah, and that's becoming an increasing focus there.
And then there are things that we haven't really tapped into yet.
Like I think we're just starting to scratch the surface on things, like what are the accessibility impacts when it comes to performance?
The Léonie Watson's been giving some great talks about how performance impacts how quickly the accessibility tree gets created.
And that's one of those things that's kinda, I think on the horizon.
Memory concerns I think are right around the corner.
You know, as we use all these heavy JavaScript architectures memory usage is climbing up and up.
So I think, you know, we're kind of on the cusp of a whole bunch of other new concerns coming away too.
Yeah.
I mean, the, the concerns that developers are dealing with today are quite a bit different just by the nature of the technologies that we're using to deliver content, which itself hasn't really been changing a whole great deal, but the ways we deliver it has certainly changed a whole lot.
And I think, you know, I, I would say in the, in the last few years a trend that we've been seeing a lot more of is noticing bottlenecks that are caused by the tooling that we've largely adopted as a community, which tends to You know, fetch content and, and, you know spend a lot of time rendering once it gets to the browser, instead of sending something that's ready to be viewed initially in the HTML, like we would traditionally, right?
Yeah.
Right.
As we rely more on that, that intermediate step that's been happening quite a bit where that client side JavaScript or whatever it has to execute sometimes to create the content sometimes to be able to figure out what resources to request you know, that's that other step in there where it's, we've puts more and more, I guess, emphasis or, it really requires the developer and the tools to carry a lot more of the, shoulder a lot more of the load for improving performance than otherwise.
Right.
Because we're kind of taking, when we do things like that, we're taking the browser and it's sort of ability to do that stuff for us kind of out of equation.
And so it puts a little bit more complexity on the tools and the developers to be able to fix that kind of stuff themselves.
Yeah, absolutely, so I thik we've seen some of those tools themselves adapt in their approach to, to address that problem recently.
Right.
We've seen you know, whereas maybe React itself might've been a popular cause of some of the heavy client side loading and, and delays, now we're seeing that community address that through getting React and similar technologies, rendering on the server before they get sent to the browsers, we get more HTML and meaningful content upfront.
So that's been a nice trend, I think.
It's been good to see.
I think it's, you know, the wheel of time goes round and round kind of a thing.
It feels like that's constantly a little bit of a circular evolution, but I think a lot of that literally has been prompted by a combination of things.
One is, you know, Google putting kind of a SEO carrot behind improving performance, right.
With the new Core Web Vitals.
That's been a massive driver of new interest in performance and it kind of makes frameworks and CMSs, and other platforms like performance as a competitive differentiator, it becomes all the more important for them.
Yeah.
Yeah.
That's a great point.
I mean, you mentioned Core Web Vitals and that's sort of a deep into the weeds.
I think of some of the, the, the metrics, the terms that we use when we're talking about performance that maybe we could start to get into a little bit.
Yeah, I think that makes sense.
So do we want to, we can just dive right into, I think let's go into a site like we've built a nice handy page with all sorts of issues and an amazing dog...
...look at this...
...yeah.
So this is a, this is an example site that we built called the metric times.
And it's a fake news website that is loaded with performance anti-patterns that are meant to cause a performance analysis tool to light up red with all of its various diagnostics and say, wow, you know this was built in such a way that is probably not ideal [laughs] for the user.
So it's, it's kind of an ideal demo site to use for talking about things that we could do better.
Maybe practices that are sort of anti-patterns from a performance perspective.
Yeah, it works really well as a nice little stress cast, a stress test for those tools and yeah, again, being able to surface a bunch of those things and it works great for demos like this.
So yeah, we ran a test on that and maybe we can kind of use this as an opportunity to start by showing how we would look at this site for performance, like some of those key metrics and some of the things that we'd be looking at in those metrics to help us figure out where we'd want to prioritize some work upfront.
Yeah.
Do you want to just show real quick how we got to this view?
So this is a WebPageTest, a result page, which you could get to, by going to webpagetest.org, a free tool, open source that Tim and I work on day to day.
And you can see on the homepage here, Tim's scrolling around and there's a field right on, right on the homepage where you can get started and run a test for free.
Tim's doing the simple configuration right now, which is a mobile configuration with Chrome browser, 4g connection from Virginia here in the United States of America.
There are a couple other presets there that we offer just for simple configurations in various locations around the world.
We've got one in Mumbai there on a slower connection.
We've got Edge browser.
So WebPageTest desolate let's you test different, different browsers.
We've got what Safari Edge, Chrome, of course, Brave, Firefox, basically any major browser that's out there should be covered pretty well.
Yeah.
So we've got this simple config, which is a good way to get up and running.
These are like, it shows off a few of the browsers.
We also were pretty deliberate about some of the profiles, like for example, like Firefox, it turns out one of the biggest countries that they have market share in is Germany.
So that's a nice test case to be running from there.
And that's the best way to kick off a test really fast.
We also have the advanced configuration with all sorts of different customizations where you can choose from any of the number of different locations we have around the world the different browsers available at each of those locations or different device profiles, but all sorts of other things that probably go a little bit beyond the scope of today's thing, but there's quite a bit you can be doing and pulling it from here.
Yeah.
But you know, just as a, at a high level, really a really neat way to be able to actually test how a site is going to load from you know, a combination of browser connection, speed, and device in real locations around the world.
So yeah.
So hitting start tests got you to there...
This is for that metric times page.
On a Moto G4 on a 4g network in this case out of Virginia.
So again, kind of using that the default profile.
Yeah.
Cool.
So what are we looking at for some metrics here in the summary?
Yeah, let's start with that.
We'll come back.
I think we'll show the opportunities to experiment stuff a little bit about how that can be a shortcut for a little bit of the analysis, but let's start by-cause I think we wanted to talk about some of those metrics in terms just start with an observed metric thing here.
Yeah.
So we've got, this is sort of a high level summary in this case of the median run.
I believe we ran three test runs.
And so what we do is we will choose the median based on speed index by default.
Although you can change that.
So if you wanted it to, let's say we want it to look at one of these, you know, Web Vitals kind of metrics as we knew we were zeroing in on that, we could toggle that out and then it's going to choose the media around based on that.
But we have the summary of these metrics here.
And so at a high level, like first byte, that's your time from when the request was first kicked off to when the server responds with the first chunk of things back.
So that's basically a indication of how efficient, how fast is your server and the server response time.
Yeah, so the HTML, right?
Usually the first first bit of HTML downloading exactly.
We have our start render time.
Yeah, yeah.
1.3 seconds.
So a bit high in this case, which we'll call out, I think a few times they give you a start render metric.
So the start render metric is, that's a metric that is available because we are recording video on the page load progress.
And so what we do is we look at the video to see how the page is loaded.
And the first time we see something change on the screen, something get painted on the screen.
That's when we mark our start render time.
Hmm.
And you can probably see that in that little film strip under the metrics there.
Yeah.
Right about three seconds or so that's when we first see something pop in...
...there it is, lines up.
Then moving to the sider there, you've got your first content full paint, which in theory should be about the same as start render.
This is okay by the way, a little bit of a gap.
We record the video.
We don't record at 60 frames per second by default because it slows everything down.
And so a little bit of a gap between these two is about right.
Is the gap due to the recording so that the...
...yes, exactly the metrics from the browser in that case, right?
Yeah.
First contentful paint comes directly from the browser.
And this is our, based on our filmstrip progression.
Sometimes you'll see a situation where one or the other is dramatically different and that's usually a hint that there's something wrong.
There's something going on in the page itself, that's causing the browser to think it's painting when it's actually not client's [they talk over each other].
Then we've got speed index, which is a, actually a WebPageTest originated metric that indicates like, you know, roughly the approximation-how fast is the most of the content get painted on onto the visible screen.
That's a good, yeah.
I liked that one as well.
It's a nice indication of that progress.
Then we've got our three Core Web Vitals here.
And two are in red to mark that they're not looking so good.
We have a Largest Contentful Paint LCP there.
And that's the time at which the, the largest piece of content in the viewport, in the initial viewport that you see when the page loads when that piece of content first appears right.
Or finishes loading.
Finishes.
Yeah.
When it's, it appears on the screen and it's finished loading, right?
So that's 7.4 seconds in this case.
And in the filmstrip, it's a little hard to tell on the mini view, but we do highlight it in red, but if you were to click on this film strip and go to our visual comparison page, then it becomes a little bit easier to see, like it's called out pretty prominently, that that's our LCP frame.
Yep.
Then you've got Cumulative Layout Shift that one's actually looking pretty good.
But that's, you know, how much does content change and shift around the page during that sort of loading process?
And then total blocking time, which is how long is the main thread of the browser blocked with long running tasks.
So typically this is JavaScript execution.
And while that, because the browser is, has that main thread, and that's what handles JavaScript execution, it's what handles user input.
It's what gets things painted out on the screen.
It's really important to keep that thread open, as open as possible because you want to be able to respond to that user interaction right away.
And so what this is telling us is that for over a second of this loading process the main thread had too much work.
So if a user tried to do something like click a button or something like that the browser would have been very slow to respond.
And then finally, we've got our total bytes.
How many bytes does the page load over the course of the time?
Kind of want to go...
...traditional?
Yeah.
Traditional metric, but still important.
You know, I, I think as, as someone who I think a lot of us are on data plans that eventually use up and start metering our speed after, you know, we download a certain amount of data each month This is this contributes to that.
So the size of sites are, you know, ballooning every year you know, with largely images, video and JavaScript a bit all contributing to that.
So yeah, it still matters.
Oh yeah.
And there was a, a couple of years ago.
The Chrome team actually published data that showed in Japan where apparently the phone contracts line up pretty well at the start of the month and the end of the month and they could see traffic differences.
So they could see is that the load times and stuff of pages over the course of the month got slower.
And as they did the pages, the number of pages that people would use, you know, and view went down dramatically.
And the reason for this was people started to hit those, you know, those network data limits and get throttled.
And as soon as that happened, they just stopped interacting with the web as much.
So definitely connects.
Interesting, all sorts of ways that performance contributes to different conditions for how people use the web.
Should we look at a waterfall?
Perhaps?
We can look at the wa...
before we do, though, I wanted to like, can we, let's talk a little bit about like, just from these metrics, we can run through the definition, but like that gap analysis that you and I like to do for this, you want to walk through a little bit on like what you look at for the gaps.
Yeah, I think so.
So I guess You know, at a high level we could start with the first gap, right?
So we have our first byte time.
That's when the HTML just starts to, start to download.
But the gap between that and the first point at which the browser can render something visually typically if there's a big gap there, which we're seeing a decent gap in this case you know, a second and a half or, you know, almost almost two seconds gap when you see a gap there, you start to think about things that are blocking that that browser's ability to start rendering anything visually.
So.
I am going to pull up the filmstrip view, cause I think that's kind of a nice way for visualizing those gaps that you're talking about.
Okay.
Yeah.
So things that block you know, if you could envision that the browser has received HTML, that it, it could decide to just show it to us.
Right.
But it may decide to wait, depending on whether or not there are additional things that it wants to fetch before presenting it visually to us.
Right.
And not just visually, but also audibly if you're browsing, you know with assistive technology, like a screen reader, for example.
So the browser decides to wait when when resources are referenced in such a way that we sort of tell it that it should wait.
Right.
So if we reference style sheets, you know, CSS files, or JavaScript files in their default sort of manner.
So a link to a style sheet or a script element referencing an external JavaScript file by default, the browser will then stop what it's doing from a rendering perspective and go out and make sure those are all downloaded and evaluated before proceeding to show us the page.
And that's good in some ways, right?
Like if it's a style sheet, we probably want a little of that blocking behavior, because we don't want to show the page visually before there's some design to apply to it.
Right.
And that comes from the style sheet.
In the case of those JavaScript files, ideally we don't want to wait to show the page while those are still being downloaded.
Ideally we could, we could show the page and then apply whatever behavior they bring a little later.
At least ideally, right?
Ideally.
Yes.
Yeah.
You don't want your content or that initial display.
It's just another point where it could break and it's another bottleneck, right?
With JavaScript.
It has to go out, it has to grab the JavaScript, download the JavaScript, execute the JavaScript all that stuff before it can put something on a screen and it's just an unnecessary bottleneck in most cases, if we can push that off and get our content out and design out without that, that's what we should aim for.
So, yeah, that first gap Scott: you're looking at it, right?
So if you look at the waterfall here, this is the, again, that filmstrip view, there's a little red line here that lines up with the filmstrip images and it also works on the waterfall.
So it kind of aligns with wherever we happen to be.
So it's a handy way to sort of zero in on these visual moments and what's been happening up until that point.
So our Time to First Byte occurs right around here.
And that's when we've got our HTML and then all the way over here, that's when we start to get something painted on the screen.
And if we look at that waterfall, we'll see exactly the kind of stuff that you were just talking about, Scott, like these requests here that occur before in this case, the green bar indicates our start render and first contentful paint times.
And yeah, you can see the requests out to be blocking-the CSS and the JavaScript.
And that those things have to come in before we can get anything on the screen.
Yeah.
So we mentioned this page is built in such a way that it's sort of a pile of anti-patterns and those CSS and JavaScript files are requested in just the right way to block rendering.
Exactly.
And we indicate that which is really common.
No, I would say the render blocking resources, you know, are one of the most common bottlenecks that we see out there.
Right.
It's one of those fundamentals.
That's been the case for years and it's probably always going to be one of the low hanging fruit things.
That is just, yeah, it's, it's always going to be a significant bottleneck when it's around.
Yeah.
And you know, when I tend to prioritize the most critical, I think performance problems, I think those are right up there because.
Especially if you're, if you're trying to request, say a JavaScript file that's on a different domain, and perhaps that domain, whatever it happens to be is having maybe some trouble responding quickly that day.
Maybe it's not responding at all, but actually hanging, that's sort of the worst case scenario and it does happen.
You know, we see CDNs go down from time to time and it makes the news because so many sites on the web break and when they break, this is kind of what's what's happening.
Right.
Those requests are just hanging.
So this is a big deal.
Yeah.
Yeah, it's huge.
And so this is that first I think gap I tend to look at, I think you tend to look at as well as that Time to First Byte to that First Paint.
And again, that's just wrapping that up.
That's usually a hint that we've got some render blocking resources that need to be taken care of.
That is almost always the bottleneck in that scenario.
The next gap is what, between that First Paint and that Largest Contentful Paint, right?
If we see a gap there, which in this case, we definitely did.
We had, what, a four second gap.
We want-ideally those metrics are happening at the same time or as close to the same time as possible.
And again, when we see a gap there, there's a hint, there's some, probably other stuff happening here.
So in this case were requesting an image, right?
There our Largest Contentful Paint element happens to be that big image of a dog in the center of the page.
And the way it's discovered seems a little problematic.
It's pretty late when that request starts in the waterfall.
So.
That that makes me think something could be optimized, right?
Maybe the maybe the way it's referenced in the HTML is not ideal for the browser to discover it sooner.
And in this case we know the secret, right?
It's uh, it's that image is referenced with a attribute that makes the browser use lazy loading, right?
So loading equals lazy is a great performance tool.
It's also a big anti-pattern depending on how you use it.
And if you, if you, in this case decide to apply it to an image that is in the viewport at the very First Paint of the website what happens with the lazy loading is the browser will wait to download any image with that attribute until it paints the layout and figures out if that image is within the viewport and needs to be shown to the user or not, and only then will it start to go download it.
And that's not the default behavior for an image, right?
Yeah.
So typically what's going to happen for the image, particularly, and this is a Chromium-based browser, and again, they've got the most complex level of priorities.
Typically, what will happen is there's an initial priority when the browser discovers the resource and then there's a later priority where it kind of adjusts it after it learns a little bit more.
The initial priority for images is going to be low because it's not a render blocking resource.
If it's an image and it's inside the body element, it's going to be that low priority initially until the browser determines whether or not it lives in the viewport.
And it doesn't know that until it's got all the CSS and it's figured that kind of thing out.
Cause it's, you know, that concept is a little out out of reach until that point.
But it'll try to cue that up right away.
And so the loading indicator lets us hint to the browser.
Like it lets us tell the browser that, "Hey, you know if we say 'loading eager', for example, you know, we think that this is something that should be fetched, you know, initially right away".
And if we do the loading lazy, which is the much more widely used one, that's saying, yeah, we don't think this is that important.
Like you can hang off on this one.
Don't worry about.
And so I think like this is where we're seeing situations like this, where we have some images get queued up a little bit earlier than this one here.
Because that loading priority that initial priority gets kind of bumped down from that loading attribute.
So it gets pushed off a bit, which is not ideal.
And then it takes a long time to load.
It looks like it's a nice, is this a nice, hefty, hefty image?
Yeah.
This is a good image image size.
There was that a Meg?
Megabyte.
Yeah.
So this is always, that gap is always interesting.
Cause if it's that gap usually tells us something like this, where it's either an, it's an inefficiency, if it's like an image, whether it's the size of the image, whether it's the loading attribute, whether it's, you know, JavaScript initiated.
But that's the hint that we've got another bottleneck sort of occurring there.
And that's the other big gap that it's nice to look at it a little bit.
Yeah.
So that's that gap.
And then yeah, I mean that covers the visual, visual gaps mostly.
There's also another one to, to pay attention to, and that's the CLS metric.
And we can see that in this in this filmstrip as well with that same image, right?
We've got a layout that doesn't really have much room for it right beforehand.
Let's switch to the vitals view for that one.
The vitals view is that page where it's specifically targeted at these metrics, you know, where it's going to call out, you know, when the, where the Largest Contentful Paint is and was triggering it and stuff.
But here also for layout shifts, we can actually see a little bit of before and after kind of happening for some of these.
So we have a little bit of a content shift that occurs for sure.
Partly from fonts actually is where we're getting hit by the most.
Yeah.
And that one depends on the metric times.
It depends on the browser size.
So I know for a fact that at a smaller window size, like a, you know, a mobile device, when this, when this layout is all one column the, that, that main image doesn't have any dimensions on it.
So it just sorta like has like a zero pixel height until the browser starts downloading it and realizes the height that it needs to make for the image.
And once that happens, it pushes the content down.
So that depending on the browser size will cause a big layout shift.
Nice.
Yeah, that's a, that aspect ratio thing on the images is one of those, you know you and I have talked about that.
I think that's one of those interesting optimizations now, where now, if we give a with an, a hide element attributes to our images, the browser can preserve the space for that image.
Something that it didn't use to do.
But it does now, every major browser does.
And it's nice because that's specifically, there's this layout shift metric where we can measure visual stability, which we haven't been able to measure before.
And so now I think that's where we start to see things like the aspect ratio and other improvements come out because that stuff's a little bit easier to catch now from a metric perspective and measure and see the impact of.
Yeah.
And that, that example I just gave is sort of a good reminder that it's worth testing your metrics, your performance across different different browsers, different browser device sizes.
Just to see how they're impacted because you might.
Even if it's a responsive site, that's one code base that fluidly works across them all, you might you might find bottlenecks at one viewport size that don't exist at another.
Yeah, absolutely.
There's a good reason, I think for testing in those different browsers and different geographies and different connection types, because that's really important because that's the, the real users are going to be coming from a spectrum of those devices and experiences.
And so testing across those to make sure you're accounting for, you know, what the experience is like.
And there's no mismatched experiences.
I think, you know, it's an important thing for...
THere's this classic performance conundrum where site owners and developers tend to test the conditions that they know their audience exists in already.
Right.
But one of the things that we like to really emphasize with a tool like WebPageTest, for example, is that you know, getting a sense of how your site performs in say regions of the world, that you don't have an audience yet might give you an idea of why you don't have an audience there yet.
Maybe you have you know, problems with accessing your site in a region of the world that blocks the, you know, the CDN provider that you use or something like that.
Or maybe it's just slower in that region of the world, because all of your files are hosted in one place.
So it's good to be able to test those things.
Absolutely.
Yeah, yeah.
That, that's, that's a pretty common, it's not as publicly documented as often as I'd like, just because companies have, sometimes they're a little secretive about it.
I know the YouTube feather story is a big one where they improve the video page, provide a lightweight version of it.
And suddenly they found that they were getting traffic from areas in the world that they had never gotten traffic before.
And because people could access it now with that smaller size.
Yeah.
And as a result, their, their overall performance looked slower.
Right.
Because suddenly people were able to access the site at all, who were never able to access it.
Yeah.
So another one of those things to keep an eye on because of the perf, if they would've just stopped at like, "Hey, our performance metrics, regressed" they would've missed that this was a great thing that happened for them.
For the countries maybe that would have produced you know, decreased developer productivity now having access to a, you know, YouTube from [unclear] holes and stuff like that.
But know so we kind of went through, I think everything we've talked about so far has been like, it's been the metrics.
We talked a little bit about the gaps, which I think is where we both like to start and showing how you can spot some of that in the waterfall, but maybe we can dive into, I think now the opportunities and experiments bit, which is the kind of the new feature here and kind of a way of hopefully making some of these things a little bit easier for folks to spot at a glance without having to necessarily read through the underlying waterfall if they don't want to.
Yeah.
Sounds good.
So there's this little summary now that that's at the top of the the test result page and it just sort of gives you at the start a little intro, a little summary of how how various diagnostics that we ran on the page ended up looking to us at a glance and they're broken into a few categories.
There's quickness, usability, and resilience.
And within those, we run a bunch of diagnostics and you can see those if you go through to the opportunities and experiments page in this case, Tim jumped down to the quickness category.
So as you scroll through this page, you can see them broken into those categories.
And basically at each diagnostic, we either we either run a test and, and decide that things look good.
And you get a check mark, or if we notice there's a opportunity to do a little better then we offer that like in the case that it Tim is highlighting here.
And from there, we either recommend some tips that you could apply to your own site to try to improve that particular bottleneck make it go away or some experiments, which is a very new thing that we just, just released where you can actually test an optimization on your live site without actually changing the live site itself, but in a, in a simulated environment to see if it, if it happens to fix the, the problem that you're dealing with.
Yeah.
So we talked a lot about, you know, those render blocking resources and stuff like that.
Right.
And so this is going to be you know, one of those experiments that then might make a lot of sense to try, right?
Because we saw a very big gap between Time to First Byte and that First Paint.
And so it might make a lot of sense for us to try and run the experiment where we block a bunch of those render blocking resources and kind of see what the impact was of doing that.
The goal being here, right, like some optimizations sometimes will be a massive impact.
Sometimes they won't impact your site at all because maybe because your site is just the way it's built or that there are other bottlenecks that are kind of get in the way.
And the idea here is to try and make it as easy as possible to iterate and find out if these are going to be meaningful things for your team, or if you're not on the development team, for your development team or engineering team to kind of tackle and prioritize.
Exactly.
And you can run these either in isolation or you could choose to run more than one at a time.
And you can see at the bottom that the number of experiments that you've selected throughout your scrolling through this page are applied at the bottom in that little filter.
So you can jump back up to them and, you know, uncheck one, if you'd like, or adjust the way you're applying one in the case where there's different, different choices, for example, maybe you don't want to apply them to every script that's blocking.
Maybe all but one that sort of thing.
So that's up to you.
In this case, we know they're all blocking rendering, so we might want to test the impact of of deferring all of them, which is essentially just a way to cause them to load on the side, right.
While the page can show visibly while they're still loading.
Yeah.
Yeah.
And then there's, you can see the other big gap that we talked about, right?
What is that gap between First Contentful Paint and Largest Contentful Paint, which was triggered in large part because of the loading lazy.
So, you know, this, again, calls that out to make it a little bit more obvious.
Like we knew that cause you know, we built a site that intentionally did bad things.
But in this case where, you know, this calls out that, "Hey, something inside that viewport had that loading attribute on and maybe that's something worth, you know, removing".
Yeah.
So those, those are all related to quickness.
Should we kick that one off?
Just to...
yeah, I got that running here in the background.
So what that's going to do is we selected both the defer-renderblocking with all those resources and the remove loading lazy.
And it's going to run two runs.
It's going to run a control run, which basically goes through our sandbox environment without any of the optimizations applied.
And then an experiment run, which applies the optimizations.
The reason for that control run is because we know that there's some impact, right.
By having it proxy through our sandbox environment.
Often it's a little faster by default.
So we want to make sure that you're comparing kind of apples to apples.
And we've got a fair comparison there.
Yeah.
Nice.
So I guess for, for some clarity around the history of running these kinds of experiments, I can say that.
You know, I think both of us have done a lot of performance auditing in the past.
Right.
And traditionally to run these kinds of experiments would mean what a number of things-either actually changing the, the live site in some way that hopefully doesn't, it doesn't show up for every user while you're testing a particular condition and how it impacts performance.
So maybe you do that through like a little query string or something and change your code on the fly.
Pretty dangerous to do on the live site.
And then kind of a more recent intermediate and better way to test experiments like this was to do it manually still, but to use these things called service edge workers, right.
Where basically companies like CloudFlare, who first made it very popular, I think and now Akamai and Netlify many companies Fastly.
Yeah, yeah, pretty much everybody's.
I mean, Lambda@Edge there's, there's some concept of edge workers on just about every major platform now.
Yeah.
Yeah.
And the way you can use them is to write a little transformer script that lets your site pass through this, this script somewhere on their server and make a little tweak along the way.
And it can do that without without changing the site itself.
So it sort of mimics how your site would perform if it was passed through one of those or if the, if the change was actually made to the site without making the change first.
So that's that's pretty good.
But you know, doing that manually was a great deal of work.
Right.
Definitely.
I mean, it requires, like, there's a combination of things that have to go into it.
Right.
You know, like you have to have the account with one of those platforms to begin with.
You have to be comfortable enough in whatever, you know, most of those are JavaScript based platforms, but there's a few exceptions, but you have to be comfortable enough in that environment to be able to actually write the code, to be able to do the proxy, apply the optimization.
And then you also have to find the optimization in the first place, like, know what you want to run that experiment on.
And so there's a lot of, like, it just requires a lot of knowledge.
Like you have to have the performance knowledge to identify the issues.
You have to have the coding knowledge to be able to apply the experiments.
And so it's really what is that?
And the control as well control.
Right.
And the controlled around.
Right.
And doing that in a way that makes sense.
Yeah.
So it's, it's a lot of requirements to pull it off.
But it's mimicking a lot of, a lot of manual work here in a pretty easy, so we can, so the test actually we're back to this now.
This is what it brings you to.
So once the test is completed in this case, this was our test again, to defer the blocking scripts, render blocking scripts, and remove loading lazy from the viewport images.
Yeah.
And right away in the, in the video.
Huge.
It's a massive impact here in the difference there.
And we see a substantial improvement in that Largest Contentful Paint, in First Contentful Paint, in speed index.
You know, pretty much every metric gets a little bit better here.
Yeah.
That's, that's a good reminder that so many of these metrics are additive in that way.
Right.
You speed up an earlier one and the later one will, will happen sooner as well.
Yeah, so it's nice.
Sometimes you won't find this.
Like I think, you know, sometimes you'll see that the impact was negative.
It went the wrong way or very, very little impact or one metric improved and another got worse.
But those are good things to find out here where it takes you a minute to try it then, you know, in your own environment where it takes you a couple of weeks to figure that out.
Right.
Right.
So, you know, I guess it's all well and good to just run, run this tool on a fake news site, but maybe we could show some real sites.
Let's, we've got a couple that we the two of us have, you know, found particularly interesting in the last couple of days.
So this is Fest foods, a grocery store chain.
Idon't know if, If this is, is this a nationwide thing?
Do they have that in New York?
I don't know of it.
They have them around here.
They're big around here.
So I actually saw on a presentation from Paul Calvano that they had a couple of issues.
So I thought it would be interesting to run a test.
Now, this is again on a 4G mobile device.
Where the big gap that calls out, like start render to Largest Contentful Paint, isn't that bad.
Like a one second gap is reasonable, right?
But this gap, Time to First Byte to start render is massive.
Interestingly, on this particular site, we just talked about all those render blocking resources, being a thing, right?
It's not actually what's happening here.
There's only a few resources that are blocking page rendering.
That's not typical.
It's not typical.
What's actually happening in this case is another issue altogether.
Which is these this-381 resources that are being preloaded, but not used during the page load.
So when we preload a resource, that's a resource hint that we tell to the browser.
And what this says is, "Hey it helps with discovery.
Like this is a resource that is either low in the page, or maybe it's dynamically injected, but whatever we think it's like, it's important.
The browser should request this as early as possible and get this down".
And so in this case, it's all these CSS and JavaScript files, but this looks like it's like a build tool fail or something like that because none of these resources are actually then used by the page.
But the browser doesn't know that.
So what happens is the browser sees the preload says, "okay, well, you know what you're talking about, I should grab all of these resources".
And it cues up the downloads right away and you can see starting request six all the way down.
These are all our preloaded resources.
And until that stuff's out of the way we cannot actually start to grab any of the actual resources the page really does need somewhere around here, we start doing that stuff.
Yeah.
So there, the preloads are not really-they would not necessarily block rendering on their own, but they're at a higher priority than the ones that will block rendering.
Right?
Yeah.
We've got a higher priority kicking in here.
They're all contending for bandwidth.
There's probably, I mean, there's probably a couple, it looks like there's what 403 resources here, that are CSS JavaScript.
There's probably a few legitimate ones that are being preloaded as well, but they have to compete with everything else for bandwidth before they can start to do anything.
And if we go down here, actually we see actually it's the bandwidth isn't too bad, but that CPU is taxed all the way through this period of time where it's trying to get all these resources, execute a few of them that are actually on the page.
And so, yeah, it basically ends up being this massive render blocking thing that's occurring.
Cool.
How would we even run an experiment to try to fix that?
Well, I'll tell you what didn't work.
I tried at first to Just like, send something to re pull out all of these, but talk about stress testing.
It turns out that trying to inject a tiny snippet to tell 381 resources not to pull out gets a little heavy and doesn't really have the, a, it doesn't work the way we would expect.
But that's where we have these custom experiments that came in handy.
For sure.
We haven't talked about these at all, but like these above here are more like, very specific to the opportunity, you know, kind of a guided performance consultant sort of view, a little bit to create experiments is more of a sandbox, like play and have fun with it, right?
Yeah.
I think it's reactive versus versus proactive, I guess, in, in some ways all this creative experiment section is it's very much a, sort of the power tools section, right.
Where you can if you know what you're doing with this particular page, you know what you're looking for, you can make modifications yourself.
Right.
So what I did for this one is I used our find and replace text and I ran using regular expressions.
Let's see if it's, I am not going to pretend that I can remember what that is.
I have the experiment results, spoiler alert here where I ran it, but this is my regex.
So basically I found anything that was a preload element and then just replaced it with an empty space for now and ran the test.
And so what that did, is it removed, it removed all the unused.
It probably removed, like I said, a few that were actual valid preloads as well But it was a good enough test to give me an indication of how impactful this would be.
And it was very impactful as it turned out.
Wow.
So it moved that First Contentful Paint back you know, almost six seconds, which is almost that entire gap, right.
That we were looking at.
I think it was, was a huge chunk of that.
Yeah.
So when you see maybe there's some additional optimizations that could be made, it looks like in the process of removing all of those, you got a few a few or four specifically render blocking requests that came in.
Maybe they, they, you know, were able to request sooner as a result of getting the loads out of the way.
Yeah.
And now it's like, if you go down to the waterfall comparison for these two, the difference is dramatic, like here's the control run where we just applied the proxy, but didn't do any optimizations right.
With all those requests.
Yeah.
And then there's the experiment, like, look how much faster we're getting to these key resources, getting that out.
And the chain is much, much shorter-118.
We aren't even out of the pre-loads yet, by that point on the other.
And yeah this is where you could start to know, you know, all the render blocking stuff would become an experiment that'd be interesting to run, right.
Like try to, you know defer the render blocking JavaScript we see here or load the CSS asynchronously.
There's a few things that now might make sense that without those preloads out of the way, probably wouldn't move the needle at all.
Yeah, you're right.
I mean, I could see like some, some Google fonts, CSS in there that could be asynchronous.
It doesn't need to load in a blocking way.
Yeah.
So you can just keep iterating from there.
Go back to the opportunities page and yeah.
And keep adding a few more and sort of build this up for sure.
Yeah.
Yeah.
Another, another example that we tested this morning.
Yeah.
This one was, you were playing with Nintendo and [talk over each other] oh, I'm sorry.
I recently recently relaunched on some new tech.
And we can see that if you look at the timeline, there's a pretty dramatic wait and not just not just white screen wait, but also when something starts to render it's not really useful, right?
So this is where metrics start to not be necessarily the only source of truth.
So even though we got to start render right around, what was it?
Six seconds.
All it was was three seconds even.
Yeah.
So things looked good from a early metric standpoint, but from a user perspective, all that the user was getting was gray boxes, like a loading screen, essentially.
And it took up until 30 seconds and this was on 4g connection in Chrome.
So pretty standard mobile connection took about 30 seconds for the page to show content.
So this was pretty interesting.
You know, we see a gap like that between First Contentful Paint and I guess, you know, our just Largest Contentful Paint I guess, would be What's showing up around 30 seconds.
You know, it could be a number of things.
It could be that the, the image that's, the largest content was taking a while to load.
Maybe it could be requested sooner, but what's happening in this page is a little different we found, right?
It's not just that the Largest Contentful Paint image is slow to load, but really all the content is not there.
Right.
And all the text on the page.
So that starts to tell us, well, maybe, maybe that text isn't in the HTML to begin with, right.
Maybe it's being requested after the back.
And that's why they're showing a, a loading screen.
And that's a pretty common pattern these days, right?
The React sites sort of introduced this popularly early on, where you get a loading screen and then React goes off and fetches and fetches and fetches and fetches.
Lots of JSON, takes it back.
You know, and converts it into HTML that it can then inject into the page and we're seeing a pretty dramatic wait for, for that scenario.
Yeah.
And this seems to be our suspect here, exactly what you were talking about on the waterfall, these yellow bars, that's JavaScript execution.
That's blocked, like triggered by that resource.
So there's a big chunk here, but the other thing is like, eh, if we go to about the time we get our LCP, right?
Like somewhere there and then just kind of go down and that's the end of our waterfall, basically, you can see all these requests that got queued up, and then there's a massive chunk of JavaScript execution here as well before we're able to do anything else either.
So yeah.
It adds more credence to your, yeah.
It seems like the contents coming back dynamically or the JavaScript, it's waiting for something from JavaScript to be able to paint that stuff onto the screen.
Yeah.
And I like to, I like to make an assumption when I see the LCP metric kind of line up with like the end of the waterfall when all the content arrives.
That's a sure sign, usually, that JavaScript is just sort of doing a voila moment, right at the end where everything gets, you know, delivered all at once.
And you get your metrics all crunched in there at the end where suddenly the site's ready to use, but it took a long time to get there For sure.
So then where did you go from there then?
I guess now you had your theory.
Yeah.
So you know, my theory was that JavaScript was, was, you know, blocking everything.
So we went into opportunities to confirm that and what I found under the usable category, or maybe it was right at the end of quickness, it was right at the end of quickness, actually that there's a nice little comparison that it offers.
Where it says a significant portion of HTML content in this case 425 kilobytes or 50% of the total page weight.
And that's a simple, just simple comparison of what was delivered upfront versus what was there at the end.
It's saying that much was, was generated after the HTML was delivered.
By JavaScript so that's a big red flag.
And we have a little experiment to offer where we can, you know, if you were to address this problem.
Well, the first thing you would think of is, well, maybe I should be serving more content in my HTML from the server to begin with.
But the process of saying something like that and actually executing on it is like months of labor.
Right.
And no experiment is going to fix that for you either.
But it can be useful to know if it's worth doing that, that work at all.
Right?
Like what if we could find out how would our page load differently if we did serve most of that end state of the, the content upfront, would it be even worth the effort?
Well, that's what this experiment tells us.
So it actually takes the the end state of that HTML page when it, when it's all complete and it stores it on the server where we can retrieve it as we're running rerunning, this experiment test, and it just serves it from the start as if Nintendo just sent it to the browser upfront.
And so we ran that experiment and you don't need to wait on it.
We have a result page here, but you know, this one was like one of the biggest, one of the biggest impacts I've seen lately where, you know, 60 seconds Largest Contentful Paint difference.
Now I should preface that in saying that it's comparing this particular control run versus the experiment.
Cause we saw in the last round, it was what, 30 seconds 35 that the LCP happened.
Right.
And that's why it's worth running a lot of tests.
Like, you know, if you're going to run an experiment, I only ran one run here and it just so happened that both of them were a little slower than that last test run.
So we see a big difference.
If I had upped the number of runs of the experiment to say, you know, six or, or even more the more runs you do, the more you can count on getting a good median run that counts out the outlier runs that are a little slower than the typical user might experience.
Right.
So this one's probably a little slower, but we see a big difference.
You know, maybe bigger than it would be realistically, but still this suggests maybe this would be worth the effort.
Right.
So, yeah.
And there's some, you know, and I think one of the things we know it on this experiment, right, is that it is going to very likely break something, right?
Because of the way this is all working, injecting that HTML.
And in this case, you know, you can see that the final state here for the final state for our getting a little click happy for our experiment run does look different, right?
Because it doesn't have some of the same kind of styling and stuff applied.
And that's fine.
Cause I think the goal of this experiment is to show the potential of like if we weren't relying on JavaScript and then we could serve something usable from the server, how much faster could we potentially get stuff on the screen?
And so even if it's not a perfect next side-by-side visual comparison metric wise, this is telling us that, yeah, there's a substantial opportunity for improvement if we can figure this out.
Yeah.
And I actually just noticed that the result you're looking at was one where I ran two experiments.
And let me share you one share you share one that had a, a better final state.
So one second.
Oh, I see.
Yeah.
So you did run in this case, disabled scripts, which is also like one of those experiments that you may want to sometimes pair with the mimic pre-render depending on the site.
Because sometimes that can actually reduce the amount of breakage in this case, though.
I think it worked better.
Yeah, sorry, I had the wrong URL there, but it's a good example of you know, that, that prior example had a couple of experiments run and it sort of messed with the the way the page rendered in the end.
Whereas this one was just an isolated experiment pre-rendered HTML and it renders, you know, roughly how it should you know, with, with little style differences here and there, but it gives you a good idea of like, especially on the timeline below how soon content would be ready.
You know, I think if you were to actually implement this change yourself, you would make sure that there's enough CSS in the page to style that initial HTML a lot sooner, right.
In this example and why I ran that second round.
You know, you would notice this and then you'd say, okay, how could I get the page styles to show sooner and defer the JavaScript?
So you start, you know, running experiments like that.
And what I learned in this one was that their, their CSS is very deeply embedded in the scripts that they're loading.
So there was no unbundling that I could do very easily.
So, you know, in reality it would be, it would be a, quite a bit of work to, to pull this off, but it shows wow.
Their users could, you know, potentially get a site on a fast cell and cell phone connection, maybe 20 seconds earlier or something like that.
That's huge.
I think like that's, you know, I think that's one of the primary things that we talked a little bit at the beginning about the amount of complexity that goes into getting stuff on the screen today.
And the complexity involved in fixing that is non trivial.
And, you know, that's hopefully where these new features, the opportunities and experiments come in to kind of help out with that a little bit and to kind of bridge that gap and eliminate some of the complexity that can come with.
And some of the whack-a-mole feeling that can come from optimizing performance sometimes.
Yeah.
And I guess we should mention as a bit of a disclosure that many of the experiments are part of the pro feature plan.
So that's a paid tier of WebPageTest that just recently launched.
But one of the most popular experiments is totally free if you have a WebPageTest account.
And that's the deferred JavaScript defer blocking JavaScript experiment.
So, you know, if you want to test this out on any old site, just get a free account.
And you can see how it works on your own work.
Yeah.
And if you do, please give us a shout out and let us know how things go.
I think one of the funnest things for me about the performance community historically is how much we all learn from each other and what we're all working on.
And it's, you know, we definitely want to continue to do that and we can refine these things and everybody can learn from failed or successful experiments.
Exactly.
Yeah.
Yeah.
That's a good point.
We didn't really show any any good examples of a experiment that didn't really contribute to a good improvement, but those are really useful to know as well.
Because you won't spend your time fixing something that isn't going to make an impact.
So Exactly.
Yeah.
Thanks everybody for tuning in hopefully everybody got a little bit out of that.
Again, I'm Scott or I'm I'm Scott I'm Tim and I'm Tim as well.
And thanks for joining us Thanks again Thanks again to Tim and Scott and keep an eye on your email inbox for a special offer from Catchpoint, if you're keen to explore how you can incorporate WebPageTest test in your performance monitoring workflow.
Thanks again for your understanding as we have this disrupted programme for Lazy Load, but we will see you again next Friday for week two of Lazy Load, where we have a whole raft of amazing presentations for you.
While you wait, why not visit the folks at WebPageTest over in their lounge for a chance to win a free access to WebPageTest Pro (includes zero-code experiments, bulk testing, API, etc) OR a free T-Shirt!
To begin, a huge apology for the disruption in the schedule to Lazy Load.
As we mentioned in our email, COVID continues to have an impact on speakers and with our speakers located all over the world-and we get them equipment to record these fantastic presentations-we can attest that the supply chain impact of COVID is definitely not entirely resolved everywhere.
But we do have something special for you today.
And then week two of Lazy Load will be next Friday.
Tim Kadlec is a long-term Web Performance expert, indeed guru, author of high performance images and Web Performance Day book amongst other books, he's also the chair of the fantastic Performance.now conference, which will return to Amsterdam in October.
So for more performance goodness, in-person, do keep an eye on that fantastic event.
And as a bonus, we have access to some past presentations from Performance.now, right here on Conffab.
Scott Jehl will be, and indeed should be very well known to many of us here.
He's the author of Responsible Responsive Design for A Book Apart, amongst other books.
He's a frequent speaker on Web Performance, as well as the author of a great online course, Lightning Fast Web Performance, which we highly recommend.
Now, WebPageTest has come up a lot at this conference and all our different speakers, because it's so foundational as a Web Performance tool.
What began life at AOL, yes, AOL developed there by Patrick Meenan for their internal use was open-sourced in 2008.
And then for the next dozen years or so was run from Patrick's basement.
His literal basement.
In 2020 WebPageTest found a new home at Catchpoint and Tim and Scott, along with a bunch of other great folks there at Catchpoint are now responsible for its continued development.
So as a bonus today, Tim and Scott will share their many years of hard earned expertise and audit a real world site for its Web Performance issues, and explore some fixes that you can make quickly as well.
WebPageTest remains open source, but there's also a hosted professional version from Catchpoint that takes care of the hassle of hosting your own version.
And now here's Tim Kadlec and Scott Jehl.
Hey everyone.
My name is Tim Kadlec and I am Scott Jael and we are going to be we both work on WebPageTest and we are going to be talking about experimenting with performance at the edge.
So a little bit about performance today and how that's changed, I think since, you know, we first kind of, started looking at performance years ago when it first became a thing a little bit about, you know, some of the things that we're seeing and how we would potentially analyze a site.
And then how we'd take that through to figuring out whether an optimization is meaningful or not.
Sounds good.
Thanks for joining us.
So, yeah.
Again, just to reiterate on that agenda a little bit about what matters today we'll talk a little bit, some of the terms, make sure that we're all using kind of, you know, everybody's on the same page for that, how we look for those bottlenecks, minding the gap, which is kind of, I think an approach that both of us, as well as a few other folks that we know like to do for helping them zero in on where the most problematic areas are.
And then that final stages that experimentation that...
...yeah.
That'll get into some new tooling.
So let's dive in.
Performance today.
It's a little bit different.
It is, things are changing.
I think you know, there are some tried and true things that still apply that have been performance concerns for quite a while, right?
Like what, like download size maybe file optimization caching, right?
Whether whether you're making sure that your your pages are cached locally in the browser so that they don't need to request files again, when you revisit something for a second time what else?
Compression, right?
That was classic concern.
So what Gzip, making sure that we send files over the wire as small as possible.
Yeah.
All those things still matter today.
Like, I think there's been one of the things that has happened in the last decade or so is we've moved towards metrics that are very, very user centric or like, it used to be metrics like total load time, download time and stuff like that have a questionable relationship with actually that user experience.
Right.
And...
...yeah, that's a good point.
Yeah.
And we've moved towards metrics that prioritize that user experience a bit.
But these things, these sort of fundamentals and traditional concerns absolutely still apply there.
So I think to, to sort of get into a little more of what wasn't, so you know observational, I guess, with old metrics would be how long a page and all of its little invisible requests took to load, even though that wasn't necessarily perceivable, right, to the user.
And why are we measuring that in the first place when we could be measuring the things that are very visible causing very visible delays for the user experience, right?
Yeah.
And I think so the metrics have matured, so we've started to focus on those things that are a little bit more visible for that user perspective.
You know, the other thing is I think, yeah, sites have gotten a lot more complex.
Like we throw more stuff at the browser than ever.
The things that we have to be aware of from performance perspective has also gotten a lot more, intense.
Like, yeah.
Yeah.
Like different, different browsers and devices for one thing that we have to care about you know, I guess that was always sort of the case, but but definitely more so today than it was 20 years ago when we had maybe two or three browsers that we were really focused on and they tended to be just on the desktop.
Now we have mobile, tablet, watches desktop browsers...
...and each of those different engines, like in an attempt to deal with the complexity of pages today, they all take slightly different approaches to how to load things up on the screen, right?
Like they all have this concept of priorities around resources, which is a huge part of performance, nowadays.
It's understanding how, you know, how to prioritize resources appropriately.
You know, in Chrome has a very, Chromium and Edge, you know, I think based on Chromium, it has a very intense prioritization scheme.
Firefox and Safari have much more simplified approaches to priorities because their goals are a little bit different, but like that's one of those things that we kind of have to be aware of now, because it has a dramatic impact on those front end performance metrics.
Yeah, I agree.
And then, you know, for all of those browsers and device types, we have different connection speeds as well to care about you know, wired connections at home wifi in public spaces with all of its spottiness and you know, a variety of of cell network style connections which are more and more kind of the, the most common use case for users of our sites today, right?
Yeah, absolutely.
Yeah.
I mean, it's hard to find any company or a site nowadays that doesn't have the majority of the traffic coming from mobile or at least that it's not trending that way.
There are exceptions.
I mean, WebPageTest tends to get a primarily desktop thing so far.
But that's exactly, but those are trending in the opposite direction for the most part.
So yeah, and that's becoming an increasing focus there.
And then there are things that we haven't really tapped into yet.
Like I think we're just starting to scratch the surface on things, like what are the accessibility impacts when it comes to performance?
The Léonie Watson's been giving some great talks about how performance impacts how quickly the accessibility tree gets created.
And that's one of those things that's kinda, I think on the horizon.
Memory concerns I think are right around the corner.
You know, as we use all these heavy JavaScript architectures memory usage is climbing up and up.
So I think, you know, we're kind of on the cusp of a whole bunch of other new concerns coming away too.
Yeah.
I mean, the, the concerns that developers are dealing with today are quite a bit different just by the nature of the technologies that we're using to deliver content, which itself hasn't really been changing a whole great deal, but the ways we deliver it has certainly changed a whole lot.
And I think, you know, I, I would say in the, in the last few years a trend that we've been seeing a lot more of is noticing bottlenecks that are caused by the tooling that we've largely adopted as a community, which tends to You know, fetch content and, and, you know spend a lot of time rendering once it gets to the browser, instead of sending something that's ready to be viewed initially in the HTML, like we would traditionally, right?
Yeah.
Right.
As we rely more on that, that intermediate step that's been happening quite a bit where that client side JavaScript or whatever it has to execute sometimes to create the content sometimes to be able to figure out what resources to request you know, that's that other step in there where it's, we've puts more and more, I guess, emphasis or, it really requires the developer and the tools to carry a lot more of the, shoulder a lot more of the load for improving performance than otherwise.
Right.
Because we're kind of taking, when we do things like that, we're taking the browser and it's sort of ability to do that stuff for us kind of out of equation.
And so it puts a little bit more complexity on the tools and the developers to be able to fix that kind of stuff themselves.
Yeah, absolutely, so I thik we've seen some of those tools themselves adapt in their approach to, to address that problem recently.
Right.
We've seen you know, whereas maybe React itself might've been a popular cause of some of the heavy client side loading and, and delays, now we're seeing that community address that through getting React and similar technologies, rendering on the server before they get sent to the browsers, we get more HTML and meaningful content upfront.
So that's been a nice trend, I think.
It's been good to see.
I think it's, you know, the wheel of time goes round and round kind of a thing.
It feels like that's constantly a little bit of a circular evolution, but I think a lot of that literally has been prompted by a combination of things.
One is, you know, Google putting kind of a SEO carrot behind improving performance, right.
With the new Core Web Vitals.
That's been a massive driver of new interest in performance and it kind of makes frameworks and CMSs, and other platforms like performance as a competitive differentiator, it becomes all the more important for them.
Yeah.
Yeah.
That's a great point.
I mean, you mentioned Core Web Vitals and that's sort of a deep into the weeds.
I think of some of the, the, the metrics, the terms that we use when we're talking about performance that maybe we could start to get into a little bit.
Yeah, I think that makes sense.
So do we want to, we can just dive right into, I think let's go into a site like we've built a nice handy page with all sorts of issues and an amazing dog...
...look at this...
...yeah.
So this is a, this is an example site that we built called the metric times.
And it's a fake news website that is loaded with performance anti-patterns that are meant to cause a performance analysis tool to light up red with all of its various diagnostics and say, wow, you know this was built in such a way that is probably not ideal [laughs] for the user.
So it's, it's kind of an ideal demo site to use for talking about things that we could do better.
Maybe practices that are sort of anti-patterns from a performance perspective.
Yeah, it works really well as a nice little stress cast, a stress test for those tools and yeah, again, being able to surface a bunch of those things and it works great for demos like this.
So yeah, we ran a test on that and maybe we can kind of use this as an opportunity to start by showing how we would look at this site for performance, like some of those key metrics and some of the things that we'd be looking at in those metrics to help us figure out where we'd want to prioritize some work upfront.
Yeah.
Do you want to just show real quick how we got to this view?
So this is a WebPageTest, a result page, which you could get to, by going to webpagetest.org, a free tool, open source that Tim and I work on day to day.
And you can see on the homepage here, Tim's scrolling around and there's a field right on, right on the homepage where you can get started and run a test for free.
Tim's doing the simple configuration right now, which is a mobile configuration with Chrome browser, 4g connection from Virginia here in the United States of America.
There are a couple other presets there that we offer just for simple configurations in various locations around the world.
We've got one in Mumbai there on a slower connection.
We've got Edge browser.
So WebPageTest desolate let's you test different, different browsers.
We've got what Safari Edge, Chrome, of course, Brave, Firefox, basically any major browser that's out there should be covered pretty well.
Yeah.
So we've got this simple config, which is a good way to get up and running.
These are like, it shows off a few of the browsers.
We also were pretty deliberate about some of the profiles, like for example, like Firefox, it turns out one of the biggest countries that they have market share in is Germany.
So that's a nice test case to be running from there.
And that's the best way to kick off a test really fast.
We also have the advanced configuration with all sorts of different customizations where you can choose from any of the number of different locations we have around the world the different browsers available at each of those locations or different device profiles, but all sorts of other things that probably go a little bit beyond the scope of today's thing, but there's quite a bit you can be doing and pulling it from here.
Yeah.
But you know, just as a, at a high level, really a really neat way to be able to actually test how a site is going to load from you know, a combination of browser connection, speed, and device in real locations around the world.
So yeah.
So hitting start tests got you to there...
This is for that metric times page.
On a Moto G4 on a 4g network in this case out of Virginia.
So again, kind of using that the default profile.
Yeah.
Cool.
So what are we looking at for some metrics here in the summary?
Yeah, let's start with that.
We'll come back.
I think we'll show the opportunities to experiment stuff a little bit about how that can be a shortcut for a little bit of the analysis, but let's start by-cause I think we wanted to talk about some of those metrics in terms just start with an observed metric thing here.
Yeah.
So we've got, this is sort of a high level summary in this case of the median run.
I believe we ran three test runs.
And so what we do is we will choose the median based on speed index by default.
Although you can change that.
So if you wanted it to, let's say we want it to look at one of these, you know, Web Vitals kind of metrics as we knew we were zeroing in on that, we could toggle that out and then it's going to choose the media around based on that.
But we have the summary of these metrics here.
And so at a high level, like first byte, that's your time from when the request was first kicked off to when the server responds with the first chunk of things back.
So that's basically a indication of how efficient, how fast is your server and the server response time.
Yeah, so the HTML, right?
Usually the first first bit of HTML downloading exactly.
We have our start render time.
Yeah, yeah.
1.3 seconds.
So a bit high in this case, which we'll call out, I think a few times they give you a start render metric.
So the start render metric is, that's a metric that is available because we are recording video on the page load progress.
And so what we do is we look at the video to see how the page is loaded.
And the first time we see something change on the screen, something get painted on the screen.
That's when we mark our start render time.
Hmm.
And you can probably see that in that little film strip under the metrics there.
Yeah.
Right about three seconds or so that's when we first see something pop in...
...there it is, lines up.
Then moving to the sider there, you've got your first content full paint, which in theory should be about the same as start render.
This is okay by the way, a little bit of a gap.
We record the video.
We don't record at 60 frames per second by default because it slows everything down.
And so a little bit of a gap between these two is about right.
Is the gap due to the recording so that the...
...yes, exactly the metrics from the browser in that case, right?
Yeah.
First contentful paint comes directly from the browser.
And this is our, based on our filmstrip progression.
Sometimes you'll see a situation where one or the other is dramatically different and that's usually a hint that there's something wrong.
There's something going on in the page itself, that's causing the browser to think it's painting when it's actually not client's [they talk over each other].
Then we've got speed index, which is a, actually a WebPageTest originated metric that indicates like, you know, roughly the approximation-how fast is the most of the content get painted on onto the visible screen.
That's a good, yeah.
I liked that one as well.
It's a nice indication of that progress.
Then we've got our three Core Web Vitals here.
And two are in red to mark that they're not looking so good.
We have a Largest Contentful Paint LCP there.
And that's the time at which the, the largest piece of content in the viewport, in the initial viewport that you see when the page loads when that piece of content first appears right.
Or finishes loading.
Finishes.
Yeah.
When it's, it appears on the screen and it's finished loading, right?
So that's 7.4 seconds in this case.
And in the filmstrip, it's a little hard to tell on the mini view, but we do highlight it in red, but if you were to click on this film strip and go to our visual comparison page, then it becomes a little bit easier to see, like it's called out pretty prominently, that that's our LCP frame.
Yep.
Then you've got Cumulative Layout Shift that one's actually looking pretty good.
But that's, you know, how much does content change and shift around the page during that sort of loading process?
And then total blocking time, which is how long is the main thread of the browser blocked with long running tasks.
So typically this is JavaScript execution.
And while that, because the browser is, has that main thread, and that's what handles JavaScript execution, it's what handles user input.
It's what gets things painted out on the screen.
It's really important to keep that thread open, as open as possible because you want to be able to respond to that user interaction right away.
And so what this is telling us is that for over a second of this loading process the main thread had too much work.
So if a user tried to do something like click a button or something like that the browser would have been very slow to respond.
And then finally, we've got our total bytes.
How many bytes does the page load over the course of the time?
Kind of want to go...
...traditional?
Yeah.
Traditional metric, but still important.
You know, I, I think as, as someone who I think a lot of us are on data plans that eventually use up and start metering our speed after, you know, we download a certain amount of data each month This is this contributes to that.
So the size of sites are, you know, ballooning every year you know, with largely images, video and JavaScript a bit all contributing to that.
So yeah, it still matters.
Oh yeah.
And there was a, a couple of years ago.
The Chrome team actually published data that showed in Japan where apparently the phone contracts line up pretty well at the start of the month and the end of the month and they could see traffic differences.
So they could see is that the load times and stuff of pages over the course of the month got slower.
And as they did the pages, the number of pages that people would use, you know, and view went down dramatically.
And the reason for this was people started to hit those, you know, those network data limits and get throttled.
And as soon as that happened, they just stopped interacting with the web as much.
So definitely connects.
Interesting, all sorts of ways that performance contributes to different conditions for how people use the web.
Should we look at a waterfall?
Perhaps?
We can look at the wa...
before we do, though, I wanted to like, can we, let's talk a little bit about like, just from these metrics, we can run through the definition, but like that gap analysis that you and I like to do for this, you want to walk through a little bit on like what you look at for the gaps.
Yeah, I think so.
So I guess You know, at a high level we could start with the first gap, right?
So we have our first byte time.
That's when the HTML just starts to, start to download.
But the gap between that and the first point at which the browser can render something visually typically if there's a big gap there, which we're seeing a decent gap in this case you know, a second and a half or, you know, almost almost two seconds gap when you see a gap there, you start to think about things that are blocking that that browser's ability to start rendering anything visually.
So.
I am going to pull up the filmstrip view, cause I think that's kind of a nice way for visualizing those gaps that you're talking about.
Okay.
Yeah.
So things that block you know, if you could envision that the browser has received HTML, that it, it could decide to just show it to us.
Right.
But it may decide to wait, depending on whether or not there are additional things that it wants to fetch before presenting it visually to us.
Right.
And not just visually, but also audibly if you're browsing, you know with assistive technology, like a screen reader, for example.
So the browser decides to wait when when resources are referenced in such a way that we sort of tell it that it should wait.
Right.
So if we reference style sheets, you know, CSS files, or JavaScript files in their default sort of manner.
So a link to a style sheet or a script element referencing an external JavaScript file by default, the browser will then stop what it's doing from a rendering perspective and go out and make sure those are all downloaded and evaluated before proceeding to show us the page.
And that's good in some ways, right?
Like if it's a style sheet, we probably want a little of that blocking behavior, because we don't want to show the page visually before there's some design to apply to it.
Right.
And that comes from the style sheet.
In the case of those JavaScript files, ideally we don't want to wait to show the page while those are still being downloaded.
Ideally we could, we could show the page and then apply whatever behavior they bring a little later.
At least ideally, right?
Ideally.
Yes.
Yeah.
You don't want your content or that initial display.
It's just another point where it could break and it's another bottleneck, right?
With JavaScript.
It has to go out, it has to grab the JavaScript, download the JavaScript, execute the JavaScript all that stuff before it can put something on a screen and it's just an unnecessary bottleneck in most cases, if we can push that off and get our content out and design out without that, that's what we should aim for.
So, yeah, that first gap Scott: you're looking at it, right?
So if you look at the waterfall here, this is the, again, that filmstrip view, there's a little red line here that lines up with the filmstrip images and it also works on the waterfall.
So it kind of aligns with wherever we happen to be.
So it's a handy way to sort of zero in on these visual moments and what's been happening up until that point.
So our Time to First Byte occurs right around here.
And that's when we've got our HTML and then all the way over here, that's when we start to get something painted on the screen.
And if we look at that waterfall, we'll see exactly the kind of stuff that you were just talking about, Scott, like these requests here that occur before in this case, the green bar indicates our start render and first contentful paint times.
And yeah, you can see the requests out to be blocking-the CSS and the JavaScript.
And that those things have to come in before we can get anything on the screen.
Yeah.
So we mentioned this page is built in such a way that it's sort of a pile of anti-patterns and those CSS and JavaScript files are requested in just the right way to block rendering.
Exactly.
And we indicate that which is really common.
No, I would say the render blocking resources, you know, are one of the most common bottlenecks that we see out there.
Right.
It's one of those fundamentals.
That's been the case for years and it's probably always going to be one of the low hanging fruit things.
That is just, yeah, it's, it's always going to be a significant bottleneck when it's around.
Yeah.
And you know, when I tend to prioritize the most critical, I think performance problems, I think those are right up there because.
Especially if you're, if you're trying to request, say a JavaScript file that's on a different domain, and perhaps that domain, whatever it happens to be is having maybe some trouble responding quickly that day.
Maybe it's not responding at all, but actually hanging, that's sort of the worst case scenario and it does happen.
You know, we see CDNs go down from time to time and it makes the news because so many sites on the web break and when they break, this is kind of what's what's happening.
Right.
Those requests are just hanging.
So this is a big deal.
Yeah.
Yeah, it's huge.
And so this is that first I think gap I tend to look at, I think you tend to look at as well as that Time to First Byte to that First Paint.
And again, that's just wrapping that up.
That's usually a hint that we've got some render blocking resources that need to be taken care of.
That is almost always the bottleneck in that scenario.
The next gap is what, between that First Paint and that Largest Contentful Paint, right?
If we see a gap there, which in this case, we definitely did.
We had, what, a four second gap.
We want-ideally those metrics are happening at the same time or as close to the same time as possible.
And again, when we see a gap there, there's a hint, there's some, probably other stuff happening here.
So in this case were requesting an image, right?
There our Largest Contentful Paint element happens to be that big image of a dog in the center of the page.
And the way it's discovered seems a little problematic.
It's pretty late when that request starts in the waterfall.
So.
That that makes me think something could be optimized, right?
Maybe the maybe the way it's referenced in the HTML is not ideal for the browser to discover it sooner.
And in this case we know the secret, right?
It's uh, it's that image is referenced with a attribute that makes the browser use lazy loading, right?
So loading equals lazy is a great performance tool.
It's also a big anti-pattern depending on how you use it.
And if you, if you, in this case decide to apply it to an image that is in the viewport at the very First Paint of the website what happens with the lazy loading is the browser will wait to download any image with that attribute until it paints the layout and figures out if that image is within the viewport and needs to be shown to the user or not, and only then will it start to go download it.
And that's not the default behavior for an image, right?
Yeah.
So typically what's going to happen for the image, particularly, and this is a Chromium-based browser, and again, they've got the most complex level of priorities.
Typically, what will happen is there's an initial priority when the browser discovers the resource and then there's a later priority where it kind of adjusts it after it learns a little bit more.
The initial priority for images is going to be low because it's not a render blocking resource.
If it's an image and it's inside the body element, it's going to be that low priority initially until the browser determines whether or not it lives in the viewport.
And it doesn't know that until it's got all the CSS and it's figured that kind of thing out.
Cause it's, you know, that concept is a little out out of reach until that point.
But it'll try to cue that up right away.
And so the loading indicator lets us hint to the browser.
Like it lets us tell the browser that, "Hey, you know if we say 'loading eager', for example, you know, we think that this is something that should be fetched, you know, initially right away".
And if we do the loading lazy, which is the much more widely used one, that's saying, yeah, we don't think this is that important.
Like you can hang off on this one.
Don't worry about.
And so I think like this is where we're seeing situations like this, where we have some images get queued up a little bit earlier than this one here.
Because that loading priority that initial priority gets kind of bumped down from that loading attribute.
So it gets pushed off a bit, which is not ideal.
And then it takes a long time to load.
It looks like it's a nice, is this a nice, hefty, hefty image?
Yeah.
This is a good image image size.
There was that a Meg?
Megabyte.
Yeah.
So this is always, that gap is always interesting.
Cause if it's that gap usually tells us something like this, where it's either an, it's an inefficiency, if it's like an image, whether it's the size of the image, whether it's the loading attribute, whether it's, you know, JavaScript initiated.
But that's the hint that we've got another bottleneck sort of occurring there.
And that's the other big gap that it's nice to look at it a little bit.
Yeah.
So that's that gap.
And then yeah, I mean that covers the visual, visual gaps mostly.
There's also another one to, to pay attention to, and that's the CLS metric.
And we can see that in this in this filmstrip as well with that same image, right?
We've got a layout that doesn't really have much room for it right beforehand.
Let's switch to the vitals view for that one.
The vitals view is that page where it's specifically targeted at these metrics, you know, where it's going to call out, you know, when the, where the Largest Contentful Paint is and was triggering it and stuff.
But here also for layout shifts, we can actually see a little bit of before and after kind of happening for some of these.
So we have a little bit of a content shift that occurs for sure.
Partly from fonts actually is where we're getting hit by the most.
Yeah.
And that one depends on the metric times.
It depends on the browser size.
So I know for a fact that at a smaller window size, like a, you know, a mobile device, when this, when this layout is all one column the, that, that main image doesn't have any dimensions on it.
So it just sorta like has like a zero pixel height until the browser starts downloading it and realizes the height that it needs to make for the image.
And once that happens, it pushes the content down.
So that depending on the browser size will cause a big layout shift.
Nice.
Yeah, that's a, that aspect ratio thing on the images is one of those, you know you and I have talked about that.
I think that's one of those interesting optimizations now, where now, if we give a with an, a hide element attributes to our images, the browser can preserve the space for that image.
Something that it didn't use to do.
But it does now, every major browser does.
And it's nice because that's specifically, there's this layout shift metric where we can measure visual stability, which we haven't been able to measure before.
And so now I think that's where we start to see things like the aspect ratio and other improvements come out because that stuff's a little bit easier to catch now from a metric perspective and measure and see the impact of.
Yeah.
And that, that example I just gave is sort of a good reminder that it's worth testing your metrics, your performance across different different browsers, different browser device sizes.
Just to see how they're impacted because you might.
Even if it's a responsive site, that's one code base that fluidly works across them all, you might you might find bottlenecks at one viewport size that don't exist at another.
Yeah, absolutely.
There's a good reason, I think for testing in those different browsers and different geographies and different connection types, because that's really important because that's the, the real users are going to be coming from a spectrum of those devices and experiences.
And so testing across those to make sure you're accounting for, you know, what the experience is like.
And there's no mismatched experiences.
I think, you know, it's an important thing for...
THere's this classic performance conundrum where site owners and developers tend to test the conditions that they know their audience exists in already.
Right.
But one of the things that we like to really emphasize with a tool like WebPageTest, for example, is that you know, getting a sense of how your site performs in say regions of the world, that you don't have an audience yet might give you an idea of why you don't have an audience there yet.
Maybe you have you know, problems with accessing your site in a region of the world that blocks the, you know, the CDN provider that you use or something like that.
Or maybe it's just slower in that region of the world, because all of your files are hosted in one place.
So it's good to be able to test those things.
Absolutely.
Yeah, yeah.
That, that's, that's a pretty common, it's not as publicly documented as often as I'd like, just because companies have, sometimes they're a little secretive about it.
I know the YouTube feather story is a big one where they improve the video page, provide a lightweight version of it.
And suddenly they found that they were getting traffic from areas in the world that they had never gotten traffic before.
And because people could access it now with that smaller size.
Yeah.
And as a result, their, their overall performance looked slower.
Right.
Because suddenly people were able to access the site at all, who were never able to access it.
Yeah.
So another one of those things to keep an eye on because of the perf, if they would've just stopped at like, "Hey, our performance metrics, regressed" they would've missed that this was a great thing that happened for them.
For the countries maybe that would have produced you know, decreased developer productivity now having access to a, you know, YouTube from [unclear] holes and stuff like that.
But know so we kind of went through, I think everything we've talked about so far has been like, it's been the metrics.
We talked a little bit about the gaps, which I think is where we both like to start and showing how you can spot some of that in the waterfall, but maybe we can dive into, I think now the opportunities and experiments bit, which is the kind of the new feature here and kind of a way of hopefully making some of these things a little bit easier for folks to spot at a glance without having to necessarily read through the underlying waterfall if they don't want to.
Yeah.
Sounds good.
So there's this little summary now that that's at the top of the the test result page and it just sort of gives you at the start a little intro, a little summary of how how various diagnostics that we ran on the page ended up looking to us at a glance and they're broken into a few categories.
There's quickness, usability, and resilience.
And within those, we run a bunch of diagnostics and you can see those if you go through to the opportunities and experiments page in this case, Tim jumped down to the quickness category.
So as you scroll through this page, you can see them broken into those categories.
And basically at each diagnostic, we either we either run a test and, and decide that things look good.
And you get a check mark, or if we notice there's a opportunity to do a little better then we offer that like in the case that it Tim is highlighting here.
And from there, we either recommend some tips that you could apply to your own site to try to improve that particular bottleneck make it go away or some experiments, which is a very new thing that we just, just released where you can actually test an optimization on your live site without actually changing the live site itself, but in a, in a simulated environment to see if it, if it happens to fix the, the problem that you're dealing with.
Yeah.
So we talked a lot about, you know, those render blocking resources and stuff like that.
Right.
And so this is going to be you know, one of those experiments that then might make a lot of sense to try, right?
Because we saw a very big gap between Time to First Byte and that First Paint.
And so it might make a lot of sense for us to try and run the experiment where we block a bunch of those render blocking resources and kind of see what the impact was of doing that.
The goal being here, right, like some optimizations sometimes will be a massive impact.
Sometimes they won't impact your site at all because maybe because your site is just the way it's built or that there are other bottlenecks that are kind of get in the way.
And the idea here is to try and make it as easy as possible to iterate and find out if these are going to be meaningful things for your team, or if you're not on the development team, for your development team or engineering team to kind of tackle and prioritize.
Exactly.
And you can run these either in isolation or you could choose to run more than one at a time.
And you can see at the bottom that the number of experiments that you've selected throughout your scrolling through this page are applied at the bottom in that little filter.
So you can jump back up to them and, you know, uncheck one, if you'd like, or adjust the way you're applying one in the case where there's different, different choices, for example, maybe you don't want to apply them to every script that's blocking.
Maybe all but one that sort of thing.
So that's up to you.
In this case, we know they're all blocking rendering, so we might want to test the impact of of deferring all of them, which is essentially just a way to cause them to load on the side, right.
While the page can show visibly while they're still loading.
Yeah.
Yeah.
And then there's, you can see the other big gap that we talked about, right?
What is that gap between First Contentful Paint and Largest Contentful Paint, which was triggered in large part because of the loading lazy.
So, you know, this, again, calls that out to make it a little bit more obvious.
Like we knew that cause you know, we built a site that intentionally did bad things.
But in this case where, you know, this calls out that, "Hey, something inside that viewport had that loading attribute on and maybe that's something worth, you know, removing".
Yeah.
So those, those are all related to quickness.
Should we kick that one off?
Just to...
yeah, I got that running here in the background.
So what that's going to do is we selected both the defer-renderblocking with all those resources and the remove loading lazy.
And it's going to run two runs.
It's going to run a control run, which basically goes through our sandbox environment without any of the optimizations applied.
And then an experiment run, which applies the optimizations.
The reason for that control run is because we know that there's some impact, right.
By having it proxy through our sandbox environment.
Often it's a little faster by default.
So we want to make sure that you're comparing kind of apples to apples.
And we've got a fair comparison there.
Yeah.
Nice.
So I guess for, for some clarity around the history of running these kinds of experiments, I can say that.
You know, I think both of us have done a lot of performance auditing in the past.
Right.
And traditionally to run these kinds of experiments would mean what a number of things-either actually changing the, the live site in some way that hopefully doesn't, it doesn't show up for every user while you're testing a particular condition and how it impacts performance.
So maybe you do that through like a little query string or something and change your code on the fly.
Pretty dangerous to do on the live site.
And then kind of a more recent intermediate and better way to test experiments like this was to do it manually still, but to use these things called service edge workers, right.
Where basically companies like CloudFlare, who first made it very popular, I think and now Akamai and Netlify many companies Fastly.
Yeah, yeah, pretty much everybody's.
I mean, Lambda@Edge there's, there's some concept of edge workers on just about every major platform now.
Yeah.
Yeah.
And the way you can use them is to write a little transformer script that lets your site pass through this, this script somewhere on their server and make a little tweak along the way.
And it can do that without without changing the site itself.
So it sort of mimics how your site would perform if it was passed through one of those or if the, if the change was actually made to the site without making the change first.
So that's that's pretty good.
But you know, doing that manually was a great deal of work.
Right.
Definitely.
I mean, it requires, like, there's a combination of things that have to go into it.
Right.
You know, like you have to have the account with one of those platforms to begin with.
You have to be comfortable enough in whatever, you know, most of those are JavaScript based platforms, but there's a few exceptions, but you have to be comfortable enough in that environment to be able to actually write the code, to be able to do the proxy, apply the optimization.
And then you also have to find the optimization in the first place, like, know what you want to run that experiment on.
And so there's a lot of, like, it just requires a lot of knowledge.
Like you have to have the performance knowledge to identify the issues.
You have to have the coding knowledge to be able to apply the experiments.
And so it's really what is that?
And the control as well control.
Right.
And the controlled around.
Right.
And doing that in a way that makes sense.
Yeah.
So it's, it's a lot of requirements to pull it off.
But it's mimicking a lot of, a lot of manual work here in a pretty easy, so we can, so the test actually we're back to this now.
This is what it brings you to.
So once the test is completed in this case, this was our test again, to defer the blocking scripts, render blocking scripts, and remove loading lazy from the viewport images.
Yeah.
And right away in the, in the video.
Huge.
It's a massive impact here in the difference there.
And we see a substantial improvement in that Largest Contentful Paint, in First Contentful Paint, in speed index.
You know, pretty much every metric gets a little bit better here.
Yeah.
That's, that's a good reminder that so many of these metrics are additive in that way.
Right.
You speed up an earlier one and the later one will, will happen sooner as well.
Yeah, so it's nice.
Sometimes you won't find this.
Like I think, you know, sometimes you'll see that the impact was negative.
It went the wrong way or very, very little impact or one metric improved and another got worse.
But those are good things to find out here where it takes you a minute to try it then, you know, in your own environment where it takes you a couple of weeks to figure that out.
Right.
Right.
So, you know, I guess it's all well and good to just run, run this tool on a fake news site, but maybe we could show some real sites.
Let's, we've got a couple that we the two of us have, you know, found particularly interesting in the last couple of days.
So this is Fest foods, a grocery store chain.
Idon't know if, If this is, is this a nationwide thing?
Do they have that in New York?
I don't know of it.
They have them around here.
They're big around here.
So I actually saw on a presentation from Paul Calvano that they had a couple of issues.
So I thought it would be interesting to run a test.
Now, this is again on a 4G mobile device.
Where the big gap that calls out, like start render to Largest Contentful Paint, isn't that bad.
Like a one second gap is reasonable, right?
But this gap, Time to First Byte to start render is massive.
Interestingly, on this particular site, we just talked about all those render blocking resources, being a thing, right?
It's not actually what's happening here.
There's only a few resources that are blocking page rendering.
That's not typical.
It's not typical.
What's actually happening in this case is another issue altogether.
Which is these this-381 resources that are being preloaded, but not used during the page load.
So when we preload a resource, that's a resource hint that we tell to the browser.
And what this says is, "Hey it helps with discovery.
Like this is a resource that is either low in the page, or maybe it's dynamically injected, but whatever we think it's like, it's important.
The browser should request this as early as possible and get this down".
And so in this case, it's all these CSS and JavaScript files, but this looks like it's like a build tool fail or something like that because none of these resources are actually then used by the page.
But the browser doesn't know that.
So what happens is the browser sees the preload says, "okay, well, you know what you're talking about, I should grab all of these resources".
And it cues up the downloads right away and you can see starting request six all the way down.
These are all our preloaded resources.
And until that stuff's out of the way we cannot actually start to grab any of the actual resources the page really does need somewhere around here, we start doing that stuff.
Yeah.
So there, the preloads are not really-they would not necessarily block rendering on their own, but they're at a higher priority than the ones that will block rendering.
Right?
Yeah.
We've got a higher priority kicking in here.
They're all contending for bandwidth.
There's probably, I mean, there's probably a couple, it looks like there's what 403 resources here, that are CSS JavaScript.
There's probably a few legitimate ones that are being preloaded as well, but they have to compete with everything else for bandwidth before they can start to do anything.
And if we go down here, actually we see actually it's the bandwidth isn't too bad, but that CPU is taxed all the way through this period of time where it's trying to get all these resources, execute a few of them that are actually on the page.
And so, yeah, it basically ends up being this massive render blocking thing that's occurring.
Cool.
How would we even run an experiment to try to fix that?
Well, I'll tell you what didn't work.
I tried at first to Just like, send something to re pull out all of these, but talk about stress testing.
It turns out that trying to inject a tiny snippet to tell 381 resources not to pull out gets a little heavy and doesn't really have the, a, it doesn't work the way we would expect.
But that's where we have these custom experiments that came in handy.
For sure.
We haven't talked about these at all, but like these above here are more like, very specific to the opportunity, you know, kind of a guided performance consultant sort of view, a little bit to create experiments is more of a sandbox, like play and have fun with it, right?
Yeah.
I think it's reactive versus versus proactive, I guess, in, in some ways all this creative experiment section is it's very much a, sort of the power tools section, right.
Where you can if you know what you're doing with this particular page, you know what you're looking for, you can make modifications yourself.
Right.
So what I did for this one is I used our find and replace text and I ran using regular expressions.
Let's see if it's, I am not going to pretend that I can remember what that is.
I have the experiment results, spoiler alert here where I ran it, but this is my regex.
So basically I found anything that was a preload element and then just replaced it with an empty space for now and ran the test.
And so what that did, is it removed, it removed all the unused.
It probably removed, like I said, a few that were actual valid preloads as well But it was a good enough test to give me an indication of how impactful this would be.
And it was very impactful as it turned out.
Wow.
So it moved that First Contentful Paint back you know, almost six seconds, which is almost that entire gap, right.
That we were looking at.
I think it was, was a huge chunk of that.
Yeah.
So when you see maybe there's some additional optimizations that could be made, it looks like in the process of removing all of those, you got a few a few or four specifically render blocking requests that came in.
Maybe they, they, you know, were able to request sooner as a result of getting the loads out of the way.
Yeah.
And now it's like, if you go down to the waterfall comparison for these two, the difference is dramatic, like here's the control run where we just applied the proxy, but didn't do any optimizations right.
With all those requests.
Yeah.
And then there's the experiment, like, look how much faster we're getting to these key resources, getting that out.
And the chain is much, much shorter-118.
We aren't even out of the pre-loads yet, by that point on the other.
And yeah this is where you could start to know, you know, all the render blocking stuff would become an experiment that'd be interesting to run, right.
Like try to, you know defer the render blocking JavaScript we see here or load the CSS asynchronously.
There's a few things that now might make sense that without those preloads out of the way, probably wouldn't move the needle at all.
Yeah, you're right.
I mean, I could see like some, some Google fonts, CSS in there that could be asynchronous.
It doesn't need to load in a blocking way.
Yeah.
So you can just keep iterating from there.
Go back to the opportunities page and yeah.
And keep adding a few more and sort of build this up for sure.
Yeah.
Yeah.
Another, another example that we tested this morning.
Yeah.
This one was, you were playing with Nintendo and [talk over each other] oh, I'm sorry.
I recently recently relaunched on some new tech.
And we can see that if you look at the timeline, there's a pretty dramatic wait and not just not just white screen wait, but also when something starts to render it's not really useful, right?
So this is where metrics start to not be necessarily the only source of truth.
So even though we got to start render right around, what was it?
Six seconds.
All it was was three seconds even.
Yeah.
So things looked good from a early metric standpoint, but from a user perspective, all that the user was getting was gray boxes, like a loading screen, essentially.
And it took up until 30 seconds and this was on 4g connection in Chrome.
So pretty standard mobile connection took about 30 seconds for the page to show content.
So this was pretty interesting.
You know, we see a gap like that between First Contentful Paint and I guess, you know, our just Largest Contentful Paint I guess, would be What's showing up around 30 seconds.
You know, it could be a number of things.
It could be that the, the image that's, the largest content was taking a while to load.
Maybe it could be requested sooner, but what's happening in this page is a little different we found, right?
It's not just that the Largest Contentful Paint image is slow to load, but really all the content is not there.
Right.
And all the text on the page.
So that starts to tell us, well, maybe, maybe that text isn't in the HTML to begin with, right.
Maybe it's being requested after the back.
And that's why they're showing a, a loading screen.
And that's a pretty common pattern these days, right?
The React sites sort of introduced this popularly early on, where you get a loading screen and then React goes off and fetches and fetches and fetches and fetches.
Lots of JSON, takes it back.
You know, and converts it into HTML that it can then inject into the page and we're seeing a pretty dramatic wait for, for that scenario.
Yeah.
And this seems to be our suspect here, exactly what you were talking about on the waterfall, these yellow bars, that's JavaScript execution.
That's blocked, like triggered by that resource.
So there's a big chunk here, but the other thing is like, eh, if we go to about the time we get our LCP, right?
Like somewhere there and then just kind of go down and that's the end of our waterfall, basically, you can see all these requests that got queued up, and then there's a massive chunk of JavaScript execution here as well before we're able to do anything else either.
So yeah.
It adds more credence to your, yeah.
It seems like the contents coming back dynamically or the JavaScript, it's waiting for something from JavaScript to be able to paint that stuff onto the screen.
Yeah.
And I like to, I like to make an assumption when I see the LCP metric kind of line up with like the end of the waterfall when all the content arrives.
That's a sure sign, usually, that JavaScript is just sort of doing a voila moment, right at the end where everything gets, you know, delivered all at once.
And you get your metrics all crunched in there at the end where suddenly the site's ready to use, but it took a long time to get there For sure.
So then where did you go from there then?
I guess now you had your theory.
Yeah.
So you know, my theory was that JavaScript was, was, you know, blocking everything.
So we went into opportunities to confirm that and what I found under the usable category, or maybe it was right at the end of quickness, it was right at the end of quickness, actually that there's a nice little comparison that it offers.
Where it says a significant portion of HTML content in this case 425 kilobytes or 50% of the total page weight.
And that's a simple, just simple comparison of what was delivered upfront versus what was there at the end.
It's saying that much was, was generated after the HTML was delivered.
By JavaScript so that's a big red flag.
And we have a little experiment to offer where we can, you know, if you were to address this problem.
Well, the first thing you would think of is, well, maybe I should be serving more content in my HTML from the server to begin with.
But the process of saying something like that and actually executing on it is like months of labor.
Right.
And no experiment is going to fix that for you either.
But it can be useful to know if it's worth doing that, that work at all.
Right?
Like what if we could find out how would our page load differently if we did serve most of that end state of the, the content upfront, would it be even worth the effort?
Well, that's what this experiment tells us.
So it actually takes the the end state of that HTML page when it, when it's all complete and it stores it on the server where we can retrieve it as we're running rerunning, this experiment test, and it just serves it from the start as if Nintendo just sent it to the browser upfront.
And so we ran that experiment and you don't need to wait on it.
We have a result page here, but you know, this one was like one of the biggest, one of the biggest impacts I've seen lately where, you know, 60 seconds Largest Contentful Paint difference.
Now I should preface that in saying that it's comparing this particular control run versus the experiment.
Cause we saw in the last round, it was what, 30 seconds 35 that the LCP happened.
Right.
And that's why it's worth running a lot of tests.
Like, you know, if you're going to run an experiment, I only ran one run here and it just so happened that both of them were a little slower than that last test run.
So we see a big difference.
If I had upped the number of runs of the experiment to say, you know, six or, or even more the more runs you do, the more you can count on getting a good median run that counts out the outlier runs that are a little slower than the typical user might experience.
Right.
So this one's probably a little slower, but we see a big difference.
You know, maybe bigger than it would be realistically, but still this suggests maybe this would be worth the effort.
Right.
So, yeah.
And there's some, you know, and I think one of the things we know it on this experiment, right, is that it is going to very likely break something, right?
Because of the way this is all working, injecting that HTML.
And in this case, you know, you can see that the final state here for the final state for our getting a little click happy for our experiment run does look different, right?
Because it doesn't have some of the same kind of styling and stuff applied.
And that's fine.
Cause I think the goal of this experiment is to show the potential of like if we weren't relying on JavaScript and then we could serve something usable from the server, how much faster could we potentially get stuff on the screen?
And so even if it's not a perfect next side-by-side visual comparison metric wise, this is telling us that, yeah, there's a substantial opportunity for improvement if we can figure this out.
Yeah.
And I actually just noticed that the result you're looking at was one where I ran two experiments.
And let me share you one share you share one that had a, a better final state.
So one second.
Oh, I see.
Yeah.
So you did run in this case, disabled scripts, which is also like one of those experiments that you may want to sometimes pair with the mimic pre-render depending on the site.
Because sometimes that can actually reduce the amount of breakage in this case, though.
I think it worked better.
Yeah, sorry, I had the wrong URL there, but it's a good example of you know, that, that prior example had a couple of experiments run and it sort of messed with the the way the page rendered in the end.
Whereas this one was just an isolated experiment pre-rendered HTML and it renders, you know, roughly how it should you know, with, with little style differences here and there, but it gives you a good idea of like, especially on the timeline below how soon content would be ready.
You know, I think if you were to actually implement this change yourself, you would make sure that there's enough CSS in the page to style that initial HTML a lot sooner, right.
In this example and why I ran that second round.
You know, you would notice this and then you'd say, okay, how could I get the page styles to show sooner and defer the JavaScript?
So you start, you know, running experiments like that.
And what I learned in this one was that their, their CSS is very deeply embedded in the scripts that they're loading.
So there was no unbundling that I could do very easily.
So, you know, in reality it would be, it would be a, quite a bit of work to, to pull this off, but it shows wow.
Their users could, you know, potentially get a site on a fast cell and cell phone connection, maybe 20 seconds earlier or something like that.
That's huge.
I think like that's, you know, I think that's one of the primary things that we talked a little bit at the beginning about the amount of complexity that goes into getting stuff on the screen today.
And the complexity involved in fixing that is non trivial.
And, you know, that's hopefully where these new features, the opportunities and experiments come in to kind of help out with that a little bit and to kind of bridge that gap and eliminate some of the complexity that can come with.
And some of the whack-a-mole feeling that can come from optimizing performance sometimes.
Yeah.
And I guess we should mention as a bit of a disclosure that many of the experiments are part of the pro feature plan.
So that's a paid tier of WebPageTest that just recently launched.
But one of the most popular experiments is totally free if you have a WebPageTest account.
And that's the deferred JavaScript defer blocking JavaScript experiment.
So, you know, if you want to test this out on any old site, just get a free account.
And you can see how it works on your own work.
Yeah.
And if you do, please give us a shout out and let us know how things go.
I think one of the funnest things for me about the performance community historically is how much we all learn from each other and what we're all working on.
And it's, you know, we definitely want to continue to do that and we can refine these things and everybody can learn from failed or successful experiments.
Exactly.
Yeah.
Yeah.
That's a good point.
We didn't really show any any good examples of a experiment that didn't really contribute to a good improvement, but those are really useful to know as well.
Because you won't spend your time fixing something that isn't going to make an impact.
So Exactly.
Yeah.
Thanks everybody for tuning in hopefully everybody got a little bit out of that.
Again, I'm Scott or I'm I'm Scott I'm Tim and I'm Tim as well.
And thanks for joining us Thanks again Thanks again to Tim and Scott and keep an eye on your email inbox for a special offer from Catchpoint, if you're keen to explore how you can incorporate WebPageTest test in your performance monitoring workflow.
Thanks again for your understanding as we have this disrupted programme for Lazy Load, but we will see you again next Friday for week two of Lazy Load, where we have a whole raft of amazing presentations for you.