Getting Off(line): Appcache, LocalStorage for HTML5 Apps that Work Offline

now you can see where I get my humor from right Thank You Maxine there we go so getting off line the idea that you know storage and hard drives and caching is particularly interesting like it's kind of a weird idea it's been something we're doing since the beginning of computing right caching is one of these things because fast stuff is expensive and slow stuff is cheap so fast networks are expensive and you know fast hard disks and fast memory are expensive but the slow stuff is really cheap so we we cheat by caching and the web's done this all along and it's a holy but other tricks in how we can make our applications and our websites faster and work without necessarily being connected for web so I'm going to talk about these things for the next 45 minutes or so and we're looking at how we can make our applications and our websites work offline and how we can make them work more quickly whether they're an application whether a website whatever you you want to however you want to describe them one of these people often say about the difference between native add apps and web apps is native apps work even when you're not connected and you know what happens when though you know in your airplanes or go in a tunnel or some other kind of magical experience where you're not connected to the web anymore like at a conference and we're doing pretty well I think today the truth is the most native apps are really just thin eight stupid clients anyway we use the Facebook app from native app Twitter and motive app when you're not connected well nothing much really happens the case of Facebook kind of nothing happens so it's arguable than in when you're connected nothing happens on Facebook but I'll leave that to one side you know that's possibly the first mention of Facebook I mean there's big things there's a trillion boxes or stuff and yet no one's talked about Facebook day which I think it's fantastic right shows who actually care about technology and implementing stuff not sort of the kind of soap opera of the web all right so we're going to look at today is we're looking at how we can cache to make our websites and applications fast so we're gonna look at how we can use local storage to keep data locally so we can work when our applications are online and we can use some events to know when a user goes online offline and we can respond we can create a different user experience we can turn off submit buttons or we can you know turn off an upload button once the user goes offline is signed so it's it's amazes really all these technologies are about the user experience all these technologies should vanish the users shouldn't care whether they're online or offline they should just they should you know the user experience should essentially be a function of the context they're currently in and one of those contexts is it contents yeah contexts is are they connected to web or not right so we as developers should work really hard that our users just don't know and don't care and it really doesn't matter to them right so I'm gonna talk about some of those technologies today now what I've been talkin what I'm talking about today we can all use and I'll talk about some little caveats and some gotchas but these technologies aren't over the horizon they're not cool things coming down the pipeline they really work in most browsers in most contexts and most devices and they're typically are good polyfills for them as well so this is stuff you can use today it's quite a lot of to get through so I'm going to get started so the first thing I'm going to talk about is talk about the application application cache and the idea behind this is that well clearly browser's cache things all right and that's why they kind of work they're basically are that I'll stick that image somewhere and later on if a person comes back I can reuse it now been doing it for a long time but we've virtually had no control over that Garan our headers we can say cases or don't catch this but in terms of once something gets on the client well we can't really do that at all until now so that's what the app cache is all about it's about creating a developer controllable cache of the resources that the user is going to tap into all right so we know the browser's already do this but we've had no control of them so what we can do with the html5 application cache is we can create that cache and we can tell the browser what to cache when to catch what to fall back to when the user isn't online and we don't have a resource and there are things we never want to cache as well so we will talk about all of that for the next ten or fifteen minutes or so the idea here is that we can create applications or websites that you work in when the user is not connected but we can also even if that's not the context of how you build things you can still get a lot of performance benefits by prefetching resources by fetching only resources you really need right now and then caching away for later use and we'll talk a little bout some of those use cases as well so even if you are kind of building things that you might think of as an as an application if you're not building something you think someone's gonna install on the home screen of their device this stuff is still really relevant to you so what we need to do to build an application thing is application cases three things we need to create an application application manifest file which describes as we'll see in detail what to case what not to case went in a how we then need to link to this from our HTML document so basically when the browser loads the document it knows that there's a cache and goes and gets in and uses that and then we have to worry about a heap of problems right so it's the third part that makes our job challenging very interesting in a way the other two are pretty trivial so we'll look at all three steps the first thing the third have got a lot more to them than the second one funnily enough all right so the first thing in application app cache manifest file is simply a text file with the dot app cache file extension afterward that's all it is simple text file I guess you know it's like that CSS HTML there we'll just text file and what they do is they specify three things what the browser must patient and only use went from the cache what the browser must never cache and always go online to fetch and then if something is missing when the user is offline what fallbacks you can specify for backs with this file you should use this fallback file and so on so the application manifest file as I said it's a simple text file it must begin with that strength cache manifest right and then it has three sections one is required the others are optional the cache section which says what should be cached what must be cache in fact it has a network session which says I can never use a cached version of these resources even if you want to do not do that right because you know otherwise the browser I just I'll just go and use this login dot HTML you know don't do that and finally the fallback safe section specifies the full-backs that the user if it should be used when the users not online and the resources have been cache and so this is what it sort of looks like this is really in a nutshell you'll have that cache manifest at the top you've got the cache section we've got the fallback section a right at the bottom you got the network section I'm going to explain each of these in details and you can see we've also got little comments there and we'll talk about all the features of this over the next five minutes or so so the cash section as I said these describes what the browser must fetch and must always use from the cache right so once something's in the cache manifest and it's been cached the browser is not allowed to go and try and get that resource again except in certain circumstances the we as a developer essentially say please go and refresh the cache so once on this cache it is there and we'll get on that just remember that once it's there it's there and doesn't matter how many times you change that file online where the JavaScript CSS image its cache I'm gonna keep using a cache because in the cache section alright so here's an example what might look like cache manifest cache images and then we have here's a relative URL he's an absolute URL it points to a resource and that resource now what you've seen to the browsers you see this image one got pin download it right now and never ever download it again until I tell you you're allowed to and similarly so you can use a relative to so that's a relative URL and it's relative to the cache manifest file itself and then you can have an absolute URL as well as I mentioned basically once that file is cached even if you change the file on the server it will not be re-downloaded and reused by the browser right that's the point of this so this is particularly problematic when you are developing so the first thing I'm gonna tell you is do not turn on app caching while you were developing it would drive you absolutely insane do not do this if I write be what be warned it's just such unbelievable pain but I've got some tricks later on how to fix that up and what we can do though is we can add something like a version number or a date stamp to our cache file in a comment and then what we do is we update change the date change the version number and that is a trigger for the browser to recache but we're getting a bit more details a bit later but the takeaway from that one is don't do this while you're developing write because you'll go insane all right now the next thing we might want to do is there might be resources we always want to use the most recent version on the most current version we never won a case version you know you think of the front page of a newspaper I think of a breaking news story you think of you know certain things like a login page at a website you know like certain Java scripts that make on being updated what we can say in the network section is do not ever catch this never even in your own little cash over here where I know you put your secret stash okay you don't allow to catch it there either so this instruction is not just about building this app cache so it's actually about what things that browser should shouldn't read in cache in its own kind of its own stash there right so I think once you put something in the network section it's saying never ever use the cached version in any way all right so what's important here also is that when the user is in an offline mode and you try for a fetch for that particular resource that will throw an error and we can manage that we can we can we can do that we can find that error and this is a bit too much detail to go into in this session but basically there are event listeners we can add for errors around the cache and we can respond to that but these are so what's the point of network sectionals for things we really they're gonna go stale very quickly we never want that browser to use a cached version so you can probably think of certain circumstances your own development again so in the first instance in the cache section we have two cache and we'll get onto this again and what the implications are in a second we have to cash resources explicitly that means we we can't have a wild card say it looks just just cache all the stuff in the images folder or or just cache everything it you have to explicitly state every resource using a direct URL or you know either either four or I'm a partial URL you know a relative URL now in in the case of the network it's slightly different you can actually also have a partial match pattern so basically you can point to part of a directory tree and anything below that will then be in the network section so we basically say you know anything inside so the images folder or anything in the JavaScript folder or wherever it happens to be you're essentially saying anything there and in subdirectories of that just don't ever cache that stuff alright there's also a wild-card the the star wildcard and this is a little bit different from the what you might think of an asterisk or star basically it says what this says is and any resource that is not explicitly cached in a cache section of this manifest file never use the offline version always use online version right including this is very importantly in the browser's own private cache all right so even for its own purposes to go on I'll go and keep this bullet no you're not gonna do that right you have to always anything that I have not explicitly cached if in the network section you have that asterisk it says anything not in this cache you must never use a cached version either my cache or your cache or anyone else cache that matter all right so you can see there's a lot of permutations combinations and that's where it gets a little bit tricky the implications of this are more subtle than you might imagine so here's what a network section might look like you know it might have signup dot HTML so that's kind of just a relative URL or you might then have in the case of the payments directory that might be a whole heap of resources that you don't want the payments any of the payment stuff being used offline it must always be the online version and finally at the bottom there we have the asterisks which says anything you haven't put you know directly in our cache manifest in the cache section never cache that so you wouldn't typically combine an asterisk with anything else in that section because essentially that ass for us takes care of everything else and finally we have a fallback section because obviously when you're working offline like we had sort of example of Wikipedia this morning with Dave and what they did with PhoneGap and obviously we keep it has got like a bazillion pages right and you wouldn't really want to make a cache manifest file that pointed every single page you can imagine where that would go right so what you might do is you might have some sort of fallback page so that anything that isn't has a link case where the person is offline you might load up a kind of a place for the page or you might have a you know if you had a directory of the thousands of staff members and you were loading their image for every page or whatever you might have a generic image for a user so when when we hadn't managed to have cached a particular you know biographical image of someone just use the generic fallback image so there's lots of ways we might want to use that now the pattern here is just a little bit different from networking cache so in the case of networking cache we're just simply pointing at a resource or a set of resources in this case what we do is we have two parts to the to the pattern in the first part we have in this case / images / which says anything in the images directory but that could also be a specific you know there could be / images / you know / kind of image ping right you could you could point a specific resource or a class of resources or set of resources and in the second part after that space we have / images / missing ping and we essentially we're saying okay anything here in this first that matches that first pattern when we're offline and you you don't have that resource cached use this specific resource knowing that in the case of this second part of the pattern what we point to a specific resource with either a relative or absolute URL right so very useful in the situation where you know we don't know ahead of time if that particular resource is going to have been cached so instead of having like a broken image you know icon or just a page missing sort of we can actually have some sort of fallback page or something you know if we don't want people to be able to sign up while they're offline because that doesn't make a lot of sense we can have a fallback page we've said you know like you know you'll have to be online and we'll just sign up to sign on so try back later or something you know we could we can respond to particular circumstances so it gives us a great deal of sophistication in handling you know the way application or our websites going to work when the user is offline that's sort of just sort of explains here you know away we might go about this like let me see a me enjoy in the case of anything else it's missing we just have sorry dot HTML so we just think you know we've if a resource is missing and obviously probably want to do this because that might apply to you know I resource it isn't an HTML page they're gonna be thoughtful about about how it's working but basically what happened to that fallback it's like a cascade it was what's the opposite of CSS it will it will match the first and if it doesn't match from the first I've got it'll go down the list right so firstly this will replace any images in the image directory and only after that will it then replace anything else with the sorry by HTML all right so yeah we can have multiple sort of responses to different resources based on where they are in the directory what we can't do they always have a wild card doc ping they kind of like Astra stop ping replace that with something that sort of wild cut doesn't work you only have this partial match pattern in the way which we can match resources and here would be a manifest file so in this case we've got a version number at the top so we start with the cache manifest that's required on was it's not a manifest file and then we have that the the hash sign which says this is a comment and we give it a version number and we'll see a little bit more wild we've done that we've got the cache section we've got a network section we've got a fallback section very very simple and the these things can get quite complicated but once you understand the basic principle I think you can start especially if you use some tools that are helpful for this you can start building quite powerful manifests so we built our manifest file we put on our server somewhere it's sitting there but the question is how's the browser not know to associate this particular manifest file with a particular HTML doc well what we do is in the HTML element we can use a new attribute e in html5 which is the manifest attribute right so it's the HTML element itself so right at the root of the document HTML and then we have manifest equals and we have a URL so that's in this case relative to the HTML document could also be an absolute URL and basically what this is when the browser loads the page it looks at manifest attribute it says oh there's a manifest for this alright and then it fetches the manifest and it builds that manifest file it downloads or excludes or whatever based on on the manifest and it's also recommend use the html5 document you can prop if you think about this for more than about 10 seconds and you build sites that are kind of responsive or you know in some way kind of a media based you can you can probably start seeing that this only allows us to have one single manifest file all right and inside the manifest file we can't say are you know in the case that this is this media download these resort right so immediately you can see there are some limitations of this technology that just using it as ease we can't we can't really use it to sort of create a cache that will be okay for a mobile device download these resolute lower resolution images or smaller images for a high resolution you know device or retina device download these large ones we can't simply do that in a manifest what we have to start doing now is playing around at the JavaScript level and then in the DOM and actually building the manifest on the fly and well I won't go into that today but just to kind of demonstrate that despite this being a relatively new technology it's a use case that hasn't been really well addressed at all and without meaning to get my hooks in too much I mean I think sometimes we're seeing this around html5 and so some extent css3 because the people who are building these technologies they build browsers well they build standards they actually don't build websites and applications and as a consequence the things that are kind of immediately obvious to people when they have to build this stuff like well what happens if we've got five different profile devices we don't have to download five different kind of image resolutions that's immediately obvious to people like us who do this all the time it's not Nestle immediately obvious is someone who's building a specification who doesn't really build sites and applications day in day out right so this that what I'm talking about here is being refined it's a pretty fully baked standard I hope though there will be some kind of recognition and changes and certainly there are some community driven kind of changes being pushed as we learn more and more about some of the limitations of this technology however this is pretty well supported all right now this is really important you must serve your manifest file as text slash cache manifest if you serve it as anything else the browser ignores it so it's not like the old kind of you know the battle days of XHTML HTML you can serve as anything alike and the browser have pretended knew what it was talking about right in this case if you serve as anything else it will get ignored now you might think this is a bad thing but it's actually a good thing we'll see what a minute all right su we can use it to our benefit the other thing is in most standards installs of Apache and things like that at the moment it's still this this association this mime type hasn't been set up so something you have the aware of because no browsers will will honor a manifest if it served as anything other than text slash cache manifest so the basic premise is we set up the cache using our cache section our network section our fallback session that says yes cache this stuff never catch this stuff and you know when something's missing either the rules great and we associate that manifest with you know a particular Journal document or set of HTML documents because we kind of link to it in the HTML element using the manifest attribute awesome and then we face a whole slew of gotchas right so this is where all the work actually comes in the first part is pretty straightforward so here's what I'm gonna do is I'm gonna cover off the next ten minutes or so some of the things that have people have stuck their toes on and burnt the fingers on and so on right because if you start working with this stuff you will fall over these things and hopefully other people like me have done that and you won't have to do it right so the first thing is real important to notice is that manifests don't expire right they're there forever and you can send any header you like so please write these things Trump headers forget your HTTP stuff once in the manifest is there are effectively is there forever right so when something's cached it doesn't know how many times you change that resource online there's no if you you know it will always the browser will always use that local version from now on as I said this makes life incredibly painful because obviously as you're developing am I'm going to change my CSS I'll reload I'll change my JavaScript I'll reload on pushing you sort of whatever the browser's is oh no no I um occasionally Scott and clearing caches are so what did I say before do not turn this stuff on while you were developing ever right just don't do that if you do I will not be responsible what happens to you right okay now this is a really important gotcha as well it's a counterintuitive but it kind of also if you think it through it's the only way this could work so imagine you've got a style sheet with a heap of background images in it and you case your Sachi awesome and then yeah this actually won't also cache all the images that are referenced from your style sheet or any imported statutes or any images referenced in your JavaScript or whatever a resource must be explicitly cached there must be referenced in a case section of a manifest for it to be chased right so what this means is basically you either have to like make a painful inventory of all the resources in your web page or you use a tool and I've built one that I'd also well I might show you a demo but I was just quickly reference to it and it actually does this all for you because I'm so nice but there is an exception to this there's one exception which is any HTML document which has a manifest attribute unit will be added to the cache associate with that manifest even if it's not referenced in the manifest right okay so this which makes intuitive sense as well because otherwise what you would have to do is you have to add all your HTML documents in the manifest and the first time that you went somebody came to your website they would start downloading every page in your website right so this lazy loading means that any webpage that it's got the manifest file is it will be cache with it having to be explosively cached they're also their strengths and weaknesses in that as well cross domain where life is made hell for good reason all the time because we have these cross domain restrictions within the case of cache manifest so you can you can you can reach any website in the world and you can tell a browser just go and get those peoples resources and stick them in your cache right video doesn't matter get all the videos from YouTube and put them in my you know no local thing right you can do that there's no restriction well as there's actually one restriction which is in the standard they say in a CVS you're not allowed to do that like HTTP you've got a you know you've got it honor the cross domain restrictions and then chrome doesn't even do that anyway right so it kind of makes it useful aspire to keep that in mind you you can resort you you don't have to like point only to stop at one domain within it doesn't have to be the same domain that the manifest is on which makes a lot of sense in the world of CD ends and the way we're kind of doing kind of different you know pulling stuff off certain repos and you know like people who are continuing to point to to jQuery and other things on you know the one the other day someone wrote about their the html5 shim how it's don't point to it on a parent so you can you can you can just point to a the mana fit yeah add that to your manifest from anywhere right so basically you can pull resources from any are on the web which is kind of good but is worth keeping in mind now I said before that once something is cached it stays in the cache forever right and even if you change that resource a million times it did browser I'm not even looking I'm sorry I'm not look I don't care if you I'm not looking because it says I'm using the case truck however once you change the cash you're gonna change your manifest file and upload it again and you have to change it you have to change the bar it has to be different it can't just be touched but once you've uploaded that the brothers all walk the occasions change I will now go and check every single resource in the cache and any which have changed since I created the cache I will download again so it's smart enough not just get everything again ooh but it has to do a holiday heat age to be traffic to go out and say oh you know what I'm just going to check every one of these things and anything that's been updated since the last time I create the cache I will do them so that's the process now for very complex reasons I will not bore you with because I don't really understand it but it makes perfect sense but it's just hurts my head the first time the user goes to your website and the browser says all the manifests change right it loads the webpage and then starts rebuilding the cache so in fact it's only the next time after that that in fact the page is refreshed you can do it actually if you you can write your own little JavaScript to actually force a refresh but if the user just visits this page with without it and reloads it it's only the second time they visit it that the page will actually use the new cached resources right which makes perfect sense and it's completely illogical at the same time html5 makes perfect sense and it's completely a logical Center it's theoretically boolean until it like but uses kind of Caring so as I said you can actually get over that by and I'm not I haven't got time to go into detail but you actually you can force a refresh in JavaScript on an object in the window in on the cache object in The Witcher window right as I mentioned this is lazy caching concept where basically if an HTML document has a manifest attribute which points to a cache that HTML document will get added to the manifest now he's a really big pain even if you mean what if I don't want that right so front page of the age all right so the front page the age right do we want to cache the whole page I don't think so but if we want we want to cache everything else in there right all all the resources are points do like at the first time serve under the age will get the javascript on the images and all the ads and we want to catch up right but because there's a link in the manifest like there's a manifest actually the page gets forced into the catcher and think oh that's alright well just whitelist this we'll say in the network do not what will this this resource as oh you can't override that so you can't and I've tested this it sounds so illogical that I actually have tested this repeatedly because I still can't believe it and it's true you'll have to trust me and I can show you the demo if you really wanted I've got online so basically trust but verify so what are these means is I got I won't occasionally some resources on my homepage of my website and my homepage of my website gets refresh frequently right which you know increasingly they do the page will get cached but we can't override that in the network section we can't say do not cache boys it's gonna catch it anyway which i think is a massive limitation and i really hope they fix it but so for now what you'll have to know is and you've probably visited newspapers we've so why is it still show me yesterday's newspaper my suspect this is why right because people are thinking caching awesome will do it this is a really subtle got shot now that's why I'm making some time to go into it you can actually then you can see force the case change using the Dom right so you can get invocation change it using the Dom but it's a that better starting do some real work so this is one of the would the really worst bits of the app cache I cannot believe I can see no justification someone would explain to me to be pretty obvious I can see no justification why you would possibly not allow me to say you know what I don't really want to cache this page you are forcing me the cash but that's how it actually is certainly in the network section right this is another really important gotcha if a single one of the resources that you point to in your cache isn't there the entire cache files right so I go to web page and I'm pointing it like ad server over here and JDO so html5 shim over there and I saw that stuff and one of those in there that cache none of it gets built the whole case is not built at all and even more subtly if there's already a cache it's being built and you go to reap so you change the cache whatever it will consider once right the old cache and not build the new cache it'll actually just keep using silently well it throws an error but in you know it's it will keep using the case it's already built because wireless resources is missing so actually verifying the consistency and the existence of all the resources you need point two in the cache section of your app cache manifest is vitally important because one or two really bad things will happen it will never get built in the first place or an out-of-date case will continue to be used right which i think is even worse because at least indication they've got two built it's gonna use the right resources but if it never refreshes that because one resource that might be on somebody's website on the other side of what were observing the other side of world isn't even there anymore changed its name the servers down for lunch if I'm making a point to get it right you can see this is an awesome technology except it's like in the same way that land mines are an awesome technology and I used to it almost immune its traditional so how's all this work in surveys it's all theory where is this practice what's happened well here we go so there's some limitations so basically this is widely supported now but different devices different browsers have different supports so you serif re there stop No Limit Mobile Safari ten Meg Chrome five Meg Android browser No Limit so that's awesome so we just have to be sort of aware of that so what it means basically is if a web page or like a website has more than 50 mega resources well actually frankly probably more than five mega resources you start worried about caching right things not a load of resources for an entire website so what you Polly gonna be doing is like building multiple caches for different aspects of your website and I hear the stuff like right now hopefully browsers are smarter say you know what even though there's 500 cases here they're pointing the same route thing so I'm not gonna read download I'm just gonna like make a reference to that stuff there and there but that's browser implementation detail all right but anyway this is just something to keep in mind right so I've sold you this fantastic technology I'm telling you like that this is this is the fine print right this is the this is the devil in the details all right I told you about how developing with appcache turning on is like a royal pain but of course it's also a royal pain to like not have a link to a manifest file in all of you your pages and then having to go back and put them in either manually or in your CMS or will build them in your templating or whatever right so what you can actually do is one or two so what I would do is action this file link to it in the manifest attribute of the HTML documents but serve it as text or some other mime type and the browser's ignore it so and then when it comes to production time or testing the production version just turn on that mime type for that resource and it gets turned on because while you are developing I'm gonna say 8 times because you will do this at your peril and you will do this because everyone does they are down below if you have it turned on evolving it you will it would just destroy you right because it will take about two days work out what the hell is what skinned occasion was not an it's not updating what isn't okay so the other one is some clever person in a class I was teaching recent workshop and so you said well why don't we just use the fact that the caching won't work if any resources are missing we just point to imaginary resource and the cache is files right so it's another potential approach all right so the case that cache gets downloaded it gets used when in fact it just gets abandoned by the browser based on the fact that it will ignore any cache that has any resources missing so it's kind of using one of the weakest aspects of cache to its advantage all right so the little couple of gotchas and techniques along the way what about support apps so IE 10 which you know as mentioned it's a bit of a shame right now the only ways we can get our hands on is Windows 8 I know Lachlan you're gonna hear that a thousand times that's alright we want to get a hands on this so we want to play with it will make it easier right I know you know that Walker all right but all these other browser support for a long time right and the really cool thing is like a browser doesn't support nothing could break right and still gonna keep doing it's all in caching whatever and this is really the most backward compatible technology known to man really I mean just for other than the wheel and even the Mayans aren't sure about that so so so it's a technology can safely use because the browsers don't support it devices don't support it they just don't get caching right so providing logic doesn't absolutely rely on caching well it wasn't there anyway right so that's one of the real important ways we can take our applications and our websites offline is using that cache now as I've mentioned one of these pains is like how many resources on your average web site by JavaScript when I do a workshop I've got a bit more time for this so actually take you know some of the audience please tell me what your website is and then we go in like use this bookmark that I've created so what the bookmarklet does and some of you I know have used these it's got manifester and these resources will be online and it just search for manifesto and you can tell it's html5 compliant because it's lacking the e I just that's all Marg to modernize though by the way so what it does is it actually looks at go to a page hit the bookmarklet and it builds a manifest for you right so go look through and says okay well here are the style sheets I'll add them here all the JavaScript all that but I will look inside your style sheet and I'll fight only at imported style sheets right only follow up one level then if you add import adding points too bad right I'm look at that far you know I'm not gonna chase you that far down the rabbit hole but it will then look at all here's all these background images and we resolve the URLs and say okay what it doesn't do right now because of cross-origin I played around with stuff is can I verify the existence of these resources right because I like doing stuff in the browser I hate going any new service right if you can do something about you should it's Mike Fossum in life so what I would love to be able to do and I can do it for resource servers that have cross origin kind of allow me to the cause right I can basically test the existence those resources if they're there I can add it to the manifest say wait this one's there but there's also another really cool tool which his name escapes me but it's online it I've written quite a bit around this you can find it they'll actually just go and verify the manifest result anything so do that always verify your manifest so the tools are there for you to do this what this will do is actually just collect everything you are pointing to in your website including all the top-level HTML pages as well I'm not saying you should just go and then use that you've got to really think about what are the things you know suppose your website like there's got like 10,000 pages do you really want to kind of force the browsing I don't like you know prefetch all of those probably not but you might want to prefetch that the top level page reaching your five main sections either website because then when somebody goes from your main page to one of the top-level pages which is the most likely flow that's almost instantly going to low because it's being pre fetched in the background Google kind of do this a little bit now in chrome with the Google with the search results they actually prefetch the results in in a Google kind of homepage search I don't have to do in other browsers I certainly do it in Google crime so you know how you like Wow that loaded quickly they're doing something along these lines so we can do that too so you gotta think about how much to cash how much to cut not to catch because this you know like if you download paper resources many website you're putting a heavy load on your server you know pic times you may be impacting performance so you've got this trade-off performance on a client versus performance on the server and then you've got traffic issues and things like that right so you've got to start thinking about how you gonna use this technology but it's there for you to use so it's pretty cool so another the other thing that we need to do with we're offline and also will help with performance is you know we know the web is essentially a stateless environment and in the past basically the only way to get some sort of long-term continuum control over our data or all kind of store on the client we can do one of two things so we want to persist data over time for a particular client is we could just basically client server it right so everything gets stored on the server everything goes backwards or forwards it's you know how we've been doing it for a long time cookies to the rescue the idea is that cookies where any kind of clients live stories if they won't even invented that by the way we're just they're kind of a hack for this and people have been using cookies there's a heap of problems with cookies around there's that send in plane tags that they get sent to server every time they're actually in you know anyway so basically should avoid these as much as possible there's Hiba challenges around this right however we now have in most modern browsers as part of html5 the ability to store with a really simple API on the client in the client and there's two aspects this we have two things one is called session storage and one it's called local storage now they have almost exactly the same well they do their work at almost exactly the same way they just work in two slightly different contexts so pay attention for the first part of this around session storage because that will then cover off on local storage so the first question is what is a session all right so now html5 has this awesome concept of a top-level viewing context they've got that which is also known as a tab so technically it's a top-level so basically a tab or window right so basically a session is any any like top-level domain or any damn I like I got any protocol plus top-level domain plus what right and also any sub sub server right so basically while that is in one tab or window I'm let's use the words had just not to get confused that is a session alright so Elizabeth I met web Direction org and I travel from web directions org to follow a link to a question comm so they're two companies you know they're actually running on the same server physical server but they're totally different in terms of domain name right so we've left we're in the same tab we go we've left the session but what we follow will come back in any way we like you know back button whatever bookmark or follow a link back and we come back to that that original tab we are now back in the same session right so it's it's the same domain etc etc in the same tab however what's really interesting is if I am in web Direction dot org and I open a new tab with a link to go or a new window that goes to web directions or all these are now two different sessions alright so when I saw something here this window can't get hold of it in in session storage alright but if I go from here to a different domain and back if I stored something go away and come back the data is available again right it makes a lot of sense it's probably you'll probably spend too much time talking about it really but basically that's what a session is so the idea behind session storage is we store we save something the browser will store something for you only while that session is alive and then while that tab is there and as soon as the user closes it then that that's it's thrown away and is not it never be used again right the idea is that we can use sort of really sensitive a user for sensitive data it's a during the signup process during a during a banking process or something who want to store that in between kind of steps in a multi-step process we can keep it there and then we don't have to worry about purging that data getting rid of it worrying about something getting hold of it the browser takes care of all the details and I my in life I believe that browsers should do the hard stuff right particularly security which is really hard right ok there's one exception to this and this is a tricky exception it's worth keeping in mind if the browser crashes with a session open which never happens it's just when you've got hundred tabs open when you restart your you your browser and it reloads you all eight million because we all have a problem the session is allowed by the spec to come back to life and it does in all browsers since I Yates right so our ie8 which supported this stuff when when about when that have got reopened after crashing the session was dead in all other browsers it's alive right would you might say that's really awesome that's what I want except if someone's the internet cafe and I'm doing my banking and my browser crashes and I'll scratch they'll go away now doesn't matter and someone comes I can reopens it that banking session is still alive all right so that's a context we have to be I think very mindful of and it's actually really tricky to know how you as a developer could decide the session should be dead maybe you have to set some sort of timer or or something you know some other way to ensure that and there's no way to basically say I'm sorry your browser has died please you know when my brother dies don't restart all my sessions all right so it's something really worth give me in mind that session is back alive which sounds awesome except that if there's anything remotely sensitive in there you know you're building a male app or you're building a banking app should you know should you actually give in you session stories for that reason or what should you put in session so I I'm not gonna answer that for you but I think you should ask yourself that question alright so there you know this is well the world of gotchas out there but I mean the gotchas are therefore often for a good reason but they're worth not well they're important to know about right so what good is a session storage well the idea is sort of as I mentioned very briefly you know we can keep data that we only want to keep for a short period of time for one session you know might be a signing in might be signing on something you know some of it is moderately a kind of sensitive data there and we know that when that sessions over the browser will scrub it clean and get rid of it it's not our responsibility as a developer but as I said there's still those issues that you want to be mindful right so in realistic Levi's building any sort of financial app or something I don't think I'd use it unless I came up with a foolproof way of knowing that someone was resurrecting from a crash and then wiping the session so just something if any browser peer out there I hope you think about this all right but anyway I'm gonna talk about how you use it because in fact everything we can do with session storage we can do with local storage and local storage is like session stories but we persist data over time right so when some back you know to another you know days later months later years later it's all stored there and and mindful of fact that obviously you know there are privacy challenges there we could use it to our benefit I think so so whatever we see in session stories we'll also see with local storage so that's why I'm gonna cut off session storage so what we can do here firstly we its object on the window called session storage someone who's checked it for his existence you know in this case windows session storage yeah it exists otherwise we might fall back to cookie years or I might use server side solution or or might do nothing right so the first thing we noticed and and kind of practice let's check whether that object exists all right so desists so the first you want to do is we want to store stuff and we store stuff in a really simple way it's a simple key value pair right so the key and the value are both strings all right so what we're going to do here for example is the person they've used Timmy's wonderful connection or 5/4 form they filled in their name right so what we do is we going to grab the value of that name field that name input and then we're going to stick it in local storage using set item and that takes two two parameters name which is the the name of the key and then name which is a variable okay so so basically whatever is the value of that variable name we've stuck in an item called name all right very very straightforward rike am almost all would have seen kind of basic key value stuff right you can't really get a much simpler kind of API for sort of databases right so not much good storing if we can't get it back so in addition to setting items we can get items right so we set item with a name and a value and then we can get back the the value of an item so we say sort of session storage dot get item name and that then gives us that value in our name variable and we can do something with it all right you'll notice it's a synchronous API right just keep that in mind for a second all right so basically it's so it's very simple to use we don't need callbacks or we just simply like grab that value with sticking in a variable and now we have our value back right that's that's all there really is to it right so what are some little gotchas so the first thing is but we're gonna do something like checking whether so if no one's ever if we haven't created an item with a certain key value and we try to get it back we won't get back an empty string we'll get back null so we want to a check for that and only do something so this is something to keep in mind you get back null rather than error if I think of throws an error as well someone correct me on that but but basically you can just check on that I think it was a throw an error I should know this play around so it's just one of those things I've forgotten anyway I you could just check with it as a null value or not right it's frozen area if it's full definitely does the trier if that key doesn't exist in the first place is that in there as well like no is it null and undefined okay thanks for catching that one all right so I will change that back okay under fine but check whether it exists or base please be aware that you know they give off in most cases probably won't matter too much but there will in some places right so we've looked at how it was session storage while that tab is alive that data will be available to the windows session storage but only for that particular domain and as soon as that's had closes it's gone forever but of course we want to persist things over time right all sorts of variables and values and we can do that with local storage rather than Stesha stories but it's the API is identical right you can get inset items in the same way completely it's a window dot local storage get item as said I'm with key value pair all right but and these things are also available for session stores but probably more about more used in local sizes we can get rid of individual items we'll remove item remove item key we'll just get your throw away the key and the value right so it no long it doesn't set that that item won't be there anymore right it's not like setting it to an empty string it just won't be there at all and we can get rid of the entire local storage we can do this with secession stories as well and that would be really valuable in a crash situation we get to take to crash and then immediately throw out that session just local storage or session storage clear gets rid of the entire local database associated with that URL all right now keep in mind you might have multiple apps at the same URL potentially that are all trying to you know communicate with one another using kind of the database so you're gonna be mindful you've thrown that out for not just for your applicants anything at that URL that's trying to use that we know dot session storage but local storage is gone now well local storage no session all right so there's something just to be really mindful it's gone for good all right now more bunches I said these stores string value so you know everything gets converted to a string right so it means if you're doing a lot of complex stuff with objects and things you got a string of fire them or you know all right if you're working with floating-point numbers and dates or just kids can't we do either you convert them or get skits gets close to a string so you got to like worry about that either way the other thing is job suit strings what's the encoding then we're not it's a really geeky question yeah you did sixteen which means that every character is two bytes so suddenly everything takes twice as much storage right and giving you don't have unlimited storage as we'll see in a second you're going to be mindful of the fact that actually it takes up twice as much storage as you might think it does all right so there there's performance challenges I hope but other challenges around this right in the fact that everything is a string okay now in the spec basically says there's a visit there's a kind of recommendation but without somebody enforcing it that the the user should it's like the browser should dump no support five make of local storage all right for a given domain now think about that if you've got lots of apps running off a particular domain you know you could end up filling out pretty quickly especially since every byte is actually two byte so we actually only got two and a half Meg right straightaway does they take half of it away without trying right so there aren't people are doing some crafty things like they're kind of fum Aska fiying strings and my zipping strings in the browser and then stuffing a zip version but you see there performance implications there as well all right so something to really be mindful of there synchronous right you saw that which makes it really easy usually just getting set variables right but it means because they're pushing stuff into somewhere which is typically into a brown into memory rather than onto the disk you know if you've got a really large amount of data you're pushing into your cache it may be use just wait form that operation and this is a criticism you do see about local obsession storage over things like indexdb I've done a whole heap of research that demonstrates in most use cases it's not an issue even in a tight loop where you're writing thousands of things it's taking kind of tens of milliseconds to like a second so the performance is already pretty good but there are some debates around it you and you should be mindful of fact that it's blocking right so you might want to just have a whole lot of writes in a tight loop you can't just go and stick it in a worker because you know it's kind of a synchronous there's a bit of trouble there right I mean you know in most use cases this is not gonna be an issue in my experience right and you can find articles I've done online about this write a couple more things before we we wrap up very quickly so um how are we online online you know I talked to the beginning about the context I mean what I would be doing at he barflies stuff when we don't really need to or if we're storing the stuff in local storage and we suddenly come online and want to do a whole lot of synchronization you know with a server and pushing data up and bring it back down bring things up to date so what we can do is we can ask the Navigator whether the object is online really simple so I never get it online through a force or we can actually get some events they get fired when you go on an offline right so I'm gonna very quickly get through these so in this case what I'm doing is I can have some sort of like UI update where I asked and navigate it is online and then I find every in this case submit button and we just disable them if the user is offline because why would you want it enabled like what way we want to let them try to submit something when they're not online right that's that's a kind of really true to the example but it's something we can do but probably better for this is an event we can listen to so we can listen on well on a few objects but I'm gonna tell you you listen on the window object and you don't listen directly but add an event listener which of course we should all be doing anyway but for offline events and online events right different browsers import different this will work across all browsers but the other ways that are in the spec only work in some browsers this one works across all browsers so you can just basically have an event listener and call this this out there this some function when the user goes online or offline alright so calling so in function we saw before but instead of having to poll constantly we say we wake to the browser tells us so events awesome you know that event-driven programming we're finally kind of getting heads around I think so there's a heap of interesting events there very briefly five mega limitation plus well actually it's really only two and a half mega unless you do some really serious data munging and that's pretty much the standard across browsers you can't lie much more than that all right very very briefly if you want big our high performance stuff you've got two options you've got indexed DB which is supported in a number of browsers and you've got web SQL which is supported in a number of other browsers and there's no intersection though that the union of the set is kind of anyway so you basically there's some pain around this for the longest time it looked like well okay so Mozilla's never go to web web sequel so we forget about that and it looked like there was some real holdouts around indexeddb but chrome have got it now in developer at least I don't know if it's come and come into the kind of mainstream chrome yet but but so basically the holdout now really is Safari around this that they're simply asynchronous they store large amounts of kind of any sort of data you don't have too much data to and from so you performance so when you've got much more sophisticated data you see that kindles online html5 app uses both of these for different browsers you know that they're not just using local storage because they you know so once you're starting to sort large amounts of data having to rewrite it a lot of the time so when you've got sophisticated applications you're going to start looking at these and there's no drop in poly field you can't say okay well this thing will give us you know a cross-platform implementation of one of the API is there two very different api's so we're still a little way out probably until recently I thought we were never gonna get there I kind of feel in the next twelve months or so we'll see can't widespread adoption so ie10 has indexeddb locking on my right I'm pretty sure yeah you said with such conviction back so basically I think when you get there around index TV but the good thing is for most of our users you know this is really felt on our edge cases right now so I'm you know for most of our users local storage is really there for us so what's the what are we looking at here well we've got three technologies I looked at AB cache local storage session storage and offline events that can really give you some more high performance more responsive applications respond to the context of the user are they online and the offline the few little glaciers around online a heap of stuff there you know what does online mean means different things the different browsers in no cases that mean you actually have an internet connection it means you've got a network connection in some cases it means you're not in private not in offline mode in other browsers so I mean it's still not fully baked and there's some new html5 sort of standards around kind of all the way down to like this is how much bandwidth you have right now right which is what we really want to start seeing we can sum that stuff's already in PhoneGap Dave you've got like so when find out we're already seeing this stuff implemented it's being standardized it's in butcher gecko they're emerging standards so we you know and I think it matters most in the mobile context a lot of the time they're the device is going on and offline they're the ones on slower networks typically so this stuff's coming online they're not universally supported a are widely supported you really have to in order to use them you can't just kind of hope and pray you really have to get into them but they will give you I think a degree of sophistication in what you're building higher performance better UI a lot of better user experience all around so I really recommend you go out and explore them this is stuff you can really be using today I use it and in fact this whole thing you've seen here very briefly so I like to walk my own talk so this thing I built called speakeasy basically it's a everyone's got their own slide deck thing right except this one you actually build the slides in the slide deck thing it's all in the browser and it uses local storage and it uses all this sort of stuff and it works offline with app cache and so basically everything you're seeing there with the exception that background image is not an image and it's all CSS and html5 transitions and all that sort of good stuff but it also uses all offline stuff as well so you can build sophisticated stuff that works right across the spectrum with these technologies and I really encourage you to go have a look at it it's not pie in the sky it's working and with that I will sit down and collapse in a screaming heap and come and ask me questions later on thank you very much feedback you