(upbeat quirky music) - Hello, good morning, everybody.
I'm super excited to be here.
First time in Australia, actually, so first time in Sydney. Really great conference so far, so thank you for inviting me, and thank you for coming.
So before I start, I would like to do a casual poll. It seems like in Australia, it's pretty common nowadays. And I would like to ask-- (audience laughing) How many of you have tried VR? Ah, that's great.
That's amazing.
Great, awesome.
And what about AR? Any kind of AR, doesn't matter.
That's great, that's pretty...
It's really exciting to me to see so many hands that's trying these technologies because it's something that I've been passionate about since I was young actually.
It seems that it's something that is very recent, but since 1996 more or less, I started playing with this technology, so I did a Master's Degree in virtual reality in year 2000.
So I've been on and off in this space for a while, doing research and also professionally.
What are the goals of the presentation? What I'd like you to take, what I mean, I'm going to bombard you with information, actually, so I usually talk pretty fast even though English is not my first language.
I would like you to take away some concepts about VR. Maybe you have tried, a lot of you have tried VR, but maybe I will just show you some things that you didn't know, right? Hopefully.
And also, what can be done on the web? That's what I'm passionate about.
I really think that there are some very interesting opportunities to take away from the presentation and from the status of technology nowadays. But also I would like to do a little bit of forward thinking, a glimpse of the future, of what is coming, and what it could be, and what it ties to the web.
So virtual reality, it seems I don't have to explain it a lot, but just to level the knowledge of the room. What is it? Mainly the idea is that all the information that you are going to get when you are in a VR experience through your senses are gonna be generated by a computer. So it's going to be everything that you're, all the senses, and mainly right now at this stage, our visual and audio senses, the information that you're gonna get is gonna be generated by a computer. So that's a more or less, very casual, bar conversation explanation of what VR is. But maybe one thing that you don't know, even if you have tried is that, and this is not an official list or anything, it's just like Iker Jamardo's list of three levels of VR quality nowadays for consumers.
There are three levels of, in my opinion, three levels of quality of VR.
The first level is three degrees of freedom. Three degrees of freedom, what it means is that you have the freedom to move in three levels. In this case, it's three axes of orientation. You have pitch, you have yaw, and you have roll. So you can move in those three angles, so to speak. And it's what is commonly known as Cardboard. Cardboard was a Google project to try to commoditize the use of VR because actually VR is leaving this new era of coming into consumers and such because of mobile phones actually. Not because you can do VR on mobile phones but because mobile phones, they have improved resolution of the screens, they have improved some of the sensors that you have inside your devices, and those two things are the basics that you need to do this kind of basic AR that I'm calling, right? So what you do is you take your phone, you use the IMUs, the sensors for rotation, orientation, accelerometers and such that you have in your device, and then you use the screen to be able to render two images, put it in a headset with some lenses to magnify the experience and try to consume some content, visual and also audio through your cell phone. And the second level of VR that is out there for consumers is actually the same thing more or less. It's also using a cell phone so it's like why are you saying that this is a different level? And why there are these platforms called Daydream from Google and Oculus Mobile, and Samsung Gear VR from Samsung and Oculus? And the main reason here is that it's exactly the same idea, that's completely true, but there are some differences.
That's why I added in the parentheses some like fancy words. One is called reprojection, that's how we call it inside Google, and the other one is ATW, that is asynchronous time warp.
What it does is, just to summarise it, this is an engineering track so I'm gonna just be a little bit engineer-y.
So what they do is try to lower the latency. So you have to imagine that you are putting a screen in front of your eyes and it has, every little subtle movement that you do and every movement that you do the device has to respond as quick as possible.
So latency is very, very important and so you don't first puke, and second of all you believe that you are in that scene.
So these two technologies, they try to modify the drivers that the devices have to be able to get these technologies, these techniques, that they reduce the latency. Just to explain it very, very quickly is what they do is if you rotate your head and you are drawing images but your system for whatever reason is not fast enough to keep up, what they do is they take the last rendered image and they move it even though it's not going to be the actual final image that you're getting, it's just moving and it's very, very fast so you get the sensation that the system is responding as fast as it should.
So that's why this, if you try Cardboard and then you try Daydream or Gear VR you will definitely see the difference.
It will be way more smoother.
And the third level is what is called six degrees of freedom and in this case we are talking about, like you have the three degrees that we talked about and three more that it's positional tracking. So you can move up in Y, X, right, and Z.
And the most known platforms for those are HTC Vive, Oculus Rift, and PlayStation VR that actually are third technologies that they connect to a powerful computer or a console. There are some ways of doing it without being fettered but that's the usual case right now. Very interesting technologies, very high end experiences because definitely you are not tied to being seated and just moving your head side by side, but you can also move.
Plus you have other ways of interaction.
But let's talk about the web and the status of the web for VR in this case.
So this is a tweet that actually it's pretty interesting because it summarises how the whole thing started.
So this is Vladimir Vukićević that was an engineer in Firefox at Mozilla and he was playing around with the DK1 from Oculus, and he mentioned that oh, I got a version of a JavaScript API running on top of this system and it's running on Firefox, right, and then suddenly there's a reply from Brandon Jones that is a good friend of mine, an engineer inside Chrome and actually he's a chair defining WebVR actually for this specification, and he's just replying like oh that's cool, I'm gonna try it too in Chrome. And they started this way.
What they did was to start like showing prototypes that were not official prototypes so people could start building things on the web from day one.
So it's like definitely a very, a way of materialising the extensible web manifesto, right? Like just as soon as things are, let's just try to put them in the hands of developers even though they are very prototype-y. And it's important to know that WebVR, even though it's in the track of being a standard is still not an open standard.
So just to make clear, but definitely it's in the process, and it's been iterating, all the draughts they usually turn from one version to another, there was the version 1.0 that nowadays there's no browsers that I know that supports it. The most or the widest supported version is 1.1 and this is more or less the landscape that we have, so we have Chrome that supports it for Android on top of Daydream and Cardboard.
Windows not yet, but there are prototypes that you can use. It's coming.
Firefox for Windows, Edge.
Oculus mobile browser, they have a browser, a modification of Chromium that they, if you have a Gear VR this is only for Gear VR. When you go to the Oculus home and you have a browser there and that browser supports WebVR, so if you go to WebVR page you are able to like get into an immersed web based content. And Safari.
Well not yet, right, but the good news is that Apple announced that they are supporting Oculus and HTC Vibe on macOS so they joined the community group.
Most likely in the future they might or might not, you never know with Apple, support WebVR.
What does WebVR provide? So in a very high level it just allows you to know, oh, what are the devices that are connected to my system? Are there any HMDs, right, so to speak.
I can actually even ask what are the capabilities of those devices.
I can request that I want to start rendering information onto those devices, and actually if you have played with HTML5 you know that there is this interesting call back to synchronise with the refresh rate of the machine, that it's called request animation frame, so WebVR has their own request animation frame mainly because these devices, they have their own pipeline to do asynchronous time warp and the reprojection, buzz words that I mentioned before. So they need to have some kind of control over the rendering pipeline, right? So that's why you should use their request animation frame instead of the regular system request animation frame. And of course they allow you to get the pose. The pose is the position and orientation of the device. And some other information that is interesting, because each of the headsets, they have different lenses, different screens, so you need to adapt all the math craziness from 3D like projection matrices and all that kind of stuff, like viewports and such, to each of the devices.
So they provide you that information too.
So as you can see it's a pretty minimal API right now, 1.1 definitely at least, but it gives you everything that you need, like the basics that you need.
And there are some tools already.
So WebVR, even though it's not a standard it's a working spec, there are tools that you can use to build things nowadays.
Like one of the tools that actually we support is WebVR polyfill, and WebVR polyfill is an interesting tool that most of the WebVR developers use, mainly because what it does is if your browser doesn't support WebVR natively, like there are still browsers that they don't have it, what it does is just polyfills it only for three level experiences.
So if you have a smart phone with a browser like Safari, for example, that doesn't support WebVR natively, with the polyfill you can get kind of the experience of Cardboard, the first level that we mentioned, for any website.
So that's good, right? I mean at least you have an option for people that they don't have still a browser that supports full WebVR natively. But there are more tools.
We also support VRView.
VRView is interesting for website that they want to imbed some content, and in this case we narrow down to 360 images and videos, even if they are 3D stereoscopic. So the most demanded content apart from high end devices with gaming is consumption of media, so consumption of, you have a camera, there is 360 camera, like the Ricoh Theta for example, and you get a picture and you want to consume it in a browser, you want to embed it inside your blog post for example, you can definitely do it, and this is a simple tool that allows you to do that.
So it's very, very narrow use case but it's very useful and a lot of people are using it. It does everything for you.
It has the polyfill and all the kinds of things that it needs, it's in there.
And there is A-Frame actually.
A-Frame is a very, very interesting tool that is built on top.
So we have the WebVR API here, then we will have some kind of like 3D rendering engine on top that most likely is three.js if you know about it, it's like an open source widespread 3D engine, and A-Frame sits actually even on top of that. So it's like a third layer that you can have. And what it does is commoditizes the building of these experiences using DOM. Instead of having to go imperative and type everything in JavaScript you can use a lot of components that the community has been building, you can even build your own of course but then you have to go imperative, but if you are okay with every component that is out there you can build your experiences with only DOM and just parameterize in some tags in the browser, in your application.
This is just an example of one of the applications that they have built on top of A-Frame that is actually pretty cool.
If you have played any of the high end 6DoF headsets that I mentioned before, one of the usual applications, because it's pretty well known and it has won a lot of prizes and such, not just because it's maintained by Google or not, is Tilt Brush, and it's a way of being able to draw in space, and whoops, that was weird, sorry. What we have here is just showing you that all of this is running inside a browser and in this case it's Mozilla Firefox, and they have built all this that you can see using DOM and a lot of JavaScript underneath of course. You can definitely build quite complex experiences as you can see.
There are a lot of WebVR, well a lot of, I would say that there are more than you would expect for like the state of the technology nowadays and the support of the browsers, websites that they support WebVR already, and these are some URLs that you can go to. WebVR.info, it's like the basics for learning what the spec is and some of the basic examples, where to start if you really want to go WebVR low level, but there is also a couple of websites, one from Google and one from Mozilla with a lot of experiences, and here is a video of one of the experiences that actually I have a device here if someone wants to play around with it afterwards, more than happy to, grab me afterwards and I can show it to them. At the release of a new album LCD Soundsystem, they created this website along with Google and some other companies to promote their album using WebVR, and actually a lot of things that are related to this talk will be explained in this video. ("Dance Tonite" by LCD Soundsystem) ♪ Everybody sing the same song ♪ ♪ It goes tonight, tonight, tonight, tonight, tonight ♪ ♪ I never realised these artists thought so much ♪ - Dance tonight is an ever-changing VR experience made by LCD Soundsystem and fans. ♪ What remains of the airwaves ♪ - I really enjoyed it.
Like it was like, I didn't expect to enjoy it. I thought kind of like, I was like...
This is not for me and it was for me.
I don't know, I think it's really neat.
- So there's this tool that musicians use that's called a loop pedal and they use this to kind of like iteratively build up a track so every loop they all play something and it loops back again and they will add another layer, and then all these layers together, they create the track.
And we had this idea of doing the same thing with motion capture, so we'll build up a loop of movements, and then we'll loop back around and we'll make a duplicate of you and we just keep adding like these versions of you into the room.
- When you're making the little videos you would be able to loop and it immediately just happens again and you have to move over and do the next guy, be the next guy.
You have to be the guy, or the gal.
- Yeah, you could say technology by nature is very structured and very rational, and human beings and the world around us is by nature chaotic and is not so easily to define or to structure, and I think that the system that we made really combines these worlds in an elegant way.
- And we really enjoy with these performances that they're not perfect.
Like people make mistakes, people are nervous. Like you feel a hesitation in their movement. - We wanted to make this experience accessible to everyone. If you're on your computer or if you're on your phone you can kind of watch as if you're in the audience.
If you have access to a VR device like a Daydream or a Cardboard you can watch as if you're on stage right next to the performers, and if you have access to a room scale device then you can be a performer and you can add your dance and your movement to the piece.
♪ And what's it you do again ♪ ♪ Oh I'm a reminder ♪ - WebVR is an open web standard that brings VR to the web. On the developer side it's really nice because it allows you to code just one project and have that kind of roll out in these different ways, so it just makes for a much easier creation, more iterative process. - Virtual reality is this, like I don't get it in the big picture, but it's sort of like going to the beach.
If somebody says like going to the beach I'm like I want to go to the beach.
Then you go to the beach, you swim around, you hang out, and you have a day off and it feels kinda great. I think it's more like that to me.
♪ That's all lies ♪ - [Jeff] Normally if you want to experience something in VR you download a giant app, but with Dance Tonite you'll just click a link. - Yeah, so as you can see it summarises a lot of the concepts regarding to the web and what is important.
It's pretty funny to see people with the headsets like doing crazy stuff, but for them it's definitely real, right? Like they're in that environment.
So interesting takeaways from this one is that as you can see it adapts to the device that you are using, so if you don't have a VR headset it doesn't matter, you can consume it somehow. It's not gonna be the full experience but you can consume it.
If you have a headset you can definitely consume it as a viewer and then if you even have a high end device you can even create new content and you can upload it. And of course there's no downloads, it's completely shareable, your creations are stored in the cloud and then people can consume it afterwards and such.
And now let's focus a little bit on AR.
What is augmented reality? So augmented reality for me is another side of the same coin.
So they are completely related to VR.
So just imagine that you have, in one extreme you have VR where everything is computer generated and in the other side you have reality that is, everything is real, right, what we are seeing right now, what we think that is real. And if you start sliding and defining how much of the reality you exposed from the virtual content or the other way around, how much virtual content do you expose, that's where you have, in between you have augmented reality.
It's over-composing digital content on top of the reality somehow.
That's like a really quick explanation.
And there are two ways of doing that right now. Even for VR we don't consider just a smartphone like this completely VR because you still see the whole reality, but for augmented reality we have what is called a smartphone based augmented reality, and we call this metaphor magic window because it's like a window to a magic world, right, that you are seeing, and the two major platforms right now, they are called ARCore and ARKit.
But there is also headsets, right, and it's important to make this distinction because there are a lot of commonalities but there are some other differentiations between them, and that is very important, and especially for the web it's going to be critical because we have to support all of them.
You can imagine with these headsets it's actually, you are not seeing a screen so you are seeing reality, we call them see through devices, and what you do is just like lay some digital information on top, like a little bit of a projection on top.
And that definitely has some implication in the rendering pipeline and such.
What about WebAR? So there is no such thing even as a WebAR spec, so if WebVR is not even an open standard but there is a spec list, for WebAR there is nothing.
It's like a made up concept but it's pretty convenient because we can talk about augmented reality on the web.
That's yet.
So maybe in the near future there will be some changes there.
But is it possible to do WebAR or like augmented reality on the web with the current stack of technologies? Yes, but mainly for a smartphone based AR, so the ones that are using the magic window concept where you are using a camera and you are rendering the camera on a screen. Because we have some APIs on the current stack like Media Capture and Streams, that allows you to access the camera and then you can do some understanding of the world to be able to do AR. And here are some examples.
Just having the camera is not augmented reality. You need to overlay some information on top and the device needs to know something about the surroundings, at least it needs to know its pose. That's the basics of even VR or AR.
So you can get your cell phone and just put it on your head and just move it around but if the device doesn't know where you are and where you are moving and it doesn't respond to it it's not VR.
The same thing happens with AR.
A technique that has been out there for a lot of years is called, identifying some specifics that the camera is capturing, in this case they're called markers, fiducial markers, and these markers they look a little bit ugly but definitely the good thing is that they have some properties with simple computer vision algorithms you can determine the relative position of the marker from the camera, and you can do things like this.
(upbeat happy music) Well as you can get the idea, the marker is telling you where the phone is relative to the marker, or vice versa, and then you can lay information on top.
Of course the marker has to be in the perspective of the camera at all times to have that tracking.
There are actually some other ways of overcoming that ugliness of those markers but just keeping up with the same idea of the technology, and actually there is a company here in Australia that is doing image based marker recognition, among other stuff.
So you can see here one of their videos.
Actually they are here and they will be talking later today. (energetic electronic music) So as you can see you can do this with your current browser right now in iOS or even in Android.
It's great.
Okay.
But those were identifying specific images that you place in front of the camera, but there is also another type of computer vision algorithms that you can use that it's recognising some other type of markers, in this case faces, right? And there are some opportunities there with the current web stack, specifically for advertising. There's this other company that is using this tracking.js framework.
(bright lively music) So as you can see there are interesting use cases, still very limited, but at least you can do it with the current web stack and you can share it directly, you have all the benefits of the web. Some other examples here, actually the first one is interesting and you can play around with it. It has face deformation, for example, that is pretty funny. And some other libraries like OpenCV.js that is a very well known computer vision library that has been imported to the web, and you can definitely use these libraries to get some kind of scene understanding and tracking from your scene using, then again, the current web stack.
But what about WebAR of tomorrow? What about this, especially if you have heard the news, if you have been following the news, Apple announced ARKit that is an augmented reality framework.
Google has been announcing Tango that is an augmented reality spec, hardware and so forth spec for OEMs to support and there are some devices out there in the market, and also ARCore was announced. And what about those technologies? Are we able to use those in the web right now? And the answer is not officially, but there are some possible solutions.
Before going there, just really quick, I'm throwing like buzz words all day, something that I think is interesting to learn about is the difference between what we have seen thus far, marker based and facial recognition based augmented reality, and what these platforms allow is to do tracking without the need of specific markers, so they use the world as a marker to be able to know what the device is. And the technique that they use is called visual inertial odometry, it's also called inside out tracking. The VR headsets that we used all before, they are doing tracking because they usually use some devices that are out there, like cameras or like Lighthouses they are called. In this case we are using the whole embedded device to do all the tracking, so you don't need anything from outside but just the world. And there are a couple of very interesting articles from Matt Miesnieks that I definitely recommend you to read. So if you just go for Matt Miesnieks and VIO you will definitely get those articles, and they explain a lot of things about like ARKit and ARCore and how they work. So what we have been doing inside Google to try to expose this, so what we did is we open sourced some prototypes, like the same way as, just imagine we are at the beginning of the conception of WebVR three years and a half ago, we are trying to do that for AR too.
And what we have done is we have exposed some of these functionalities on top of WebVR 1.1. As I said before, there are a lot of commonalities between AR and VR so it made a lot of sense to us just to stand on the shoulders of giants and reuse everything that has been done for VR to expose some capabilities of AR.
And these products are called WebARonARCore, really a mouthful but, WebARonARKit, WebARonTango, and we also released a library called three.ar.js on top of three.js to be able to commoditize some of the use of this API and make things a little bit easier for developers.
So we have some examples there.
You can just play around with them, but you need these specialised browsers.
You cannot go to Chrome right now and use these technologies.
You need to go to a specific APK or app that you have to download.
Actually even in the case for ARKit you have to build it yourself.
So this is open source.
You download the S-CODE project and then you have to build it yourself for iOS for now. So very prototype-y, very early stage, but they enable very interesting functionality. So what are the things that we have been exposing on top of everything that I listed before from WebVR 1.1? We exposed the camera feed, we synchronised that camera feed with the pose that you are getting, we get some scene understanding.
We're exposing right now three elements, the concept of HitTest that is a really high level abstraction to be able to, I don't know what the device is understanding about the world but I'm gonna throw a ray from the device, right, a virtual ray, and then you are going to tell me, the system's gonna tell me if it hits something in the reality. So just imagine that for example, in the case of Tango that is able to recognise a lot of things from the environment, you throw the ray and it will just hit against this real object and it will just tell me that position there, like the position, even the orientation.
So that is something that we expose.
In the case on ARKit, on ARCore they expose planes right now, so they don't have specific hardware to identify more information. So what they do is they try to recognise horizontal planes for now, so that's at least the basics that you need to just put an object on the floor or on top of a table, and we're exposing those planes in JavaScript, and we are also exposing a point cloud, right? That is something that in this case only Tango provides. Tango is a specialised hardware for AR that has an infrared camera.
An infrared camera is just, well it has an infrared metre, the infrared metre sends some light, it bounces back and then the infrared camera captures it and it can understand the depth of things.
It's similar to Kinect if you have played around a little bit with it or similar to what the security cameras usually do but they don't get the depth.
You get more or less the idea, but nobody else sees those infrared light rays. We are also exposing anchors.
Anchors is an interesting concept.
I am not gonna go very deep into it but anchors, what allows you to say is like, look visual inertial odometry system, I know that you are going to start moving and you are going to correct yourself, so I'm going to put a virtual object here and if you correct yourself you have to notify me because if you correct yourself I need to correct my object too.
So that's the concept of specifying I want an anchor here in the world and I want that to be something significant for your system.
And it's an important concept that you have to like use when you are using these kind of technologies nowadays.
I know, a lot of explanation, really high level explanation. If someone wants to know a little bit more about it I'm more than happy to respond about it. If you just look around about it you will find a lot of information.
And also we're exposing a lot of functionalities like markers, so this is a little bit, sometimes it sounds a little bit weird because we were trying to get away from markers and suddenly we are exposing markers.
Markers are a great way of doing a lot of, and enabling a lot of use cases.
Like for example you can do tracking but suddenly if you identify a marker you can launch the experience there.
You can use a marker to synchronise multiple devices at the same time.
So you can do a lot of things with markers, and especially if we are exposing, in this case, with our print work we're exposing QR codes, and QR codes can embed information in it. Like for example URLs, very interesting for the web. So markers, definitely interesting.
So we're exposing all those features.
Just one note also, like because we're on an engineering track, for the ARKit we're doing a lot of hacking here.
So for ARCore and Tango we have full-fledged Chromium builds, so it's a full browser with everything in it to do AR.
For iOS we cannot build a whole browser because Apple doesn't allow us, so what we are doing is we are cheating and what we are doing is we're using a WebView that is just a component that is in every operating system nowadays and we are putting it on top of the actual native implementation of ARKit and we are making that WebView transparent. So you can see the camera underneath and you can put the WebView on top.
Just to let you know that we are cheating there a little bit.
But everybody's doing that so we're not the only ones, because it's limitations of the platform, not the limitations of the technology. And we released these projects actually at Google I/O, and I'm gonna go a little bit nuts and I'm gonna defy the demo gods and I'm gonna do a live demo here and show you the prototype that we released at Google I/O this year.
So this is a prototype exposing a Chromium build with AR capabilities on top of Tango, and let's see it. If it works.
It worked when we were...
Okay, there you go.
So what this prototype allows you to do is we connect to the Wayfair, Wayfair is a US online Ikea so to speak with a lot of 3D models actually and they have an open API where you can access the content that they have, and what we did is just build a, this is a website, so you have like the regular elements that you would have in a website, but if you press on this space search what it's doing is, in this case we are using the HitTest element that we mentioned before and we are identifying that there's a floor there, so the system is able to identify it. This floor is not great for this but so what I'm gonna do now is I'm gonna define a volume here and what it's going to do is it's going to filter the elements that they fit inside that volume and in this case we can, even I'm going to use my weird English accent, but we also have a speech recognition API on top of the web stack, table. Oh good, it worked.
So it just shows some tables and then you can just select one of them, it just loads it, and you can just see there, and see how it fits, and you can just rotate it and just go for another model and so on and so forth.
So this works actually on ARCore, on ARKit, and on Tango. So with our prototype, specific prototypes. See if it'll go back.
Good.
With Tango we can do a lot of things.
Like for example, occlusion.
Occlusion is something that is pretty complicated with the current state of AR but using these IR camera that I mentioned before you can do things like this, you can occlude a virtual object underneath a real object.
This is another example just to show how you can get the whole point cloud.
This is only Tango specific done again.
On ARKit and ARCore what you can do is just you can acquire horizontal planes for now, and these are some other examples built on top of the web that we were doing. This is outside of our office and one of our colleagues' house, and you can do these kind of experiences that you are just getting into a different dimension on a portal and that's still the reality that you are seeing. Some other examples here of some other builds that we did, so I'm kind of like playing around with how interesting it is to have six stuff to be able to draw in space, have controls.
Easy transitions between a 2D page to an augmented reality experience.
This is actually using CSS 3D transformation so you don't have to use complex WebGL if you don't want to, and even the community has been building some very interesting use cases. These are, we didn't build these examples actually. They were built by the community.
Another example from the community, in this case a company took the Tango device and mapped the whole office that they had and they sent all the information to their servers and they have their own office mapped, and this is actually React VR that is a framework in JavaScript to be able to build things in WebVR that they implemented when we released it, they implemented their Hello World in AR.
And then those were mainly on top of Tango but we also, with the same API we exposed ARKit and ARCore, so you can see an iPod in one side and you can see a pixel device on the other side just running AR experiences. And now where we are is like we have all these technologies and we are still exposing a little bit more features.
What we are very interested in is like knowing what you can build on top of it.
What are the new interactions that you can do? And this is an example of how to transition from a 2D website that looks like Wikipedia for example or a blog post into an AR to be able to see the model in real size, like so you can see and you can go with a very, very intuitive way of moving around and see the details of that model. So there are some other experiments out there that are worth mentioning.
There is, specifically for iOS because it's a little bit easier, that hack that I mentioned before. There are these two projects, one for Mozilla and one from another company that they built some AR exposure in JavaScript.
The only thing is that they are not using exactly the same API that we are using. They are using something else.
As you can see there's no standard so everybody does whatever they want.
So us included.
But what is next? One of the ideas that has been playing around is to use the term WebXR, and WebXR is something that it could be the intersection of AR and VR in this case or even more technology.
WebVR now is in the works for like the version 2.0 and actually the community has been thinking about what are the needs for AR inside the WebVR 2.0.
So the approach that we have building on top of WebVR, it seems like the community is receiving it pretty well and they are trying to define some things.
Like for example, coordinate systems.
The concept, this is very low level but it's like being able to understand that you are, when you are here you have a coordinate system, when you walk away and you just go like three blocks farther you will have a different coordinate system because the tracking is not gonna be good enough to keep up and be accurate enough. So you need different anchoring and defining different coordinate systems, and that is being discussed inside the WebVR 2.0.
Especially pushed by Microsoft, that they have this HoloLens, very interesting AR device. And also there's support for for magic window. WebVR 1.1 didn't have the explicit support for the magic window idea of like using a device to just move around, so WebVR 2.0 is gonna explicitly understand that concept and that use case so it's very, very interesting.
We are working to create an actual spec.
So we need that, and we definitely will be working on adding more things to our prototypes that you saw before, and we want to experiment more.
And we want the community to experiment more. So we are actually, that's the main reason why we released those prototypes.
And that was pretty much everything that I wanted to talk about.
A lot of information, I know.
But before I go I wanted to play a little game and it's just a game of like forward looking and having a little bit of imagination of the future. So we may not everybody agree but for the sake of this game let's agree that in the future we will have some kind of device, it could be a contact lens, or a headset, or some kind of like very lightweight glasses that allows us to access all these AR capabilities in a very simple way.
And maybe even go to VR if we want to.
Just occlude the whole vision and just go to a different place or different environment. But let's go for like the AR use case.
So we have those glasses and we are most likely going to access a lot of information. So we will get some information that are similar to our cell phones right now, information that we are interested in only ourselves and we don't want to, we just want that information to be displayed, but also we would like to maybe, as we are doing with cell phones nowadays, when we are walking down the street maybe some information will pop up.
Like when we are passing by a restaurant or there's a new offer or something that is on sale when we are looking for something specific. So we want that information.
That information may just pop up in our vision. So if we're imagining the future what are the things that we need to build that technology stack? So definitely there will be massive amounts of content. Like we are talking about billions of nodes because we are talking about like the whole world. It could be here in Sydney, or in San Francisco, or in Spain, in my home country.
It has to be taking information from multiple sources and it has to be simultaneously. I mean I can just get information from this local store or I can even get some information from a weather service that is just telling me that in this specific area at Sydney the current weather is this, and this is the forecast for the future.
So multiple sources.
It has to be instantaneous.
I don't want to download an app every time I'm walking down the street because there is something interesting to me.
And it has to be serendipitous.
I don't know where I'm going to be.
It can be anywhere.
And it has to be definitely safe and secure because we are exposing a lot of things about ourselves. Just imagine that the demo that I showed you was knowing that there was some kind of floor here and actually if I was pointing it to something like a chair it would just know the chair, and if I'm pointing it to a person it will know that there is a silhouette of a person, so we're exposing a lot of information.
It has to be safe and secure.
And it has to be interoperable.
What does that mean? That maybe I have the fancy Apple glasses or Google glasses or whatever I have in the future that are $1,000 worth, but maybe I'm just with my old cell phone and I want to consume that information, or even in a laptop somewhere, and I would need to access that futuristic world of information.
I don't want anybody to be left out.
So I don't know about you but that sounds like the web to me, like the web is interoperable, you can access different sources of information, it's serendipitous and ubiquitous, and it's definitely secure. Thank you.
(audience applauding) (upbeat quirky music)
