- Mark's talked a lot in theory, and in many ways practise, how we can build and what we can build in a virtual reality world. We're gonna hear about some very very practical applications of this technology that you can take back into your workplace, to your clients, today, the sort of things that can give you very actual ROI. It should also be quite a lot of fun as well. So Aaron Spence has been running Panedia for 20 years, has been doing a lot of this sort of work for many many years.
He has content on Google's, Google Earth platform, if you're not familiar with it, well we're all familiar with that, but in fact there's an enormous amount of 360 content there. And he's done work for any number of tourism organisations, the Municipality of Monaco, and all sorts of other folks there.
He's done work here with Qantas and others in very practical, practical kind of actionable VR stuff. And we're gonna learn a little bit more about that today. So would you please welcome Aaron Spence.
(audience applause) - Thank you, thank you, am I on, everyone can hear me well? (clears throat) Excuse me, sorry.
So I think we've done maybe 50 or 100 VR demos out there in our little stand there.
How many, can I see a show of hands, who's actually come and had a go? Okay that's a lot of people, I think we're done. Okay, thank you. (audience laughter)
So we're gonna do some stuff here, some stuff that you haven't necessarily seen. This presentation's gonna be a bit of Google slides, a bit of web, and a bit of VR.
So it's very multimedia.
And we'll see how it goes.
I had a different tack on this and just in the last few hours and last night I decided to change it round a bit and make it a little bit more free form and just really answer the questions that everyone's been asking over the last few days. So I'll just say very quickly who we are, we got a good intro there.
That's Carlos, my general manager of Panedia. Sorry, I'm Aaron Spence, I'm the CEO of Panedia, which started about 10 years ago.
I've been producing 360 content for about 15 years and have been a professional photographer since the 80s, so a very long time.
Carlos, my GM, has almost the exact same story, but he was doing all of the same things in Portugal and then a few years ago he came across to Australia and now we continue doing this with a team of about 10 people.
So, just very quickly, we work, we developed the technology and we do all the post production in all the systems for Google Underwater Street View. You would have heard of Street View and probably a lot of you have seen Underwater Street View, so that's the first prototype device in 2012. So we developed all the photo technology and we do actually all the production from the work that's shot around the world in our offices up in Queensland.
That's a little bit later on, the more advanced devices, and that one's human powered, and up above there you can see we've got the Google Trekker. So we do work together directly with Google sometimes but most of the time our people, or our partners are out shooting that content, and then we give it to Google once we've done all the work on it.
In addition to that that content goes on the Global Reef Record, the URLs there, globalreefrecord.org.
That's a project with the University of Queensland, so there's hundreds of thousands of 360 photos which cover thousands of kilometres of coral reefs around the world.
And a fantastic project, I haven't got time to go into it. We also put our on content onto Google Maps, Google Street View.
Our content has been viewed about 300 million times on Google Maps, and when combined with our partners content, we've had about 600 million views. That's all of our above water content, so some good things happening there.
And then a couple of years ago we've been dabbling with 360 video for a number of years, or quite a few years, but in 2014 we got more serious about it and started doing some real projects, like this one for Olympus to show off an operating theatre. And some of you guys who have done the VR would have see a little bit of that.
We won't go into it tonight but that was our first, or today I should say, but that was our first real project. And what I'm gonna show today are all real projects that we've done for real clients and they're actually using them.
What I tend to find in most conferences I go to is when people are talking about VR they're often talking about what you could do with VR and what VR might be useful for, but this is what we're actually doing with real clients. So, yeah.
VR terminology, it's important, it's a highly contentious issue, Carlos and I were having an argument about it just two days ago, about all the terms in VR. And so I'm not gonna say I'm the authority on terminology but this is what we use.
This isn't a new thing this terminology problem, it's gone on for at least 15 years.
So I'm not sure it's gonna be solved any time soon but for our purposes the VR we produce is what I consider live action VR.
Not gonna do that anymore.
Live action VR.
So we film with real cameras and we produce what's technically 360 video, and then that can be viewed on the web and all sorts of places in the Oculus Rift, on the Gear VR. So that's what we're calling what we do is live action VR as opposed to what I would consider VR, which is more computer generated.
So games, simulations, some of the stuff that Mark was talking about on the web, that's all computer generated content and VR. And importantly I think that content has six degrees of freedom, which means you can move anywhere within that space. You've got the freedom to do what you want. With live action VR you don't have that freedom. Because it's been filmed you can only go where the content has been filmed.
So that's one of the major distinctions between those formats.
The other thing is that VR, in terms of computer graphics, can be fully interactive, you can interact with your environment.
And again with 360 video, or live action VR, you really can't interact with the environment unless you bring in 3D elements.
So it's really not interactive in that respect but I'll show you some stuff we're doing, that allows a level of interaction, later on. AR, augmented reality, Google Jump obviously, everyone's familiar with that, Google Jump. Pokemon Go, I don't know where Google Jump came from, my apologies. (audience laughter)
Google on the brain.
Google Jump is a device that has 16 GoPros in it and it films live action VR.
Augmented reality is overlaying data onto your real world view.
Pokemon Go, perfect example of that, and there's plenty of others.
And then this is where it gets really contentious is that companies are, companies obviously like to grab at terminology and claim it as their own and try and make it the standard use of that term. So Microsoft has coined mixed reality, with apologies to Mark for the HoloLens.
And so what they're saying is that mixed reality is overlaying holograms on your real world view. I mean, are they holograms, are they not, that's contentious.
I'm not a big fan of that but it's an acronym that's being thrown around now.
So if you hear it in general talk about AR, VR, MR, that's what they're talking about, is mixed reality and they're saying it's a more advanced form of augmented reality.
I'm not saying that's the case I'm just saying that's a term you may hear. So that's the terminology, this is the master format of VR.
This is what's called an equirectangular image. It's 360 degrees wide by 180 degrees high.
So if you take a sphere and you unwrap it that's what you have, in fact I think this one we've cropped a little bit off the bottom, but it's basically the format.
This is the Brooklyn Bridge in New York which is a really good example to show you the curvature and the distortions and all the crazy stuff. So when you first see this sort of content it's a bit mad. Carlos and I and our team have been working with this content for a very long time, so to us, this looks like a normal picture. But that's an equirectangular master and you'll see on the screen out there, the edit tutorial out there on Adobe Premiere, we're editing, when we're editing the video, the video format is in equirectangular format, so it's distorted like this.
Once this gets played in some format, web, VR, wherever it's gonna be played, it changes to something that's, what's called rectilinear perspective, or normal perspective.
And I'll give you a quick look at that, we'll just switch across to the web.
So this is that same image in our web player. So, as you can see, the straight lines are now straight. Obviously you can see all the way up, you can see all the way down.
Touchpads with this sort of stuff are terrible. Yeah, so you can have that experience.
Ooh there we go, I can use the arrows, I never do that. So that's what an equirectangular image looks like once it's converted for you to look at as a 360 photo. Let's switch back to, here we go, so that's that. And then this is an example, just a quick example, is that, when you, you can still edit and do things with 360 video like you do with normal video, there's a lot of considerations. But here's an example of a project we did with Carnival Cruises.
The day that was picked for the filming, which was not optional, could not be renegotiated or changed, was truly terrible, as you'd expect. And so we've had to go in and make it look like a decent day and there's a quick animated GIF of that.
Most of you wouldn't be familiar with animated GIFs, it's advanced technology and we won't get into that (audience laughter) at this point in time.
In fact they used to be called CompuServe GIFs is that correct, once in the olden days? No.
- [Man] JIFs.
- [Aaron] JIFs, sorry, is that really, is it really, JIF? - [Man] No I don't know.
- [Aaron] Okay.
- [Man] I don't know that.
- [Aaron] Okay, so let's have a quick, let's have Carlos up on stage.
Everyone say hello to Carlos my lovely assistant. (audience applause) So what we're gonna do is, let's see, no we don't want that, we'll just exit out of this. Sorry this is where we've got a, mixed reality, MR, presentation.
So let's see if we can get this working.
You guys can see that? Excellent.
- [Woman] The Compassion Project worker came, And she got me out of there.
(bright upbeat music) - Hi, I'm Oliver Clarke from Olympus.
And welcome to our ENDOALPHA 3D OR.
What you're looking at now is the Olympus ENDOEYE 3D FLEX, which is the world's first and only flexible 3D laparoscopic camera.
(bright upbeat music) - [Aaron] That huge mall is on a ship.
That's about a 15 story high atrium.
We'll go into the Qantas stuff a little bit, a little bit more in a minute.
A couple enjoying some privacy in a spa with a 360 rig right there next to them.
(audience laughter) It's the future, everywhere you'll go there'll be someone with 360 something filming everything you're doing.
(bright upbeat music) Okay.
So that's actually what's showing outside at the moment, if anyone wants to see the full version, or experience the full version of that, they can do that later on.
So that's the sort of work we're doing, I apologise that the audio wasn't working at the start, but these are some of the clients that we're producing 360 video for, or live action VR. And they're using that content in real ways. And that's playing again, let me just stop that. Okay.
So, they're using that content now, and they're doing all sorts of useful things with it. But this, this is the money shot if you like, this is why you're here to listen to my talk in particular today.
It's what we counsel our clients on as a VR distribution strategy.
So if you've got clients that you're bringing to you, if you're doing trade shows, if you are doing sales meetings, if you are in any situation where you have control and you're bringing people to you, then at this point in time for the live action VR the best device you can have is the Samsung Gear VR. It's a phone, a non-exploding Samsung phone, this particular one, (audience laughter) and the device.
The devices are about $150, the phones, there's about six different phones you can use, probably start at about $800, and a set of headphones. We recommend Bluetooth headphones 'cause then you've got a totally wireless solution. You almost always see people with wires hanging around and plugging them in, which is for me is crazy talk when you've got all this wireless gear.
That's the best experience that you can have right now. Oculus Rift is also fantastic but it is obviously a wired solution and you need a very expensive piece of hardware to run that. And Cardboard that you'll be familiar with there is also VR, live action VR, and real VR, but the experience is nowhere near as good as these dedicated devices.
So the strategy is this.
If you're bringing people to you you give them them the best experience you can, which is with this type of device, with the Google Daydream which is coming and launched as of a day or so, those type of devices do a fantastic job.
That same content, depending on your strategy, can then go onto YouTube and onto Facebook and get wide distribution so millions of people can see it. And then, depending upon the situation, you can then take that same content and make it available on Google on Cardboard, you can see those ones we've made for Monaco there, and those can be handed out, so people can have that experience on any sort of device. Is there anyone here who has an Apple device? No, no, I didn't think so.
There's a sea of Apples here, I'm actually quite hungry. But Apple, unfortunately, are just not really doing anything in VR at the moment.
We expected them to with the iPhone 7 launch a few weeks back but absolutely nothing.
So we're gonna have to wait to probably the iPhone 8 before we get to VR.
So at the moment an Apple device will work in Google Cardboard but it won't work in of these sort of more advanced systems. And so therefore you can't get that sort of immersive experience, the sort of presence that you get with these devices. The other issue of course with Apple devices is that the screens are lower res than the Samsung, so they're more pixelated even if you do do a VR experience with them.
So distribution strategy.
Hi quality, bringing people to you, put it out through the web, Facebook, You Tube, you can put it on your own website.
And Cardboard, in terms of being able to give away stuff that people can use with their own devices. In terms of, in terms of, actually I didn't want to do that, oh that'll do. Let's go to the live web.
In terms of views 360 content's very popular on Facebook at the moment, they're pushing it a lot. So this particular one that we did for Princess Cruises has been viewed, you may not be able to see that, 1.1 million times.
250 likes, 99 comments, 430 shares.
And so that same content that we were looking at a moment ago or a version of that content just runs in your browser.
If you load that up in the Facebook app on your phone you can do the thing where you look around with the gyro on the phone.
So it's certainly not this experience but it's still a valid and useful VR experience and it gets the biggest number of views, which is, depending on your client, is the metric that they care about.
That is distribution of VR content, live action VR content 101.
The devices we talked about, Gear VR, Oculus Rift, the Vive, the HTC Vive is very similar to the Oculus Rift, again it needs a computer.
The PlayStation VR launched a couple of weeks ago, it's a device that's like this but plugs into a PlayStation 4 or the new PlayStation 4 Pro, which has just been, or just about to be, launched. And then you've got on the end there Google Daydream which is Google's play in the Oculus, Samsung VR space. Just to give you an idea, this is Oculus software on a Samsung device.
Oculus of course is owned by Facebook, Facebook bought Oculus a couple of years ago for $2 billion. So essentially this is Oculus's play in the mobile VR space at the moment.
The Google Daydream is Google's play in that same space. And so that's, this stuff literally changes weekly and you need to pretty much discount, I would say 90% of all the announcements that you hear concerning VR, AR, MR, as massively overindulgent, and just look at what's available and has been available for a while.
A lot of people are announcing a lot of gear, most of it doesn't happen.
In terms of production, people are wondering, "How do you do this stuff?" It's multi camera arrays basically to shoot live action VR. So it's a bunch of cameras facing all different directions, filming at the same time. That content gets stitched together into that equirectangular format we looked at earlier, and then you've got VR once you play it back on some sort of device.
Here we've got a picture of the, it's that Samsung rig there, it's a custom rig we designed which produces really high quality.
You've got GoPro rigs, you've got a lot of stuff. In this space as well, again, there's announcements at least monthly, if not weekly, of some company that's gonna make some amazing 360 camera.
And again you wanna discount 99%, 99% in the camera space most of them don't happen, or if they do they are terrible.
So, let's look at Cruiseabout.
So Cruiseabout's part of Flight Centre, it's a big organisation, they sell cruises funnily enough. So we filmed a bunch of content for them, and for the ships, on Royal Caribbean, Carnival, P&O, and Princess Cruises.
So what I wanna do now is have a quick look at a commercial product, beginning to end.
I'll bring up my lovely assistant and we'll see if we can make this a little bit smoother. Okay, so we're gonna have a quick look at P&O, actually we probably won't look at all of it but. (bright upbeat music) - [Narrator] Hey there cruiser.
Welcome aboard P&O Cruises Pacific Aria.
We're setting sail for a new adventure so say goodbye to Sydney and hello to a place like no place on Earth.
First up let's head to the dome, the ultimate spot at the top of the ship with 180 degree views of the ocean.
Look around, you're in Salt Grill by Luke Mangan. Exclusive to P&O you'll love this unforgettable dining experience featuring his signature dishes, the crab omelette and the famous licorice parfait with lime syrup. (upbeat acoustic guitar music) Now let's check out your room before exploring the rest of the ship.
- [Aaron] So what's happening with (P&O narrator narrating) P&O is, actually, sorry I'll turn that down a little bit.
What's happening with P&O, sorry not P&O, sorry Cruiseabout, Cruiseabout are using some of this content in their stores.
So they're enabling people to do virtual tours of some of the ships, and they're enabling people to make buying decisions in-store.
And they tell me that people are actually making those decisions.
So what you just saw was actually a $10,000 a night penthouse.
Not many people are making that buying decision but it's one, it's one you could make.
And interestingly it's not as though you can get to see that, you can't hope on the ship and say, "Show me the penthouse." It's not available.
And so we've filmed it so people can actually check that out.
But people are making real buying decisions. You can go into the room and you can decide, "Do I want just a window room or do I want a balcony room?" And when you've seen it in VR you can go, "Yeah that balcony is really worthwhile, "I'll pay the extra money and have the balcony." Which is the point of this medium, is to put people in places that they can experience something that they would have to go to in real life.
So here you want to sell them that experience, you need to put them in it beforehand so they can make that decision.
So that's working well for them, I won't run through the whole thing because we're gonna run out of time.
So if you wanna just stay, oh, he can't hear me. (audience laughter)
If you wanna say anything bad about him he can't hear a thing at the moment.
So P&O's a family ship, so the talent is a family, they're doing stuff all round the ship.
People can enjoy that experience.
It's not, it's not a story, it's not really a narrative piece but there's, the family is woven through it. So it's a three minute piece, upbeat music, voiceover telling you about the ship.
(audience laughter) Yeah, and they have real, real, real big pepper shakers. So yes.
Our talent was great although dancing wasn't their strong suit.
Not that it's mine but it's not theirs either. Okay.
Let's jump back into wherever we were.
He can continue on, he doesn't know, oh wait a minute, wait, wait.
He does know we've left.
Okay just wait there for a second 'cause we're gonna, we're gonna come back again soon. I shouldn't have said that about him.
So this project is one we did for Compassion, they're a multinational NGO.
They take sponsorship's from the public and they use that money to support children in, all over the world, often third world countries, not exclusively. And so the interesting thing about VR is that some people have called in an empathy engine. Because you can actually be placed into situations where you can't otherwise go and the immersion makes it feel like you're there. So it gives you an opportunity to tell stories in a way that really makes people feel them. Particularly at this point in time, for my money I think eventually people will get enough of those sort of stories and that, that power will reduce, dissipate, my lips aren't working.
But at the moment in particular these stories are really powerful.
We shot some stuff over in Thailand, it was a wonderful experience, we got to meet that actual girl from the story. We got to go to her actual home where she used to live and experience pretty much what she used to live, everything she used to have in her life, and how her life is now which is, for us it was humbling and it was a really wonderful experience.
But the VR experience is great.
So what Compassion are doing with it, they're not putting it on the web, they're not pushing it out there.
Their thought process is around, they don't want people to see this on YouTube or on Facebook and then not have the opportunity to see it in real life because, sorry, see it in real life, see the virtual reality version on a Gear VR, because they've seen it.
"Oh yeah I've seen that before, I know all about it." But the thing is if you've seen it on YouTube or on Facebook that's fantastic but you don't get that impression that you are actually there.
It doesn't have as much power.
So for the time being at least they've got these kits that you can see there, they've sent them out to their field offices and they're doing real demos, exactly like we're doing upstairs, for people in all the different venues that they attend. And so that's working really well for them. I expect that they will put it online at some point but for the time being they wanna make it an exclusive experience.
We're gonna jump out quickly again and uhmmm. - [Girl] These are my words and this is my story. I was happy then.
I could go to the Compassion project.
I could go to school and church with my brothers. We were all together.
And my mom had left.
I haven't seen her again.
(melancholy music) - [Aaron] So this is her actual home that she used to live in which was literally a shack. And it was well over 40 degrees when we were there filming and it wasn't even summer.
And again in VR, you're standing in that room, and you get to see what it's like.
And it's not pretty.
That's the kitchen.
So the story she's telling there is that one day she woke up in the morning with her brothers and her mom was gone and she's never seen her again since.
And her father then wanted her to learn how to cook and once he taught her how to cook he disappeared as well. So it's, it's a long story, it goes on, there's more to it than that but, yeah, it really is powerful, the medium.
And in this case it does work out in the end, in this case, which is great.
- [Girl] The Compassion project worker came. And she got me out of there.
- [Aaron] In Thailand you don't need to wear helmets, which is great for photography but terrible for safety. So we'll stop that there.
And I'll close it this time so it's not running in the background.
There we go.
Learnt something.
Okay, so that's, so a commercial project with Cruiseabout, P&O. We've got a story, a narrative, something that really is powerful to drive people to give in this instance.
And now we're gonna look at what, for my money, is probably the best possible uses of live action VR at this point in time, and that's training and learning. We've got another company we're setting up, VR Learn, specifically for this, and we're working with Qantas at the moment in this space. So what we're looking at here, again training in live action is really interesting, because you can put people into situations where you don't necessarily want to put them on a regular basis.
It might be expensive or time consuming, all sorts of issues with putting people on a tarmac to watch a plane load.
Certainly when it's your job, that's fine, but when it's not your job it's extremely difficult to get trainees to do this sort of stuff.
And in fact we were hearing the other day when we were chatting with Qantas about some other stuff that there's not a specific training programme for this it's more on the job training.
But what they're wanting to do is allow people to have this experience of being on the tarmac and doing this work before they actually go out there.
So what these experiences, or these training scenarios really give people, is a level of situational awareness.
They can really start to feel that, "Okay, there's the plane and there's this, "and then around here there's that "and then I've gotta look out for this." Because in a lot of situations you need to have your head on a swivel to really make sure you don't get injured.
Into those, we're filming a few different things with Qantas. Into those training scenarios we put the assessment.
So you do a training scenario and then you do the assessment in the VR as well.
And the assessment then communicates with the LMS and lets the trainer know that the trainee has actually done the scenario and they have done the assessment and they passed it or they didn't pass it.
And there's a lot of cool stuff that we're gonna be doing with that as well in terms of being able to, not only collect timings, like how long did they take to do this, were the questions right or wrong, and whatnot. But where were people looking while they were doing the training? There's a lot of really useful data that we don't really have access to at the moment. And certainly next year when we start to see devices that not just have directional information but also eye tracking then we can really focus in on what exactly are people looking at while they're doing their training scenarios. So I'll get my lovely assistant back up one more time. What we're gonna look at now is just the, is the video, the scenario, we're not gonna look at the assessment 'cause the assessment is an app for the Gear VR at the moment, it's not an app for the Oculus Rift.
We don't really recommend the Oculus Rift for training because you need all of this as opposed to just this.
So what this scenario in particular is about it's about retraining, or, you know, retaining a supervisor's eyes to see the things that are being done incorrectly.
On any given day you're not usually gonna see things done wrong because most people do things right most of the time.
And so over time you just stop looking for things to be done wrong.
But in this particular scenario there's a bunch of things that are done wrong, there's a bunch of things that are done right. And in the assessment at the end of that you need to determine what was done right and what was done wrong. So it literally is to try and retrain your eyes to go, "Okay, yeah, I saw that, "so I'm aware of that but I didn't see that "so I need to think about that in the future." The other thing is of course is that it's 360. So most times when people take, do this, the training and the assessment for the first time they get it wrong, because they miss things that were happening behind them. He's carrying a few too many items there for example, he stepped over the bar.
I know this now, all these things are wrong. This guy here's on his phone while he drives away, which is not good.
Oop, he. (audience laughter)
That's what happens to your luggage on a regular basis, no question.
This guy has been dropped off late to the plane and he's just gonna walk up the back and drop off his bag. As we were shooting this I was thinking, "Does this really happen?" And they say, "Yeah, yeah, this, all this really happens." So the guy walks up, he's like, "Get out of it, "what are you doing, get outta here, okay?" And he's, "Oh what, this place is mad." (audience laughter) Up, you might not see it up in the window here there's actually, ooh, is he there? Yeah, you probably can't see it on that view. There's actually a guy in the window with his phone filming all this going on 'cause you know that's gonna be on YouTube and everyone's gonna be complaining about the service. So this guy then has to get off the plane because someone's brought too much gear on. And in fact I think I missed him.
A guy just walked straight through the prop-arc which is where the prop spins, extremely dangerous. Oh here we go.
I've ruined it, I've ruined the surprise, I'm sorry, I'm sorry.
So he walks straight through the prop-arc which is very dangerous, comes up, drops off the gear. So that's not everything, there's a bunch of things being done wrong.
And then, like I said, the assessment goes in and asks you to find what's right and what's wrong. I mean that's at its simplest form, we're filming a bunch more scenarios this month and they're gonna have more advanced stuff like actual, questions that are happening that are time and location specific within the experience. So you need to notice this within a time frame otherwise you've not noticed it and so therefore you've failed on that particular item. So there's gonna be the ability, the questions coming in and out, assessments at the end, and yeah, a whole bunch of stuff. So we're really we're just figuring out for Qantas what works really for them, for training, what is actually helpful and what isn't.
This particular scenario is already being sent out to the airports so, I'm just gonna turn that down a little bit more. So just like the Compassion kits you saw before Qantas have got a bunch of kits made up and they're sending those out to the airport, to different airports, leaving them there for a weeks or so, and then sending them on to other airports so that the staff can do some upkeep if you like, upkeep training in the stuff that they're already doing.
They're getting fantastic feedback from that and they're looking forward to producing some more content so that people can start doing some more training.
So that's that.
And I think I may be finished.
Yeah, that is, whoops, that is me.
Let's get that correct.
And I've only touched on so many topics here, if you have more questions there's my email address. Feel free to shoot me an email, happy to answer those.
If you wanna shoot it, we'll put all this info together, and if you wanna grab it shoot me an email and I can send all this info and more back to you. So that is it, thank you very much.
(audience laughter) Appreciate you having a listen.
