Augmented Reality, Virtual Reality, Mixed Reality, and the Metaverse

So, what do those terms mean? AR and VR have different histories, but they are beginning to become more entwined.

The joke is that in AR you trip over because you are not watching where you are walking. You might be playing Pokémon Go on your smartphone, and trip over the curb due to being engrossed in the game. The joke is that in VR your trip over because you cannot see where you are walking. You might be frantically battling virtual monsters in Resident Evil 4 and trip over your living room sofa.

The concern that AR could contribute to injuries and traffic fatalities is real, with academic studies to investigate if this is a larger issue.

Augmented Reality 

However, AR stands for “Augmented Reality” – a way of embedding a dashboard in your visual display. A good example of this is a HUD (Heads-Up Display) in an airplane, that allows the pilot to see important data without having to look away from the window.

Google Glass is a relatively recent example of personal Augmented Reality. Glass was a type of smart glasses launched by Google to the general public in 2014, The system was essentially a very lightweight computer integrated into a pair of eyeglasses. A small battery powered the unit, an integrated touchpad on the side allowed the wearer to respond to prompts, a button on the frame activated the camera for photographs, the forward-facing camera could record video and photographs, and a prism reflected the Glass display onto the wearer’s right eye. This provided a heads-up display, in which the Glass wearer could see information superimposed over their normal view. The Glass frames were designed to work with prescription lenses.

However, Glass immediately became a controversial technology. NameTag launched an app that would use facial recognition technology to identify someone the Glass wearer could see via matching against photographs on social media sites. The system promised to mine criminal record information too. Google reacted by barring NameTag from its MyGlass app store, but there are always ways to bypass this type of restriction.

The privacy controversy continues to this day. Clearview AI is a facial recognition company that provides software services to companies and corporations, notably law enforcement, across many countries. There is ongoing debate and legal action to provide guidance on who can use this and in what context. There is considerable opportunity for this type of technology to be abused. At launch, Glass promised less-controversial apps for wearers. Glass could be used to translate other written languages to English on the heads-up display. This was a significant boon.

As with any other new technology there were issues:

• The prism could be damaged in hot or humid weather, rendering the device unusable.
• Battery life was limited to a few hours, necessitating bringing a portable battery charger.
• Recorded video had an upper limit of about 20 minutes.
• Glass limited the user network to about ten people.
• Glass wearers were called offensive names (such as “Glasshole” and “Bluedouche”) as many found the potential privacy issues of a Glass wearer in their midst problematic.

So, the first iteration of Glass became a consumer failure. The device was costly, lacked a killer application, had a cumbersome user interface, was difficulty to purchase, and was not properly explained to a mainstream audience.

The enterprise edition has been more successful. The device is more robust, and the users of these updated device better positioned to have a real need to use in the workplace.

Microsoft has a competing device, the Microsoft Hololens. Larger, heavier, and considerably more expensive, this is a device that is focused on organizational use, rather than the consumer. Again, this is a device that has not been warmly accepted by the target audience. It has niche use.

At present, there is a bifurcated trend in the marketplace. Enterprise AR headsets are expensive, with slow iterative improvements, and focused on development integration with the end-user. The manufactured of consumer devices have learned from the Glass experience, with more rapid improvements, a more limited set of inputs (doing one or two things, rather than the many that Glass did), brand affinity, smartphone integration, and designed to blend into the environment (rather than advertising that the user is wearing a smart device).

For the enterprise market, there are probably about five major players in the U.S.:

Within the college, there are four disciplines where there would be clear use of AR headsets:

• Entrepreneurship
• Finance (Fintech)
• Hospitality Leadership
• Marketing

With the most likely areas for use and experimentation:

• Best practices for use with customers/consumers
• Data visualization
• Product and service design

The consumer AR headsets are focusing on individual use and less-ostentatious design:

• Camera glasses (Snapchat, Facebook Ray-Ban Stories)
• Alexa glasses (Echo Frames)
• Bluetooth glasses (Bose, Echo Frames, Vocalskull, Vue)
• Smartwatch data glasses (Cosmo, ENGO, Julbo)

Of note are

• Facebook Ray-Ban Stories
  Smartglasses that partner Ray-Ban and Facebook to record the wearer’s environment back to social media.
• Echo Frames
  Smartglasses that communicate with Amazon’s Alexa services to provide quick answers and audio services to the wearer.
• VocalSkull
  Smartglasses that provide bone conduction audio. This allows the wearer to hear audio from a Bluetooth device (such as smartphone or computer) without alerting those nearby.
• Cosmo
  Smartglasses that project data to the wearer from a smartphone. Targeted, at present, for cyclists.

To a certain extent, all these headsets operate like spy devices from James Bond movies of the past. The implication to the college is that they could be subverted to surreptitiously record audial and video in areas that had an expectation of privacy and be used to cheat during exams – both on-campus and online.

At the cheapest end of the market, certain apps can be used to provide AR services on smartphones. Examples include:

• Star Walk: When used to view the night sky, Star Walk will overlay constellation data.
• IKEA Place: The free app allows the user to superimpose furniture over their view of spaces in the home and at work. This way a prospective IKEA customer can better decide what products work within their space before making a purchase.
• Holo: Here the app works in a similar way to IKEA Place, but superimposes entertaining characters, creatures, and objects into the user’s immediate environment.

Virtual Reality 

So, let’s talk in more detail about VR, Virtual Reality. 360° Video is one way of providing an immersive experience of real-life environments, such as surfing or scuba diving, where the viewer can decide where to direct their gaze.

With 360° Images, the viewer sees a static image, but again can look around at any point within that image.

The process to generate these images is to typically use a camera that takes multiple overlapping pictures simultaneously, and then stitches these images together. These cameras range from relatively affordable consumer cameras to wildly expensive professional rigs.

This is an area in which there is a natural fit for the college in at least two disciplines: Hospitality Leadership and Real Estate. The Real Estate industry frequently uses such tools to market properties for sale or rental. Hospitality Leadership can benefit from immersive viewing simulations that prepare service professionals to understand environments before visiting them in real life. This can all be easily created to operate in D2L by working with the Center for Teaching and Learning to place H5P content (HTML 5 Package) in online courses. Below are two examples of the type of item that can be easily created:

• Virtual Tours: A series of 360° images can be linked via hotspots. The viewer simply clicks on a hotspot to move to the next location. This example demonstrates the filming locations in our Mini-Studio. 
• VR Movies: The viewer can place themselves in an immersive video, where they can look around the environment. This very basic example was filmed in a classroom. More professional examples would improve the lighting and sound.

Movies like the Matrix or Ready Player One have presented a high-fidelity representation of what Virtual Reality could be, but the experience in real-life has been a little more cumbersome. In 1991 Virtuality created coin-operated units for video game arcades and festivals. These units typically required a human handler to help players navigate the experience and technology. The fad soon dissipated. Nintendo’s Virtual Boy launched in 1995 and promised to bring virtual reality games to a home audience. Most players found playing these games to be nausea-inducing, and this market experiment crashed. That same year the Forte VFX 1 was launched, a precursor to the Meta Quest of today. The VFX 1 was bulky and lacked the necessary processing power, so again this product failed to make traction in the market.

Things changed considerably in 2012. A successful KickStarter campaign saw the development of a relatively affordable VR headset that connected to a personal computer for gaming. Facebook purchased the company in 2014 and released Oculus Rift headsets to consumers in 2016. Gamers loved this headset, which transformed first-person video games into an even more riveting experience.