XR through the years: From the nausea-inducing Virtual Boy, to Apple’s era-defining Vision Pro

• The history of Extended Reality (XR) gaming started with the Nintendo Virtual Boy in 1995, and it was not an auspicious beginning, plagued by high prices, monochrome displays, and a generally uncomfortable experience.
• A decade and a half later the Oculus Rift started to show what this most engaging of formats could deliver.
• Apple’s Vision Pro has taken this era on another step, offering 3D capture capability and the potential to truly immerse ourselves in virtual worlds.

For the last 30 years, the XR (Extended Reality) industry has been on a quest to discover ever more engaging experiences for its users. From Meta’s haptic gloves to Sony’s gaming gloves and handheld XR devices, innovations have continually set new levels for wow-worthy tech. While some cinematic dreams have proven elusive, the trend has been upwards, forwards and all around, turning living rooms, bedrooms and anywhere with a wifi connection into virtual worlds for their protagonists.

So how did it start, and where will it end up? We take a look at the progress of XR technology through the years, from an uninspiring debut to the potential to relive our own memories.

A blast from the past: Early VR interactions  

In 1995, Nintendo took a bold step with the Virtual Boy, an early VR concept. Despite initial excitement, it faced a swift downfall: plagued by high prices, monochrome displays, and a generally uncomfortable experience. Its bulky design and less-than-stellar ergonomics didn’t help. In the end, it was more of a horror movie prop than a VR device.

In less than six months, the promise turned into disappointment. Nintendo couldn’t find a way around the dizziness and polarisation effect that Virtual Boy caused. Nintendo made one more attempt at turning things around by introducing a tripod. The goal was to reduce the neck strain and dizziness that gamers were experiencing.

But playing Virtual Boy games remained just a small step away from a migraine.

Even more unappetising was the fact that the product offered a three-dimensional visual experience yet it was mainly played with traditional gaming controls.

After less than a year, Nintendo decided that Virtual Boy was better off as a museum piece.

Nintendo’s problems weren’t unique to them in the early stages of VR development. Hardware manufacturers grappled with bulky equipment that tend to overhead. Worse, the options for in-game interaction was limited to traditional controls.

Also read: VR company transfr raises $40M to change how we train our workforce

Introducing possibilities: the Oculus Rift  

Around 2012, a newcomer renewed the excitement for VR. Oculus, led by an energetic founder named Palmer Luckey, introduced the Oculus Rift prototype.

The Oculus Rift offered realistic video and 3D experiences without bulky controls. The exciting possibilities of the device attracted a $2.4 million investment, which Luckey and his team put to use by improving Rift’s headset and tracking software.

The potential proved too irresistible for Facebook. In 2014, the company that we now call “Meta” offered Oculus a $2 billion handshake.

The Oculus Rift lacked manual tracking controllers and couldn’t support complex games. But it did provide solid-enough three-dimensional experiences. VR had officially moved forward.

Shortly after, Google VR Box and Samsung Gear joined the parade.

Samsung’s contribution? A touchpad on the device’s side that users could use without having to look at the controls. Additionally, the Samsung Gear could be connected to a Bluetooth controller when playing certain games.

Also read: Samsung’s new AD borrows the story of Romeo and Juliet to call on Apple to support RCS

Shifting gears

Between 2012 and 2016, VR interactions started evolving. Although they still couldn’t function as standalone devices, VRs shifted towards user-friendly designs.

After 2016, VR hardware began emphasising hand and head tracking, upgrading from 3DoF to 6DoF. 6DoF (Six Degrees of Freedom) tracking defines XR interaction by tracking the positioning of the head and hands. However, exciting new things are coming our way and 6DoF is just the start. HTC Vive, Valve, Quest series, and PICO Neo 3 blazed the paths for more accurate and highly interactive controls.

“Imagine putting on a headset and being transported to your child’s first steps – where you can walk around the space and see them and experience them all over again”

MK Bedosky, co-founder and head of marketing at Gen City Labs

The quest for natural interaction  

Balancing cost and technology remains a challenge, keeping natural interaction concepts in check. As a result, VR controllers remain the prevailing mode of interaction.

Founder at Red Roof Industries, Angela Simoes points out: “When it comes to balancing cost and technology in VR, specifically for software, developers should use toolkits and component technologies that already exist to make their job easier, faster and more affordable. For example, if a developers needs to bring in buildings, furniture, vehicles or other physical objects, there are CAD data exchange toolkits to bring the 3D models of those physical objects into VR, and then also graphics toolkits to help make the 3D models photo-realistic.”

The next big bite into VR development came from Apple’s Vision Pro. The release of the Vision Pro brought a quantum leap in VR interaction. Natural auxiliary methods like voice, gesture, face tracking, and full-colour perspective have become integral. Voice recognition, a rising star in artificial intelligence, is gaining ground and promises continuous XR interaction upgrades.

Also read: Apple’s ‘Cheaper’ vision pro isn’t exactly cheap

However, Mark Zuckerberg was less than impressed with Apple’s achievement. He believed that the invention didn’t really fall far from the tree of its predecessors.

Zuckerberg believes that his own line of VRs, the Quest3 being the latest, is just as good at a much cheaper offering: as of June this year, the Quest 3 was priced at $499. In contrast, Vision Pro stands with a more forbidding price tag of $3,499.

But Vision Pro isn’t just about reinventing the wheel. It’s about perfecting every aspect of XR interaction. From eye and hand movements to voice commands, Vision Pro offers a holistic set of interaction methods.

Apple isn’t the inventor of eye tracking, but it’s the first to make it a core interaction method.

MK Bedosky, co-Founder and head of marketing at Gen City Labs, told BTW: “The biggest impact of the Apple Vision Pro may not be the user interface, gesture recognition, or pass-through camera. I believe that it will be the 3D capture capability – the ability to get a ‘point cloud’ 3d model (with some help from AI) instead of a flat two dimensional photo. This means that users will be able to ‘walk around’ their memories, and instead of purely looking at a flat image, immerse themselves in them. Imagine putting on a headset and being transported to your child’s first steps – where you can walk around the space and see them and experience them all over again. Or your wedding. Or a birthday. The ability to create an immersive environment will be game changing and is likely to be the ‘killer app’ that catalyzes broad adoption for consumers.”

Vision Pro gets rid of the “gamepad” external controller. Users can interact with the device just with their hands, voice, and eye movement.

It succeeded where PS VR 2, Quest Pro and PICO 4 Pro failed. The arrival of Vision Pro opens up new horizons, redefining how eyes, gestures, and voice collaborate in the XR space.

Moreover, voice interactions are making huge strides in VR interaction. All of these new advancements were made in great part to the ones that failed in the past.

Virtual Boy failed but it showed us what’s possible. Oculus renewed our excitement for extended reality. Now, Meta and Apple are head-to-head in reinventing our digital realities. It’s hard to imagine what’s possible but we probably won’t need to: the next set of XR advancements will arrive in your news feed soon enough.