Over the last few years a wide range of Consumer focused devices have come on to the market enabling users at low cost to experience Virtual, Augmented and Mixed Reality.
The majority of these devices have been focused on Gaming, Social Media and Entertainment. However, Enterprises have also been looking in to how these technologies, collectively called XR, can help improve business processes. Enterprises particularly large Automotive, Aerospace, Defense and Research organisations have been using Virtual Reality for over 30 years using CAVE’s and Powerwalls. This article discusses how the technology has changed and how the new consumer XR devices could be used by the Enterprise.
Technology Background
First of all we need to define what is meant by the various technologies of which Virtual Reality is the most mature.
Virtual Reality (VR) is an interactive computer-generated experience taking place within an entirely simulated environment, incorporating auditory and visual feedback, but can also include other types of sensory feedback like haptic. The computer-generated simulation of a three-dimensional image or environment can be interacted with in a seemingly real or physical way by a person using special electronic equipment, such as a helmet with a screen inside or gloves fitted with sensors. In a literal sense, VR can make it possible to ‘experience’ anything, anywhere, anytime. It is the most immersive type of reality technology.
In an industrial context, Commercial VR has been evolving for around 30 years. Developed by specialist niche software companies, to serve large product development and manufacturing businesses, the immersive experience can be created through complex ‘CAVE’ environments where projectors are directed onto three to six of the walls of a room-sized cube. This produces full-size renderings, giving the user a feeling of being immersed in a 3D world that can be interacted with through 3D glasses and a hand controller or glove. CAVEs are, however, extremely expensive, costing anything around $1M+.
Powerwalls have attempted to reduce the required investment (to around $100k, possibly more if a 4K resolution is sought). Like a CAVE, a Powerwall creates a display through projection or more recently through a matrix display. Again, like in a CAVE, 3D glasses are used to create the illusion of three dimensions. However, despite the resolution, inevitably the sensation is one of working ‘on’ a 2D screen, albeit at full-scale.
It is also worth noting that just presenting the data in 3D does not make an immersive system. It is only through motion capture tracking technologies that the system becomes actually immersive. The markers are seen by the system and the image on the screen is then rendered to that person’s point of view. This means that the tracked user is properly immersed. However, it can only track one person. Accordingly, for everyone else watching, the view is slightly offset, unless they are standing directly behind the tracked person. So, your commercial CAVE or Powerwall is really only truly immersive for one person at a time; when you consider this fact, these seem even more expensive!
Commercial VR in the form of CAVEs and Powerwalls has been closely aligned to the Product Lifecycle Management (PLM) software developers, such as Siemens, Dassault Systèmes and PTC, presenting another high-cost of entry for product development companies attempting to leverage VR. Similarly, the preparation of data for use in these Commercial VR systems is a difficult and time-consuming process, resulting in larger businesses deploying dedicated data-preparation teams to support the demand.
Interestingly, Virtual Reality has seen a parallel development path in the consumer world; Consumer VR has been evolving for around 20 years. Unlike Commercial VR, the technology companies behind Consumer VR are global tech’ giants, such as Samsung, HTC, Google and Facebook. The objective has been wide-scale availability and low cost. Most significantly however, unlike Commercial VR, and despite multiple hardware giants playing in this field, the software behind Consumer VR has come from the gaming industry, and it is dominated by two massive players in the gaming world; Unity and Unreal. Consumer VR is a totally immersive technology, with the user seeing a completely simulated environment through head mounted displays. The upside is that the users can be ‘transported’ to a full-sized simulation of any environment, and then interact with that environment through linked handsets. The simulation also allows multiple users to interact within the simulated world. The downside for some is that complete immersion can be disorientating, resulting in ‘VR sickness’.
Consumer VR has further evolved into two aligned technologies, Augmented Reality and Mixed Reality. Augmented Reality (AR) has been around for about 15 years but, in the last 5 years has grown in adoption and use with the advent of more powerful laptops, tablets and mobile phones.
Using the same graphical engines as Consumer VR, but now being driven by development tools such as PTC’s Vuforia, Apple’s ARKit and Google’s ARCore.
Augmented Reality (AR) provides the ability to overlay and view digital data onto the real world, using traditional devices such as smartphones and tablets. The overlaid data can be text to “augment” what is being seen through the devices camera or it can be 3D geometry and the two datasets combined to add further value.
While dominated today by gaming applications, AR is seeing rapid uptake in applications such as training, where digital assembly or service instructions can be presented by anchoring digital images onto real world ‘locators’. A number of training solutions developers are helping manufacturers to exploit this opportunity. In areas like IOT for instance it can be used to overlay performance data on top of a machine which is running e.g. providing data on running speed and temperature as captured by the sensors on the machine.
The disadvantage can be a cognitive distance created by having to place the screen to one side while working, or the hindrance of holding a screen while trying to work with the images displayed. Head-mounted devices, such as VUZIX, Google Glass or the DAQRI helmet, are presenting some hands-free options to address this problem. Despite restricting the user to a relatively ‘flat’ viewing experience (through a screen), it is a substantial step forwards in terms of contextual relevance when viewing data. AR is seeing significant growth, and it is predicted by industry observers to ultimately be the winner over Consumer VR1.
Mixed Reality is the newest technology appearing around 5 years ago. It is similar to AR in that as well as seeing your digital data you see it in your environment through the devices camera. The difference though is the digital content image is a hologram. What that means is you can walk around the hologram as you would a physical object, this gives a much more visceral feeling to the content. Like AR the digital content can be 2D text and or 3D geometry, so MR “augments” like AR but does so in a more “physical” manner.
Until recently the MR space has been dominated by the Microsoft HoloLens, but recently challenged by a competitive offering from Magic Leap (with high profile investment from Google, Alibaba, and AT&T), both providing a Head-Mounted-Display (HMD) that blends digital [holographic] data with the physical world. This digital and physical blend is the closest step yet towards the provision of true spatial context for the user, placing full-sized 3D digital data ‘into’ the user’s world, and removing the need to ‘view through’ a screen. With the distinct advantage of being able to see other physical users in the same space, and even to present virtual ‘avatars’ of users into the room, MR is the closest solution yet to removing the cognitive gap.
While the advantage of bringing engineers closer to their designs may be obvious, the scale of investment required for Commercial VR running in CAVEs and Powerwalls, and the complications of applying it, has and does present a barrier for many product development businesses. Consumer VR, and the related Augmented and Mixed Reality technologies, present an opportunity to overcome the financial barriers to entry.
These technologies, collectively known as ‘XR’, also represent the potential to provide engineers with better solutions to interact with full-sized digital representations of their designs in a spatially relevant context. Whilst for application in the engineering field, the Unity and Unreal gaming ‘engines’ are not focused on supporting product development (CAD) datasets there are methods to solve that problem.
The result is engineering and manufacturing can now take advantage of low cost consumer XR technologies, this democratizes access to these technologies and so opens up a wide range of new ways to tackle existing problems and offers the chance to solve problems which could not be addressed with commercial VR.
1.XRDC (2018) AR/VR Innovation Report, San Francisco, October 2018.How can engineers gain advantage?
For Engineering and Manufacturing companies to take advantage of the new devices industry needs relevant applications. Theorem Solutions along with a number of other vendors have been creating solutions for:
- Visualization
- Design Review
- Manufacturing Build
- Production Layout
- Training
- Remote Assistance
The result is affordable XR, enabling small, medium and large scale engineering companies to reap the benefits of these technologies to improve existing processes and tackle problems where previously the economics were prohibitive.
This may also be of interest: Exploring the Cognitive Gap and the potential of XR technologies.
