Our everyday behavior has already been altered as technology changes how we interact with our physical environment. Our mobile devices can serve as layer or medium, through which augmented reality (AR) applications overlay digital information onto the real world. You have probably seen the growing trends of augmented reality in increasingly popular mobile apps, such as Pokémon Go. Or consider newer car navigation systems that use AR to overlay information in a way that is less distracting.
In industrial settings, AR can help manufacturers in a number of ways. From operating and maintaining complex equipment, to powerful training content with proven results, AR is delivering quantifiable business improvements and cost savings. AR is even being used to showcase 3D product demonstrations—allowing prospects and customers to “see” a product without it physically being there. These experiences are highly accessible and easy to distribute via smartphone-driven AR apps.
But how exactly does augmented reality work and what does it do? How is it delivering such rapid value for manufacturers—particularly in relation to other types of technology investments.
AR technology takes relevant digital information such as 3D objects, sounds, graphics and videos, and applies them contextually to your physical surroundings. In the industrial context, an AR-enabled headset will register the machine in front of the employee and overlay the appropriate work instructions to carry out a task or procedure. Unlike VR, where users only see a digital screen, AR’s purpose and value is in presenting digital information within the context of a physical environment. This provides the benefit of the digital information, while allowing the user to safely interact with physical objects.
Augmented reality simply requires an end user, a physical device (such as a smart phone or purpose-built wearable), and AR technology capable of rendering an AR content experience—such as PTC Vuforia. AR technology relies on computer vision to recognize pre-determined targets; these can include printed markers, CAD geometries, QR/bar codes and GPS or camera-based sensors. This target recognition enables the device to overlay the digital content experience onto a physical environment.
One of the benefits of AR is that users require almost no technical expertise. For example, PTC created an augmented reality experience to help primary school students study and explore the monarch butterfly lifecycle on their tablets. The student’s tablet camera captures in the view of the classroom and renders the pre-authored augmented reality experience, turning the space into an interactive monarch butterfly garden.
The low technical bar also applies to AR content creation. Authoring these experiences is easy for anyone to use with today’s out-of-the-box solutions, and does not require a background in coding. Content creators can leverage existing 3D CAD models, first-person video, and IoT data to build interactive instructions, 3D diagrams, or display real-time performance data. There are collaborative AR assistance applications, like Vuforia Chalk, that requires no pre-configured content.
Augmented reality experiences are usually triggered by recognition of a target—including pre-determined markers, CAD geometries, QR/bar codes or location-based or sensor inputs. Computer vision recognizes the appropriate target, allowing the AR data to be displayed correctly. Correctly in this case means that the data is applied to a specific object, and in many cases, locking onto it to with 3-D spatial awareness. This aligns the geometries, points, measurements and features to bring the use case to life so the end user can interact with it. On the plant floor, some examples that may be created are 3-D virtual work instructions, annotations on a piece of equipment, or remote guidance that also connects you to an expert. Without computer vision to register the surrounding world, AR content would lack the spatial context that gives data value and makes it actionable by the user.
There may come a situation where the user is looking at a machine or part that they may not recognize. Computer vision enhanced by artificial intelligence (AI) and deep learning offers a solution that takes augmented reality applications to a new level. As demonstrated by PTC CEO Jim Heppelmann at Liveworx 2019, this evolved AR experience can automatically recognize a spare part that does not have any bar codes, or target markers, in just a few seconds.
In this example, deep learning embedded into augmented reality takes existing CAD files and trains the networks in the cloud by feeding artificial 3D CAD models of the part in different positions to create an inferenced AI model that is displayed on the AR application. The transformed CAD data enables AI to quickly recognize objects in the surrounding physical environment. A hybrid of traditional computer vision algorithms and deep learning are now being used to push the full potential of AR applications.
With a basic understanding of how augmented reality works, let’s walk through a brief overview of the four main ways you will see AR in practice.
The end user is provided with an immersive mixed reality experience and an enhanced way to view equipment—including the ability to visualize product features and demonstrations invisible to the naked eye. These experiences are often immersive and hands-free involving wearables such as the Microsoft HoloLens. PTC’s customer, Howden leverages PTC’s Vuforia Studio to create mixed reality experiences that provide employees with an enhanced view of equipment –including the ability to visualize what’s going on in the machine.
Augmented reality does not need to be tied to an existing asset. With Vuforia Expert Capture, seasoned employees can bottle up knowledge by capturing first-hand videos and images as they go step-by-step through complicated tasks and procedures—all hands-free. These experiences can be easily authored on the Vuforia Capture app using pre-built templates to build out AR-guided content such as digital work instructions. When the employee is ready to use them, they can access these experiences anytime through headsets and mobile devices in 2D or 3D on the space in front of them. These experiences can be tailored to be location-specific content for multi-step, multi-location procedures.
New employees can access up-to-date instructions anywhere, anytime in the factory, which is essential to a smooth, efficient onboarding and training process. In the event of a road block or challenge, employees can connect with an expert on Vuforia Chalk to so they can both see and discuss the situation at hand and draw digital annotations on a mobile screen or desktop that will stick to anything on the space in front of them.
When using augmented reality, you can visualize a product on a flat or level surface in front of you without it physically being there. Whether it be a factory floor or retail store, augmented reality experiences powered by Vuforia can perform 3D scans of a location and place content in that environment. Vuforia Studio allows users to leverage existing CAD data and brings products to life. These virtual 3D product demonstrations are an asset for trade shows to create engaging experiences that allow customers to see what a product looks in their own work environment.
Unregistered augmented reality experiences are suited to industrial environments and often utilize wearables, such as the Realwear headset and M300 Smart Glasses. These can be worn with safety equipment like helmets and goggles and deliver 2D work instructions in the wearer’s view to provide more situational awareness. This device is voice-operated and conducive to a hands-free work experience. Employees often can gain on-the-job guidance by using these headsets to access step-by-step work instructions to help safely carry out new or complicated tasks.
Augmented reality is appearing everywhere—from sports, entertainment and gaming to manufacturing and training. Augmented reality offers new ways to transform sales and marketing from shorter sales lifecycle to creating new interactive customizable experiences that will change the way we shop for products. In the industrial sector, augmented reality is changing how manufacturers carry out operations, workforce training and field service. With enhanced information delivery, faster knowledge transfer and immediate access to remote expertise, augmented reality is critical to scaling operations and improving business outcomes. As computer vision continues to advance its understanding of the world around us, the possibilities for the future of AR are endless.
With a better grasp on how augmented reality works, it's easy to understand why more and more sectors are adopting this immersive technology. Businesses across verticals face common obstacles like productivity inefficiency, a growing industrial skills gap, and increasing operating costs. Augmented reality incorporates analytics and automation equipping manufacturers with an innovative solution to confront these workforce challenges.
Check out this interactive infographic to learn more about how augmented reality technology can target each of these areas for improvement while aligning to key business goals.