AR for the Digital Industrial Workforce
00:00 Thank you for joining. My name is JJ Licklider. I'm the general manager for the Vuforia business at PTC. I was excited when I saw the spot that I got to talk to you all today because it was right after the keynote. Hopefully you all had a chance to attend. It's been several years since I've been able to get up on a stage at LiveWorx and excited to share with you the the evolution of the Vuforia platform, the business. Since many of us last talked, I see a lot of familiar faces customers,
00:32 partners, colleagues and we have a lot of news to share. We've been busy over the last several years, many of us actively on a journey together. But today I want to take the material from the keynote and the announcements that that Jim made there one layer deeper and relate it back to augmented reality. What's been happening with Vuforia, the benefits that our customers are seeing, the benefits that that you can realize if you deploy augmented reality to the industrial workforce. We'll do a Q and A at the end you can come up to
01:04 the mics here in the at the podium and ask those. So Jim did a did a pretty good job of covering the breadth of what the PTC vision is. He talked about the digital thread. He talked about how all the different technologies fit together. He talked about how the how the digital thread flows from the physical to the digital. But let's let's go a little bit deeper and talk specifically about how augmented reality and Vuforia helps to power that.
01:34 So who are we talking about? Where do we fit into that story? What customers? What end users are getting benefits when they use our. Now, 75% of the global workforce is actually on the front lines. These are workers at many of your companies that are using their hands that don't have access or the luxury to sit behind a computer screen all day with all the information they need at their fingertips. They need that information on demand when they're on the factory floor, when they're out in the field in order to do their jobs more effectively.
02:08 A Forrester study here would tell you that 77% of those front line workers say they don't have access to that technology when they need it. They don't have the ability to get the information that exists that they know exists somewhere at the point that they need it. And that leads to mistakes that can be made or time that can be wasted. And so with our we endeavor to change all of that, it's a technology that can provide information to a front line worker when and where they need it. And when you look across the PTC portfolio,
02:38 as illustrated in the infinity diagram I'll speak to that are is the point that you connect that digital thread to the physical world. It's a visualization technology and it's solutions that are inherently usable by front line workers. We spend a lot of time with our user experience designers, our product teams talking to customers to make sure that anything we design has the front line worker in mind that it's a powerful technology can sound futuristic, but it's very important that it's accessible, that it's easy to use. It's a big focus.
03:12 Now. How do we connect? Digital or physical with are we? How do we augment it? We can capture procedures that are in the minds. That's the expertise of a seasoned veteran that's been on the job for 20 or 30 or 40 years and distribute it to a frontline worker. That may just be getting started or may be coming from another discipline. We can recognize complex objects to overlay and virtualize process critical information, live sensor data. We can guide workers to take the right steps,
03:44 to find the right parts, to perform the right tasks. And we can remotely support frontline workers that are somewhere else in the world. We can use our tools to help provide accurate guidance to those frontline workers so they can get the job done right the first time. It's bringing together all of these different capabilities that can really optimize a frontline workforce. And there are a lot of possibilities. There is places to deploy are all across the enterprise, all across the value chain from engineering new products to sales and marketing,
04:17 showing end users and customers. What products look like after they'll be configured, after they're built, building products in the factory, becoming more efficient, servicing products, operating products, helping your customers learn how to use your products. There are literally hundreds of different use cases that augmented reality can address, and many of you have tackled those. But what we found and really where we've been focused since we last got together here, is on the use cases that are the most valuable today.
04:48 The use cases where our customers are seeing the biggest return on their investment, where they're driving the most workforce efficiency. And so that's what I want to focus in on today. I'll explain these use cases. Then I'm going to give you some customer examples where these use cases have been addressed in the field today, and many of them are also on the floor. So you can you can see and touch and experience them live because augmented reality is a very visual technology. It's helpful to see and hold and experience it firsthand.
05:19 One area where we've seen a lot of traction is an end of line inspection, specifically in the automotive industry. This is where there are highly complex, highly variable products, many different configurations that are put together and you have front line workers that are inspecting them mid or end of line, ensuring that they're put together accurately, noting where there's inaccurate inaccuracies or something hasn't been done correctly. Augmented reality can can really supplement and empower that use case finding parts
05:49 a service technician. Lots of turnover in the service industry. New front line workers, worker shortages. There's a huge demand to transfer knowledge and get new service workers up to speed. Augmented reality can help them understand the components that they need to replace the procedures that are associated with those components. Machine builders. Our customers who produce heavy equipment, they sell them into factories, factory operators that are for a range of different possibilities.
06:19 They can create augmented operations manuals, manuals that use augmented reality for an end customer to understand how to how to operate the machine, how to troubleshoot simple, simple errors that come up with the machine. This can be a supplement to an existing service offering for these customers. Training, product familiarization, visualizing products in full scale in 3D, understanding how they operate before a service technician travels to go fix them.
06:51 Getting up to speed, new employees getting trained up, knowledge transfer using the existing digital product data. And another area where we see a lot of interest and more and more customers deploying is in process manufacturing. The pharmaceutical industry using augmented reality and work instructions in order to remain compliant and train up a new workforce, use cases like line changeover and line clearance. So those are those are five key areas that hopefully get your mind thinking about ways
07:22 that you could deploy augmented reality. And when we attack those use cases, there's a couple principles that we keep in mind, a couple of different areas that that we invest to generate customer value and align very directly with things that you saw this morning in Jim's keynote. It is critical that there are ways to capture knowledge. I talked about augmented reality as a tool to take knowledge from a frontline worker and distribute it to an entire thousands, thousands of people in an organization.
07:53 But to truly achieve scale, to create something sustainable, it's important that you can leverage all of your existing 3D product data In augmented reality, it's critical that you're able to connect back into the source of that data, which for many of the attendees at LiveWorx is a PLM system. When you can connect augmented reality to PLM, you can get that configuration specific product data for use in AR. That means that you can actually have thousands
08:24 or even millions of different unique AR experiences that are programmatically generated to communicate to a frontline worker how to do a job. We are consistently advancing the state of the art, investing in new computer vision and artificial intelligence technologies that can assist frontline workers when they're performing their job with their hands , acting as another set of eyes, another input to ensure that everything is done right the first time. I'll share an example. And then as Jim highlighted again, PTC is,
08:57 is moving towards SaaS. There are use cases that are deployed on premises. There are use cases that can are deployed in the cloud, but there are huge benefits in user experience and scale that can be reached when you invest in a SaaS transformation. And that's that's what we're helping our customers with. Now, the result for for Vuforia, again, since since we've last been together has been consistent recognition that we're that these are the right areas to invest, to provide the most value, to help customers, to get to get to scale, to provide the most complete platform possible.
09:30 So we're very proud of that. And I wanted to dive a little deeper into the digital thread or the model based digital thread. As Jim said this morning and explain how AR fits in. So this looks pretty familiar. If you if you watched the keynote on the left, you have the digital world or the digital representation representation of the physical products before they're built. Maintained in Windchill all the different configurations are accounted for. Products are designed and then produced for use in the physical world.
10:02 As I just said, as those products are produced, as they evolve, as they're designed. All of that information already exists. And Vuforia is the conduit to take that CAD data, that 3D product data and visualize it in the physical world, to take that data and configure it to be used as an augmented work instruction for a range of different use cases, service, manufacturing, training. Once you do that, once you have a frontline worker empowered with all that data,
10:32 you can you can leverage years of product design, investment on a frontline and entirely new workforce that can realize different benefits, sometimes far greater benefits depending on the the industry. But it doesn't stop there as those products exist in the world. And Jim talked about ServiceMax and what that means as those products exist in the world and they change and they're serviced. We have a new system of record and a new part of this digital thread that involves service technicians that are providing feedback back into the design process.
11:04 So these service technicians are changing things. They're fixing things. They're working on products based on a serial number or a Vin. And the learnings that they have there can be captured in augmented reality and then sent back upstream to further improve future designs. It's really a process of continual improvement. In a very powerful one. So let's look at a couple customer examples that are helping to illustrate the advancements along those three different pillars. The first is Volvo.
11:35 So if you've been on the floor, you could not have missed a giant electric Volvo truck that's in the center of the Vuforia booth. And I'd urge you to go check out this demo firsthand. But Volvo and PTC have had a long collaboration was discussed again this morning as it pertains to augmented reality. There's a huge opportunity to leverage all of that product data that exists in Windchill downstream for frontline workers. As I've been discussing, Volvo is exploring a range of different use cases, but the one that I'm highlighting here is as part of
12:08 a first responder application that uses Vuforia augmented reality for these Volvo electric trucks to help the first responders understand critical safety information when they arrive on the scene, where the batteries are, where the cabling is, what they should touch, what they shouldn't touch. And the example that you're seeing on the screen here is guiding a frontline worker through the steps to the safe parts of the truck. They have the ability to see into the vehicle so they don't take apart the wrong panel so that they can troubleshoot the specific information related to that truck.
12:41 And what's really powerful here is that this application, which you can download, by the way, it's live on for iPads on the App Store is available and supports 1500 different configurations of that Volvo truck today can scale to hundreds of thousands. And those 1500 weren't all hand offered. No one sat down and made it through 1500 different configurations. Volvo established a bridge to their PLM system so that they can programmatically get that three d information that exists in their
13:12 PLM system into augmented reality for a frontline worker to use. It's a great example of the potential that you can reach when you connect that piece of the digital thread. Similarly Peterbilt. Another example is service example. Here their service technicians are using Vuforia in a wire harness use case. So complex wire harnesses that exist in the Peterbilt trucks, sometimes as much as 40 plus pages to explain the configuration and how to find
13:43 the appropriate part of of the of the harness, how to access, how to how to follow the instruction could take a long time. They implemented are experiences were able to cut at least 20 minutes off of each service procedure that was performed where you're working with the wiring harnesses sometimes as much as 24 hours when it was an incredibly complex issue. So real efficiency realized by Peterbilt connecting the digital thread again back into their PLM system. Automotive is an area where we see a lot of pull for this use case because there's
14:13 a lot of configurations. There's large frontline workforces feel the needs of all the service challenges that that I'm sure many of you experienced today. This is applicable to any any industry with with heavy machinery, with complex equipment where there's variability. It's an important stage of the digital thread. So that's how we that's how we're seeing customers get to scale with augmented reality connecting that digital thread. Now, I want to I want to talk a little bit about a new technology that we're introducing into Vuforia, and we call it step check.
14:46 So step check is a is a technology that's been in beta for the last several quarters and pleased to announce at the show today is going commercial. It's technology that you can access through Vuforia expert capture to act as a second set of eyes for a technician that's performing an inspection procedure uses AI in order to tell that technician when something has been put together correctly and when it's not. Step check will find that error and offer a workflow for a technician to report that
15:17 so that they can improve and catch errors before before the parts leave the factory. One of the customers that again, you can you can experience firsthand on the floor that we've been working with throughout the beta period is Magna. Magna is an automotive supplier. They make parts that they supply to all the major automotive providers and so they have highly configurable, highly variable parts with hundreds of points of inspection and they have frontline workers coming in and turning over like everyone else, that they need to train on what an appropriate configuration looks like at
15:51 the end of the line when they're inspecting it before the part leaves the factory. They're very good at catching these errors already. But the fact remains that mistakes still do happen. And step check is again that second set of eyes for these frontline workers to understand when something has been put together correctly. What you see on the screen is actually looking through an iPad and you see this red check here. When this piece of when this cable wasn't plugged in correctly. That's the that's the the eye that is telling the worker that that's actually not in the right place.
16:23 That's not what it's supposed to look like. This can get very detailed and very specific. It can help with issues related to soft connections where you have a cable that's not even plugged in all the way. Step, check, can can find that in use cases like this before the part leaves the factory. When something is correct, you see a green check mark and you can move on to the next step. When it's not, you can document it, fix it, and move on. This is a great one to see in person. I would I would experience it first hand. It's a pretty, pretty magical technology.
16:56 Step checks, working to transform how inspection is handled more efficiently, making sure that inspections are done right the first time, catching these errors. It's a visual guide. Go here. Look at this. This is done right. That's not You should. You should. You should. Take another look here. You can start to start to get that really that I support to get inspection done right the first time. And what's important here is I'm saying words like I and are
17:26 and computer vision throughout this presentation to all of you but this is a very simple technology to use. It does not require prior knowledge of how augmented reality or artificial intelligence or neural networks or any of that, any of that stuff works. It's designed to be used by a frontline worker. The same experts that put these put these faces together correctly in the case of Magna can train. The network just by taking pictures. And we have a very simple interface for redundant for for doing that.
17:56 This is put together correctly. This is how it works. So beyond technical innovation and beyond connecting the digital thread, one area that is very critical to driving adoption for these industrial enterprises is ease of use, good user experience, the ability to scale, to get all of that content that you've got when you connect to the digital thread, step, check all of that technology in the hands of your frontline workers. And there's no better way to do that than SaaS.
18:28 Lots of benefits. Already discussed this morning. But but for us, SaaS allows us to take advantage of very powerful cloud compute capabilities like Skepchick. It allows us to create new distribution mechanisms so that frontline workers throughout thousands and thousands person service organizations can all get access to content instantly. It's instantly available, instantly available on the devices that frontline workers use mobile devices, eyewear devices and PCs as well.
18:58 Scheffler is a is another automotive supplier partner that is using Vuforia expert capture for training. So they're addressing head on knowledge transfer and training optimization, making it more efficient for their workers who come in to understand the tasks that they're about to perform. Training is a big a big use case for Vuforia. They're early in their journey, but they're seeing amazing results and they're getting ready to to move further. So I talked on the use cases about process manufacturing and mentioned mentioned pharma.
19:32 One of our one of our great initial customers in this area and a great partner is Merck. And Merck is using for expert capture, for training, for line clearance, for line changeover to make their workers more productive. But what's unique about Merck are not unique, but what's important to Merck is that they can do all that in a compliant fashion. When you're in the pharmaceutical industry, you have to adhere to something called GMP or good manufacturing practices.
20:03 That brings a whole nother level of rigor and discipline to the procedures that are followed throughout their business every day. And so we've worked with Merck and other partners to ensure that expert capture is GMP compliant, that we can keep those customers GMP compliant. As they utilize Vuforia for work, instructions for training and line clearance and line changeover. There's a there's a big opportunity here if you work in life sciences.
20:33 Swing by the booth. We have folks there who can tell you more about it. The last example that I'll give here is Acoma. So again, another one you can see on the floor. What you'll see kind of in between the Iot area next to the Vuforia area is a large packaging machine from Harpochloa. Harpochloa is what what I call a machine builder. They produce equipment and then they sell them to those operating factories.
21:05 And Harpochloa is a is a is a is a great customer. They're very forward looking and advanced and far along, frankly, in their digital transformation journey. They really utilize all of the Vuforia portfolio and PTC portfolio to really digitize all aspects of their go to market. The example here specifically that I want to talk to is how Harpochloa uses Vuforia to provide instructions to their end customers that operate their machines in the factory. So I talked about the use case already,
21:35 but this is this is very powerful because to a customer this is going to lead to increased satisfaction. I was able to figure out how to operate this aspect of the machine. I was able to troubleshoot something that I didn't understand. This didn't require another support incident. I was able to to immediately resolve an issue. I was able to keep the line running. Those types of benefits can be passed along as part of service offerings, you know, from from from customers into end to end customers in the factory in this case.
22:05 So there's really there's really value all along that that that life cycle, if you will, from product creation to product operation into service and then all the way all the way down to end use from customers as well. So with that, I'll wrap up and if there's any questions, feel free to use the mics here.
22:37 After recognition step. Is that a separate database or is that need to be connected to your your PLM your your Creo or. Yeah. The question was, is the expert capture instructions that a separate database or does that need to be connected to PLM? It does not need to be connected to PLM. It can complement information coming from PLM. So you can you can attack the problem from both sides if you'd like, but it's not dependent. Expert capture is designed to as as we named it,
23:10 capture the expertise from your subject matter expert. Use you know, typical SAS tooling to make sure you can get that that knowledge that expertise to to anyone so you don't don't need to connect to PLM. Hi, Jay. Thanks for the presentation. The expert capture I, I think it was a model recognition.
23:42 Is there a way to use ThingWorx or something like this for tracking? Yeah. So expert capture doesn't require 3D models. You can you can unlock some new capabilities with it with in it like step when you use a 3D model. But there's just general spatial tracking when you're creating, when you're creating a procedure, it uses our underlying technology, which we call area targets that map just an environment. So it's completely markerless in that regard. When you create the procedure, you can capture those those environments.
24:14 But what if I have like slightly different like manufacturing stations? Will it still work? Yeah, it should. It's highly accurate. Yeah. I mean. Come, come talk to us. Let's let's dig into the use case and learn more about the environment. Okay, We'll do. Thank you. Hi, Jiji. Ajit from. Great presentation. In the example of the Volvo truck.
24:44 The user could see the whole all the parts within the truck. How was that done? Was that purely through expert capture, or were there any other technologies involved ? Yeah. So the it's a good question. So in that case, the Volvo example, all of that product data, all of the parts, all of the visualization visualization information was pulled out of the PLM system, out of Windchill, which is which is really powerful from a scale perspective. And it's up to the designer of that experience at the end of the day or
25:18 the application developer to decide how much they want to expose to the end user. You can also see an interesting related use case. If you look at the Acoma demo where you can actually see all of the part information and then tap on the different parts to understand what it is, what the part number is, maybe you need to order it, maybe you want to you want to learn specifically about that part. So there's sort of different ways that you can use that visualization when you have all of that info.
25:49 It's more broadly with the Vuforia suite as soon as you plug into PLM. I think that there's there's there's there's information you can capture always that's going to come from a person or an experience, which is where expert capture fits in. And then and then when you want to connect to PLM, you can get sort of a different level of fidelity and, and a different type of scale.
26:19 Hello. Adam Bielecki from Robert Bosch Company. I would like to focus on the module of artificial intelligence because we have made a lot of POCs at Bosch. And what we discovered is to show people not only what to do during the instruction, but especially what they should not to do. Like one of the most important and and funny situation was that we created
26:49 an instruction and we ask someone who has no experience how this machine works to just put the HoloLens on the head and follow the instructions. And the first thing that he did was to push the big red button in the middle, which turned everything off. And afterwards we needed to turn it on and wait 15 minutes until it works again. So what I would recommend is to add some also possibilities for the hand tracking
27:27 or to to recognize your hands. And even though it shows you please do not press this button. It shows it. Supposed to show you that your hand is not in the right place. And be better to be careful because you can accidentally press this button and you can spoil everything. So when it comes to this, this artificial intelligence, I really hope it will help in our cases because it's really useful.
27:57 But it needs to be enhanced and we need to be sure that nobody will spoil that, because otherwise, if only one person per day will do the So we will have a lot of money. We could lost a lot of money. So then people in factories will decide, yeah , better to have a teacher next to him. Yeah. Instead of having these glasses. It is a great suggestion and it's something we're, we're actively working towards.
28:29 So there's a, there's a follow on session. Ryan Orwall back there our head of product raise your hand Ryan is giving. The tomorrow. 1:00 today where he'll give it a little bit of a peek into where we're going on some of these different dimensions of of the technology. And and our evolution of AI is one of those. But going beyond whether something is done correctly and to whether something is being done correctly or whether the actions that are being taken are correct is exactly where we're headed. Thank you.
29:07 Okay. Well, thank you. Thank you, everyone, for attending. And again, come see us at the booth. Or find me. Thank you.