Digital and Physical Worlds Collide in Future HMIs

PTC is moving beyond its CAD roots, leveraging innovative augmented reality tools to create new ways for humans to interact with machines.

Digital and Physical Worlds Collide in Future HMIs
Digital and Physical Worlds Collide in Future HMIs

For years, the manufacturing industry has been focused on “empowering operators,” which means arming them with more information gleaned from intelligent devices and data analytics to enable them to make better decisions. The next step in this era of “Operator 4.0” is the addition of augmented reality that allows an on-site service technician to don a hands-free headset, like Microsoft HoloLens, to access 3D work instructions and maybe even stream in a remote expert via Skype to help with troubleshooting.

All of this is great, and highly productive, but for forward-thinking companies, like PTC, it’s just not enough. The future human machine interface (HMI) is a symbiotic blend of people and technology. It’s the ability to interface to the physical world on a digital layer. A bi-directional exchange of information to create complex interactions between objects.

To get to the next generation of HMI, PTC has been building out its augmented reality product portfolio, investing in cutting-edge technology that will redefine the way in which humans interact with machines.

The PTC acquisition of ThingWorx in 2013, a platform for building and running applications for the Internet of Things (IoT), was a turning point for the company known for its CAD software, serving as a foundation for building connected products. That deal was followed by the acquisition of Vuforia augmented reality (AR) technology in 2015, allowing the company to leverage the two platforms to blend AR and IoT. And, late last year, PTC unveiled its Reality Lab test center, an incubator for emerging AR technologies led by researchers who will create applications that leverage augmented, virtual, mixed and physical realities. The purpose is to change the way we interact with our physical environments, adding a digital layer onto everything we do.

All of this is coming together with even more out-of-the-box thinking, which was on display earlier this month during PTC LiveWorx. In particular, the company demonstrated how AR technology from MIT Media Lab tech startup Waypoint Labs, which PTC recently acquired, can chronicle the real-time movements of a person wearing an AR headset by monitoring the individual both audio-visually and spatially in 3D. According to the company, the technology enables users to capture this information through a knowledge processing engine and convert it into any number of formats, including an interactive website with photos and videos, Microsoft Word documents with photos, or an interactive AR training where trainees wear a similar headset.

Waypoint and another recently acquired technology, called Reality Editor, currently make up the PTC Reality Lab. Reality Editor, born from the MIT Media Lab Fluid Interfaces Group, is a web-based tool for digitally controlling the physical world.

Referred to as “a digital screwdriver to fix and manipulate how things behave,” Reality Editor works as if everything has a digital layer on top of it that allows a person to adjust information about the environment around them. For example, with the visual programming editor within the application, people can create complex interactions between objects. In a simplistic example, a child can use this open web technology interface to program a Lego car by holding an iPhone over the discrete components, then simply pointing on the phone screen where they want the car to go. The physical object then responds, rolling to the location that it was digitally programmed to go by the child through the phone interface.

Honestly, it seems so easy that it’s a hard concept to grasp. But the work that PTC is doing with the MIT Media Lab, which it is now a member of, promises to change the way in which we design, program, deliver and maintain the products we make.

And it is not just about changing manufacturing, but changing mankind. It was uplifting to hear that PTC also recently announced a project with Boston Children’s Hospital, the number one pediatric hospital in the U.S. Together, the organizations will explore the use of augmented reality to help children with autism and related developmental disorders, finding new ways for them to communicate and learn more effectively.

The goal, overall, is to use AR to make a difference in business and in people’s lives. I like where this is going.

Companies in this article
More in Control