Professor Mark Billinghurst from the Auckland Bioengineering Institute (ABI) has joined a very exclusive global club of pioneers honoured for their role in building the AR (Augmented Reality) and VR (Virtual Reality) industry.
He has been inducted into the inaugural Augmented World Expo (AWE) XR Hall of Fame for his groundbreaking work in AR and VR, something he describes as both a great surprise and an honour.
AWE is the oldest and largest professional XR conference, and the XR Hall of Fame recognizes 101 people worldwide who played pivotal roles in creating the modern AR/VR industry. Billinghurst is the only person from NZ in the list and one of only 15 academics globally chosen. He joins such famous people as Ivan Sutherland, the developer of the first AR head mounted display, Neal Stephenson, the author who coined the term Metaverse, and John Carmack, creator of the videogame Quake.
Billinghurst is director of the Empathic Computing Laboratory (ECL) at the University of South Australia and at ABI in the University of Auckland. He researches how virtual and real worlds can be merged, publishing over 750 papers on AR, VR, collaboration, empathic computing, and related topics.
His current research focuses on using empathic technology to connect and collaborate in new ways. “Empathy is about seeing with the eyes of another, listening with the ears of another and feeling with the heart of another. Our goal is to develop computer systems that recognise and share emotions which help people better understand one another.”
He cites ECL’s Empathy Glasses as an example. This AR headset relays emotional and physiological information about the wearer, such as their gaze, facial expression, and heart rate, to a remote collaborator. “I could be in Adelaide working on a task using the Empathy Glass technology and my colleague in Auckland could be seeing, hearing and feeling what I’m feeling in real time. In this way we can we work together with a closer connection than typical conferencing tools.”
Billinghurst’s world is one of the fastest moving industries as AI and VR grow exponentially in capacity and reach. He sees a near future of revolutionary change in the way we communicate with one another where AR and VR enhance face-to-face and remote collaboration, making AR technologies more accessible and practical across various domains.
“EC systems will be able to tell whether I’m paying attention on a video call or whether I’m becoming stressed by a situation. We’ll see the emergence of large-scale, online collaborations, enabling people to connect together in very different ways that will lead to deeper understanding.”
The combination of AI with AR and VR provides even more opportunity for new types of collaboration. Billinghurst expects AI to drive the development of virtual people in the collaborative space in the next five years. “They will look and sound like us, creating yet another significant effect on how we communicate. We will develop systems where AI creates enhanced communication layers between people on video conferencing tools such as Zoom, enabling real-time translations for each participant, an instant transcript, a meeting summary, and a list of action items from the meeting.”
However, he says AI is not without challenges. “What I’ve described is how we think AI might be deployed but in reality we don’t have any accurate sense of what will happen. AI is like an accelerant tossed onto a fire, and the technology is progressing extremely quickly.”
“Just four or five years ago, we had AI systems that could have conversations but there was an unnatural time-lag, and they had limited conversational ability. Last week, Open AI released ChatGPT-4o which can conduct real-time conversations in a very natural way, suggesting applications as diverse as marketing, game play and teaching.” The ECL is using this technology to create virtual characters that can facilitate meetings and assist in brainstorming.
Early in his career Billinghurst was a co-developer of the first open-source computer vision AR tracking library, ARToolKit. He invented the award-winning AR MagicBook which seamlessly transports users between reality and virtuality. He also pioneered multimodal input methods for AR/VR interaction, combining natural language and AI techniques to allow interaction with an intuitive mix of voice, gesture, speech, gaze and body motion.
Since then, he has published more research papers in AR than any other academic globally, and his work has had a significant impact on the field. As a result, in 2013 he was elected as a member of the Royal Society of New Zealand, and in 2023 became an IEEE Fellow. In 2022, he became only the second New Zealander inducted into the SIGCHI Academy of the ACM SIGCHI, after Andy Cockburn from the University of Canterbury in 2015. ACM SIGCHI is the largest international society for professionals, academics and students in Human-Computer Interaction (HCI). It is part of the Association for Computing Machinery, the world's largest scientific and educational computing society, with over 110,000 members.
For more information about the AWE XR hall of fame, and for the full list of awardees, go to https://www.awexr.com/hall-of-fame
Comments