An explorer of extended reality

Anton stands on the rooftop terrace of an office building in Erlangen, Germany. Yet, his mind is inside a virtual medical examination room. He uses hand gestures to control a virtual X-ray machine under which the leg of a virtual patient lies. "Probably the greatest advantage of extended reality is that you can use the applications whenever and wherever you want," says Anton.

The 32-year-old, who self-mockingly calls himself a "tech-savvy nerd", joined Siemens Healthineers over six years ago. He initially worked in the Customer Services unit on the Education Project which delves into digital and immersive training opportunities, advancing product development in cooperation with customers.

Meanwhile, Anton Ebert has switched to the Enterprise Transformation & Collaboration team as a Senior Technology Manager. Together with his colleagues there, he's driving the advancement of digitalization with a focus on extended reality, and developing ideas for XR applications. 

Anton is also Senior Key Expert for Extended Reality (XR), working to advance the company-wide extended reality strategy in joint efforts with colleagues in the XR community, dubbed the "all sorts of reality" (ASOR) community. Their ambitious goal is to establish this technology throughout the company and, in the long term, steadily integrate it more firmly into the core business of Siemens Healthineers across the various business segments and product lines.

Anton studied media engineering at university, a field at the interface of digital media informatics and media design in which students are taught the fundamentals of digital technology as well as design. At university and the first stations in his professional career he was involved in game design, user experience (UX), camera work, audio design, and 3D modeling. 

This mixture prepared him well as a "cross-border commuter" between various disciplines for doing what Anton does as strategist and idea developer within the XR community: he can share ideas and information with programmers just as well as he can with UX designers, and summarize the technical requirements of an XR application as well as its challenges from a design standpoint. 

Perspectives "outside the box", too, are key to keeping pace with the pulse of latest developments: Anton and his colleagues in the XR community frequently collaborate with outside partner firms and startups. Especially important to Anton is the exchange he maintains with his working students, who mostly hail from the fields of media engineering and psychology, and who join in the work on all key projects: "They bring a fresh mindset to things, and new, innovative ideas."

Another aspect is the instinct to play – virtual gaming: "All my colleagues who work with augmented and virtual reality play video games in their leisure time – I do too, of course," admits Anton with a laugh. "It's good for our job, so that we understand: How do games work? How do we succeed in making people feel enthusiastic and happy when gaming? How do we keep them involved for longer periods of time? After all, what we don't want is for people who use our training applications to stop after two minutes because they're getting bored." 

Anyone who has ever put on a VR or AR headset can experience the feeling for themselves: the incredible curiosity and enthusiasm that these technologies can unleash. These emotions can be harnessed for creative and sustainable learning experiences. Studies have been conducted that demonstrate the success of these technologies in education.

So, where and how, exactly, can extended reality be used in medical technology? When asked this question in our #Futureshaper interview, Anton pauses briefly to gather his thoughts. He then reveals why he reflected: The scope of current and future possible applications is so wide-ranging and diverse that the abundance is almost a bit overwhelming.  

Anton explains that, for him, extended reality is one of the key technologies for bridging the human world and digital world, and for designing digital content interactively in natural ways. With its capacity for visualizing and simulating devices and processes, extended reality offers a vast range of benefits for all kinds of target groups. Here are few examples that Anton points out to us:

Already today, customers of Siemens Healthineers, hospitals, physicians and medical staff as well as university medical schools and their students are benefiting from the training scenarios made possible by virtual and augmented reality:

At the School of Medicine of University College Dublin (UCD) in Ireland, radiography and medical students train with the aid of Skilitics VR X-RAY. With this mobile, VR-based radiographic training unit, the students practice applying the suitable radiation dose for anatomically correctly simulated patients on virtual X-ray machines. Up to 50% of the practical training on actual physical X-ray units could be replaced in this way.

With Augmented reality concepts like the ExpertGuidance mobile training application for tablet computers that Anton helped facilitate clinical staff efforts to onboard new equipment systems from Siemens Healthineers, train new colleagues, and undergo regular continuing education and training.

Simulation and augmented reality can also help support efforts to better visualize and understand the human body: At University Hospital Erlangen, physicians and medical staff in the Department of Pediatric Cardiology work with a combination of the photo-realistic visualization technology called cinematic rendering and augmented or mixed reality. This enables them to transform detailed 3D views of patients' hearts into holograms that can be zoomed and observed from every possible perspective. This helps doctors to better plan medical interventions in children's small hearts.

Siemens Healthineers has developed an immersive virtual reality training program in partnership with PrecisionOS: this software enables surgeons and technicians to authentically practice their joint work procedures with the mobile 3D C-arm Cios Spin for intraoperative 3D imaging in a virtual-realistic environment.

Solutions like those described above can contribute to optimizing clinical processes, save time, cut costs, and prevent errors in clinical reality.

And all that could even be expanded much further, says Anton: "Moving forward, we're striving to be able to simulate combined workflows of multiple medical devices and multiple physicians and medical staff." In the more distant future, virtual patients could then be added to the mix.

"Real" patients would of course also benefit from these solutions. The future will bring much more potential for developing this concept, too: "With the aid of virtual reality, patients with a fear of medical examinations, for example, could be 'teleported' to a pleasant or relaxing virtual environment, or distracted somewhat from their pain."

In the longer term, extended reality could take the art of collaboration within a global company like Siemens Healthineers to a whole new level. The XR community is already experimenting in ways of coming together in VR meeting rooms with aid of avatars: "This technology helps us interact with one another more naturally because it lends us the feeling of being physically in the same room – significantly better than what a video stream can do today," explains Anton.

Employees of Siemens Healthineers as well as customers could benefit from the myriad of possible virtual training and onboarding scenarios. Scenarios with augmented realty headset or tablet computer are imaginable for further optimizing the manufacture of medical equipment, for example by means of a virtual checklist with step-by-step instructions that could be displayed in overlay on the real plant environment to support workers in production processes. 

XR could also revolutionize the way in which we as a company work in future on innovations: Siemens Healthineers maintains multiple innovation centers, including ones in Shanghai, China and Bangalore, India. These are physical locations where researchers focus their work on new technical innovations.

Extended reality technologies should in future play to their strengths in this area as well, says Anton: "We're currently planning a virtual innovation center. The idea is to enable our researchers worldwide to collaborate independently of their respective locations with the aid of intelligent environments and virtual avatars." 

The world today is more interconnected than ever before. And yet, social distancing is sometimes imperative – a lesson we learned quite starkly from the Covid-19 pandemic. XR can create new spaces in which people can interact with one another – no matter where they happen to be in the real world. Extended reality technologies can help prevent unnecessary travel, and thereby cut costs and reduce our carbon footprint.

XR opens up opportunities not least in connection with other future technologies such as digital twinning, explains Anton: "XR can render visible what initially were abstract digital twins of devices or people, and make them 'tangible'. Thanks to extended reality, we could interact with them at some point."

Even revolutionary future concepts like Microsoft's digital parallel universe metaverse currently hyped in Silicon Valley could theoretically offer extensive opportunities for medical technology: a complete digital world consisting of digital representations of people, things, and places. It's a new form of Internet, so to speak, in which, with the help of virtual reality, we could interact almost exactly as we do in the physical world. 

Yet, this beautiful new virtual world also harbors numerous risks: "A good many fundamental questions will have to be clarified before we as a company would be able to develop realistically implementable concepts," Anton explains. Questions such as: What form of metaverse will prevail in the end? Whom does this metaverse belong to? Who regulates a metaverse? And, how secure are sensitive data there?

Extended reality cannot offer the experience of actual touch. Or rather – not yet. Research work is already underway in this field, as well: "Research is delving into vibration sensors that would let you think you are touching something, or exoskeletons that can simulate resistance," explains Anton. Future developments in this field could enable surgical training applications, for example, to be designed much more realistically.

In his private life, Anton Ebert prefers things down to earth, rather than "nerdy". There, he doesn't putter around with his hands in empty space, but rather in his own hobby workshop: with wood. And entirely without VR headset: "After all, that's also a part of life."