Virtually amazing

Imagine being a University of Michigan student or a learner anywhere in the world, logging on to an online course. Instead of seeing a talking head delivering a lecture, you see your instructor walking through ancient Cairo, in an operating room in Tokyo, or on a construction site in Rome.

When instructors step into the new extended reality stage at U-M’s Center for Academic Innovation, ​they can transport learners anywhere or any time in the world. The center is building online learning opportunities so students can engage and dive into an immersive virtual journey. ​There are no more limits to teaching and learning.

The XR studio can incorporate augmented, virtual and mixed reality into virtual production and was built upon the technological milestones achieved by the Disney+ series The Mandalorian. Producers figured out how to use large LED screens and cameras to render 3D environments where the cast is immersed in real-time. The technology produces high-quality video content and doesn’t require a lot of post-production work after filming is completed.

The center’s XR studio is one of six studios at its new facility in downtown Ann Arbor dedicated to developing accessible online learning experiences. While that is a long way from virtual production’s Hollywood origins, the new studios have the potential to change the way people teach and learn just as virtual production changed the film and TV industry. That transformative power comes from the combined expertise of U-M’s faculty and the center’s digital artists, media producers and learning experience designers.

“The studio opens a new chapter on scalable, personalized and immersive learning technologies, giving us an opportunity to pull students into different environments that would be very difficult to create on campus,” said James DeVaney, associate vice provost for academic innovation and founding executive director of the center.

“This allows our learners to practice in high-stakes environments while reducing risk. It helps students develop a sense of belonging in complex areas and solve complex problems in an increasingly uncertain world.”

Jonathan Rule, assistant professor at the U-M Taubman College of Architecture and Urban Planning, set foot on the new stage looking for a more engaging way to teach his classes in construction and design.

“My research has always been aligned toward studying things that would be beneficial for use in practice,” he said. “That might be how we use new technologies to shape teaching and practice. Then, once a student finishes school, he will be prepared to thrive in architecture.”

Rule leads “Augmented Tectonics,” a new project that leverages VR and AR and demystifies construction concepts currently presented through abstract drawings and images. Rule is exploring the use of XR to redefine the meaning of immersive and hands-on education for students and learners beyond campus.

“By creating an immersive environment, construction concepts become more tangible and supplement current learning methods through drawings and images,” he said. “Teaching on the XR stage, I can categorize, animate and highlight things. I can pick up augmented reality objects and move them around. You can actually pick up the virtual elements as if they were physical.”

In control

Eric Schreffler, XR developer at the Center for Academic Innovation, works in the XR studio control room. (Image credit: Jeremy Marble, Michigan News.)

“I can pick up the intersection of two walls, for example, so one is perpendicular to the other,” Rule said. “It allows me to show viewers how those members come together to create a corner and where you can attach the finished material to it. Things that are complex layers are easily made visible through these types of technologies.

“This also begins to grab the attention and sparks the interest of a student or a viewer, as opposed to a static image. It’s interactive. It’s in motion.”

The new studio is set up like a broadcast facility that can support asynchronous and live multi-camera productions as well as cinematic-quality projects. In combination with augmented reality, camera tracking brings an unprecedented level of interactivity and engagement to course creation, making it a more engaging experience for learners.

“We have Hollywood here in Ann Arbor, and our faculty and student communities have access to this technology,” said Jeremy Nelson, senior director of XR, media design and production at the center. “This technology will shape the future and is a step forward in making education accessible to everybody.”

According to Nelson, some universities have similar studios to teach film students how to use this technology to create movies and broadcasts. But U-M’s new XR studio is the first in North America to use technology to support teaching and learning goals in online courses for all subjects and disciplines.

The center is experimenting with different types of environments from digital twins of the U-M Diag or the Ford Nuclear Reactor to tiny worlds like the inside of a DNA helix to fantastical worlds on other planets or deep in space.

Another add-on of the new stage is the ability to “teleport” another person from anywhere in the world.

“If a scholar is home, let’s say, in California and has a green screen behind him, we can beam him onto the stage in Ann Arbor,” Nelson said. “We are using the studio to collaborate and create courses in engineering, mathematics, science, dentistry, medicine and many other fields. We have a studio here to create online learning of the same caliber as ‘The Mandalorian.’”

Business education

The center’s XR studio builds on the university’s first investment in digital education studios at Michigan’s Ross School of Business. Since 2019, students in the Online MBA and Executive Education programs have benefited from instruction delivered by faculty in the school’s studio. Featuring graphics technology similar to immersive video games, the studio creates a 3D environment where online students become part of a fully engaged virtual classroom community.

“Our XR studio expands our ability to connect with students around the globe, allowing learners with busy jobs and lives to still invest in their growth and development from afar,” said Lindy Greer, U-M professor of management and organizations. “The technology makes the classroom experience seamless, so faculty and students can interact fluidly and leverage the benefits that technology-fueled education can offer.”