User immersion, between 360° video and virtual reality

When it comes to creating immersive environments, designers have a wide range of tools at their disposal. Mixed reality, which allows users to be immersed in a more or less interactive environment, encompasses everything from augmented video to fully synthesized 3D images. To determine which is the best option, researchers from Carnot TSN (Strate école de design) and Carnot M.I.N.E.S (Mines ParisTech and IMT Mines Alès) joined forces. They compared, on several use cases, the differences in user engagement between a 360° video device and full 3D modeling - i.e. virtual reality.

" On the Carnot TSN side, and together with engineers from Softbank Robotics interested in the project, we worked on the case of a prototype museum," explains Strate researcher Ioana Ocnarescu. A room containing artefacts such as the Minitel, tools linked to the development of the Internet, photos of major robotics researchers and robots was set up at Softbank Robotics to create a mediation on the register of science and technology. Once the room has been set up, a 3D copy is made, and a path is laid out between the various objects. This scenario serves as the basis for both the shooting of the video in a 360° device guided by a mediator, and the creation of a virtual guide - in the form of a Pepper robot - evolving in the 3D scene to accompany the spectator. In both cases, the user is immersed in the environment using a mixed-reality headset.

The test 360 video mediation is animated, like the 3D one, by a Pepper robot.

Freedom or reality: the choice is yours

Beyond the naturally different graphic aspects of video and 3D modeling, there is a fundamental difference between the two: freedom of action within the scene. " In 360° video, the viewer is rather passive," describes Ioana Ocnarescu. " He follows the guide, can zoom in on objects, but is not free to move around. His only room for maneuver is reduced to turning his head and making choices to linger on one object rather than another. To achieve this, the video has been cut several times, allowing a decision tree to be constructed that links to specific sequences according to the user's choices.

3D modeling, on the other hand, gives viewers a great deal of freedom. They can move freely around the scene, choose whether or not to follow the guide, turn around the artifacts on display, and look at them in any orientation they wish - whereas 360° video is limited by the position of the camera. " Feedback shows that some content is better suited to one device or another," reports Strate's researcher. For a painting or a photo, for example, the usefulness of moving around the object is negligible, and the viewer will prefer to be confronted with the artifact in its environment with as much fidelity as possible. " 360° video will therefore be more suited to museums organized in corridors with paintings hanging on the walls," she points out. Conversely, 3D modeling is particularly suited to the study and contemplation of 3D artifacts such as statues.

These experiments are invaluable for design researchers, not least because they involve real users. " Knowing what people do with the devices offered to them is at the heart of our thinking ", stresses Ioana Ocnarescu. For over 5 years, Strate has been studying the interaction between users and machines to develop more effective interfaces. In the case of this project, the people immersed can give their feedback directly to the Strate team. " That's what's most valuable about our work; when everything is controlled in the laboratory, the information we draw is less meaningful. "

Tests still need to be pursued to incorporate maximum feedback from the widest possible range of populations. Once completed, the results will be compared with those of other use cases explored by Carnot M.I.N.E.S. " For their part, Mines ParisTech and IMT Mines Alès are comparing the same two devices, but in the case of autonomous cars and the exploration of the Chauvet cave ", explains the researcher.