The new demo, which took place May 4th, 2022, involved the streaming of hundreds of 3D objects in a complete 3D scene, creating a seamless 3D experience for the viewer. Now, the viewer will be able to watch and interact with longer form 3D content on multiple immersive displays.
For this demo, we will employ both Unreal and Unity game engines as clients. Over 90% of developers and content providers creating content for immersive displays are using these game engines to create content, so this is a major step towards bringing 3D streaming to a wider audience. The use of Unreal and Unity runtimes paves the way for targeting current and future light field based holographic displays such as the recently announced SolidLight display from IDEA founding member Light Field Lab.
This is a new method of 3D streaming to current and future immersive displays. The demo will start from an ITMF scene file created in OctaneRender software. Developed by an engineering team from IDEA Member Companies Charter, OTOY and CableLabs, the demo will start with a small-footprint thin client (the Unreal Engine application). The server cluster and Network Orchestrator will leverage the output from the newly released IDEA open-source library “libitmf” to help deliver the ITMF 3D scene (meshes, materials, transforms) from an edge network. Reversing the paradigm of how video is streamed to high-definition displays, 3D streaming provides resolution agnostic 3D content with low bitrate and bandwidth requirements. The tradeoff is compute power at the client versus video codec based streaming which increases bitrate and bandwidth as resolutions increase. The demo will be targeting a recently released holographic display from IDEA member company Looking Glass Factory. This consumer-friendly display provides the viewer with the ability to view 3D content without wearing special glasses or eye tracking.
In this talk we describe how we streamed a light-field from the cable network edge to a holographic display that resolves multiple viewer perspectives simultaneously. The system is also capable of changing camera perspectives based on user gestures. This demo was shown in October 2020 at the Society of Cable Telecommunication Engineers (SCTE) Tech Expo, which is a pre-eminent venue for thought leadership in broadband network evolution. We shall review the video of the demo and delve into the challenges we had to overcome to build the end-to-end delivery architecture.
Mixed Reality experience demands the highest performance graphics computation using dedicated GPUs to enable truly immersive experience for fast, smooth, and high fidelity visuals; concomitant with high precision spatial mapping for the fusion of the real and virtual worlds. Such performance is not possible to achieve on stand-alone un-tethered mobile devices such as AR/VR/MR headsets owing to the necessary power, thermal, and weight limitations.
New display technologies capable of reproducing realistic 3D images are now under development. In the near future, it may even be possible to bring the fictional “Holodeck” to reality – creating a truly immersive, glasses-free visual experience. In this segment, you’ll learn about recent developments in these Immersive Displays, including multiplanar, volumetric, wavefront holography and light field. This range of technical approaches can be described through a taxonomy of optical design based on generating either points of light, rays of light or waves of light. These methods each have advantages and disadvantages, so will likely support different applications and use-cases.
Forest Gibson, Co-founder of Pluto VR shares insights and lessons learned about Immersive Communication. The talk will cover: What is Immersive Communication and how it relates to current remote communication tools. The Uncanny valley and its importance in Immersive Communication. The basic building blocks required to produce compelling in person like connections.
Visby (visby.io) takes you behind the scenes of our light field video shoot for Charter’s SCTE Cable-Tec Expo keynote demonstration. The cutting edge shoot represents several technical firsts, including outdoor light field video. We take you through preproduction, production and postproduction to demonstrate the process of going from creative concept to holographic video, streamed over DOCSIS 4.0 into a Looking Glass 8K display. We introduce Radiant Images’ 100 Sony camera 4K array, bring you on location to the shoot, discuss technical and logistical challenges of light field capture, and reflect on the state of the art and readiness for widespread adoption.
Real-life use cases for immersive displays today.
The ITMF 1.0 Specification launches a media format for the interoperability of immersive 3D scene data and its assets. It establishes the footing for a standard industry format for mastering, carriage, interchange, and distribution of immersive media. ITMF’s assets are device agnostic enabling the media to be transcoded in a media and application aware network to match the end user device. In this session, we break down the suite of ITMF specifications, look at its role in comparison to currently used scene description formats, and preview some features of the next version.
How will immersive technologies improve our ability to live, work, learn and play? Immersive media will enable content creators to change narratives and presents new opportunities and challenges to production. Tune into to this informative discussion with Eric Klassen, Executive Producer at CableLabs, to hear (and see) how he envisions the near future that immersive media will bring, and how the mission of IDEA is critical to making this near future a reality.