NVIDIA, one of the tech sector’s power players, is pushing the Universal Scene Description protocol as the foundation of interoperable content and experiences in the metaverse. In a recent post the company explains why it believes the protocol, originally invented by Pixar, fits the needs of the coming metaverse.
Though the word metaverse is presently being used as a catchall for pretty much any multi-user application these days, the truth is that the vast majority of such platforms are islands unto themselves that have no connectivity to virtual spaces, people, or objects on other platforms. The ‘real’ metaverse, most seem to agree, must have at least some elements of interoperability, allowing users to seamlessly move from one virtual space to the next, much like we do today on the web.
To that end, Nvidia is pushing Universal Scene Description (USD) as the “HTML of the metaverse,” the company described in a recent post.
Much like HTML forms a description of a webpage—which can be hosted anywhere on the internet—and is retrieved and rendered locally by a web browser, USD is a protocol for describing complex virtual scenes which can be retrieved and rendered to varying degrees depending upon local hardware capabilities. With a ‘USD browser’ of sorts, Nvidia is suggesting that USD could be the common method by which virtual spaces are defined in a way that’s easy for anyone to decipher and render.
“The most fundamental standard needed to create the metaverse is the description of a virtual world. At Nvidia, we believe the first version of that standard already exists. It is Universal Scene Description (USD)—an open and extensible ecosystem for describing, composing, simulating, and collaborating within 3D worlds, originally invented by Pixar Animation Studios,” writes Nvidia’s Rev Lebaredian and Michael Kass.
“[USD] includes features necessary for scaling to large data sets like lazy loading and efficient retrieval of time-sampled data. It is tremendously extensible, allowing users to customize data schemas, input and output formats, and methods for finding assets. In short, USD covers the very broad range of requirements that Pixar found necessary to make its feature films.”
Indeed, CGI pioneer Pixar created USD to make collaboration on complex 3D animation projects easier. The company open-sourced the protocol back in 2015.
USD is more than just a file format for 3D geometry. Not only can USD describe a complex scene with various objects, textures, and lighting, it can also include references to assets hosted elsewhere, property inheritance, and layering functionality which allows non-destructive editing of a single scene with efficient asset re-use.
While Nvidia thinks USD is the right starting point for an interoperable platform, the company also acknowledges that “USD will need to evolve to meet the needs of the metaverse.”
On that front the company laid out a fairly extensive roadmap of features that it’s working on for USD to successfully serve as the foundation of the metaverse:
In the short term, NVIDIA is developing:
- glTF interoperability: A glTF file format plugin will allow glTF assets to be referenced directly by USD scenes. This means that users who are already using glTF can take advantage of the composition and collaboration features of USD without having to alter their existing assets.
- Geospatial schema (WGS84): NVIDIA is developing a geospatial schema and runtime behavior in USD to support the WGS84 standard for geospatial coordinates. This will facilitate full-fidelity digital twin models that need to incorporate the curvature of the earth’s surface.
- International character (UTF-8) support: NVIDIA is working with Pixar to add support for UTF-8 identifiers to USD, allowing for full interchange of content from all over the world.
- USD compatibility testing and certification suite: To further accelerate USD development and adoption, NVIDIA is building an open source suite for USD compatibility testing and certification. Developers will be able to test their builds of USD and certify that their custom USD components produce an expected result.
In the longer term, NVIDIA is working with partners to fill some of the larger remaining gaps in USD:
- High-speed incremental updates: USD was not designed for high-speed dynamic scene updates, but digital twin simulations will require this. NVIDIA is developing additional libraries on top of USD that enable much higher update rates to support real-time simulation.
- Real-time proceduralism: USD as it currently exists is almost entirely declarative. Properties and values in the USD representation, for the most part, describe facts about the virtual world. NVIDIA has begun to augment this with a procedural graph-based execution engine called OmniGraph.
- Compatibility with browsers: Today, USD is C++/Python based, but web browsers are not. To be accessible by everyone, everywhere, virtual worlds will need to be capable of running inside web browsers. NVIDIA will be working to ensure that proper WebAssembly builds with JavaScript bindings are available to make USD an attractive development option when running inside of a browser is the best approach.
- Real-time streaming of IoT data: Industrial virtual worlds and live digital twins require real-time streaming of IoT data. NVIDIA is working on building USD connections to IoT data streaming protocols.
Nvidia isn’t alone in its belief that USD has an important role to play in the coming metaverse. The idea has also taken hold to some extent at the newly formed Metaverse Standards Forum—of which Nvidia and thousands of other companies are members—which has also pointed to USD as a promising foundation for interoperable virtual spaces and experiences.