Bringing physical world to life in the Metaverse

The metaverse is much more than just 3D meetings and collaboration. In fact, you could already take steps on building the metaverse, because it is about connecting the physical and virtual worlds together – taking advantages of both and enabling organizations to succeed. The base for this has happened in the industry for a long time already – think process automation technologies in various factories, breweries and other manufacturing facilities: probes and other physical devices create digital measurements like temperature, amount of flow and there are physical devices that manage the process by controlling valves and motors. There are other examples like office / home automation (temperature, security) and traffic control on both ground and air. Think all those tinkerers who used Raspberry Pie devices to measure and control things. Some log that data and use it in Power BI reports.

All these are already providing signals and data that could be input for Digital Twins. And these Digital Twins are building blocks for connecting and synchronizing the physical world with the metaverse. While the data and integration could come from various sources, Microsoft Metaverse stack also provide ways how to make physical and virtual worlds work together.

Of course Azure plays a big part in that. Intelligent Cloud and Edge technologies have been there for some time already and they are improving all the time. Intelligent Edge can bring smartness to the location, for example computer vision that runs on the edge and it can be used, for example, to detect spills by analyzing the video with machine learning. All this provides signals to Azure that can be processed with the help of AI. The synchronizing part means that the results – either by automated or human acted actions – are fed back to the process via integrations to the service bus to actuate physical devices controlling the process.

Anecdote: In late 80s when I chose my education path, I selected process automation engineering technology. I worked my trainee periods in a paper mill (automation process field worker fist, then in the IT department) and later on participated in building a paper mill solutions before changing domain, after a few years, to finance and fully into Information Technology (developer, databases, reporting, Microsoft stack and so on). My reason to take automation field, instead of information technology, was simply connecting physical world and computers. Little did I knew that this would give me understanding to work on the Metaverse almost 30 years after I graduated in mid-90s.

All this sounds a bit complicated of course, but it can be said simply as well: connecting and synchronizing physical world with the metaverse using Azure Services and using Mesh for Teams to enable users to interact with it.

Mixed Reality / Metaverse picture by courtesy of Pouneh Kaufman, Microsoft

Receiving signals and data, processing it and making actions back to that. It could be on the scale of home, office, factory or span globally or even universally. That is a big difference: we can have and process vast numbers of data with the help of Azure and AI that wasn’t feasible in “old days”. This means we can analyze and predict much more and do it much more quickly.

Using Azure IoT enables to create and connect physical world to Azure.

Microsoft Metaverse enables the next level for the physical world

Once we have the data we can go forward to start creation of Digital Twins by modeling the real world environments, using, IoT and line-of-business systems and giving everything a location with Azure Maps. Synchronizing data between digital and real world means we start gaining business insights and can analyze that data.

Analyzing the data, creating predictions and comparing data to history are one step to level up the reality. Ability to run simulations to your model also means faster results (fail fast..) without affecting your physical world. Examples of this can be, besides process automation that has used those simulations for decades, 3D walk-in models or devices in the virtual world – reviewing designs and performance. All these simulations, predictions and comparisons save money and time because you don’t need to interact with the real world. Of course it takes effort to build the Digital Twin, but as usual: scales differ: in some cases you need only measurements and not the visual model (or the model can be crude, until there is a use-case to improve it).

That is all IoT, logic, data storage and algorithms. Analyzing camera feeds to virtualize spaces, events and watching out for triggers that require actions. That is Azure, Machine Learning and AI. The next level-up comes with an easy way for users get access to all that. That is where you can bring in Power Platform: to create low-code tools for users to interact with metaverse. Embedding Power Apps, process flows and reports where users, controllers, maintenance or business decision makers need them.

Microsoft Mesh adds the new User Experience level

The final step is to make all these accessible and usable to your team and organization. When making decisions you need collaboration, seeing simulation results, history data and work on details. Your team may be scattered in different locations. This is where Microsoft Mesh and especially Mesh for Teams steps up. Adding the immersive Metaverse experience to Microsoft Teams makes it easy for everyone needed to participate actively – be it via a headset (Mixed Reality, HoloLens, Virtual Reality), desktop or mobile.

Microsoft Mesh environment can contain and interact with relevant Digital Twins, information and tools that are available to people, represented by their digital twins: avatars, using it . No matter what the physical locations is, they can occupy the shared virtual space along with the Digital Twin to collaborate and make better business decisions. Some people could be on the factory floor level and some analyzing the data from other countries.

Azure Digital Twins combined to Microsoft Mesh and to Teams (people) is how Microsoft Metaverse connects and synchronizes physical world to be part of Metaverse.

I recommend you to watch this Microsoft Video :

You can already takes steps building the Metaverse with Microsoft tools by starting building Digital Twins. This is the Microsoft metaverse technology stack available: