A perfect Field Worker day

Welcome back to the my Metaverse day! This time I am going to walk you through to a day in modern Field Workers life and how Metaverse tech, that includes mixed reality and other devices, can make their day a perfect one. We can call our fictional field worker Marilyn in this example and follow her to a field trip onto fictional factory site.

Morning

The first thing Marilyn does, right after required amount of coffee and getting through her morning routines, is checking out on Microsoft Teams what kind of jobs are waiting there.

For this her company has provided a custom Power Platform application inside Teams to highlight schedule and tasks that are being managed in their Dynamics 365 Supply Chain Management and Dynamics 365 Field Service. It makes sense because her company is using D365 Supply Chain Management solution to manage their manufacturing process – having it’s IoT connected to the Microsoft Cloud for real time information and is also syncing information with Dynamics 365 Field Service and Azure Digital Twins for more advanced features. The company is using D365 Field Service to manage work orders and Supply Chain Management to make certain that maintenance is done during optimal slots regarding the manufacturing process to minimize downtime and optimizing factory throughput.

She sees that there are several tasks awaiting for her, but the most critical task is the replacement of controller units that are showing stress and needs to be diagnosed and replaced. Marilyn uses embedded Teams chat to ask from maintenance engineering team details about the timeslot. Meanwhile she is getting herself more details about the controller units and technical room where she is going to do the service work. She pulls out the 3D digital twin of the room and familiarizes herself with the rack design, cable routes, notes from previous technicians who worked on that and also sees her target control units highlighted there. She also pulls out information about control units and checks temperature history & lifecycle prediction- as well as nearby units to make sure there isn’t any wider problem with the rack or the room. She can view documentation, installation and service history of those units and makes sure she has the right units waiting for her at the pickup point.

After she has gotten the confirmation of the service time slot she can schedule her drive and required service time to the factory into her calendar. She realizes it has been a while since she was visiting the factory last time in person – she pulls up the 3D digital twin of the whole factory to remind herself about her route to the target location nearest parking areas and uses the tool to show her the route from the parking to the technical room. She also views safety requirements and passes required to access – and is happy when she sees all her stats showing green. Which was assumed so, since she was appointed to perform this task. Marilyn has learned from the experience it is better to check requirements and routes before getting to the location – that is why she watches the route in advance so she doesn’t have to rely on mixed reality guide while in the location.

She can do this using integrations with the Supply Chain Management, Field Service, Digital Twins and company’s databases. When digital twins and other services are created it is possible to map everything installed into the factory to documentation and locations. 3D scanned factory area and indoors can help to manage and plan service slots but also work as the interactive tour to the area.

As Marilyn recognizes, and confirms with her boss, the maintenance task at the factory has priority over other tasks that would not fit the day with this service task. Those are quickly delegated to other technicians. Marilyn drives to the pickup point and gets replacement parts as well as factory-certified HoloLens 2 set. Out of experience she makes sure it is fully charged. This is something she has learned from the experience – some people return devices without charging them.

Lunchtime

Since she has still time before she needs to start the drive, she uses the time to catch up with some learning. Opening Viva Learning in Teams she continues her certification towards senior technician and attends a on-demand training in 3D environment. Since she has her 3D headset at the other office, she uses her laptop to join the training using Microsoft Teams – while 3D experience would be more immersive and focused she can still join it in the 2D display and enjoy 3D Metaverse environment. Artificial Intelligence bots – or eTeachers or TeacherDroids as some call them – guide her through the training and answering her questions – and in the end of the training her knowledge is tested. As she is happy to pass and has a half an hour time left, she jumps to her team’s Metaverse space to see if any of her colleagues are present. As it happens, there were couple of technicians and she can go ahead and tell them she is one step closer towards senior designation. James, one of her colleagues, pops out a few questions on her previous work item and site since he is heading out there later and she is happy to provide him with information & tips before she needs head for a quick lunch and head out to the factory.

With upcoming Microsoft Mesh and Mesh for Teams enables people to join meetings, trainings and persistent spaces in 3D spaces (we call them Metaverses, some may prefer to use Virtual Reality spaces) with various devices. In this example Marilyn doesn’t have her VR headset with her so she joins the space with her laptop. She could be using also her phone or tablet if she didn’t have her laptop with her. In this fictional example she doesn’t want to use HoloLens for that, because she wants to make sure it is fully charged. We are still waiting for Microsoft Mesh to be published, but it is already possible to create motivating training and learning experiences with the use of HoloLens.

Afternoon

Marilyn drives to the factory using smart route she was assigned by Field Service. Entrance to the factory was easy – her car was recognized by AI and she confirms to the gate her identity and task there. She she familiarized herself with the are outdoors it was easy to find the parking area nearest the technical room. She picks up her backpack and tools and heads out towards the technical room. As she is isn’t really sure about the route (she doesn’t have THAT good memory) she takes out her phone and opens the factory guide app. Using Mixed Reality the app uses GPS and camera to see where Marilyn is and pops out a guide droid to the screen that lets her know where to turn to and helps her to reach the technical room. When the area has been 3D scanned it is possible to create virtual tour or mixed reality guide to help find the destination.

In the technical room Marilyn unloads her tools, equipment and puts on the HoloLens. As she activates the maintenance application she sees the information about her work floating in the air. She can see the application already recognized the location and the rack she needs to operate – it gets highlighed in her visor. It also shows her what tools she will be needing for next steps. Using guides she opens the rack and sees her target controller units highlighted in the HoloLens.

As she progresses with replacement of controller units she gets guides into her HoloLens as well as information about required tools, highlighted parts and step by step instructions.

In case she had any trouble, or there were anything missing from guides, she could also contact maintenance engineer who could help her from operations office. Using Dynamics Remote Assist feature.

After doing the physical maintenace work she also updates the information about new controller units to the documentation by scanning their codes in and verifying the installation. She then follows the guides to go through activation and diagnosis of those new units. After all those steps are done she picks up her gear and finally 3D scans the rack with her mobile phone to update the digital twin, as well as signing off the maintenance work done – with her notes of course.

On her way out she gets an urgent task and message from her boss to go to factory operations room before she leaves. She needs to help factory technician on troubleshooting the process flow. As she knows this is a task she really wants to do she is happy to accept and gets assigned a new task. Since she doesn’t know the operation room location she uses the virtual guide on her phone to direct her there.

With the digital twin of factory in place, she begins troubleshooting the issue with local technian. During the process she can use all those tools we already went through during her earlier mainenance work. With the exception that both she and the tecnician can see the digital twin and real time IoT information when using the HoloLens. They can see the service application recognizing (with the help of digital twin data) specific components in the physical world and make it possible for them to pull out more details about those components – seeing the history or service history of specific components.

Since they need more details, they also ask the principal maintenace engineer to join them virtually. They all are using HoloLenses, so they can be virtually in the same space despite the engineer being physcially in the other town. Marilyn and technician can see the engineer’s avatar standing next to them and the engineer can see their avatar. This allows them to collaborate together on the problem while the engineer can see what on site technicans see. They can add notes and bring out schematics & information about components and utilize the digital twin for testing out scenarios. Engineer can authorize the tasks and oversee the work being done

This speeds up the troubleshooting and enables new ways how to optimize the production.

At home, later in the evening, Marilyn thinks it was a really wise decision to make sure her HoloLens was fully charged before coming to the factory. This was really a perfect day for her – one passed certification, succesfull service work and helping the factory to save big money by troubleshooting the issue before it created downtime. It surely looks good on her KPIs and reports.

Marilyn used a wide variety of technologies in this example, showing the importance of digital twins, mixed reality and other Metaverse tech. Like many other people in the office, she didn’t use VR headset but other devices at her use mobile phone, work laptop and HoloLens mixed reality device. The Microsoft Cloud (and Metaverse solutions) is already making lots of examples described possible – using tools such as Dynamics 365 Field Service, Guides, Supply Chain Management, Power Platform, Azure, Mixed Reality and HoloLens can already improve productivity, meaningfulness and utilization for various industries – from manufacturing to services. Using tech in a smart way to enable new kind of , better and more productive, work day experiences for everyone.