Adding machine learning optimization analytics to the mix also increases productivity, by guiding personnel through tasks in an optimal way, as determined by machine learning optimization modeling. Reinforcement machine learning can optimize the order of tasks, note how best to perform each task, and help schedule and route crews to maximize productivity and reduce costs. Humans often exhibit biases that lead to suboptimal performance; using machine learning to find the optimal path, then using VR to train humans to overcome their biases, leads to higher productivity.

In reinforcement machine learning, programmers create a software agent that uses a set of actions to interact with an environment, with the goal of maximizing a target value. The target value can relate to business objectives such as faster processes, lower costs, lower defect rates, or fewer accidents. Over time, the software agent learns what actions to take to alter the environment and maximize the target value.

The software agent isn’t programmed to take specific actions in response to specific situations or input; it is programmed with a range of possible actions to take and the ability to observe the results of those actions. Each action the software agent takes generates data about the change in the environment resulting from the action. Each action also generates some level of reward related to the target value.

Over time, the agent discovers which actions generate the most long-term reward by using two approaches: exploration and exploitation. Using the exploration (trial and error) approach, the agent tries an action and then observes the result. When the exploitation approach is applied, the agent uses an action that has worked in the past, then observes how well it works in a new situation. Through these actions, the software agent improves its performance and maximizes its target value.

To apply exploration and exploitation, reinforcement machine learning needs ways to observe and influence its working environment. This isn’t a trivial exercise when it comes to maintenance operations, especially for assets and locations that don’t require frequent maintenance. While it is possible to develop these models with GPS tracking, careful logging, and a variety of sensors, it would still be a challenge to generate enough data—enough exploration—to refine the model. Virtual reality presents a unique solution.

Immersive Visualization: Generating Data to Drive Analytics

Virtual reality simulation has long been used for training, allowing personnel to tackle complex and at times dangerous tasks in a controlled, safe environment. It allows jobs to be mocked-up and observed, so potential mistakes can be identified and corrected.

VR also presents a unique opportunity to “analyze action.” It can help users understand human perceptual constraints. For example, an environment can be created and what workers notice and what they miss can be analyzed. The environment can then be modified and more trials run. This reduces the time required to design safety alerts and protocols, and then train workers to recognize and follow them. In a sense, VR can be used to develop “muscle memory” for safe practices.

In a properly instrumented VR environment, a stream of data can be produced on which to perform analytics. In a training environment, the user’s activity stream can be analyzed and checks can be performed to ensure all steps in a protocol were completed correctly. Efficiency can be analyzed. How many steps did it take the user to perform the task? Did they select the right tools? Did they perform unnecessary tasks? A performance score or a safety score can be calculated, and a user can be compared to various norms, accounting for experience, role, even geography.

A potential pitfall is that not everyone performs optimally in a VR environment. Differences based on past VR experience or even a tendency for some people to get nauseous in a VR simulation need to be considered.

These performance analytics can help in the development of a machine learning model to predict what an individual’s score will be in the simulation. By combining simulation performance data with demographic and other employee data, a sophisticated model that can predict who will perform well on any given set of tasks can be developed. It might also be used as a screening tool to understand which new hires are likely to succeed in a given role.

VR and AR can be used to train personnel to perform a given set of tasks in an optimal way, but to develop the machine learning model, developers need to understand all the different ways a task could be performed. One way to get that data is through VR training simulations.

The machine learning models need data such as how long it takes to perform a task, the starting locations of equipment and tools, and any constraints on performance such as safety protocols, union rules, and availability of personnel. It may be possible to collect this data over time from real-world maintenance activities, but VR simulations allow collecting far more data in a shorter period of time, and environmental parameters can be managed very quickly. In the real world, it’s impractical to repeatedly relocate assets, change signs, or reorganize a truck’s tools; in VR, almost anything is possible.

The data generated by the simulation is fed to a path analytics model that will find the optimal path through a set of tasks. An example of a path analytical model that may be familiar to readers is the “ant colony algorithm.” The process was developed based on the method ants use to find an optimal path to food—relying on exploration and exploitation.

As previously mentioned, a VR simulation can be used to understand the results of taking various paths through a series of steps. However, if a path analytical model is to find the optimal way to perform tasks, exploration must also be encouraged.

Gamification might be one way to provide the needed incentive to keep users looking for the ideal way to complete a simulation. Gamification is, as it sounds, a way to turn a set of tasks into a game. For example, points might be assigned to a given way of performing a task or for taking certain steps in a certain order. The points awarded could be reduced as more users perform steps the same way (exploiting the discoveries of others). More points could be awarded for trying something new, which would encourage exploration. As the model matures and exploration is no longer producing gains such as faster times or lower costs, the point system could shift to reward compliance with the optimal model.

This two-way, iterative approach to combining VR and analytics should be a potent operational optimization tool. VR can help managers understand present operations and train employees in current protocols. It can then produce data that ultimately enables improvement of operations and protocols. Operators can train employees in the new methods, creating a circle of operational improvements.

Virtual reality provides the context to use big data to make fast, confident decisions. Combined with advanced machine learning analytics, immersive environments become a powerful tool for optimizing operations. Asset-intensive organizations such as utilities, transportation networks, and manufacturing will use virtual and augmented reality to understand and take action on their data to improve the performance of their operations and assets. ■

—Jeff Weitzman leads marketing and product strategy at Space Time Insight Inc.