Evolution of mine rescue virtual reality training

Thursday, February 27, 2020

“The Mine Rescue Experience” VR demo still in use at Dynamic Earth

Screenshot of mine rescue virtual reality training scenario with volunteer helping injured workerMine rescue is Ed Wisniewski’s virtual reality (VR) playground. And the NORCAT director of technology’s VR mine rescue playground is getting bigger, faster.

“The ball that we pushed down the hill five years ago is really rolling,” Wisniewski said, noting technological advances released last year moved the virtual playground into a more practical realm with significant potential for mine rescue training.

“In fact, in a few months, it will be even more amazing,” he said, though issues remain to be resolved before VR mine rescue training becomes an actual reality. “It’s just a matter of time.”

Wisniewski said NORCAT had been engaging with clients for more than two decades, attempting to introduce tech-enabled methods of training and development, however momentum only started to build in 2016 after the release of their mine rescue simulation. 

“Initially, industry saw virtual reality as gaming technology,” he said. “They would respond saying that health and safety is not a game, and we’re not interested.”

“But just because it’s gaming technology . . . doesn’t mean we can’t use gaming technology to train within health and safety.” 

Retaining knowledge through meaningful experiences

The technology provides opportunities to engage learners, and virtual reality allows them to gain and retain knowledge through meaningful experiences.

The pendulum began to swing in 2016, when Science North approached NORCAT to develop a “Mine Rescue Experience” for Dynamic Earth in Sudbury. For Wisniewski, it was mandatory that Ontario Mine Rescue (OMR) be involved to provide guidance and industry reflection.

The Mine Rescue Experience, still in use at Dynamic Earth, was a hit.

“At that time, the experience we built didn’t incorporate a lot of VR technology, but used avatar technology in the style of Call of Duty,” explained Wisniewski, referencing the popular video game. 

That meant a large screen monitor and controller rather than a VR headset and arm and body control.

And “we got everyone’s attention because OMR jumped on board.”

Demand for further projects

The simulation showed what Wisniewski and his small team of designers and developers could do, and the team began to experience interest and demand for further projects. New VR training programs, including circle checks, hazard recognition, and scaling, went into development, while the team continued to build and refine the mine rescue experience.

Their vision for mine rescue, however, was difficult to achieve because the technology required to bring it to life didn’t exist. A powerful, portable and lightweight computer with extended battery life, multiple exterior sensors, and more detailed and extensive image assets were just a few of the elements required.

The NORCAT team “just kept plugging away at it . . . making sure we were staying up to date in whatever the technology was,” Wisniewski said. Notebook and laptop computers became more powerful, smaller, and efficient, and “the stuff that came out in May changed the world for us overnight.”

With the introduction of the Oculus Quest, cameras and forward sensors were mounted in the headset, and eliminated the need for exterior sensors and allowed greater untethered movement. 

“I can now just put the goggles on and grab hand controllers.”

In fact, NORCAT managed to fit a notebook computer in a Dräger BG4 to create high fidelity VR visuals without a tethered connection, and is now looking at doing a mine rescue simulation where the participant is in full gear, and wearing a facemask equipped with VR goggles, Wisniewski said.

Ultimately, “it will be an operator (mine rescue officer) designing their scenario on a PC, and the trainees (mine rescue volunteers) solving and training in that VR scenario.” The MRO would pick a scenario, a mine location, the injury suffered by a casualty, the team’s starting point, and more, Wisniewski explained.

“What you would see from an external point of view is five guys all in their spaces . . .  doing something. But what they would be seeing is everyone huddled around the casualty in VR. The illusion that they are all beside each other is there.”

Featured in "VR for Good" initiative

The efforts caught the attention of Facebook, the owner of Oculus, which asked NORCAT to take part in their “VR for Good” initiative “to spotlight how non-traditional VR technologies have the ability to make the world a better place,” Wisniewski said. 

The video, which showcased the collaboration between NORCAT and OMR, was shot at a Sudbury area mine with OMR volunteers. It features the NORCAT team; Shawn Rideout, OMR’s chief mine rescue officer; and Alex Gryska, secretary of the International Mines Rescue Body and former OMR director.

The next enhancement within several months, Wisniewski said, will be the addition of real hands into the experience. Instead of hand controllers, users will have the ability to pick up and manipulate objects in VR using their own hands and fingers.

Currently hand controllers are used to do actions in VR, but they will become mine rescue devices, he said, an MX6 gas monitor, a thermal imaging camera, or a Sentinel, hanging from the front of a BG4 for use when needed.

And more is yet to come.

“It’s just a matter of time before we get down to full body recognition,” Wisniewski said, and participants will see facial actions and emotions in VR characters.

Until then, Wisniewski and his team continue to build, expand and experiment with the mine rescue virtual reality, with the ultimate goal of transforming it into a stand-alone training product for mine rescue.


Ontario Mine Rescue