How Realised Realities helped Head Set build a watertight pre-production plan for their VR Journalist Training app Extreme Weather
Head Set specialize in training professionals who will work in high risk and dangerous social environments, to provide a more accessible, cost effective, and easily deployable alternative to the existing in-person training courses. They offer highly immersive situational experiences as part of their training sessions to recreate high-pressure situations and environments through immersive technologies such as VR. They empower their users (news journalists, humanitarians, aid workers and others) to practice vital skills they will need in the field, and to learn from peers and instructors without leaving home or the newsroom.
Considerations of safety, cost and logistics mean that there are limits on how immersive and realistically these situations can be recreated through real-life training scenarios. It was of prime importance for the team that their users be able to ready themselves for the emotional impact of these events, and experience some of the boots-on-the-ground intensity ready for if and when they had to face similar scenarios in real life.
A recent study by PwC revealed that VR learners are more confident in applying what they’re taught. Learners trained with VR were up to 275% more confident to act on what they learned after training. This is in comparison to improvements after learning in a classroom (40%), and over e-learning (35%). Additionally, VR Learners were 375% more emotionally connected in VR than through classroom learning. Head Set use various monitoring tools (including telemetric data, a replay function and biometric data such as heart rate) to monitor and understand the emotional responses from their users. Understanding each user’s responses can give valuable insight for trainers to work with the group and one-on-one, and enable them to tailor their training accordingly.
Realised Realities first worked with Head Set in a mentoring capacity when they were a bustling start-up, and over the intervening years we have been thrilled to be their choice to provide immersive design and planning support for a number of their exciting projects.
One of the things that makes Head Set such an exciting company to work with is that they adopt a different artistic and creative approach for each of their experiences. If there’s a running-theme that marries these approaches, that house style can perhaps always be defined as ‘unique and evocative’ – they look to find the right style and the right fit that best evokes the mood and intensity of the real life experience in each individual case. Alongside this, they offer a deep but intuitive curated tool set for the trainer to manage sessions in-person or remotely.
For their previous piece, Civil Unrest, the look and feel is one of dark, busy, electric escalation. The crowds and even your fellow journalists are built from a network of shards and data points. It lends the piece a feeling of authenticity, like a digital reconstruction, rebuilding the scene from extrapolated data. It’s a style that evokes the emotions of the moment brilliantly, and also allows the experience to exist on today’s standalone headsets. It’s an impactful, thoughtful approach from award-winning Creative Technologist Nick Pittom, allowing journalists to experience an explosive, hostile protest gathering at ground zero.
For Extreme Weather, the team developed a very different creative style that beautifully evokes both the rural town and their inhabitants with a dirty, washed out bleakness and exhaustion. It feels wet and ruined and dangerous. It is filled with desolation, despair, and desperation. The town is populated by locals displaced from their homes and businesses, trapped by the floods, searching for lost family member, queuing for food and water, desperate to charge their phones to make contact with their loved ones.
"It’s hard to explain the impact of this until you see it. It’s like psychology meets safety training.
To harness the emotional aspects of a scenario like this is something that you otherwise just can’t do as a trainer. Plus the tech is super easy to work with." Chris Post, Trainer
The Challenge
The concept for Extreme Weather was fascinating; an immersive and emotionally engaging evocation of being present in a small town after a disastrous storm had torn the area apart and flooded the streets. That in itself would be a challenge for any developer to conjure, but there were additional requirements to consider.
Kate and Aela had wished for a more immediate and convenient way to make changes to the design. Transforming their vision for earlier projects into the virtual experience they wanted had been more of a challenge than they thought it should be because iterating ideas, layouts and interactions in VR had been a case of working directly with coders and engineers and waiting for their changes to come through in later builds. Trying to finesse locations, layouts and character positions this way had proven slow and sometimes frustrating.
Another challenging aspect was that we were building this segment ‘out of time’ as it were – this had to work as a stand-alone piece, but it was also to be one of the later chapters in a much larger overall narrative that covered the whole timeline of the event; from the initial forays into the town pre-storm as the residents and your team make their preparations, through the visceral experience of the scramble to safety as the storm itself hits the small term, and then seeing the town emerge, ravaged and broken in the aftermath. The pre-storm and intra-storm sequences would be developed as part of the longer roadmap; the trainee will eventually be able to experience all of these chapters in a linear narrative, so we had to design the final ‘aftermath’ with this in mind. We had to think through what was there before, what would get destroyed, and where it would end up.
The topography of the landscape had to work as both a believable small town and as a location to teach the skills, but it also had to be believably flooded post-Storm, yet still allowing users to navigate through the wreckage. This also meant that it was necessary to consider the full narrative timeline. The earlier chapters would build familiarity with the town and it’s inhabitants so the user could feel the emotional impact of the storm; the floods, the damage, the loss of life. In a way it was like the sort of crazy exercise you might find on a story writing course, we were starting with the third act of the story, and we knew the previous chapters would have to dovetail into this one seamlessly. We had to avoid painting ourselves into corners that weren’t going to be built until further down the line.
The other important challenge we set ourselves was to design the project with reuse in mind. It’s a core part of Head Set’s approach that these experiences can be tailored for the training needs of different clients. In a real life disaster, there are lots of rescuers, aid workers, reporters and other groups that will respond with different responsibilities – this approach means Head Set can support different perspectives and training needs for different client groups, engaging with the core scenario in different ways.
This version was aimed at news journalists, for example, but at the same time we were helping co-develop a different version for training humanitarian aid personnel: using the same town, the same events, but with a different focus on the user’s role and responsibilities. We worked with expert consultants from both sectors, as well as collaborating with training, safety and disaster experts to develop all aspects of the experience and make sure each area was providing narrative opportunities to support the different training requirements of the groups.
The Process
Aela and Kate at Head Set already knew the outline of the narrative and had a good idea of what the environment should look and feel like. We took that narrative journey and began to research and plot out the locations where each scene would take place, to understand the shape and size of the journey that the user would undertake in this post-storm version of the town. From this, we sketched out layout descriptions for the town, paying attention to scale and distances to make sure each location could be seen and reached from the last one.
At the same time, we knew which scenes took place at the waterline and which were on higher ground, so we overlaid the flood waters where we needed them to be for the narrative, then designed the topography of the town center to facilitate the water to have flooded where we needed it. This was an important step, because we wanted to make sure that when the earlier chapters were built, users would be experiencing this area in it’s pre-storm form, so the layout and building heights had to make sense when they are first encountered. Being able to relate to the town and its inhabitants was a key objective, so that earlier visit ahead of the storm would be important for establishing characters and locations to add emotional resonance when see the town and its inhabitants irrevocably altered after the storm has subsided.
The 2021 storm season in the US provided lots of reference which, while difficult to watch sometimes, gave us lots of insights we could fold into the narrative of the scenario. It also prompted a stronger understanding of Storm behavior, prompting the team to decide to re-plot the course of the storm partway through pre-production and change the directions we had envisioned the flood waters coming from, and how they were draining away. It was important to get the details right, because we wanted the user to believe and become emotionally invested in the scenario – and the deeper the immersion, the more valuable the experience becomes for the user. All of this reinforced the very real need that Head Set’s immersive training addresses – multiple news outlets, national and local, had crews reporting on the disaster and destruction from ground zero, putting themselves at the heart of dangerous locations, amongst inhabitants who had suddenly lost everything and are looking for anyone to help. These are difficult, unknowable scenarios that are almost impossible to train for in the real world, but where preparedness and experience can make all the difference.
Once this was planned out, the most useful phase of the process took place. The team were keen to meet regularly in-headset, in the world, and to experience, assess and tweak it together. Their resident code whiz, Priyadarshini Krishnan, imported the 2D layout map into Unity, blocked out the buildings in their correct places, scaled it to real-world size and imported it into Altspace, so the entire team could enter together and use the app’s in-built tools to discuss tweaks and make changes. This is a much easier and more accessible way to do this than building a custom network solution, and got to resolve the necessary design decisions quickly and efficiently. Together, we checked sight-lines, travel distances, nudged and jostled buildings into the right positions, added in figures for the main characters we would encounter, and determined where the user would stand and what they would see. It was a very convenient and fast way to plan out the town with full confidence that everyone was on the same page with the layout and design. In this primitive form we were even able to plan out the mo-cap movements so that the mo-cap performers on the day could pop on the headset before each scene, and understand perfectly the world they were inhabiting, and the spatial cues and characters around them - something that’s always challenging where performers need to act and react to the invisible.
“ It was super useful to lay out the world in this low cost way, so that everyone from the development team through to mocap producers and the actors could visualise and see the environment like a movie set. We’ve integrated this into our workflows and processes to save time and money.” Aela Callan, Director
Finally, when everything was set and all the team were confident with what we were building, the project transitioned into production mode. Priya’s simple block-model town was the skeleton upon which the full version of the town was now being built and detailed, and it was one that the team had explored and finessed in VR over multiple design sessions and had full confidence that every sight line, scene composition and mocap requirement was already settled. Technologist Nick Pittom could understand the processor overheads of each location and could confidently design the striking art style of the experience to be respectful to those constraints, having already been able to identify where the most challenging scenes would be from the early VR walkthroughs and making sure the engine and the art style would handle them.
The Results
Approaching pre-production this way was a big success for the project. By taking the time to fully plan and prototype as much as possible through straightforward pre-production approaches, Realised Realities were able to help Head Set finalize and lock down almost all of the design, layout and interactions in advance of production and give them high confidence moving into the production phase.
The actual benefits were felt throughout development. We had very few changes to make from the version we had settled on in pre-production, and the production period for the project was significantly reduced from early estimates / previous projects. And crucially, much of that time was spent elevating the experience, polishing and finessing every aspect of it, rather than dealing with the typical VR development problems of scaling back the details and simplifying shaders to hit frame-rate targets. ‘Polishing’ time is always at a premium with XR projects, where teams want to be spending as much time as possible, but not where they are always able to focus when deadlines are looming and the niggles are still being worked out. This felt like the proof that the process and approach the team had worked out with Realised Realities had been the right choice.
And this was borne out in the feedback from the users;
Using tools of narrative engagement and technological immersion, there's no question that personnel who have to insert themselves in these situations due to their roles and responsibilities will always benefit greatly from getting experience under their belt. Facing real, believable scenarios through simulation in this way is something immersive VR training can enable in ways that haven't been possible before.
Most of us hope we never have to experience a storm so terrible that it destroys the homes and livelihoods of our local communities, but with the changing state of climates around the world, the chances of this happening are increasing. It's reassuring to think that technology is now delivering timely tools that can be help prepare those who will need to face these situations more regularly than most.
One of the many services we offer at Realised Realities is professional help in translating your immersive vision into a viable virtual reality. With over a decade of experience in leading XR design thinking and planning, we help and guide our clients to make the right choices for their immersive projects. Supporting clients of all sizes, we're here to help.