VRSlide
The history of this crazy ride
In mid-2017, to gauge interest in the waterpark industry for VR Snorkeling, Stephen Greenwood and I attended the 2017 IAAPA Asia Themepark and Waterpark Tradeshow in Singapore, where we met Rainer Maelzer, the then-CEO of Wiegand Waterrides (WWR) who suggested we collaborate on adding VR to waterslides.
We spent the next 6 months in Germany designing and building a virtual reality system that could work on any existing waterslide. We worked with a pilot partner, at Therme Erding, just outside Munich, the largest indoor waterpark and wellness center in Europe, and formed a partnership with WWR to build this system. Since then we have installed many more VRSlide worldwise, and hundreds of thousands of users have paid to experience these rides.
Although the bulk of the system was designed for the first installation, we found obstacles unique to subsequent projects, for which subsets were redesigned or new features were added.
This is the story of that design journey.
Collaborators
Atlas Roufas : Atlas was responsible for the ML aspects (HiveMind), and some parts of the tracking algorithms.
Serhii Yolkin : Built a lot of the Android level interface software, e.g. to be able to lock the screen to prevent water from interfering with it, etc
Stephen Greenwood : Project manager
Wiegand.Waterrides : Manufactures sensor attachment parts and conducts electrical installation, and was a sales partner within the EU
The Recce Trip
During our first visti to Galaxy Erding, we choose a slide (‘Space Glider’) and took measurements on it to develop some hypotheses around which we could design the product
Living in the Bay Area, we didnt have easy access to waterslides where we conduct beacon tests, so I built this lego rig with a Samsung S8 running some prototype software to zoom past some beacons!
We tried this bluetooth beacon on the slide on-site, but found that the latency was too variable for it to be reliable
Here's Atlas all decked out in mobile phones to capture IMU data both from the head and the chest mouted position to see which was less noisy. Turned out that the head mount would work fine
Here we take measurements on the slide, while a passive acoustic beacon has been stealthily attached with some amazing duct-tape!
Surveying our slide-to-be : Space Glider at Galaxy Erding, Germany
System Design
Our first and most important hypothesis was that accurately tracking users as they go down the slide would be paramount. From our recce data we found that there was an immense amount of variability in timing of the same rider descening multiple times, due to changes in water pressure, raft pressure (and therefore friction), rider position, etc.
Sensor Selection Process
We chose a combination of 2 sensors, in order to have redundancy :
Detection of acoustic signals from fixed beacons on the slide : discrete data
Machine learning classification of IMU data that would be compared to a model of that slide : continuous data
Grand Central
This is where all the millisecond-level 3D tracking would happen
Tracking System Hypotheses:
Acoustic beacons could transmit encoded position information while devices flew past at speeds of upto 15 m/s within very noisy and wet environments
Each slide had a unique acceleration signature, which could be derived from the IMU data and teased out by a machine learning system
We could combine these signals to generate a smooth 3 dimensional position and 3 axis rotation values, with low latency
Comparing the data from HiveMind (ML of IMU data) and Redshift (beacons)
Technology overview : How the VRSlide is able to track riders as they go down the slide
Grand Central Architecture
Modular design, where the tracking module, GrandCentral, can be plugged into any content piece within Unity
RedShift Design
Doppler Shift
My initial idea was to have passive sensor : waterproof speakers attached to the slide that simply played a frequency encoded tone, and as riders went by, the Doppler Effect would distort the sound such that the frequency would first increase as it was approached, and then decrease after you went past it. On detecting the exact moment this change happened would allow one to know the exact position of the beacon.
Video Above : Doppler shift prototype in development. I built the app in Unity to detect the moment doppler shift changes polarity, and display the encoded ID, and the ways I tested it.
However, the reliability of this method was low, and although I was able to get it to work well eventually with much more sophisticated algorithms, at the time I abandoned it, and moved onto designing active sensors, which have their own advantages.
Sensor Calibration & Control
After achieving some stability in triggering the sensor and receiving the encoded ultrasonic data, I set about to make the system deployment friendly. This meant that I or any trained personnel should be able to calibrate each sensor quickly without having to resort to command-line. I used the Blynk platform to develop this process and the first version of the sensor stack firmware was developed in NodeJS
Cassini in action
An early version of the sensor control app, built using Blynk. This allowed me to fine tune the sensor settings, such as encoded ID, trigger distance, or reboot them, etc remotely after they were setup.
Active Sensors :
Production Versions
versions 1 & 2
v1 : Improved on the housing and mounting features
v2 : Upgraded to use a LIDAR sensor that would allow a completely sealed housing
v1 mostly added a more serious housing, but still had all the circuitry exposed, but conformally coated to prevent water intrusion
v1 Sensor mounted on a slide in Prague
Steel housing designed to be flush with the ceiling of an enclosed slide. The transmission voltage was upgraded to 12V DC to improve electrical transmission over longer distances
Steel housing mounted on a slide in Austria. Note the retro LEDs!
Giant steel box custom made for this particular slide by Wiegand Waterrides
v2 Sensor installed on a slide in Thailand
v2 in the new sealed enclosure and upgraded speaker and amps
Tests with different LIDAR sensors
In order to make a really compelling user experience, the VR content needs match the somatic experience, and therefore should start as early as possible. To this end, I made a special sensor that was triggered by the conveyor button press
This particular slide has a conveyor belt start that is initiated by the operator pressing a button.
Here you can see a test video where the button press (and corresponding blue light) kicks off the barebones VR experience
Tracking System Is A Go
Optimizing for User Experience
Once these pieces were in place, focus shifted towards providing the best user-experience possible. We surmised that the following were crucial:
A comfortable, lightweight headset that could be sanitized easily
Ability to achieve high throughput on slides : simple & quick experience launch and calibration methods
Exciting content!
Headset & Charging Design
v1 production version
Maximizing ergonomics
This is the initial version, that has some unique features like weep holes to let water out, and hole on the door to expose the microphone. The latter turned out to be bug, as water would stay in the hole and prevent the mic from functioning properly! It weighs in at just under 400gms, making it one of the lightest VR headsets.
v2 Final
Evolving
We found that a lot of phones IMU would be toast in 6-9 months, and since we did not see the same problem with our DIVR devices, i theorized that it might be due to g-forces and rough handling of these headsets. In v2, the VRSlide and DIVR headsets converged and now these phones were protected inside a hard case as well. This added some weight, but also robustness.
Wireless Charging
For both operational efficiency, as well as reducing electrical hazards, we didnt want to have the operators taking the phones out of the headset to plug them into charge. One of the reasons I choose the S8 was chosen because it could be wirelessly charged. The image above shows the wireless charging dock I designed, which eventually became much smaller and cheaper.
Charging Cabinet
One of the downsides of wireless charging is that it generates a lot of heat, which if not dissipated, will reduce the lifetime of the battery significantly. So I designed a wirelses charging cabinet to be installed with air conditioning, and software that would report battery temperatures to our backend, so alarms could be raise if they get too hot. This is still one of the most used backend features.
“Forget about VR in the living room; this summer it’s on waterslides..”
- Rachel Metz, MIT Technology Review
We designed an incredible system
It had to survive the rigors of a corrosive, demanding environment.
In order to pull it off, we designed and built:
The world’s first waterproof VR headset
A way to launch experiences within a waterpark setting (single tap NFC launch, no screens, no buttons)
A way to sanitize headsets and transport them between the end of the ride and starting point
Beacons that communicate with headsets with a fixed low latency
Trained and built a machine learning algorithm to analyze acceleration signatures in real time
A library of content that can be remotely updated to headsets anywhere
Ability to remotely monitor the health of the sensors and headsets
Wireless charging system
Make everything rugged enough to survive temperatures between -20C and 40C, chlorine, squabbling kids and undertrained waterpark staff