Augmented Reality

Thought-powered VR is becoming real!

By  | 


Thought-powered VR is becoming real!

A dream came true last week-end at Montreal’s Hacking Health hackathon: our team hacked a VR headset to make it EEG/ECG (electroencephalography and electrocardiography) fully equipped; a first class equipment blending Vrvana‘s headset -the Totem- and OpenBCI‘s.

For a very long time at the University of South California, Dr A. Rizzo works at the intersection of VR and brain imaging. I can’t remember how many times I hear people quoting him: “I have no question that Oculus will revolutionize virtual reality for clinical purposes.” Early 2015, I experimented with the Dk2 and the EPOC headset and, despite the fact that it is kind of a mission impossible to wear the rift and the EPOC without the help of a friend, I’ve been amazed by the potential of such a  combination. In march 2015, Mindmaze created a surprise announcing a thought-powered virtual reality headset: MindLeap. But we didn’t hear of them since then. Well, it happens that we have VRvana here in Montreal, a company that is developing their own VR/AR headset. Logically, we thought of customizing their headset, the Totem, in order to achieve what MindLeap is supposed to be doing.

For the hackathon, our plan was to create a guided meditation VR/AR application. And that’s what we did! On our table, we had a DIY respiratory belt, a pulse sensor, an OpenBCI board, EEG/ECG sensors, and, of course, the Totem. This headset seems designed for a VR/EEG dedicated set-up, especially because you can attach/detach velcro orange pads on the headset really easily (see pictures below). For the purpose of the hackathon we actually sewed a strap with the EEG sensors to one pad. In the near future, we can imagine different pads would be available on the market, each one equipped with different sensors. For example, if I’m playing a game that doesn’t require any sensors, I would mount the headset with the regular pad. If I want to play a guided meditation game that works with EEG or ECG, then I would replace the regular pad by the EEG/ECG pad, the hook and loop fastener system making it user friendly.

P1500825 copie

P1500898 copie


We worked with OpenBCI and LabStreamingLayer for networking, time-synchronization, real-time access, and we built our scene in Unity3D.

Within a day, our team created a guided meditation game that works with respiratory belt, pulse, EEG & ECG. As the whole operation happened in a day, the actual game is pretty basic: the user starts in the augmented reality mode, a voice guides him and asks him to close his eyes, and open again. When the user opens his eyes, he is in the virtual reality mode, in an outdoor scene with a campfire in front of him. The user can’t move, only look around. When he inhales, the campfire’s intensity goes down, when the user exhales the campfire’s intensity goes up. At the end of the session of meditation (3 minutes for the demo) the voice asks the user to close his eyes and open again, back in the augmented reality mode. All the data recorded from EEG/ECG, pulse and the belt during the sessions are archived and available through an application on the user’s smartphone. Also, at the end of each session, the user can see an interpretation of all the datas recorded as a computer-generated painting (something between J. Pollock and fractals).

There is one main problem dealing with EEG, it is noise. The shorter your cables are, the better. Rapid head movements can also generate noise, which is the last thing we want to record -even tho it is still a potentially useful input-.  Other than noise, everything works well. You are more than welcome to download the project (soon online) and make it yours, please do! The more creators we are, the merrier, and the sooner we will be able to scientifically approve VR therapeutical potentialities.

P1500751 copie


Epilogue: That hackathon has been a blast! Our team is still working on the project. We can’t wait to share the open project with the Github’s community, probably within two weeks, once we adapted it for the Oculus Rift Dk2 -as for now the Totem headset is not available-. We actually won the first prize from the kids jury! The professional jury had a chance to try our demo on sunday morning and each ones definitively got the potential of neurogaming at large. I suspect we didn’t get any prize from them because we didn’t use the products that they were promoting for the hackathon: IBM’s Watson and Bluemix. I loved it when my mate Jeremy told the organizers that we would certainly not use IBM engines as they are too slow for our project: “It’s not good enough for our project, we deal with real time signals and your internet is too slow.” Boom, in your face.

The team that won the first prize develops an application that lets parents aware of what is going on on their kids facebook’ page. No needs to tell you what I’m thinking about that. Still, hackathons are a lot of fun and if you have a chance, just go for it! You would meet great and talented people that could help you greatly with your projects or, even better, become friends!


P1500835 copie

P1500941 copie



Alecz is currently initiating HIGHWAY101, Experiential Techgnology Community, a VR-focused hub based in NYC that represents visionary creators in NYC, builds a network of industry influencers, and actively participates in industry events and festivals. He's co-organizer of The Virtual Art Collective Meetup in NYC that focuses on painting, sculpting and animating in VR.

You must be logged in to post a comment Login

Leave a Reply