Augmented Reality

Thought-powered VR is becoming real!

By  | 


Thought-powered VR is becoming real!

A dream came true last week-end at Montreal’s Hacking Health hackathon: our team hacked a VR headset to make it EEG/ECG (electroencephalography and electrocardiography) fully equipped; a first class equipment blending Vrvana‘s headset -the Totem- and OpenBCI‘s.

For a very long time at the University of South California, Dr A. Rizzo works at the intersection of VR and brain imaging. I can’t remember how many times I hear people quoting him: “I have no question that Oculus will revolutionize virtual reality for clinical purposes.” Early 2015, I experimented with the Dk2 and the EPOC headset and, despite the fact that it is kind of a mission impossible to wear the rift and the EPOC without the help of a friend, I’ve been amazed by the potential of such a  combination. In march 2015, Mindmaze created a surprise announcing a thought-powered virtual reality headset: MindLeap. But we didn’t hear of them since then. Well, it happens that we have VRvana here in Montreal, a company that is developing their own VR/AR headset. Logically, we thought of customizing their headset, the Totem, in order to achieve what MindLeap is supposed to be doing.

For the hackathon, our plan was to create a guided meditation VR/AR application. And that’s what we did! On our table, we had a DIY respiratory belt, a pulse sensor, an OpenBCI board, EEG/ECG sensors, and, of course, the Totem. This headset seems designed for a VR/EEG dedicated set-up, especially because you can attach/detach velcro orange pads on the headset really easily (see pictures below). For the purpose of the hackathon we actually sewed a strap with the EEG sensors to one pad. In the near future, we can imagine different pads would be available on the market, each one equipped with different sensors. For example, if I’m playing a game that doesn’t require any sensors, I would mount the headset with the regular pad. If I want to play a guided meditation game that works with EEG or ECG, then I would replace the regular pad by the EEG/ECG pad, the hook and loop fastener system making it user friendly.

P1500825 copie

P1500898 copie


We worked with OpenBCI and LabStreamingLayer for networking, time-synchronization, real-time access, and we built our scene in Unity3D.

Within a day, our team created a guided meditation game that works with respiratory belt, pulse, EEG & ECG. As the whole operation happened in a day, the actual game is pretty basic: the user starts in the augmented reality mode, a voice guides him and asks him to close his eyes, and open again. When the user opens his eyes, he is in the virtual reality mode, in an outdoor scene with a campfire in front of him. The user can’t move, only look around. When he inhales, the campfire’s intensity goes down, when the user exhales the campfire’s intensity goes up. At the end of the session of meditation (3 minutes for the demo) the voice asks the user to close his eyes and open again, back in the augmented reality mode. All the data recorded from EEG/ECG, pulse and the belt during the sessions are archived and available through an application on the user’s smartphone. Also, at the end of each session, the user can see an interpretation of all the datas recorded as a computer-generated painting (something between J. Pollock and fractals).

There is one main problem dealing with EEG, it is noise. The shorter your cables are, the better. Rapid head movements can also generate noise, which is the last thing we want to record -even tho it is still a potentially useful input-.  Other than noise, everything works well. You are more than welcome to download the project (soon online) and make it yours, please do! The more creators we are, the merrier, and the sooner we will be able to scientifically approve VR therapeutical potentialities.

P1500751 copie


Epilogue: That hackathon has been a blast! Our team is still working on the project. We can’t wait to share the open project with the Github’s community, probably within two weeks, once we adapted it for the Oculus Rift Dk2 -as for now the Totem headset is not available-. We actually won the first prize from the kids jury! The professional jury had a chance to try our demo on sunday morning and each ones definitively got the potential of neurogaming at large. I suspect we didn’t get any prize from them because we didn’t use the products that they were promoting for the hackathon: IBM’s Watson and Bluemix. I loved it when my mate Jeremy told the organizers that we would certainly not use IBM engines as they are too slow for our project: “It’s not good enough for our project, we deal with real time signals and your internet is too slow.” Boom, in your face.

The team that won the first prize develops an application that lets parents aware of what is going on on their kids facebook’ page. No needs to tell you what I’m thinking about that. Still, hackathons are a lot of fun and if you have a chance, just go for it! You would meet great and talented people that could help you greatly with your projects or, even better, become friends!


P1500835 copie

P1500941 copie



Primarily interested in the transdisciplinary intersections among cognitive sciences, gamification and media philosophy, Alecz has a deep background in science, art and technology. After researching on intertextualities within the transhuman discourse at the era of the Psychedelic Renaissance, and developing serious games dedicated to VR headsets such as the Oculus Rift & Samsung Gear VR, Alecz is currently initiating HIGHWAY101, Experiential Technology Community, a creative hub that encourages the praxis of experiential technologies as therapeutical engines.

1 Comment

  1. immersive

    4th July 2016 at 8:02 am

    Its like you lean my thoughts! You appear to grasp a lot approximately this, like you wrotge
    the guide in it or something. I think that you jyst can ddo with
    a few % to drive the message home a little bit, however other
    than that, this is magnificent blog. A fantastic read.
    I’ll definitely be back.

You must be logged in to post a comment Login

Leave a Reply