Interactive Dynamic Video Removes Barrier Between User And Content
The past few years have seen a significant shift in the direction of technology. Sizably due to the emergence of virtual and augmented reality, and the capabilities they offer. We’ve witnessed the effect that augmenting additional information into the real world can have on people, check out any of the statistics attached to the new Pokemon Go craze for an example of this notion. However, adding to the environment isn’t the only goal in the minds of those involved with expanding the reach of this, new, giants arm.
Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory have been developing a new system that will allow for the manipulation of images and video captured in the real world. The technique has been coined ‘Interactive Dynamic Video’ and lets users bend, pull, flick and manipulate objects in a virtual setting.
The system uses an algorithm to analyze video content and picks up on small vibrations, referred to as ‘vibration modes’. It then uses this information to understand the physical properties of the object, allowing it to simulate its reaction when exposed to different forces. Interestingly, the algorithm only needs a short amount of footage to work with to achieve this effect – in one example as short five seconds.
Additionally, the system can also work with objects that have an undefined 3D shape. Another instance in the video, released by MIT, shows a small shrub being moved around virtually with the click of a traditional mouse button. The effects of which are quite remarkable. – Prior to this, a complicated 3D model would have needed to be captured to achieve the same effect.
Some of the applications cited by the developers include, ‘Structural health Monitoring’ – A process used to analyze the structural integrity of old buildings to ascertain how safe they are. Another example is in the Film industry. One of the problems faced by those in the world of Hollywood filmmaking is getting virtual (CGI) animations to interact with the space they inhabit. Interactive Dynamic Video could overcome this hurdle, traditionally, resolved with big budget production, and could do so at a significantly cheaper cost.
One of the most immediate and obvious applications for this technology in the VR world, is, of course, 360 video content. It is often argued that 360 video content isn’t true virtual reality. The general basis of this is that you are unable to interact in with the environment in the same way as you would with a 3D rendered, virtual space. However, MIT’s new technique may be a starting point for bridging that gap.
“The ability to put real-world objects into virtual models is valuable for not just the obvious entertainment applications, but also for being able to test the stress in a safe virtual environment, in a way that doesn’t harm the real-world counterpart,” says CSAIL Ph.D. student Abe Davis,
“When you look at VR companies like Oculus, they are often simulating virtual objects in real spaces,” he says. “This sort of work turns that on, its head, allowing us to see how far we can go regarding capturing and manipulating real objects in virtual space.”