Unfortunately, 3-D modeling is expensive, and can be nearly impossible for many objects. Ultimately, people want stuff to do that proves VR is worthwhile, and this CSAIL project could eventually mean that VR video becomes a more engaging two-way interaction. What’s more, just five seconds of video footage is enough for IDV to work. By identifying the vibration modes’ shapes, the researchers found they could better predict the way the models would move in a realistic simulation.
Advertisement
It’s called interactive dynamic video (IDV) by its creator, PhD student Abe Davis, who is publishing his dissertation on the research this month.
There’s likely a long way to go before we see this type of technology hit mainstream applications, but it could make games, movies and videos far more interactive than they now are. “Davis and his colleagues have provided a simple and clever way to extract a useful dynamics model from very tiny vibrations in video, and shown how to use it to animate an image”, James said.
Davis says that IDV has many possible uses, from filmmakers producing new kinds of visual effects to architects determining if buildings are structurally sound. A video shows mock ups of a Caterpie crawling out of a bush and a Pikachu moving around a jungle gym. He even demonstrated how he can make his own hand appear to telekinetically control the leaves of a bush.
The technique could also encourage augmented- and virtual-reality developers to think of the simulation in a different way.
Researchers say that the tool has many potential uses in engineering, entertainment, and more.
Engineers could also use the system to simulate how an old building or bridge would respond to strong winds or an natural disaster. This new tech could make it easy to blend real video with CG creations, which of course has applications far beyond Roger Rabbit and awful, bad spiritual successor Cool World, the 1992 movie Brad Pitt would like you to forget he was ever in.
But get ready for it, because researchers from MIT have kicked the augmented reality game up a notch by inventing a program that allows virtual objects like Pokémon to interact with real-world environments.
This new method could be perfectly timed to ride the wave of interest and investment in virtual and augmented reality tech.
At the moment these forces are applied by clicking or pushing a mouse, but if paired with a virtual reality helmet or viewer motion tracking systems, like, for instance a Microsoft Kinect system, viewers could literally reach out and start poking and pushing objects they see on the screen.
Advertisement
Pokemon Go superimposes virtual monsters over a live view of the world from a smartphone’s camera, although the Pokemon do not interact with real-world objects. Chen also received support from Shell Research through the MIT Energy Initiative.
