Tech Converting 2D videos into VR videos

[xxxclusive]

Anyone who can do this with AI?

[Expert1980]

Can use AI, what kinda project do you have?

[woodmanjack]

Yes, it's possible if you know what you're doing. But most of it lives on the cutting edge, in open source projects, and it depends on understanding how the steps break down.

1. You're taking a monoscopic source and converting it for two eyes, so you have to generate a second eye. The path is: depth map, then displace that depth to synthesize a second viewpoint.

2. You're taking flat footage and putting it into a format where you can turn your head. The footage only covers the angle the lens gave you, so the rest of the sphere is empty. Nobody has the time, the compute, or the technology right now to reliably generate the edges of a video, which means your usable field of view is capped by what was originally shot. Practically, you want the original plate as close to 180 degrees as you can get, and then sit the viewer inside it at around a 35 degree window, so there's plate left over on every side when they turn their head.

So: shoot wide, generate a depth map, displace to get the second eye, project onto a 180 degree sphere. That's roughly where the state of the art sits today.

[woodmanjack]

Can use AI, what kinda project do you have?

To be clear about scope: AI would only be in there for generating depth. I wouldn't pitch anyone on generating the edges of their video.

What are you, sitting on a repurposed bitcoin mining facility in the Czech Republic with a rack of idle GPUs? That's the level of compute you're talking about.

And if you are, hit me up. I have a custom approach to it that nobody else is taking. It just doesn't scale on conventional hardware.

[xxxclusive]

seems rendering existing 2d videos into so called side-by-side VR is best and easiest way