In the latest blog post, Stability AI announced its new AI model, Stable Virtual Camera, which is capable of converting 2D images into 3D videos. The model is currently available in the research preview under a Non-Commercial License. The AI Company claims that this new model can convert up to 32 images into 3D videos. The other specifications of this multi-view diffusion AI model include user-defined camera trajectories and 14 dynamic camera paths, including 360°, Lemniscate, Spiral, Move, and Roll.
The most promising feature of this AI model is to convert 2D images into videos without complex reconstruction or scene-specific optimization, which was previously required. Additionally, the company claims to add realistic depth in the AI model.
Stable Virtual Camera for Filmmakers
Stable Virtual Camera is presumed to assist filmmakers and animators by combining control of traditional virtual cameras with AI to produce precise and intuitive 3D videos. It is different from traditional 3D video models by generating novel views of a scene from one or more images at user’s specific camera angles. The AI model could be commercially used because it produces seamless trajectory videos, providing smooth 3D videos to the users.
Working Principle
This newly launched AI model works by taking input views of a scene and generating realistic videos from different angles. It is capable of handling a fixed number of input and output views, however, while processing it can adapt to more number of views according to the requirement. For this, it takes two pass sampling procedure. Firstly, it generates few key ‘anchor’ views and then uses these anchor views by using ‘multi-view diffusion model’ to convert them into final targeted views in small chunks, ensuring high-quality output videos.
Limitations
Currently, the company has enlisted a few limitations of the newly launched AI model. It says that initially the model may produce low-quality videos in certain scenarios or flickering artifacts. However, the company has welcomed researchers to use this AI model and give their feedback for an improved version of this AI video generating model.
