DaVinci MagiHuman

DaVinci MagiHuman is an AI tool that can create talking videos from a single picture and some audio. It was created by Sand.ai and GAIR Lab at Shanghai Jiao Tong University. This tool is open-source, meaning people can look at its code, run it on their own computers, and even use it for business purposes as long as they follow the license rules.

Benefits

DaVinci MagiHuman makes it easy to create videos where someone speaks. It combines audio and video creation into one step, so you don't need separate tools for speech and video. It works with just one photo and can sync lips to speech in many different languages. It's also very fast, able to create a short video in about two seconds using a powerful computer. Tests show that the videos it creates have very clear speech and people prefer them over videos made by other similar tools.

Use Cases

To use DaVinci MagiHuman, you need a clear photo of a person's face looking straight ahead and a script or audio recording. You can then choose the quality of the video you want, like 256p or higher if your computer has enough memory. The tool will then create the talking video. You can download the finished video. If you want to run the tool yourself, you can download the necessary files from Hugging Face Hub and find instructions on GitHub. There's also a demo you can try online.

Vibes

Published evaluations show that DaVinci MagiHuman has a much lower word error rate compared to other public models. It also wins a large number of head-to-head comparisons when people are asked to choose which video they prefer.

Additional Information

DaVinci MagiHuman is an open-source model released under the Apache 2.0 license. This license allows for commercial use, but you need to give credit and follow certain rules. The use of the videos created depends on your specific situation, any rights others might have in the original photos or audio, and the law. If you find any problems or have ideas for new features, you can report them on the project's GitHub page.