Note: This article was first published on 15 October 2024.
Adobe MAX 2024 Firefly Video Model announcement. Photo: HWZ
Adobe made a significant leap forward in creative generative AI with the announcement of its Firefly Video Model at Adobe MAX 2024. This breakthrough expands the company’s Firefly offerings to video for the first time.
The feature, which is in beta right now, introduces Text to Video and Image to Video options, enabling video creators to generate stunning footage from text prompts and still images. Users are also able to have further control over the generated videos through a selection of camera angles, video motion, and zoom levels.
Adobe MAX 2024 Firefly Video Model extremely detailed prompt demo. Photo: HWZ
Firefly's expansion into video also includes groundbreaking tools like Generative Extend in Premiere Pro. This feature allows users to seamlessly extend video clips by generating entirely new frames in order to fill gaps and smoothen transitions for more refined, polished video edits.
Another new functionality shown during the Adobe MAX keynote was a content-aware subject tracking, where a user can easily create a motion mask of a moving subject in video with basically one button. The function seamlessly masked out human dancers moving in and out of frame.
Firefly Image 3 model enhancements also improve efficiency, generating images up to 4x faster than previous versions. Adobe also introduced a Generative Workspace (beta) in Photoshop, which allows designers to iterate on ideas more intuitively. In Illustrator, Firefly’s Vector Model advancements offer new features like Generative Shape Fill and Text to Pattern, empowering designers to quickly create unique vector designs and adjust pattern density with ease.
Adobe is also rolling out new enterprise-level tools under Firefly Services, including AI-powered Dubbing and Lip Sync, allowing companies to translate spoken dialogue into various languages while maintaining lip sync accuracy. Another major release, Bulk Create (beta), promises to streamline tasks like resizing and background removal for high-volume image production, improving workflow efficiency for content creators.
Adobe continues to prioritize responsible AI use, and one of the key points that was constantly stressed upon is the fact that Firefly’s generative AI tools, including the new video model are all developed and trained on licensed content, and are thus safe for commercial use. Through Adobe’s Content Authenticity Initiative and AI ethics, Adobe does not train its AI models on artist proprietary content or crawl the web for data. Even 2-3 second clips created with the Generative Extend feature in Premiere Pro are automatically tagged as AI Generated to avoid confusion with the source clips and maintain transparency.
The Firefly Video Model is now available through a limited public beta, though you'll have to sign up on a waitlist, with video generation tools offered for free during this period. More information on pricing and full feature availability will be shared when the model exits beta. For now, creatives and enterprises alike can explore the next generation of content creation at firefly.adobe.com.
Our articles may contain affiliate links. If you buy through these links, we may earn a small commission.