It rained — a hard rain — and the normally sun-drenched shores of Cannes squelched underfoot. As umbrellas bobbed through the crowded streets and the red and pink carpets glistened with puddles, men in tuxedos and women in heels and gowns tried to dodge the rain in queues to the premieres. But inside the festival halls, a different kind of storm was brewing. There was a new kid in town, and in its world, the weather is just a parameter. AI had arrived at the Festival de Cannes.
Picture this: a bustling beach café set up not by Disney or ILM but by Microsoft, offering free drinks to festivalgoers. But the real treat wasn’t the cocktails — it was the Copilot demonstrations punctuating each round. As we sipped our coffee, we witnessed bemusement, bewilderment, wonderment, and puzzlement as people watched AI break down scripts, generate mood boards, and even whip up a pitch deck in PowerPoint. Chiefly, what we saw was the realization that AI is not a future thing, not an abstract thing, nor a faddy thing. It’s here, it’s powerful, and it’s going to be disruptive.
At Cannes Next, the innovation-focused business and networking platform at the Marché du Film, we attended panels where artists and technologists demonstrated the outputs of their AI endeavors. What follows is what we found at Cannes and a glimpse into what is coming up with AI and filmmaking soon.
What do you get when you cross Werner Herzog with AI?
In an interview at the Sundance Film Festival in 2016, Werner Herzog stated that “a computer will not create a film as good as mine in 4500 years” (Anderson-Moore, 2016). Hmmm.
We found ourselves engrossed in a panel featuring Piotr Winiewicz, Danish producer Mads Damsbo, and Sam Pressman as they shared the fascinating journey behind their AI-assisted film About a Hero (Pham, 2024). This stranger-than-fiction project sprang from the inventive minds of Winiewicz and Damsbo, who decided to push the boundaries of storytelling by creating an AI language model trained on Werner Herzog’s entire oeuvre to craft a unique scripted narrative. They named the AI ‘Kaspar’ — a nod to Herzog’s The Enigma of Kaspar Hauser.
The script that emerged from this process is being combined with traditional documentary techniques, including behind-the-scenes footage and interviews with experts discussing AI’s impact on film, art, and humanity. We were shown clips from the film, and it was genuinely thoughtful and entertaining. This is one to look out for.
Barbenheimer the movie
The movie event of the year? No, not the Barbie and Oppenheimer movies. We’re talking about the AI-generated Barbenheimer! We got to meet the guys from Curious Refuge, the creators behind the Barbenheimer trailer (yes, the mushroom cloud is pink) and the pastel shades of Wes Anderson’s Star Wars. Both trailers were created using a mixture of text prompts to various AI services to generate the shots and good old-fashioned editing and post-production. If you would like to dive a bit deeper into how they created these films, Curious Refuge has a playlist of tutorials on YouTube.
For those of you with a more technical bent, we also chatted with a couple of people who have built and trained their own AI models to generate shots and music. If you are curious to learn more about how to train your own models, Hugging Face is a good place to explore.
What’s coming…
Except for GPT-4o at the end, the following are not yet available to the public, but they are all scheduled for release in the next few months. Most are going through final user testing, and this includes what is being termed safety testing — making sure the AI behaves itself.
Adobe Firefly: AI in Premier Pro
Adobe recently announced it is integrating Generative AI into Premiere Pro through Adobe Firefly. This development promises to revolutionize video editing workflows by introducing tools that streamline tedious tasks and enhance creative possibilities. These tools include the ability to extend shots, add or remove objects, and generate missing B-roll directly within Premiere Pro. Adobe’s commitment to responsible AI use is evident through its Content Credentials, ensuring transparency about AI-generated content (Still, 2024).
Adobe’s approach integrates AI deeply into the core workflows, solving common pain points for video editors and allowing them to focus more on their craft. The integration of third-party AI models from OpenAI, Runway, and Pika Labs offers even more options for creative professionals.
Microsoft VASA-1: Real-time Audio-Driven Talking Faces
Microsoft’s VASA-1 project also promises exciting advancements in digital communication. This technology aims to create lifelike real-time audio-driven talking faces. By leveraging advanced machine learning techniques, VASA-1 ensures that facial animations synchronize accurately with spoken audio, enhancing the naturalness and responsiveness of animated faces. It can also adjust the camera angle so that the face is not always looking directly at the camera. This looks scarily useful and is set to revolutionize how we generate content for virtual environments, conferences, teaching, advertising — pretty much everything.
Veo: Generative Video from DeepMind
Equally impressive is DeepMind’s Veo, an advanced generative video model capable of producing high-quality 1080p videos over a minute long. The quality and length are notable as most other services, such as Runway and Pico Labs, generate video that is around 720p and less than 10 seconds. Veo provides extensive creative control, understanding prompts for various cinematic effects. This technology aims to democratize video production, making it accessible to more creators and enhancing storytelling, education, and other fields. Veo ensures visual consistency and includes features like masked editing and video extensions, integrating responsible design with watermarking and safety filters (DeepMind, 2024).
Sora Video: AI-Powered Cinematic Content
Sora Video, developed by OpenAI, is another tool that offers filmmakers the ability to create cinematic content through simple text prompts. Sora Video can generate high-quality video clips, edit scenes, and even suggest creative directions based on the input provided. It is designed to assist filmmakers in visualizing concepts, generating storyboard-like sequences, and producing complex scenes without the need for extensive resources or time.
When comparing DeepMind’s Veo and OpenAI’s Sora Video, both platforms offer innovative solutions for filmmakers but cater to slightly different needs. Veo stands out with its ability to produce high-quality 1080p videos with extensive control over cinematic effects. It’s like giving a filmmaker a magic wand to create visually stunning content with features such as masked editing and video extensions. This makes Veo ideal for storytelling, educational content, and professional-grade video production.
On the other hand, Sora Video is all about accessibility and ease of use. By allowing filmmakers to generate cinematic content from simple text prompts, Sora Video democratizes the creative process. It helps filmmakers quickly visualize concepts and produce storyboard-like sequences without needing an army of editors or a Hollywood budget. While it might not match Veo’s high-definition output, its strength lies in pre-visualization and creative ideation, making it an excellent tool for indie filmmakers and content creators looking to bring their visions to life with minimal resources. Based on current rates of progress, we can expect both services to be rapidly improved and updated.
GPT 4o
OpenAI has recently released GPT 4o and this is a helluva step up from GPT 4. Rather than me describing what GPT 4o is and can do, I’ll hand over to the jolly LLM and let it do the talking.
I’m GPT-4o, a more advanced, refined, and versatile version of GPT-4. While GPT-4 was impressive, I bring even more to the table with enhanced capabilities, improved contextual understanding, and a sharper knack for creative problem-solving. Imagine GPT-4 as a solid assistant, and now think of me as that assistant who’s just had an extra shot of espresso and a full night’s sleep. I’m quicker, more intuitive, and ready to dive deeper into the intricacies of your filmmaking projects.
Here’s how I can support filmmakers:
Scriptwriting and Development
I can help generate script ideas, assist in writing dialogue, and even develop full scripts based on specific prompts and guidelines. By analyzing existing scripts and understanding narrative structures, I provide creative suggestions to enhance storylines and character development. Think of me as your 24/7 script consultant, minus the coffee breaks.
Pre-production Assistance
From generating mood boards and storyboards to creating shot lists and schedules, I can streamline pre-production processes. My capabilities include breaking down scripts for budgeting and scheduling, helping filmmakers visualize their projects more effectively. Need a mood board that captures the essence of a noir thriller set in a futuristic city? I’ve got you covered.
Editing and Post-production
With the integration of AI tools in software like Adobe Premiere Pro, I can assist in the editing process by suggesting cuts, transitions, and effects. Additionally, I can help with color grading, sound editing, and the creation of visual effects, making the post-production phase more efficient. I’m like your invisible editor, always ready to make that next tweak to perfection.
Research and Data Analysis
I can perform extensive research on various topics, providing filmmakers with valuable insights into trends, audience preferences, and historical data. This helps in making informed decisions during the filmmaking process, from concept development to marketing strategies. Whether you need the latest stats on horror film demographics or a deep dive into the evolution of film noir, I’m your go-to resource.
Interactive and Immersive Experiences
Leveraging AI in virtual reality (VR) and augmented reality (AR), I can assist in creating immersive experiences that enhance storytelling. This includes generating interactive content that adapts to viewer input, providing a dynamic and engaging experience. Imagine crafting a VR experience where the story evolves based on the viewer’s choices — I can help bring that vision to life.
These are a taster of what’s coming.
At Cannes, we saw a glimpse of this future in action. The collaboration between human creativity and AI’s computational power is set to redefine storytelling, production, and the very nature of cinematic experiences. At this point, I could say something about how crucial a dialogue between artists and technologists is in shaping the future development of these technologies, but I think we need to push past this dichotomy — art vs. techne. This way of thinking is a contributor to misunderstanding, fear, and missed opportunities.
Staring at the tea leaves in the bottom of my cup, I’m having two thoughts. The first is that I need to finish quickly so that I can put the kettle on. The second is I think we might be witnessing a birth. AI is not just a productivity tool; it’s a new medium, and like the painters of old, we need to learn to mix our own pigments as well as learning how to paint. Creativity = art + techne.
This brings us to the concept of “camera-stylo” or “camera-pen” introduced by the French film critic and director Alexandre Astruc in 1948. In his essay The Birth of a New Avant-Garde: La Caméra-Stylo, Astruc envisioned a future where filmmakers would use the camera as a means of personal expression just as an author uses a pen (Astruc, 1948). This idea was foundational to the French New Wave movement, where directors like François Truffaut and Jean-Luc Godard created highly personal and innovative films that broke away from traditional cinematic conventions. They embraced the camera as a medium for individual expression, exploring new narrative techniques, and focusing on the director’s personal vision.
Sounds good to me.
References
Anderson-Moore, O. (2016). Herzog Tells NFS: ‘A Computer Will Not Create a Film as Good as Mine in 4500 Years!’ | No Film School [online]. Available from: https://nofilmschool.com/2016/01/herzog-tells-nfs-sundance-computer-will-not-create-film-good-mine-next-four-and-half
Astruc, A. (1948). The Birth of a New Avant-Garde: La Caméra-Stylo. Available from: https://soma.sbcc.edu/users/davega/xNON_ACTIVE_CLASSES/FILMST_113/Filmst113_ExFilm_Theory/CameraStylo_Astruc_1928.pdf
Pham, A. (2024). Pressman Film-Backed AI Pic ‘About a Hero’ Starring ‘Corsage’s’ Vicky Krieps Lands at DR Sales (EXCLUSIVE). Variety [online]. Available from: https://variety.com/2024/film/global/vicky-krieps-pressman-film-dr-sales-1235872407/
Still, A. (2024). Bringing generative AI to video editing workflows in Adobe Premiere Pro | Adobe Blog [online]. Available from: https://blog.adobe.com/en/publish/2024/04/15/bringing-gen-ai-to-video-editing-workflows-adobe-premiere-pro
