Interesting find of the week. Robot conductor? Take a look…

In this issue:

🎬 ADOBE FIREFLY VIDEO: NEW AI-POWERED VIDEO GENERATION AND EDITING TOOLS

🎮 AI IN VIDEO GAMES: UPCOMING TITLES FEATURING AI-ENHANCED GAMEPLAY

📡ON OUR RADAR:

• SUNO SCENES: AI-GENERATED MUSIC FOR PHOTOS AND VIDEOS

• GS^3: EFFICIENT RELIGHTING WITH TRIPLE GAUSSIAN SPLATTING

• HART: HYBRID AUTOREGRESSIVE TRANSFORMER FOR VISUAL GENERATION

• EFFICIENTVIT: MULTI-SCALE LINEAR ATTENTION FOR HIGH-RESOLUTION TASKS

• CTRLORA: FRAMEWORK FOR CONTROLLABLE IMAGE GENERATION

• SGEDIT: SCENE GRAPH-BASED IMAGE EDITING WITH LLMS

• FLEXGEN: FLEXIBLE MULTI-VIEW GENERATION FROM TEXT AND IMAGE

• SAR: SET AUTOREGRESSIVE MODELING FOR IMAGE GENERATION

• TEX4D: ZERO-SHOT 4D SCENE TEXTURING WITH VIDEO DIFFUSION

• AI ALGORITHM FOR MASTER-LEVEL PIANO PERFORMANCES

If you’re trying to get to inbox zero but you still want to read this later:

ADOBE FIREFLY VIDEO MODEL

Adobe has unveiled its Firefly Video Model at it’s Adobe MAX 2024 event, introducing a suite of AI-powered video generation and editing tools. The centerpiece of this release is the Text-to-Video and Image-to-Video generators, now available in limited public beta through the Firefly web app. These tools can create short video clips up to 5 seconds long at 720p resolution and 24 frames per second, based on text prompts or reference images. Users can fine-tune their generated videos using various "camera controls" that simulate different angles, motions, and shooting distances, offering a range of styles from realistic footage to 3D animation and stop motion.

Complementing the web-based tools, Adobe has integrated AI capabilities directly into Premiere Pro with the Generative Extend feature, currently in beta. This tool allows editors to extend the beginning or end of a clip by up to two seconds, or make mid-shot adjustments to correct issues like shifting eye-lines or unexpected movements. While limited in duration, this feature could potentially eliminate the need for reshoots to fix minor timing issues. Generative Extend also works with audio, capable of extending sound effects and ambient noise by up to ten seconds, though it cannot generate spoken dialogue or music.

Adobe emphasizes the responsible development of these AI tools, stating that the Firefly Video Model is trained exclusively on licensed content, making it "commercially safe" for professional use. This approach aims to address concerns about copyright and ethical AI training practices that have plagued some competitors. Additionally, Adobe is integrating Content Credentials into videos created or edited with these tools, allowing for transparency about AI usage and ownership rights. While the current capabilities may seem limited compared to some other AI video generators, Adobe's focus on integration with existing workflows and tools like Premiere Pro positions these features as practical additions to professional video production pipelines, rather than standalone novelties.

‘The Verge’ Source

‘Adobe Blog’ Source

AI IN VIDEO GAMES

The integration of artificial intelligence in video games continues to open up exciting new possibilities for player experiences! Two upcoming video games are set to offer unique gameplay experiences in their respective genres. "Echo of Mobius" is a role-playing game (RPG) that allows players to create and customize characters with diverse backstories and abilities, engaging in turn-based battles and adventures. "Auravale" is a life simulation game where players guide characters through daily activities, social interactions, and personal growth in a customizable world. Both games incorporate artificial intelligence technology to enhance various aspects of gameplay, from character creation to narrative generation.

Echo of Mobius:

Developed by studio D, Echo of Mobius is an upcoming RPG slated for release in November 2024. The game's standout feature is its use of in-house AI models for character creation and storytelling. Players can imagine and bring to life any character concept, from brave orc fighters to dancing cookies, with the AI generating unique appearances, backstories, and personalities. The game also employs AI to craft adventure narratives and dialogues, promising an ever-evolving world shaped by player choices. While Echo of Mobius embraces AI technology, the developers emphasize their commitment to originality, using only AI trained on their own artist-created datasets for character visuals.

Auravale:

Auravale, developed by Incite, reimagines the life simulation genre with AI at its core. This game allows players to guide their characters, called "auras," through text-based instructions. The AI interprets these commands to determine character behavior, needs, and interactions. Auravale features both a graphical interface and an alternate "text mode" for a more narrative-driven experience. The game emphasizes social interactions, allowing players to form relationships, customize their homes, and even create their own scenarios using natural language inputs. With plans to support local AI models for reduced costs, Auravale aims to offer a deeply personalized and dynamic living world where every player's story unfolds uniquely based on their written inputs and choices.

PUT THIS ON YOUR RADAR