• NewsLounge
  • Posts
  • đź’ˇGroundbreaking idea-to-video platforms

đź’ˇGroundbreaking idea-to-video platforms

Stable Video Diffusion Has Come a Long Way

newslounge.co · Nov 30, 2023

Good Morning. 

Here’s what’s on the menu today:

  • Unity lays off 275 of Weta engineers

  • AI short film / Pika Labs

  • Stable Video Diffusion has come a long Way

  • Super realistic Megahuman head

  • Reddit’s IPO(finally!!)

  • TikTok long-form videos

YouTube ad revenue, more opportunities for artists and more :)

-Ardy Ala, Dave Funston, Brandon Webb

was this email forwarded to you? you can sign up here.

HEADLINE

Unity Lays off Hundreds of Weta Digital Engineers

Chung Sung-Jun / Getty Images

in December 2021, Unity spent a cool $1.625 billion in cash and stock to purchase the tech division (Weta Digital’s tools, pipeline technology) of the Peter Jackson-led Weta Digital, taking in 275 company engineers in the process.

In addition to the layoffs, Unity will shutter 14 of its offices and scale down its remaining locations.

What happened?

  • Unity believes the Weta Digital team are remarkable, but Unity needs to become leaner as it focuses its expertise on its core business.

  • Unity has terminated its obligations to provide certain services to Weta FX and also amended certain intellectual property rights between the parties.

  • Pandemic Hiring Reversal: many companies are now reversing the pandemic hiring surge, often dismissing long-term, high-wage employees.

The recent tech layoffs involve real people losing their jobs due to companies not meeting their revenue numbers. This affects not only the employees but also the strategic direction of these companies, as they adjust to changing market conditions and technological advancements

In somewhat happier news, Weta FX has pledged to “hire back as many crew as we can” from the now-closed Weta Digital.

CASE STUDY

Short Film Made Entirely with AI Generated Images

AI Film / William Bartlett

William used Midjourney for most of the 'base' images along with krea.ai to create the 'picture in picture' shots.

  • Photoshop - generative fill to extend and adjust - as well as the neural filters tool to create depth maps.

  • Pika Labs to add movement and 'animation'. Pika 1.0 is a groundbreaking idea-to-video platform that is set to revolutionize how we bring our creative visions to life.

  • Nuke to add camera moves using the depth map from Photoshop.

  • The final shot of the 'hand drawn' earth was created using Stable Diffusion through the ComfyUI interface.

What’s the story behind Pika?

Last winter, Demi Guo and several Stanford computer science Ph.D. classmates spent their winter break trying to make a movie using generative AI. Buzzy AI video editor startup Runway, was offering a $10,000 grand prize for the best submission to its inaugural “AI Film Festival” and they were confident they had a shot. But they didn’t even place. I was so frustrated, Guo said.

So in April, Guo and Chenlin Meng, a fellow Ph.D. student, dropped out of Stanford to launch Pika to build an easier-to-use AI video generator.

Pika competes against generative AI video tools and models from the likes of Runway and Stability AI. But with Pika 1.0, Pika’s looking to elevate its game with several differentiating features.

  • ships with a tool that can extend the length of existing videos or transform them into different styles, like “live action” to “animated”

  • Another module edits video content using AI, like changing someone’s clothing or even adding another character.

In a recent report, IDC projected that generative AI investments will rise from $16 billion this year to a whopping $143 billion in 2027.

Stable Video Diffusion Has Come a Long Way

Animation & Article By Brandon Webb

In April 2023, video diffusion was in its infancy, with early tools offering frame-to-frame coherence, yet they offered limited control over the finer aspects of video manipulation. They were seen more as novelties, not yet useful for serious applications. However, this field evolves rapidly, and keeping up with the pace can be challenging.

How we got here?

Fast forward six months, several independent developers have crafted extensions for stable diffusion. Initially, we had Modelscope, promising but limited by its dataset and heavy watermarks. Zeroscope followed, offering better resolution, RAM efficiency, and a cleaner dataset. Around mid 2023, these, along with commercial releases like RunwayML, became available, still with limited control - either a text prompt or an image as a base.

The real game-changer has been the combination of extensions like Animatediff and Controlnet.

  • Animatediff enhances video diffusion with longer clips, adjustable prompts, and Controlnets compatibility.

  • Controlnets, on the other hand, lets you preprocess images for depth, line art, and pose estimation, directing the output more precisely.

This combo allows for more purposeful video generation, edging closer to a usable tool, though cohesion remains a challenge.

follow this link to see the entire workflow plus the project files to get started with Stable Video Diffusion in ComfyUI and achieve the top GIF animation.

SNIPPET

  • Reddit is considering going public as early as the first quarter.

  • Google(beginning Friday) will start deleting accounts that have been inactive for at least two years.

  • Unity To Lay Off 265 Employees At Weta Digital As Part Of "Company Reset"

  • YouTube is expanding its presence in gaming, rolling out 30+ online minigames for its Premium users across Android, iOS, and desktop.

  • Amazon has unveiled Q at their re:Invent conference in Las Vegas. It’s an AI-powered chatbot for businesses designed to interact, generate content, understand specific business needs.

Ten24, a 3D Artist and Designer from Sheffield, England, known for developing the popular asset library 3D Scan Store, has made significant strides in creating a photorealistic human head rig.

TRENDING

TikTok Creators Can Make More Money

TikTok

Creators can make more money and get more followers on TikTok by uploading longer content to the platform. In late October, TikTok invited dozens of creators to its New York office for a private event aimed at mobilizing them to create more videos at least a minute long.

Why it matters- Longer videos create more opportunities for advertising, which some in the creator space think could mean that TikTok videos may include midroll ads in the future.

In 2023, TikTok and YouTube are generating significant revenue through ad sales. TikTok's ad revenue is projected to reach $13.2 billion according to some estimates​​, while other sources anticipate it could be as high as $18 billion.

On the other hand, YouTube's ad revenue for 2023 is forecasted to rise to $30.4 billion​​.

As these platforms show more appetite for long-form content, there will be new channels for visual artists and video creators to monetize their work, leveraging the unique features and audience engagement these platforms offer.

We hope everyone has a great day. see you next week.

-newslounge