At AWS re:Invent, Amazon Web Services (AWS) announced its partnership with Twelve Labs, a startup pioneering multimodal AI to enhance video content analysis. By building its foundation models on AWS, Twelve Labs empowers developers to create applications capable of deep video understanding, revolutionizing industries reliant on multimedia data.

1. Unlocking the Potential of Video Content:

• The Challenge: Nearly 80% of global data resides in video format, yet much of it remains unsearchable.
• The Solution: Twelve Labs’ AI models map natural language to video content, enabling applications to search, classify, and summarize videos effectively.

2. Transformative Applications Powered by AI:

• Semantic Video Search and Text Generation:
  • Available on AWS Marketplace, these tools are tailored for industries like media, entertainment, sports, and gaming.
  • Example: Sports leagues use this technology to catalog footage or create highlight reels automatically.
• Personalized Viewer Experiences:
  • Content creators can compile tailored highlight reels, such as action sequences starring a favorite actor.
  • Coaches can analyze athletes’ performance using precise video moments.

3. Key Foundation Models by Twelve Labs:

• Marengo and Pegasus Models:
  • Provide text summaries and audio translations in 100+ languages.
  • Enable contextual searches to identify specific events or moments in video content.
  • Example: Matching spoken words to visual elements in sports or entertainment videos.

4. Advanced Technology Behind the Models:

• Amazon SageMaker HyperPod:
  • Facilitates training of multimodal models using parallel compute instances.
  • Capable of processing videos, images, speech, and text for deeper insights.
• Global Deployment:
  • With AWS Activate, Twelve Labs scales its technology globally while optimizing machine learning performance.

5. Strategic Collaboration for Future Growth:

• Three-Year Agreement with AWS:
  • Enhances Twelve Labs’ model training capabilities.
  • Expands video intelligence services to new industries via AWS Marketplace.
  • Focus on reducing model training costs and improving operational efficiency.

6. Industry Impact:

• Media and Entertainment: Automated content creation and tailored highlight reels.
• Sports: Precision coaching tools for performance improvement.
• Global Reach: Advanced video intelligence solutions delivered to a diverse customer base.

Twelve Labs, powered by AWS, is breaking barriers in video content understanding with multimodal AI. By integrating advanced technologies like Amazon SageMaker HyperPod, Twelve Labs is enabling industries to unlock actionable insights from video data, driving innovation and enhancing user experiences worldwide.