Meta unveils SeamlessM4T AI model to enable real-time language translation and transcription

San Francisco: Meta Platforms, the parent company of Facebook, has introduced an advanced AI model named SeamlessM4T, designed to facilitate real-time translation and transcription of speech across numerous languages.

This innovation holds the potential to revolutionise cross-lingual communication tools, allowing seamless interactions between individuals irrespective of language barriers.

In a blog post, Meta announced the features of the SeamlessM4T model, highlighting its ability to support translations between text and speech across nearly 100 languages. Moreover, it offers comprehensive speech-to-speech translation functionality for 35 languages, combining features that were previously available only in separate AI models.

Meta's CEO, Mark Zuckerberg, envisions these tools as fundamental to realising his vision of the Metaverse, a network of interconnected virtual worlds that could enable global user interactions. The company's commitment to fostering an open AI ecosystem aligns with Zuckerberg's strategy to harness the collaborative efforts of the community to develop user-centric tools for its social platforms.

The SeamlessM4T model is being made available to the public for non-commercial use, reflecting Meta's dedication to democratising AI technology and making it accessible to a broader audience.

Throughout this year, the prominent social media conglomerate has introduced an array of AI models, often offered for free. Notable among these is the Llama language model, which poses a significant challenge to proprietary models from Microsoft-backed OpenAI and Google's Alphabet.

Zuckerberg's emphasis on openness stems from Meta's belief that crowd-sourcing the development of consumer-focused tools for its social platforms holds greater value than charging for access to AI models. This strategy, while advantageous, also brings Meta into the realm of legal scrutiny concerning the training data employed to create its AI models.

In recent legal actions, comedian Sarah Silverman and two other authors filed copyright infringement lawsuits against both Meta and OpenAI, asserting that their literary works were used as training data without authorisation.

For the SeamlessM4T model, Meta's researchers divulged in a research paper that they accumulated audio training data from a repository containing 4 million hours of "raw audio" sourced from publicly available web data. The specifics of this repository were not disclosed. Textual data used in training was extracted from datasets formed last year, drawing content from Wikipedia and related websites.

Meta's spokesperson refrained from providing additional insights into the source of the audio data.