Google Translate Can Translate Conversations In Real Time And Help You Practice A New Language

Google on Tuesday announced two new features for its popular Google Translate app that leverage advanced Gemini AI models to improve translation experiences for users. Google Translate will support live translation of real conversations on Android and iPhone, a feature that should make it even easier to get around when you don't speak the language. On top of that, Google Translate will get a new learning mode where users can practice a foreign language, similar to the Duolingo experience, albeit more limited.

Google explained during a briefing that Gemini models with "advanced reasoning and multimodal capabilities" have been used to improve the Google Translate experience and bring live translations for conversations to the app. For the learning experience, Google Translate will use AI to generate interactive practice tests on the spot and adapt in real-time those experiences to match the user's proficiency. This isn't the first time Google upgraded Google Translate with AI. A few years ago, Google added AI-powered contextual translation abilities to Google Translate.

The two new features will be available for free to Google Translate users, though the learning experience is available in beta and limited to a few languages. Google did not mention pricing details for the language-learning practice mode once the feature graduates from beta.

The Android and iPhone Google Translate apps will get a "Live translate" button following Tuesday's announcement. The app will support translations for more than 70 languages, including Arabic, French, Hindi, Korean, Spanish, and Tamil. Users in the U.S., India, and Mexico will receiv the new live translation features first.

Once you press the "Live translate" button, you can begin speaking, and the AI will translate what you said in real time. The translation will be spoken aloud, and a transcript will appear on the screen in both languages. The translation will continue once the other person responds.

Google

Google's new Gemini AI models can transition smoothly from one speaker to the next and identify conversational pauses. The AI will also understand accents and intonations. It will filter out background noise to focus on the actual speech. "This means you get a high-quality experience in the real world, like in busy airports or at a noisy café in a new country," Google said in a blog post.

The new live translation feature will require an internet connection to work, which implies that the translation will not happen on-device. As for the AI voice used to speak translations aloud, Google Translate will support a single voice at launch.

The more surprising Google Translate feature Google announced on Tuesday is the learning mode that allows users to practice their listening and speaking skills when learning a new language. Google said during the briefing that one-third of Translate users come to the app to seek help for learning a new language. That's what convinced Google to build the new AI-powered language learning features inside Google Translate rather than launching a standalone app.

People learning a new language can use the new learning features in Google Translate by tapping a "Practice" button in the app. They'll set their skill level and goals, and then Translate will generate custom scenarios for listening and speaking. Google says the scenarios were created with learning experts, taking into account the latest studies in language acquisition. The exercises will track your progress and adapt to your skill in real time.

While the experience seen in the clip above may seem similar to what you get from Duolingo, a popular app for learning new languages, Google Translate won't quite match that experience. For starters, the experience is available in beta, and it's limited to a few pairings. English speakers who want to practice Spanish and French; and Spanish, French, and Portuguese speakers learning English will be able to use the new features. The practice feature will be available on Android and iPhone.