ARTFEED — Contemporary Art Intelligence

Open TTS Leaderboard Launches Objective Metrics for Multilingual Text-to-Speech and Voice Cloning

ai-technology · 2026-09-30

A new leaderboard for Open TTS has been introduced to assess text-to-speech models through objective criteria, aiming to reduce fragmentation in TTS assessments. It evaluates intelligibility by calculating word and character error rates with Qwen3 ASR, measures speed using inverse real-time factor and time-to-first-audio on H200 GPU and CPU, and assesses speaker similarity through cosine similarity of WavLM embeddings. The time required for evaluations has been cut down from weeks to mere hours. The top models, based on macro-average WER for English splits of Seed TTS Eval and CV3 Eval, include hexgrad/Kokoro-82M, Supertone/supertonic-3, and fishaudio/s2-pro. Multilingual capabilities feature k2-fsa/OmniVoice and FunAudioLLM/Fun-CosyVoice3-0.5B-2512. This leaderboard prioritizes open-source models and intends to release evaluation scripts. As of September 30, 2026, only 16 out of 92 models on Artificial Analysis are open-weight.

Key facts

  • Open TTS Leaderboard uses objective metrics: WER/CER, RTFx, TTFA, and speaker similarity
  • Evaluation time drops from weeks to hours
  • As of Sep 30, 2026, only 16 of 92 models on Artificial Analysis are open-weights
  • hexgrad/Kokoro-82M, Supertone/supertonic-3, and fishaudio/s2-pro lead on English WER
  • k2-fsa/OmniVoice, fishaudio/s2-pro, and FunAudioLLM/Fun-CosyVoice3-0.5B-2512 are strong multilingual models
  • kyutai/pocket-tts performs well for streaming on GPU and CPU
  • Leaderboard includes Listen and Streaming tabs, with voting requiring HF login
  • Evaluation scripts will be open-sourced like the Open ASR Leaderboard repo

Entities

Artists

Institutions

  • Hugging Face
  • Artificial Analysis
  • Voice Arena
  • Open ASR Leaderboard
  • Seed TTS Eval
  • CV3 Eval

Sources