Whisper is an automatic speech recognition (ASR) system developed by OpenAI, designed for high robustness and accuracy using a vast, diverse dataset.

What languages can Whisper transcribe?

Whisper can transcribe speech in multiple languages and translate these languages into English.

How does Whisper handle input audio?

Input audio is split into 30-second chunks, converted into log-Mel spectrograms, and then passed into an encoder to predict text captions.

What are the special tokens used for in Whisper?

Special tokens in Whisper are used for language identification, phrase-level timestamps, multilingual speech transcription, and translation into English.

How does Whisper compare to existing ASR systems?

Whisper is more robust, with 50% fewer errors in zero-shot performance across diverse datasets due to its large and diverse training dataset.

Is Whisper specialized for certain benchmarks?

While Whisper doesn't excel in specialized benchmarks like LibriSpeech, it outperforms state-of-the-art models in zero-shot translations on datasets like CoVoST2.

What resources are available for Whisper?

OpenAI has made the paper, model card, and code for Whisper available for more detailed understanding and experimentation.

What impact does Whisper aim to have?

Whisper aims to enable developers to add voice interfaces easily to various applications due to its high accuracy and ease of use.

Whisper (OpenAI)

Name: Whisper (OpenAI)
Brand: Whisper (OpenAI)
Rating: 5 (1 reviews)
Author: OpenAI

By OpenAI

0 reviews

Free

Claim Tool

What is Whisper (OpenAI)?

Whisper is a cutting-edge automatic speech recognition (ASR) system created by OpenAI. Trained on 680,000 hours of multilingual and multitask supervised data from the web, Whisper boasts improved robustness to accents, background noise, and technical language. It provides transcription services in multiple languages and translates those languages into English. Whisper uses an encoder-decoder Transformer architecture that captures 30-second audio chunks, converts them to log-Mel spectrograms, and predicts corresponding text captions. Its large and diverse dataset helps Whisper outperform existing systems in zero-shot performance across diverse scenarios.

Speech-To-Text24 favourites

Whisper (OpenAI)'s Top Features

Key capabilities that make Whisper (OpenAI) stand out.

High robustness to accents and background noise

Supports multiple languages

Translates languages into English

Encoder-decoder Transformer architecture

Processes 30-second audio chunks

Predicts text captions with special tokens integration

Improved zero-shot performance

Open-source with detailed resources

Enables voice interfaces for applications

Outperforms on CoVoST2 for English translation

Key Details

Category: Speech-To-Text
Pricing Model: Free
Website: Visit Whisper (OpenAI)
Last Updated: August 8, 2024

About the Maker

The organization behind Whisper (OpenAI).

OpenAIai lab

Creating safe AGI that benefits all of humanity

234 Tools6 ModelsFounded 2015San Francisco, CA

View full profile

AI Models by OpenAI

Large language models from the same organization.

Model	Context Window	Price (In / Out per M)	Capabilities
GPT-5.4Current	1.1M	$2.50 / $15.00	textvisioncodetool use+2
GPT-5.3 CodexCurrent	400K	$1.75 / $14.00	textvisioncodetool use+2
o3Current	200K	$10.00 / $40.00	textvisioncodetool use+1
GPT-4o miniCurrent	128K	$0.15 / $0.60	textvisioncodetool use
GPT-4oCurrent	128K	$2.50 / $10.00	textvisioncodetool use+1
GPT-5	400K	$1.25 / $10.00	textvisioncodetool use+1