Voxtral Mini · 4B Realtime
Voxtral Mini 4B Realtime 2602 is a multilingual, realtime automatic speech recognition model from Mistral AI, released under the Apache-2.0 license. It features a natively streaming architecture with a causal audio encoder and sliding window attention, enabling configurable transcription delays from 240ms to 2.4s (and reportedly 80ms to 2.4s). The model has approximately 4B parameters (≈3.4B language model and ≈970M audio encoder) and is described as optimized for on-device deployment. It supports 13 languages and is presented as achieving accuracy comparable to offline systems with less than 500ms delay, specifically matching leading offline open-source models and realtime APIs at a 480ms delay. The model card recommends a temperature of 0.0, using websockets for audio streaming, and a delay of 480ms as a balance between performance and latency.
What do you want to do?
Choose a task. Get the software, files, settings, and steps to run it.
Open a guide for its software, hardware requirements, and setup steps.
At a glance
- Parameters
- 4B
- Architecture
- Causal audio encoder and language model
- License
- Apache 2.0
- Disk space
- 8.3 GB
- Software
- vLLM, Transformers
Will it run on your machine?
Save your machine to see a personalized rating and its reasoning.
Add my hardwareBest for
- Private meeting transcription
- Live subtitle creation
- Real-time assistants with speech understanding
- Multilingual transcription tasks
- Streaming speech-to-text applications
Tradeoffs
- Model card states inference is false, indicating no direct inference support without additional deployment setup
- Requires vLLM for deployment as library_name is vllm
- Recommended deployment practices include specific configurations such as temperature 0.0 and max-model-len settings, which may require tuning
- The model card notes that pre-allocations of RoPE parameters among other things limit max-model-len in practice
- No details provided on performance in languages beyond the 13 listed, or on accent robustness, noise conditions, or domain-specific accuracy
Ways to run it
Transformers · See official guide
Repository-specific command in publisher documentation; confirm dependencies and hardware in the source.
from transformers import VoxtralRealtimeForConditionalGeneration, AutoProcessor
from mistral_common.tokens.tokenizers.audio import Audio
from huggingface_hub import hf_hub_download
repo_id = "mistralai/Voxtral-Mini-4B-Realtime-2602"
processor = AutoProcessor.from_pretrained(repo_id)
model = VoxtralRealtimeForConditionalGeneration.from_pretrained(repo_id, device_map="auto")
repo_id = "patrickvonplaten/audio_samples"
audio_file = hf_hub_download(repo_id=repo_id, filename="bcn_weather.mp3", repo_type="dataset")
audio = Audio.from_file(audio_file, strict=False)
audio.resample(processor.feature_extractor.sampling_rate)
inputs = processor(audio.audio_array, return_tensors="pt")
inputs = inputs.to(model.device, dtype=model.dtype)
outputs = model.generate(**inputs)
decoded_outputs = processor.batch_decode(outputs, skip_special_tokens=True)
print(decoded_outputs[0])Explore its uses
Keep exploring
YuE2 · 3B
Create a full song from style and lyrics with a publisher Python package on a Linux NVIDIA GPU; outputs include a playable FLAC.
granite speech 5.0 470m turboctc
Granite Speech 5.0 470M TurboCTC is a compact 470 million parameter English automatic speech recognition model released by IBM under the Apache 2.0 license. It uses a conformer acoustic encoder with block self-attention, self-conditioning, and temporal downsampling, with an output layer of 16,384 BPE units. The model was trained on approximately 60,000 hours of English audio from public corpora using Connectionist Temporal Classification (CTC), and inference is non-autoregressive with greedy decoding. It is designed for low-latency and high-throughput English speech-to-text transcription, and is described as suitable for deployment on laptops, smartphones, and other edge devices. Native support is available in transformers>=5.16.0, with additional usage options for Apple Silicon via mlx-audio and local GGUF inference via transcribe.cpp.
Whisper
Explore 6 versions of Whisper. Compare model sizes, packages, hardware requirements, and available setups.