AIToolScan

audio.cpp Overview

audio.cpp (0xShug0/audio.cpp) is an all-in-one, pure C++ inference engine for audio models, powered by ggml — the same approach as llama.cpp, but for audio. It replaces the usual Python stack (dozens of packages, Conda environments, dependency conflicts) with one shared native runtime that runs TTS, voice cloning, voice conversion, ASR, diarization, VAD, source separation, forced alignment, codec, music generation, and more. No Python dependency is required for inference.

• All-in-One Audio Engine: One framework covers text-to-speech (Qwen3-TTS, Fish Audio S2 Pro, Supertonic 3, VibeVoice, VoxCPM2, MioTTS), speech recognition (Nemotron 3.5 ASR, Qwen3-ASR, Fun-ASR-Nano, Voxtral, Parakeet-TDT), VAD, diarization, voice conversion (RVC, SeedVC, Chatterbox), source separation (HTDemucs, BS-RoFormer), music generation (Stable Audio 3, MiniMax-H3, HeartMuLa), and more — 49 model families and 70+ variants in release 0.6.
• CLI Usage:audiocpp_cli --task tts|asr|vc|vad|diar|sep|gen|align|... --model <path> --backend cpu|cuda|vulkan|metal|hip — for example TTS with --task tts --family pocket_tts --text "Hello" --out out.wav, transcription with --task asr --family qwen3_asr --audio sample.wav, plus batch folders (--batch-text-dir, --batch-audio-dir), long-audio chunking, and JSON pipelines (--pipeline) for multi-step workflows like redubbing or narration.
• Server & OpenAI-Compatible API:audiocpp_server --config server.json exposes /v1/audio/speech (TTS), /v1/audio/transcriptions (STT), /v1/models, and /health, with lazy model loading for production-style services.
• Built-in WebUI:audiocpp_server --ui ships an embedded SvelteKit interface compiled into the binary — TTS, cloning, ASR, generation, conversion, separation, VAD, and diarization from the browser with microphone recording and near-live ASR, no Python and no separate frontend files.
• GGUF Quantization: Every released model family supports GGUF loading; tested Q8 packages run up to 1.53x faster while cutting peak VRAM by up to ~37%.
• Best on NVIDIA GPUs: CUDA is the optimized path — multiple TTS routes already run 1.8x to 8x faster than their Python references with 45%-85% lower latency; Supertonic 3 generates about 10 hours of audio in 3 minutes on an RTX 5090 (200x+ real-time), and VibeVoice 1.5B renders a 93.9-minute podcast in 18.2 minutes.
• AMD GPUs: Early HIP/ROCm support (ROCm 6.4 for full coverage incl. RX 7600, ROCm 7.1 recommended for RDNA4) lets AMD users run CUDA-class models with the ROCm runtime bundled.
• Apple Silicon: Metal backend with optimized ops — tested VoxCPM2 end-to-end runs up to 2.56x faster on Apple silicon; great for on-device TTS and ASR on MacBook and Mac Studio.
• CPU-Only Machines: Fully works without any GPU — Supertonic 3 stays over 6x real-time on CPU, making real-time TTS viable on ordinary laptops, mini PCs, and servers; Vulkan and CPU backends cover portability and testing.
• Easy Setup & Docker: Prebuilt Windows (CUDA/CPU) binaries on the Releases page, Homebrew install on macOS (brew install audio-cpp), official CUDA and CPU Docker images for CLI and server, and a model manager (tools/model_manager_v2.py) that downloads ready-to-use GGUF packages.
• Open Source: Apache 2.0 licensed, 1.7K+ GitHub stars, actively developed with a fast-moving contributor community (releases 0.3 through 0.6 landed within two months).