microsoft/VibeVoice

VibeVoice is a modular speech processing system that provides both text-to-speech (TTS) and automatic speech recognition (ASR) capabilities. It features a streaming TTS engine for real-time audio generation and an ASR module for transcribing audio and video files. The system supports efficient model adaptation through LoRA fine-tuning and offers high-performance inference via a vLLM plugin, all wrapped in web-based demos and deployment scripts.

This period focused on the modular restructuring of the VibeVoice model to support real-time streaming text-to-speech and automatic speech recognition. The work introduced new architecture classes, processors, and scheduling components to enable low-latency inference. Additionally, a comprehensive web-based demo and associated utility scripts were added to showcase these new capabilities.

7 changes

This period focused on integrating the VibeVoice ASR model into the vLLM serving infrastructure, enabling high-performance inference with support for audio and video inputs. The work included adding LoRA fine-tuning scripts, generating tokenizer files, and implementing a Gradio-based demo for transcription tasks.

5 changes

microsoft/VibeVoice was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point the surveyor at a repository you know and see whether you agree with it.