Complete Technical Paper Content — Speech Recognition Projects
1. Executive Abstract, Problem Formulation & Background
Speech recognition (automatic speech recognition (ASR), computer speech recognition, or speech-to-text (STT)) is a sub-field of computational linguistics concerned with methods and technologies that translate spoken language into text or other interpretable forms. Speech recognition applications include voice user interfaces, where the user speaks to a device, which "listens" and processes the audio. Common voice applications include interpreting commands for calling, call routing, home automation, and aircraft control. These applications are called direct voice input. Productivity applications include searching audio recordings, creating transcripts, and dictation. Speech recognition can be used to analyse speaker characteristics, such as identifying native language using pronunciation assessment. Voice recognition (speaker identification) refers to identifying the speaker, rather than speech contents. Recognizing the speaker can simplify the task of translating speech in systems trained on a specific person's voice. It can also be used to authenticate the speaker as part of a security process.
1982 – Dragon Systems, founded by James and Janet M. Baker, was one of IBM's few competitors.
1987 – The back-off model enabled language models to use multiple-length n-grams, and CSELT used HMM to recognize languages (in software and hardware, e.g. RIPAC). At the end of the DARPA program in 1976, the best computer available to researchers was the PDP-10 with 4 MB of RAM. It could take up to 100 minutes to decode 30 seconds of speech. Practical products included:
1984 – the Apricot Portable was released with up to 4096 words support, of which only 64 could be held in RAM at a time. 1987 – a recognizer from Kurzweil Applied Intelligence 1990 – Dragon Dictate, a consumer product released in 1990. AT&T deployed the Voice Recognition Call Processing service in 1992 to route telephone calls without a human operator. The technology was developed by Lawrence Rabiner and others at Bell Labs. By the early 1990s, the vocabulary of the typical commercial speech recognition system had exceeded the average human vocabulary. Reddy's former student, Xuedong Huang, developed the Sphinx-II system at CMU. Sphinx-II was the first to do speaker-independent, large vocabulary, continuous speech recognition, and it won DARPA's 1992 evaluation. Handling continuous speech with a large vocabulary was a major milestone. Huang later founded the speech recognition group at Microsoft in 1993. Reddy's student Kai-Fu Lee joined Apple, where, in 1992, he helped develop the Casper speech interface prototype. Lernout & Hauspie, a Belgium-based speech recognition company, acquired other companies, including Kurzweil Applied Intelligence in 1997 and Dragon Systems in 2000. L&H was used in Windows XP. L&H was an industry leader until an accounting scandal destroyed it in 2001. L&H speech technology was bought by ScanSoft, which became Nuance in 2005. Apple licensed Nuance software for its digital assistant Siri.
Speech recognition systems are based on HMMs. These are statistical models that output a sequence of symbols or quantities. HMMs are used in speech recognition because a speech signal can be viewed as a piecewise stationary signal or a short-time stationary signal. In a short time scale (e.g. 10 milliseconds), speech can be approximated as a stationary process. Speech can be thought of as a Markov model for many stochastic purposes. HMMs are popular because they can be trained automatically and are simple and computationally feasible. An HMM outputs a sequence of n-dimensional real-valued vectors (where n is an integer such as 10), outputting one every 10 milliseconds. The vectors consist of cepstral coefficients, obtained by a Fourier transform of a short window of speech and decorrelating the spectrum using a cosine transform, then taking the first (most significant) coefficients. The HMM tends to have, in each state, a statistical distribution that is a mixture of diagonal covariance Gaussians, which give a likelihood for each observed vector. Each word, or (for more general speech recognition systems), each phoneme, has a different output distribution; an HMM for a sequence of words or phonemes is made by concatenating the individual trained HMMs for the separate words and phonemes. Speech recognition systems use combinations of standard techniques to improve results. A typical large-vocabulary system applies context dependency for the phonemes (so that phonemes with different left and right context have different realizations as HMM states). It uses cepstral normalization to handle speaker and recording conditions. It might use vocal tract length normalization (VTLN) for male-female normalization and maximum likelihood linear regression (MLLR) for more general adaptation. The features use delta and delta-delta coefficients to capture speech dynamics, and in addition might use heteroscedastic linear discriminant analysis (HLDA); or might use splicing and LDA-based projection, followed by HLDA or a global semi-tied covariance transform (also known as maximum likelihood linear transform (MLLT)). Many systems use discriminative training techniques that dispense with a purely statistical approach to HMM parameter estimation and instead optimize some classification-related measure of the training data. Examples are maximum mutual information (MMI), minimum classification error (MCE), and minimum phone error (MPE).
Dynamic time warping was historically used for speech recognition, but was later displaced by HMM. Dynamic time warping measures similarity between two sequences that may vary in time or speed. For instance, similarities in walking patterns could be detected, even if in one video a person was walking slowly and in another was walking more quickly, or even if accelerations and decelerations came during one observation. DTW has been applied to video, audio, and graphics – any data that can be turned into a linear representation can be analyzed with DTW. This could handle speech at different speaking speeds. In general, it allows an optimal match between two sequences (e.g., time series) with certain restrictions. The sequences are "warped" non-linearly to match each other. This sequence alignment method is often used in the context of HMMs.
Neural networks became interesting in the late 1980s before beginning to dominate in the 2010s. Neural networks have been used in many aspects of speech recognition, such as phoneme classification, phoneme classification through multi-objective evolutionary algorithms, isolated word recognition, audiovisual speech recognition, audiovisual speaker recognition, and speaker adaptation. Neural networks make fewer explicit assumptions about feature statistical properties than HMMs. When used to estimate the probabilities of a speech segment, neural networks allow natural and efficient discriminative training. However, in spite of their effectiveness in classifying short-time units such as individual phonemes and isolated words, early neural networks were rarely successful for continuous recognition because of their limited ability to model temporal dependencies. One approach was to use neural networks for feature transformation, or dimensionality reduction. However, more recently, LSTM and related recurrent neural networks (RNNs), Time Delay Neural Networks (TDNN's), and transformers demonstrated improved performance.
Researchers are exploring deep neural networks (DNNs) and denoising autoencoders. A DNN is a type of artificial neural network that includes multiple hidden layers between the input and output. Like simpler neural networks, DNNs can model complex, non-linear relationships. However, their deeper architecture allows them to build more sophisticated representations that combine features from earlier layers. This gives them a powerful ability to learn and recognize complex patterns in speech data. A major breakthrough in using DNNs for large vocabulary speech recognition came in 2010. In a collaboration between industry and academia, researchers used DNNs with large output layers based on context-dependent HMM states that were created using decision trees. This approach significantly improved performance. A core idea behind deep learning is to eliminate the need for manually designed features and instead learn directly from input data. This was first demonstrated using deep autoencoders trained on raw spectrograms or linear filter-bank features. These models outperformed traditional Mel-Cepstral features, which rely on fixed transformations. More recently, researchers showed that waveforms can achieve excellent results in large-scale speech recognition.
Automatic pronunciation assessment is the use of speech recognition to verify the correctness of speech, as distinguished from assessment by a person. Also called speech verification, pronunciation evaluation, and pronunciation scoring, the main application of this technology is computer-aided pronunciation teaching (CAPT) when combined with computer-aided instruction for computer-assisted language learning (CALL), speech remediation, or accent reduction. Pronunciation assessment does not determine unknown speech (as in dictation or automatic transcription) but instead, compares speech to a reference model for the words spoken, sometimes with inconsequential prosody such as intonation, pitch, tempo, rhythm, and stress. Pronunciation assessment is also used in reading tutoring, for example in products such as Microsoft Teams and Amira Learning. Pronunciation assessment can also be used to help diagnose and treat speech disorders such as apraxia. Assessing intelligibility is essential for avoiding inaccuracies from accent bias, especially in high-stakes assessments, from words with multiple correct pronunciations, and from phoneme coding errors in digital pronunciation dictionaries. In 2022, researchers found that some newer speech to text systems, based on end-to-end reinforcement learning to map audio signals directly into words, produce word and phrase confidence scores closely correlated with listener intelligibility. In the Common European Framework of Reference for Languages (CEFR) assessment criteria for "overall phonological control", intelligibility outweighs formally correct pronunciation at all levels.
ASR is now commonplace in the field of telephony. In telephony systems, ASR is predominantly used in contact centers by integrating it with IVR systems. It is becoming more widespread in computer gaming and simulation. Despite the high level of integration with word processing in general personal computing, in the field of document production, ASR has not seen the expected increases in use. The improvement of mobile processor speeds has made speech recognition practical in smartphones. Speech is used mostly as a part of a user interface, for creating predefined or custom speech commands.
Aerospace, e.g., NASA's Mars Polar Lander used speech recognition technology from Sensory, Inc. in the Mars microphone on the lander Automatic subtitling with speech recognition Automatic emotion recognition Automatic shot listing in audiovisual production Automatic translation E-discovery Hands-free computing Home automation Interactive voice response Mobile telephony, including mobile email Multimodal interaction Real-time captioning Robotics Security, including usage with other biometric scanners for multi-factor authentication Speech to text Telematics, e.g., vehicle navigation systems Transcription Video games like Tom Clancy's EndWar, Bow-wow Battle, and Lifeline Virtual assistant such as Siri
Vocabulary size and confusability Speaker dependence versus independence Isolated, discontinuous, or continuous speech Task and language constraints Read versus spontaneous speech Adverse conditions
2. Related Work, Literature Survey & Algorithmic Foundations
Error rates increase as the vocabulary size grows: e.g. the 10 digits "zero" to "nine" can be recognized essentially perfectly, but vocabulary sizes of 200, 5000, or 100000 may have error rates of 3%, 7%, or 45% respectively. Vocabulary is hard to recognize if it contains confusing letters: e.g. the 26 letters of the English alphabet are difficult to discriminate because they are confusing words (most notoriously, the E-set: "B, C, D, E, G, P, T, V, Z (when "Z" is pronounced "zee" rather than "zed", depending on region); an 8% error rate is considered good for this vocabulary. Speaker dependence vs. independence: A speaker-dependent system is intended for use by a single speaker. A speaker-independent system is intended for use by any speaker (more difficult). Isolated, discontinuous or continuous speech With isolated speech, single words are used, which is easier to recognize. With discontinuous speech, full sentences separated by silence are used. The silence is easier to recognize similar to isolated speech. With continuous speech naturally spoken sentences are used, which are harder to recognize. Task and language constraints can inform the recognition The requesting application may dismiss the hypothesis "The apple is red." Constraints may be semantic; rejecting "The apple is angry." Syntactic; rejecting "Red is apple the." Constraints are often represented by grammar. Read vs. spontaneous speech When a person reads it's usually in a context that has been previously prepared. When a person speaks spontaneously, recognition must deal with disfluencies such as "uh" and "um", false starts, incomplete sentences, stuttering, coughing, and laughter) and limited vocabulary. Adverse conditions environmental noise (e.g., in a car or factory). Acoustic distortions (e.g. echoes, room acoustics) Speech recognition is a multi-level pattern recognition task.
Acoustic signals are structured into a hierarchy of units, e.g. phonemes, words, phrases, and sentences; Each level provides additional constraints; e.g., known word pronunciations or legal word sequences, which can compensate for errors or uncertainties at a lower level; This hierarchy of constraints improves accuracy. By combining decisions probabilistically at all lower levels, and making ultimate decisions only at the highest level, speech recognition is broken into several phases. Computationally, it is a problem in which a sound pattern has to be recognized or classified into a category that represents a meaning to a human. Every acoustic signal can be broken into smaller sub-signals. As the more complex sound signal is divided, different levels are created, where at the top level are complex sounds made of simpler sounds on the lower level, etc. At the lowest level, simple and more probabilistic rules apply. These sounds are put together into more complex sounds on upper level, a new set of more deterministic rules predicts what the complex sound represents. The upper level of a deterministic rule should figure out the meaning of complex expressions. In order to expand our knowledge about speech recognition, we need to take into consideration neural networks. Neural network approaches use the following steps:
Digitize the speech – for telephone speech, 8000 samples per second are captured; Compute features of spectral-domain of the speech (with Fourier transform); computed every 10ms, with one 10ms section called a frame; Sound is produced by air (or some other medium) vibration. Sound creates a wave that has two measures: amplitude (strength), and frequency (vibrations per second). Accuracy can be computed with the help of WER, which is calculated by aligning the recognized word and referenced word using dynamic string alignment. The problem may occur while computing the WER due to the difference between the sequence lengths of the recognized word and referenced word. The formula to compute the word error rate (WER) is:
W E R = ( s + d + i ) n {\displaystyle WER={(s+d+i) \over n}}
where s is the number of substitutions, d is the number of deletions, i is the number of insertions, and n is the number of word references. While computing, the word recognition rate (WRR) is used. The formula is:
W R R = 1 − W E R = ( n − s − d − i ) n = h − i n {\displaystyle WRR=1-WER={(n-s-d-i) \over n}={h-i \over n}}
h = n − ( s + d ) . {\displaystyle h=n-(s+d).}
MLX is an open-source machine learning framework developed by Apple and designed primarily for Apple silicon. It is used for training and running machine learning models and is optimized for Apple silicon's unified memory architecture, allowing the CPU and GPU to work with the same data without copying it between separate memory pools. MLX can be used for applications including large language model training and inference, image generation, and speech recognition.
Speech Recognition Project Introduction This project is a part of the Summer Semester 2024/2025 Deep Learning course at the Warsaw University of Technology, Faculty of Mathematics and Information Sciences. The aim of this project is to implement several deep learning architectures (mainly Transformers) to classify basic speech commands. Prepare the dataset Since this dataset comes from a Kaggle competition, the official test data is provided without labels. Therefore, this project only uses the train data and splits it into train , validation , and test sets. You are, of course, more than welcome to use the unlabeled test data provided by Kaggle for your own experiments. Download the dataset The dataset can be found here . You need to find the download button (usually in the section Data , at the bottom) Extract the dataset Extract the downloaded archive. Inside, you will find a train.7z archive. Extract it as well. Copy the audio/ folder Inside the extracted train/ folder, you will find an audio/ directory. Move this audio/ directory to the root folder of this repository. You can now safely remove the rest of the downloaded archive. Create unknown class (optional) Run create_unknown_class.py Run split_dataset.py Run the script to split the dataset. The processed files will be saved under a new dataset/ directory. Once the process is complete, you can remove the original audio/ directory to save disk space. Setup Linux : Create virual environment: python3 -m venv venv source venv/bin/activate Install required libraries: pip install torchaudio numpy matplotlib scikit-learn torchsummary torchcodec Ensure you have ffmpeg installed. If not, run: sudo apt update sudo apt install ffmpeg Usage After downloading the dataset, run split_dataset.py to split the dataset into train, validation, and test parts. (See the Prepare the dataset section). Training Select the desired parameters in train.py and run the file. Models will be saved under ./models/ , with each model in a separate directory. The model (and optimizer) will be saved after each epoch. Create confusion matrix To create a confusion matrix, run create_confusion_matrix.py . First, select the model (path and iteration) inside the create_confusion_matrix.py file. The confusion matrix will be saved under the images/ directory. Example confusion matrix: Create a learning curve To create a learning curve, run create_learning_curve.py . First, select the model inside the create_learning_curve.py file. The learning curve will be saved under the images/ directory. Example learning curve: Create a spectrogram You can also create a spectrogram (image) of a given sound file. This process is independent of deep learning. To do so, run create_spectrogram.py (you can first select the file ID inside it). It will also be saved inside the images/ directory. Example spectrogram: Results Experiments were conducted to determine the optimal number of blocks and the number of attention heads in each block. The results are presented below (the larger number represents the F1 score, and the smaller number below it indicates the number of parameters).
This project is a part of the Summer Semester 2024/2025 Deep Learning course at the Warsaw University of Technology, Faculty of Mathematics and Information Sciences.
The aim of this project is to implement several deep learning architectures (mainly Transformers) to classify basic speech commands.
Since this dataset comes from a Kaggle competition, the official test data is provided without labels. Therefore, this project only uses the train data and splits it into train , validation , and test sets. You are, of course, more than welcome to use the unlabeled test data provided by Kaggle for your own experiments.
3. Dataset Ingestion, Preprocessing & Stochastic Augmentations
After downloading the dataset, run split_dataset.py to split the dataset into train, validation, and test parts. (See the Prepare the dataset section).
Select the desired parameters in train.py and run the file. Models will be saved under ./models/ , with each model in a separate directory. The model (and optimizer) will be saved after each epoch.
To create a confusion matrix, run create_confusion_matrix.py . First, select the model (path and iteration) inside the create_confusion_matrix.py file. The confusion matrix will be saved under the images/ directory. Example confusion matrix:
To create a learning curve, run create_learning_curve.py . First, select the model inside the create_learning_curve.py file. The learning curve will be saved under the images/ directory.
You can also create a spectrogram (image) of a given sound file. This process is independent of deep learning. To do so, run create_spectrogram.py (you can first select the file ID inside it). It will also be saved inside the images/ directory.
Experiments were conducted to determine the optimal number of blocks and the number of attention heads in each block. The results are presented below (the larger number represents the F1 score, and the smaller number below it indicates the number of parameters).
huggingface / transformers Star 168k 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training. audio python nlp machine-learning natural-language-processing deep-learning pytorch transformer speech-recognition glm pretrained-models hacktoberfest gemma vlm pytorch-transformers model-hub llm qwen deepseek Updated Oct 29, 2026 Python
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
ggml-org / whisper.cpp Star 54.8k Port of OpenAI's Whisper model in C/C++ inference transformer speech-recognition openai speech-to-text whisper Updated Oct 28, 2026 C++
mozilla / DeepSpeech Star 26.8k DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers. machine-learning embedded deep-learning offline tensorflow speech-recognition neural-networks speech-to-text deepspeech on-device Updated Jun 19, 2025 C++
DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.
SYSTRAN / faster-whisper Star 26.3k Faster Whisper transcription with CTranslate2 deep-learning inference transformer speech-recognition openai speech-to-text quantization whisper Updated Oct 25, 2026 Python
4. Neural Network Backbone, Layer Specifications & Loss Formulations
m-bain / whisperX Sponsor Star 24.9k WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization) speech speech-recognition speech-to-text whisper asr Updated Oct 22, 2026 Python
modelscope / FunASR Star 20.9k Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving. audio pytorch speech-recognition chinese speech-to-text punctuation transcription asr speaker-diarization emotion-recognition voice-activity-detection streaming-asr paraformer vllm real-time-asr openai-compatible-api funasr mcp-server multilingual-asr whisper-alternative Updated Oct 28, 2026 Python
Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
leon-ai / leon Star 17.6k 🧠 Leon is your open-source personal assistant. nodejs python bot text-to-speech automation privacy ai offline chatbot artificial-intelligence speech-synthesis assistant speech-recognition personal-assistant speech-to-text leon voice-assistant virtual-assistant ai-assistant ai-agent Updated Oct 29, 2026 TypeScript
kaldi-asr / kaldi Star 15.5k kaldi-asr/kaldi is the official location of the Kaldi project. shell c-plus-plus cuda speech speech-recognition speech-to-text kaldi speaker-verification speaker-id Updated Sep 22, 2025 Shell
alphacep / vosk-api Star 15.2k Offline speech recognition API for Android, iOS, Raspberry Pi and servers with Python, Java, C# and Node android python raspberry-pi ios privacy deep-neural-networks deep-learning offline voice-recognition speech-recognition speech-to-text kaldi stt speaker-verification asr speech-to-text-android deepspeech speaker-identification google-speech-to-text vosk Updated Oct 15, 2026 Jupyter Notebook
Offline speech recognition API for Android, iOS, Raspberry Pi and servers with Python, Java, C# and Node
NVIDIA / DeepLearningExamples Star 14.9k State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure. nlp translation computer-vision deep-learning mxnet tensorflow pytorch speech-synthesis speech-recognition forecasting drug-discovery recommender-systems paddlepaddle tensorflow2 large-language-models Updated Aug 12, 2024 Jupyter Notebook
State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.
abus-aikorea / voice-pro Star 13.4k Gradio WebUI for creators and developers, featuring key TTS (Edge-TTS, kokoro) and zero-shot Voice Cloning (E2 & F5-TTS, CosyVoice), with Whisper audio processing, YouTube download, Demucs vocal isolation, and multilingual translation. text-to-speech translator audiobook podcasts tts speech-synthesis subtitles speech-recognition webui speech-to-text karaoke transcription gradio whisper voice-conversion voice-cloning yt-dlp faster-whisper whisperx Updated Jul 13, 2026 Python
Gradio WebUI for creators and developers, featuring key TTS (Edge-TTS, kokoro) and zero-shot Voice Cloning (E2 & F5-TTS, CosyVoice), with Whisper audio processing, YouTube download, Demucs vocal isolation, and multilingual translation.
kmario23 / deep-learning-drizzle Star 13k Drench yourself in Deep Learning, Reinforcement Learning, Machine Learning, Computer Vision, and NLP by learning from these exciting lectures!! machine-learning natural-language-processing deep-neural-networks reinforcement-learning computer-vision deep-learning optimization probability deep-reinforcement-learning medical-imaging speech-recognition artificial-neural-networks pattern-recognition probabilistic-graphical-models bayesian-statistics artificial-intelligence-algorithms visual-recognition geometric-deep-learning explainable-ai graph-neural-networks Updated Aug 22, 2026 HTML
5. Training Protocol, Convergence Behavior & Experimental Evaluation
Drench yourself in Deep Learning, Reinforcement Learning, Machine Learning, Computer Vision, and NLP by learning from these exciting lectures!!
PaddlePaddle / PaddleSpeech Star 12.7k Easy-to-use Speech Toolkit including Self-Supervised Learning model, SOTA/Streaming ASR with punctuation, Streaming TTS with text frontend, Speaker Verification System, End-to-End Speech Translation and Keyword Spotting. Won NAACL2022 Best Demo Award. tts speech-synthesis transformer voice-recognition speech-recognition whisper asr vocoder conformer sound-classification kws self-supervised-learning code-switch voice-cloning speech-translation punctuation-restoration wav2vec2 streaming-asr speech-alignment streaming-tts Updated Aug 12, 2026 Python
Easy-to-use Speech Toolkit including Self-Supervised Learning model, SOTA/Streaming ASR with punctuation, Streaming TTS with text frontend, Speaker Verification System, End-to-End Speech Translation and Keyword Spotting. Won NAACL2022 Best Demo Award.
speechbrain / speechbrain Star 11.9k A PyTorch-based Speech Toolkit audio deep-learning transformers pytorch voice-recognition speech-recognition speech-to-text language-model speaker-recognition speaker-verification speech-processing audio-processing asr speaker-diarization speechrecognition speech-separation speech-enhancement spoken-language-understanding huggingface speech-toolkit Updated Oct 14, 2026 Python
QuentinFuxa / WhisperLiveKit Star 11.2k Real-time, local speech-to-text with streaming ASR, speaker diarization, translation, and OpenAI/Deepgram-compatible APIs. python streaming real-time translation websocket pytorch speech-recognition automatic-speech-recognition speech-to-text whisper speaker-diarization fastapi Updated Oct 23, 2026 Python
Real-time, local speech-to-text with streaming ASR, speaker diarization, translation, and OpenAI/Deepgram-compatible APIs.
openvinotoolkit / openvino Star 11.1k OpenVINO™ is an open source toolkit for optimizing and deploying AI inference nlp natural-language-processing ai computer-vision deep-learning transformers inference speech-recognition yolo recommendation-system performance-boost good-first-issue openvino diffusion-models stable-diffusion generative-ai llm-inference optimize-ai deploy-ai Updated Oct 29, 2026 C++
espnet / espnet Star 10k End-to-End Speech Processing Toolkit text-to-speech deep-learning chainer end-to-end machine-translation pytorch speech-synthesis speech-recognition kaldi voice-conversion speaker-diarization speech-separation speech-enhancement spoken-language-understanding speech-translation singing-voice-synthesis Updated Oct 29, 2026 Python
QwenAudio / SenseVoice Star 9.6k Open-source SenseVoiceSmall model for Mandarin, Cantonese, English, Japanese, and Korean ASR, language ID, emotion recognition, and audio event detection. multilingual pytorch audio-analysis speech-recognition speech-to-text cantonese transcription asr emotion-detection cross-lingual language-identification speech-emotion-recognition voice-ai llama-cpp funasr sensevoice audio-event-detection multilingual-asr speech-understanding whisper-alternative Updated Sep 30, 2026 C
Open-source SenseVoiceSmall model for Mandarin, Cantonese, English, Japanese, and Korean ASR, language ID, emotion recognition, and audio event detection.
Uberi / speech_recognition Sponsor Star 9k Speech recognition module for Python, supporting several engines and APIs, online and offline. audio python speech-recognition speech-to-text Updated Sep 2, 2026 Python
Speech recognition module for Python, supporting several engines and APIs, online and offline.
Model Figures & Evaluation Curves (4 Images)
Deep Learning Execution Pipeline
(Preprocessing)
(Feature Extraction)
(Self-Attention)
(Loss Optimization)