Multilingual DistilWhisper

Language experts for whisper-small

About

Multilingual Distilwhisper allows for better ASR performance in target languages by adding lightweight CLSR modules on top of whisper-small. This project was developed at NAVER LABS Europe. The model is shared under the MIT License.

Link to resources

  • Our collection of models: Hugging Face
  • The code: GitHub

Citing us

When using any of these resources, please cite the publication below:

@INPROCEEDINGS{10447520,
  author={Ferraz, Thomas Palmeira and Boito, Marcely Zanon and Brun, Caroline and Nikoulina, Vassilina},
  booktitle={ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
  title={Multilingual Distilwhisper: Efficient Distillation of Multi-Task Speech Models Via Language-Specific Experts}, 
  year={2024},
  volume={},
  number={},
  pages={10716-10720},
  keywords={Adaptation models;Costs;Training data;Signal processing;Logic gates;Multitasking;Robustness;knowledge distillation;multitask speech processing;automatic speech recognition;multilingual speech processing;language experts},
  doi={10.1109/ICASSP48485.2024.10447520}
}