Instructions to use nvidia/nemotron-speech-streaming-en-0.6b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- NeMo
How to use nvidia/nemotron-speech-streaming-en-0.6b with NeMo:
import nemo.collections.asr as nemo_asr asr_model = nemo_asr.models.ASRModel.from_pretrained("nvidia/nemotron-speech-streaming-en-0.6b") transcriptions = asr_model.transcribe(["file.wav"]) - Transformers
How to use nvidia/nemotron-speech-streaming-en-0.6b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="nvidia/nemotron-speech-streaming-en-0.6b")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("nvidia/nemotron-speech-streaming-en-0.6b") model = AutoModel.from_pretrained("nvidia/nemotron-speech-streaming-en-0.6b", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Can we expect an ONNX quant?
4 or 8 bit would be awesome and I'm not sure I'm smart enough to do a good job!
Any plans to release ONNX, GGUF or other quant's for this model?
Thanks. 🫶🏼
Also supporting this, it would be super helpful.
Thank you for the feedback, @SuperPauly and @tmssmt . We don’t have quantized versions to share at the moment, but this is great input and definitely something we’re interested in exploring and considering for future updates.
If anyone is interested in:
I exported it as onnx 2 weeks ago https://huggingface.co/altunenes/parakeet-rs/tree/main/nemotron-speech-streaming-en-0.6b
usage:
https://github.com/altunenes/parakeet-rs/blob/master/examples/streaming.rs