Optimizing models using ONNX

[arakesh]

I came to know Hugging Face use optimized onnx models for inference on cpu. I tried to do something like that using keras VGGNet16 pretrained model using keras-onnx package (see this github issue) but couldn’t see any performance benefits. Can I know how exactly Hugging Face is optimizing models under the hood?

[valhalla]

These two blog posts might help

Accelerate your NLP pipelines using Hugging Face Transformers and ONNX Runtime

Faster and smaller quantized NLP with Hugging Face and ONNX Runtime