rolf-mozilla commited on
Commit
cd00055
·
verified ·
1 Parent(s): 096fa0e

v0.2.5: sbsd5 checkpoint, quantized with reduce_range=True (Bug 2064781)

Browse files

Weights from Metaflow run sbsd5 (autofill-tiny-supportedv2-bb-tinybert-13ep-lr1e4).
NOTE: main previously held the jt4qd checkpoint, so this is a checkpoint change
as well as a quantization fix. id2label is identical (66 labels), as is the
tokenizer and every architecture parameter.

Quantized with --per_channel --reduce_range; see the embed repo commit for the
VPMADDUBSW int16 saturation mechanism.

Weight range: 0/4682058 over +-64 (max |w| = 64). Previous: 308683 over, max 127.

Accuracy on testing-supported.txt (4914 rows), sbsd5 checkpoint:
variant kappa acc bal_acc w_f1
fp32 0.876357 0.887057 0.722377 0.885980
q8 per-channel 0.873763 0.884615 0.704006 0.883869
q8 per-channel+reduce 0.874187 0.885022 0.723430 0.884059

reduce_range is better on every metric; balanced accuracy recovers the full gap
to fp32 (0.704 -> 0.723), i.e. clipping was mostly hurting rare field types.

Mirrors rolf-mozilla/tinybert-address-autofill v0.2.5.
Toolchain: transformers.js 3.8.1 scripts.quantize, onnxruntime 1.20.1, onnx 1.17.0.

Files changed (1) hide show
  1. onnx/model_quantized.onnx +2 -2
onnx/model_quantized.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7db5b0a13d83076c678276343321e01be0345b962458849ddcb2753f2765a27f
3
- size 14529113
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c786543239a644ceb877ecba5a3d05c591425f9098a9118930f9fd819228c895
3
+ size 14601388