Jina Code Embeddings 1.5B — calibrated MXFP4 for MLX

This is an experimental, native MLX MXFP4 conversion of jinaai/jina-code-embeddings-1.5b, created by RESMP.DEV from upstream revision 39aeb4fb9b60f930934c78ae5d749a46287c248a.

It uses activation-calibrated block-GPTQ within each native 32-value MXFP4 group. Values are E2M1, scales are unsigned E8M0, and the resulting weights load directly into MLX QuantizedLinear layers. This is not a repackaged third-party MLX quantization.

What the results support

On a complete held-out 949-pair code retrieval set, calibrated MXFP4 recovered most of the quality lost by native round-to-nearest MXFP4 while reducing artifact size by 62.2% and peak MLX memory by 42.4% relative to BF16.

Variant	Artifact	Peak MLX	MRR	Recall@5	Recall@10
Canonical BF16	3.09 GB	3.56 GB	0.58466	0.66807	0.73235
Native RTN MXFP4	1.17 GB	2.05 GB	0.56345	0.64700	0.70390
Calibrated block-GPTQ MXFP4	1.17 GB	2.05 GB	0.57833	0.66175	0.71549

The calibrated model does not beat BF16 overall. It does beat both controls on the small held-out TypeScript slice:

Language	Samples	BF16 MRR	RTN MRR	Calibrated MRR
Python	434	0.68911	0.68454	0.68843
Rust	249	0.72659	0.69627	0.71486
Metal	124	0.19079	0.16678	0.18486
C++	110	0.47359	0.43903	0.45829
TypeScript	22	0.60911	0.60811	0.63047

The TypeScript result is interesting but has only 22 held-out examples. In a paired analysis, calibrated MXFP4 improved four queries, regressed two, and tied BF16 on sixteen. A paired bootstrap 95% interval for the MRR delta was [-0.00576, +0.06939] (76.0% of bootstrap draws were positive), so the apparent win is not statistically conclusive. Treat it as a result to reproduce on larger external TypeScript corpora, not a universal superiority claim.

Calibration

32,768 calibration tokens sampled from a language/repository-balanced code corpus
exact and token 8-gram near-duplicate removal before split assignment
exact 32×32 activation Hessian within every hardware MXFP4 group
GPTQ error propagation within each group
calibrated E8M0 scale search with native RTN retained as a fallback candidate
196 matrices calibrated on an NVIDIA RTX A6000 in 206.72 seconds
aggregate activation-weighted objective improvement versus RTN: 42.47%

Torch/CUDA and MLX produced byte-identical packed weights and scale bytes on a matched test. Canonical Torch BF16 and direct MLX BF16 embeddings had cosine similarity 0.999150.

MLX usage

The maintained loader, calibration pipeline, and OpenAI-compatible server live in RESMP-DEV/calibrated-code-embeddings. Model repositories contain weights, configuration, and evaluation receipts only.

git clone https://github.com/RESMP-DEV/calibrated-code-embeddings.git
cd calibrated-code-embeddings
uv sync --extra mlx
CODE_EMBEDDING_MODEL_PATH="RESMP-DEV/jina-code-embeddings-1.5b-mxfp4-block-gptq" \
CODE_EMBEDDING_MODEL_NAME="jina-code-embeddings-1.5b-block-gptq-mxfp4-32k" \
uv run code-embedding-serve --host 127.0.0.1 --port 1235

Use Jina's task prefixes. The bundled server accepts input_type: "query" or input_type: "passage" and applies them exactly:

curl http://127.0.0.1:1235/v1/embeddings \
  -H 'content-type: application/json' \
  -d '{
    "model": "jina-code-embeddings-1.5b-block-gptq-mxfp4-32k",
    "input": "How is an HTTP router implemented?",
    "input_type": "query"
  }'

Embeddings are 1536-dimensional, last-token pooled, and L2-normalized.

Reproducibility receipts

Model safetensors SHA-256: 9cadd568e6757e15358d960e4688bb0792a7edcd9889ef3d37f2361f4db6b756
Calibration JSONL SHA-256: 3bc24ff6718944f15be98a93785fa04de388fe6fe45e3101ebe6e40d0abd1d6f
Test JSONL SHA-256: 8ca65a8b8fad6b166294bf3ba3e811dfef10bca9b3083de003a49f23d7d0ac27
Calibration token SHA-256: 1594ce4891fc875a15b5c0126b429ceeab9bd602c3a0d2c736d50ff69c765315

Machine-readable BF16, RTN, and calibrated benchmark receipts are in benchmarks/.

License and attribution

The model weights are a derivative of Jina AI's checkpoint and remain licensed under CC-BY-NC-4.0. They are for non-commercial use under those terms. See WEIGHTS_LICENSE.md and the upstream model card. The separately maintained GitHub tooling is MIT-licensed by RESMP.DEV.