Matricxon supported models

Architectures

Note

The list of supported models grows over time. New architectures and quantization types are added as the project develops, so a model that is not listed today may be supported later.

Architecture	Examples	Status
llama	Llama 2/3, TinyLlama, Mixtral (MoE), LLaVA (vision)	checked on real weights
mistral3	Ministral	checked on real weights
gemma4, granite, granitemoe, command-r, falcon	Gemma 4 (dense and MoE), IBM Granite, Cohere Command R, Falcon	tiny-model tests only
nemotron_h	NVIDIA Nemotron-H (hybrid Mamba-2)	tiny-model tests only
qwen2, qwen3	Qwen 2.5, Qwen 3	checked on real weights
qwen35	Qwen 3.5 (hybrid: Gated DeltaNet and attention), text and vision	checked on real weights
phi2	Phi-2, moondream2 (vision)	checked on real weights
starcoder2	StarCoder2	checked on real weights
bert, nomic-bert	Embedding models	checked on real weights

Checked on real weights means it ran on a real downloaded model and the output was verified: logits compared with Hugging Face transformers or llama.cpp, or coherent and correct generation.

Tiny-model tests only means the wiring is tested on small synthetic GGUF files, but the numbers are not yet verified on a full-size model.

Any other architecture is rejected with a clear error and never run approximately. Model families plug in through an architecture registry, so more can be added without touching the rest.

Quantization types

• F32, F16, BF16
• Q4_0, Q4_1, Q5_0, Q5_1, Q8_0
• K-quants Q2_K to Q8_K
• I-quants and ternary types: IQ1_S, IQ1_M, IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_S, IQ4_NL, IQ4_XS, TQ1_0, TQ2_0

Not supported: Q8_1 and the raw integer types, which fail with a clear error.

Tokenizers

Byte-level BPE, WordPiece, SentencePiece BPE and Gemma’s rank-based BPE.