Matricxon tools and vision

What a model can do

GGUF files have no standard field that lists a model’s abilities, so Matricxon works them out. GET /api/tags and POST /api/show report them as capabilities.

Capability	When it is reported
`completion`	Any supported text architecture
`embedding`	Encoder models such as `bert` and `nomic-bert`
`tools`	The model’s chat template uses a tool-call format Matricxon can read
`vision`	A projector file is installed next to the model, and the architecture can use it
`chat_format_unverified`	No confirmed chat template, so replies may be poor
`architecture_features_unsupported`	This file uses a feature of its family that Matricxon does not run yet

pAIring uses these to show badges and warnings in its model list.

Tool calling

Send a tools list to POST /api/chat. When the model decides to call a tool, the reply comes back with a structured message.tool_calls entry, each with an id. Run the tool, then send its result as a message with role: "tool".

While the reply streams, normal text passes through at once. Anything that starts a tool call is held back, so its markup never shows in the visible message. When the reply ends, the held text becomes tool_calls.

The format is picked from the model’s own chat template:

Family	How the model writes a call
Qwen 3, Qwen 2.5, Hermes	`<tool_call>{"name": …, "arguments": {…}}</tool_call>`
Qwen 3.5	The same tags, with an XML body: `<function=NAME><parameter=KEY>…`
Mistral	`[TOOL_CALLS]name[ARGS]{…}`, or a JSON list after `[TOOL_CALLS]`
Llama 3.x	The whole reply is one JSON object with `name` and `parameters`
Gemma 4	`<\|tool_call>call:NAME{key:value}<tool_call\|>`

If the text does not parse as a call, it is shown as a normal reply. Parameters that the tool declares as string stay strings, even if they look like numbers.

Vision

A vision model comes as two files: the text model and a projector (an mmproj file holding the image encoder). Pull both. They pair by convention: a projector in the same folder as the text model belongs to it. The order does not matter, so a text model becomes vision-capable as soon as its projector arrives.

Architecture	Models	How images are fed in
`llama`	LLaVA	CLIP encoder, one base-resolution view per image
`phi2`	moondream2	SigLIP-style encoder
`qwen35`	Qwen 3.5	Its own vision encoder

Images go in the message’s images field as base64, the same as Ollama. Each image is resized and normalized with the values stored in the projector file.

A message with images sent to a model with no paired projector still gets a reply, but the model does not see the picture. In pAIring, a default vision model answers any message with an image, so this rarely happens there.