Matricxon configuration

Set environment variables, or put them in a .env file in the project folder. Real environment variables win. All of them are listed in app/config.py.

Variable	Default	Meaning
MATRICXON_HOST, MATRICXON_PORT	0.0.0.0, 8420	Where the server listens
MATRICXON_MODELS_DIR	./data/models	Where models are stored
MATRICXON_MAX_LOADED_MODELS	2	Models kept in RAM at once
MATRICXON_DEFAULT_KEEP_ALIVE_SECONDS	300	Idle time before a model unloads
MATRICXON_MEMORY_SAFETY_MARGIN	1.2	Free RAM needed beyond a model’s size (1.1 to 1.8)
MATRICXON_ENABLE_QUANTIZED_NATIVE_COMPUTE	true	Multiply packed weights directly
MATRICXON_GEMV_BACKEND	native	native (C kernels) or numba
MATRICXON_TORCH_THREADS	all cores	CPU threads. Fewer runs cooler
MATRICXON_DEVICE	cpu	cpu, or cuda / cuda:N (experimental GPU mode)
MATRICXON_GPU_WEIGHT_MODE	dequantized	On a GPU: dequantized (fast, more VRAM) or packed (less VRAM, slower)
MATRICXON_ENABLE_FUSED_OPS	true	One native call for RMSNorm, RoPE, decode attention and sampling
MATRICXON_MAX_DECODE_BATCH	8	Replies that run at once per model
MATRICXON_PROMPT_CACHE_SLOTS	4	Conversations cached per model
MATRICXON_PROMPT_CACHE_BUDGET_MB	2048	Memory those caches may use together
MATRICXON_PERSIST_PROMPT_CACHE	true	Keep chat caches on disk (first-run default, runtime changes are saved in prompt_cache_policy.json)
MATRICXON_PERSIST_CACHE_BUDGET_MB	4096	Disk space for saved caches
MATRICXON_PERSIST_CACHE_TTL_HOURS	168	Hours before a saved cache expires
MATRICXON_PROMPT_CACHE_DIR	./data/prompt_cache	Where saved caches and the generated prompt_cache.key live
MATRICXON_CACHE_KEY	generated	Encryption key. Set it to supply your own instead of the generated file
MATRICXON_PROMPT_CACHE_CONTEXT_SHIFT	true	Keep a chat’s cache when its oldest messages are trimmed
MATRICXON_ENABLE_MIXED_PRECISION_LOADING	false	Mixed float32/bf16 layers (Mistral 3 only). Measured slower, so it stays off
MATRICXON_ADMIN_SECRET	unset	Signs admin routes. Unset leaves them open. See API
MATRICXON_LOG_LEVEL	0	0 warnings, 1 per-request steps, 2 per-layer trace

For more detail, see Prompt cache and Kernels and GPU. The cluster variables are on the Cluster page.