pAIring deployment layouts

pAIring does two different jobs:

1. The app. The web UI, users, chats, documents, settings and the database. This is light work.
2. The inference engine. Ollama or Matricxon. It loads the models and writes the replies. This is the heavy work: it needs lots of RAM and a fast CPU or GPU.

The big choice is simple: do both jobs on one node, or move the heavy job to other nodes? If you move it, there are two ways to do it, and they are very different.

The short version

	1. Single node	2. Matricxon cluster (BETA)	3. Separate inference servers
Where the heavy work runs	On the same node as pAIring	Several Matricxon nodes working as one group	Several separate nodes, each on its own
Models	On that node	Shared: one models folder for all nodes	Not shared: install them on every server
Chat cache	In that node’s memory	Shared: saved to a shared folder, so any node can continue a chat	Not shared: stays on the server that handled the chat
Load balancing	Not needed	Real balancing: a chat can move to any free node	Basic: requests are spread, but chats tend to stick to one server
If a node fails	Everything stops	The other nodes keep serving	The other servers keep serving
Needs	One node	A shared network folder, a few Matricxon nodes	A few nodes
Engine	Ollama or Matricxon	Matricxon only	Ollama or Matricxon
Setup	Easiest	Most work	Medium

You can start with a single node and move to the other options later. The mode is a setting in the UI.

Option 1: Single node

Everything runs on one node.

  Browser  ───►  ┌────────────────────────────────┐
                 │  One node                      │
                 │  pAIring + engine + models     │
                 └────────────────────────────────┘

How it works. pAIring installs, starts and stops the engine for you. The engine listens on the same node, and the models are stored there. In Settings → External servers the mode is Local.

Options

• Engine: Ollama or Matricxon. You can switch under Settings → External servers.
• Images: the image engine (stable-diffusion.cpp) can also run on the same node.
• Database: SQLite works out of the box.

Limits to plan for

• Memory is the ceiling. Every model you offer must fit in this node’s RAM or VRAM. On a 32 GB computer, offer two or three models that fit together.
• Users share one node. More people means slower replies for everyone.
• One failure stops everything.

Good fit: home use, a small team, a lab or a demo.

Option 2: Matricxon cluster (BETA)

Several Matricxon nodes work as one group and share their models and their chat memory.

                          ┌─► Node A ─┐
  Browsers ─► pAIring ────┼─► Node B ─┼──►  Shared folders
                          └─► Node C ─┘     (models + chat caches)

How it works. You turn on cluster mode on pAIring’s Cluster page and add nodes. The nodes use:

• one shared models folder, so every node can load every model,
• one shared folder for saved chat caches,
• a built-in agreement protocol (Paxos), so they all agree on who is in the cluster, even when a node goes down.

Some nodes can act only as voters and take no chats. The Cluster page shows a live map of every node.

Why this balances load better. A chat has a cache: the processed form of its conversation so far. With the cache, the engine continues right away. Without it, the engine must read the whole conversation again, which is slow for long chats.

In a cluster, the cache of a chat can be saved to the shared folder. If the node that had the chat is busy, another node picks up the cache and continues, with no re-reading. So pAIring can send each message to the least busy node, and users do not pay for it with slow replies.

Limits to plan for

• Needs shared storage. All nodes must reach the same network folders, such as NFS or Ceph. Models load over the network, so the first load of a model is limited by network speed.
• BETA. It is designed for hundreds of nodes, but only small clusters have been tested so far.
• Matricxon only. Ollama does not support cluster mode.
• Changing quickly. Cluster mode is under active development, so details and options may change between versions.

See the Matricxon overview.

Good fit: a team or organization that needs real, even load balancing and steady speed with many users.

Option 3: Separate inference servers

pAIring runs as the app, and the heavy work moves to engine servers that do not know about each other.

                            ┌─► Engine server A (own models, own cache)
  Browsers ─► pAIring app ──┼─► Engine server B (own models, own cache)
                            └─► Engine server C (own models, own cache)

How it works. In Settings → External servers, set the mode to Remote and add the address of each engine server (up to 120). pAIring then:

• checks that each server is healthy,
• picks a server for every request,
• skips a server that fails for about 30 seconds and sends the request to another one,
• sends a follow-up message back to the server that already holds that chat’s cache in memory (Matricxon, with more than one server).

This works with Ollama and with Matricxon. The pAIring node no longer needs a big GPU or lots of RAM.

What is not shared

• Models. Each server has only its own models. pAIring does not track which models each server has, so install the same models on every server you add.
• Chat cache. A chat’s cache lives in the memory of the server that handled it. Nothing moves it to another server.

What that means for load balancing. pAIring sends a follow-up message to the server that holds the chat’s cache, because that is fast. But if that server is busy, there are two bad choices: wait for it, or use another server, which must read the whole conversation again. So the load is spread only roughly evenly, and busy chats can pile up on one server. It works well when chats are short or when you mainly want failover. For truly even balancing, use a cluster (option 2).

Good fit: adding capacity cheaply, mixed hardware, or Ollama setups.

Other ways to add capacity

Cloud GPUs through a connector. A connector adds an engine that runs on a cloud GPU service. Today there is one, for RunPod. This is optional, and pAIring runs fine fully private without it.

Scaling the app itself

The app is light, but you can run more than one copy:

• Several copies on one node. pAIring has a built-in balancer, so you do not need a reverse proxy. Set it under Settings → System → Proxy mode.
• Copies on several pAIring nodes. Put a reverse proxy such as nginx or Caddy in front, and use MySQL so all copies share one database.

Which one should I choose?

• Trying it out, or one person or a few friends: option 1, single node.
• Many users and you need steady speed: option 2, a Matricxon cluster. Keep in mind it is BETA and needs shared storage.
• A few extra servers for capacity and failover, or you use Ollama: option 3, separate servers.
• You want burst capacity without buying hardware: add a cloud connector.

Models and memory

In every layout, a model must fit in the memory of the node that runs it. Splitting work across servers adds capacity. It does not let one node run a model larger than its own memory.