The model and Kvasir

The wizard does this for you. This page is what it does, for when you would rather do it yourself.

1. Serve a model

On the machine with the card, serve a model on the loopback with anything that speaks the OpenAI completions API. Whatever you use must answer at a /v1 address, for instance http://127.0.0.1:30000/v1.

Nothing about NILS depends on which server it is. Kvasir can download the model’s files for you: see local models.

2. Kvasir

git clone --depth 1 --branch v1.0.0-alpha.10 https://github.com/kineuro/kvasir
cd kvasir && npm ci && npm run build
cp kvasir.example.json kvasir.json

Edit kvasir.json: where it listens, how callers are known, and where it keeps its state. Keep the purposes the example declares, one for each of the assistant’s stations. The models are not in the file: Kvasir keeps them in its own database, and refuses a kvasir.json that still names backends.

{
  "bind": "127.0.0.1:7100",
  "origin": "http://127.0.0.1:7100",
  "auth": { "mode": "off" },
  "store": "kvasir.sqlite",
  "pepperFile": "kvasir.pepper",
  "sealKeyFile": "kvasir.seal"
}

How callers are known follows the desk. With no login, auth.mode is off, and the caller holds everything. When an identity provider decides, auth.mode is oidc with the same trust entries as the engine’s, the desk’s and the provider’s:

{
  "auth": {
    "mode": "oidc",
    "trust": [
      { "issuer": "https://nils.example.org", "audience": "nils", "jwks": "https://nils.example.org/.well-known/jwks.json", "keepSubject": true },
      { "issuer": "https://auth.example.org/application/o/nils/", "audience": "<client id>", "jwks": "https://auth.example.org/application/o/nils/jwks/" }
    ],
    "groupsClaim": "roles",
    "roles": { "reader": "reader", "reviewer": "reviewer", "operator": "operator", "admin": "admin", "assist": "assist" }
  }
}

When the desk keeps the people, the desk’s entry alone is enough, with the desk’s own origin. The desk’s tokens carry each person’s grants and detail, which Kvasir reads as they are, so the same person is the same person everywhere and reaches Kvasir through the desk with what they may do. nils-desk register prints the whole block.

• keepSubject: true keeps a subject that already names its host, which only the desk’s entry should: every other entry names the caller sub@<issuer host>.
• groupsClaim is the claim the groups of a provider’s own token arrive in, groups unless set, and roles gives each group one grant or one old name, which stands for its set, as groups and grants says.
• A token in auth.tokens names its principal and what it holds, as "principal@node:kvasir:work,assistant:use", and keeps working beside the trust list, which is how the wizard reaches the doors it uses. A token that names nothing is refused.

A caller that holds no grant is refused with 403 no_grant, and so is a caller without a door’s grant, with the grants that door needs. Everything that changes Kvasir needs kvasir:work: models, downloads and local models, keys, where stations go, and admission. So do a backend’s address and the whole ledger; every other caller reads their own rows of it.

node dist/main.js --config kvasir.json

3. Add the model

With Kvasir running:

node dist/main.js models add --config kvasir.json \
  --id card --url http://127.0.0.1:30000/v1 --locality local \
  --model the-model-id --name "The model" --context 32768 --max-tokens 4096 --reasoning

Kvasir first sends each model one short request, and holds the backend only once every model answered. Otherwise it says what each one did: nothing answered, the key was refused, there is no model by that name, or the server refuses for now. models test asks the same and keeps nothing, and with no --model it lists the models the server serves. A provider’s key is read from a file with --key-file, sealed in Kvasir’s database, and never shown again. models list shows what Kvasir holds, and models remove --id card lets go of a backend with its key and the policy rows that sent a purpose to it.

With work on Kvasir, the same is done on the desk’s Language models page.

locality is the promise that matters: local means the prompt does not leave the machine, remote means it does. Purposes are opened to remote backends deliberately, one at a time, rather than by default.

A local model is warmed until its first token, then run through Kvasir’s admission, a short suite that asks the model for tool calls, a schema and a stream. It is listed to the assistant only once it passed, and a station nobody moved goes only to a local model that passed: until one has, such a station is refused, and the refusal names the model still being checked or refused. Kvasir runs the suite when the model is added, and runs it again with:

node dist/main.js admission run --config kvasir.json --backend card

A model that leaves its reasoning inline in its words, between markers such as <think>, Gemma 4’s channel or gpt-oss’s harmony channels, is read apart by Kvasir, so the assistant shows the reasoning as thinking and the answer alone as the answer. --inline-reasoning says how: markers, the default, reads reasoning only where such markers open the output; open also reads it from the very start, for a model whose prompt opens its thinking itself; off leaves the text as the backend sent it.

With a commercial provider as the only backend, a purpose goes nowhere until it is mapped to that backend, and a purpose that reads rows of the archive needs an acknowledgement that the rows leave the site. The desk’s Language models page maps them, under the name of the person who does it; the wizard maps the purposes that read no rows for you.

4. ChatGPT subscriptions

Kvasir has one backend of its own, chatgpt, which streams through a person’s own ChatGPT subscription. A purpose is moved to it on the Language models page. A person who holds both assistant:use and kvasir:see then signs in to their own subscription on the Language models page, with a code and a link, and chooses the model it answers with; seeing whether it is signed in, and signing out, need only that they are that person. It serves only their conversations, and only while they hold both: otherwise, as for a person with none signed in, the station answers with the default model in your systems. A purpose that reads rows goes to ChatGPT only with a written reason, and identifiers never do. A key or a program that calls a subscription’s doors is refused with not_a_person, since a subscription is a person’s.

Where nobody signs in, with auth.mode set to off, the install has one subscription, signed in from the command line:

node dist/main.js subscriptions sign-in --config kvasir.json

It shows a code and a link, and waits until the sign-in is approved. subscriptions list and subscriptions sign-out do what they say.

5. Local models

Kvasir downloads a model’s files from the Hugging Face Hub into a location you choose, and where a runtime is installed beside it, it starts a downloaded GGUF model itself.

node dist/main.js local lookup --config kvasir.json --repo OWNER/NAME --include "*Q4_K_M.gguf"
node dist/main.js local download --config kvasir.json --repo OWNER/NAME --include "*Q4_K_M.gguf"
node dist/main.js local list --config kvasir.json
node dist/main.js local start --config kvasir.json --id N

A lookup lists the files the patterns choose, with their sizes, and keeps nothing. A download goes into <location>/<owner>--<name>/<commit>/, one file at a time, each checked against the sha256 the hub lists, and is refused where the location lacks the room it needs and a gibibyte more. A download stopped with local pause --id N, or by Kvasir closing, carries on from where it was with local resume --id N or when Kvasir starts again. local remove --id N deletes the model’s files.

Starting a model. The runtime is llama.cpp’s server in its router mode. The wizard installs one pinned build, checked against its published sha256, and runs it as a service beside Kvasir with no model loaded, listening only where Kvasir reaches it, with a key only the two of them read. local start --id N has the runtime load the model and follows it until it is loaded or has failed. A loaded model is held as a local model on the backend llama-cpp: Kvasir warms it and runs its admission like any local model, and a station nobody moved goes to it. A model that failed to load keeps the runtime’s exit code and the last lines of its log. local stop --id N unloads it, and a started model is refused removal until it is stopped. One model runs at a time: starting another stops the one that runs. A model that was serving is loaded again once when Kvasir or the runtime starts again; a load the runtime did not survive is left failed, most likely for lack of memory, until it is started again. It gets an 8-bit cache (never a 4-bit one, which spoils tool calls), its own chat template, and a context of 32,768 tokens or the model’s own if that is smaller, or what --context N names. Where the runtime computes on the processor, Kvasir first checks the memory the load needs against what the machine has free: it shortens the context down to 8,192 tokens where that is enough, and otherwise refuses the start and says how much is needed. local list shows the context and slots the runtime settled on.

The wizard names the runtime in kvasir.json:

{
  "local": {
    "runtime": {
      "url": "http://127.0.0.1:7110",
      "keyFile": "/home/you/nils/kvasir/runtime/runtime.key",
      "presets": "/home/you/nils/kvasir/runtime/models.ini",
      "log": "/home/you/nils/kvasir/runtime/runtime.log"
    }
  }
}

and runs it as:

llama-server --models-preset ~/nils/kvasir/runtime/models.ini --no-models-autoload --models-max 1 \
  --api-key-file ~/nils/kvasir/runtime/runtime.key --host 127.0.0.1 --port 7110 --no-webui \
  --log-file ~/nils/kvasir/runtime/runtime.log

Kvasir writes models.ini itself, one section for each model it starts. Without a runtime, as on a server where you run SGLang, a downloaded model shows the command each model server runs it with, such as llama-server -m for a GGUF file or vllm serve for a folder of safetensors, and the server is added as in step 3. A folder of safetensors shows those commands everywhere.

New downloads go to models beside Kvasir’s database until you change the location, with local location --set /srv/models or on the Language models page. A model downloaded before stays where it is. A gated or private model needs a Hugging Face token, set with local token --file FILE, sealed, and never shown.

6. Mint the assistant’s key

The assistant is a principal like any other, and its key names the purposes it may ask for and nothing else:

node dist/main.js keys mint --config kvasir.json \
  --principal nils-assistant@this-host \
  --purposes assistant.ask-help,assistant.concierge,assistant.keyword-tune,assistant.identity-check,assistant.title,assistant.operator \
  --class rows > /srv/nils/assistant/kvasir.key
chmod 600 /srv/nils/assistant/kvasir.key

The secret is printed once, on standard output. Put it in a file the assistant reads, mode 600, and nowhere else. Mint another and the old one keeps working until you revoke it with keys revoke. The wizard mints the key for every purpose the stations use, and mints it again, revoking the old one, when a station is added.

Next: the assistant itself.