How to use Asgard and Midgard

Getting in step by step, what you get, where things are, what is yours to protect, and what to do when something breaks.

Status: v1, 2026-08-29 · For: everyone with an account · Read and acknowledged at onboarding · Companion to: server_design_v1.md (the design; this document only tells you how to live in it)

1. Getting in, step by step

You need three things, in this order: an SSH key (yours to make), a four-line SSH config (given to you), and a kineuro account for the browser (made for you). Steps 1 and 2 take ten minutes.

If you have no machine on Asgard, you need only Step 3

Not everyone with a kineuro account computes here. If your work is the portal, the dashboards, the website and GitHub, your account is the browser one and nothing else: go straight to Step 3, and skip Steps 1 and 2 and everything about containers. Your portal start page says which you are; it names your container if you have one, and says you have none if you do not. If you later need your own Linux machine with the GPUs next to the data, ask an administrator and then come back to Step 1.

Step 1: your SSH key

Nobody types a password to reach the servers; your key is the only way in.

1a. Check whether you already have one.

macOS or Linux, in a terminal:

ls ~/.ssh/*.pub

Windows, in PowerShell:

dir $HOME\.ssh\*.pub

If a file such as id_ed25519.pub is listed, you have a key: skip to 1c. If nothing is listed (or the folder does not exist), go to 1b.

1b. Create one, only if 1a found nothing. Same command on all three systems:

ssh-keygen -t ed25519 -C "<your name>@<this computer>"

Accept the default file. Give it a passphrase when asked. This makes two files: id_ed25519 (private; stays on this computer) and id_ed25519.pub (public; the one you hand over).

1c. Register the public key. Print it and paste the one line (it starts with ssh-ed25519); it is not a secret.

If your kineuro account already exists, do it yourself and no administrator is involved: https://gate.kineuro.se > kineuro login > Credentials (left) > Public keys > Add > paste the line > label it with the computer (anna@laptop) > Save. The same page is where you revoke the key of a computer you have lost, which is the only thing that actually stops a login. Otherwise send the line to an administrator.

macOS or Linux:

cat ~/.ssh/id_ed25519.pub

Windows:

type $HOME\.ssh\id_ed25519.pub

Warning: never send, copy or back up the private key

id_ed25519 (no .pub) never leaves the computer it was made on: not by e-mail, not in a chat, not copied to a second laptop. A second computer gets its own key (§3). If a private key was ever exposed, tell an administrator the same day.

Step 2: connect

The administrator answers with your username (<you>) and the container name (<you>-lxc1). Put this block into ~/.ssh/config (create the file if it does not exist; on Windows it is C:\Users\<login>\.ssh\config):

Host lxc1
  HostName ssh.kineuro.se
  User <you>:<you>-lxc1
  ProxyCommand openssl s_client -quiet -connect ssh.kineuro.se:443 -servername ssh.kineuro.se 2>/dev/null
  ServerAliveInterval 30
  ServerAliveCountMax 3

The last two lines matter: without them a connection that goes quiet, such as a VS Code window you leave alone, is dropped by the gateway and VS Code sits in a reconnect loop. If you already have a block without them, add them.

Copy the ProxyCommand line exactly, -quiet included. It is not cosmetic: it makes openssl ignore EOF, and without it a large transfer through the tunnel drops the connection at random, roughly four times in ten. If you see openssl printing Connecting to ... and a list of certificates when you connect, your line is missing it.

Then:

ssh lxc1

That opens a shell in your container from any network that allows HTTPS, hospital included; no VPN. The prompt reads <you>@<you>-lxc1. VS Code: Remote-SSH, choose lxc1. Windows users need openssl on the PATH (it comes with Git for Windows).

What the User line means: <you> is your account on the gateway, <you>-lxc1 is the target it may connect you to. The gateway (Warpgate) checks your key, then opens your container with its own key. You cannot reach anything else, and nobody can reach your container by any other route.

Warning: ten failed logins ban your address for an hour

A wrong key or a typo in the User line counts as a failed login. After ten in ten minutes the gateway blocks your network address for an hour. If ssh lxc1 fails, stop and check the block and the key rather than retrying; an administrator can see the exact reason in the gateway log.

Step 3: the browser

Your kineuro account (username <you>, the password an administrator gave you) logs you into everything under *.kineuro.se: the gateway page https://gate.kineuro.se, Grafana at https://kineuro.se/grafana/, and later NILS and JupyterHub. On the first login you are asked to enrol a second factor (an authenticator app or a hardware key); after that, every login asks for it. Change the password at https://auth.kineuro.se after the first login.

2. What you get

Who you are	What you get
Staff and tech team	a GPU-enabled container `<you>-lxc1` on Asgard: your own Linux machine with `sudo` and Docker inside, and the CPUs, memory and GPUs of your seat (your portal start page shows them); your own `/home/<you>` and `/scratch/<you>`; read and write in `/work`, `/data/results` and `/data/shared`, which all staff share; `/data/bids` read-only; `/data/source` only where a study grant names you
Students	JupyterHub in the browser at `hub.kineuro.se`, with the course datasets; no shell (not built yet)
External collaborators	one capped VM, or one app, as granted, with an expiry date
NILS curators	NILS at `nils.kineuro.se`, and the `nils` group rights on `/data/source`, `/data/bids` and `/data/shared`
Administrators	everything, with a hardware key

A second container. If you need a separately installed environment (another CUDA stack, a service that must keep running), ask for one: it is <you>-lxc2, alias lxc2, with the same home and data. Copy the block from Step 2 and change lxc1 to lxc2 in both places.

3. Two or more computers

One key per computer, never a copied private key. After your first key is in, you manage the others yourself:

On the new computer, do Step 1 (check, then create if needed).
Open https://gate.kineuro.se, log in with your kineuro account, and under Credentials > Public keys paste the new computer’s .pub line.
Put the Step 2 block into that computer’s ~/.ssh/config. ssh lxc1 works from each.

Lost or replaced computer: delete its key on the same page (or tell an administrator the same day); the other computers keep working, nothing else changes.

4. Where things are, and where you can write

Path	On	What it is for	Who can write	Backed up
`/home/<you>/`	Asgard NVMe	your code, environments, configuration	only you	no; 48-hour undo
`/scratch/<you>/`	Asgard NVMe	temporary files	only you	no; files not modified for 30 days are deleted, a week after a notice
`/work/`	Asgard NVMe	work in progress a team iterates on	all staff, inside a folder	no
`/data/results/`	Midgard	results, and anything shared that must be kept	all staff, inside a folder	yes, snapshots on Midgard
`/data/shared/`	Asgard NVMe	staged copies of BIDS releases for fast pipeline reads, and files a colleague’s pipeline should read fast	all staff, inside a folder	no
`/data/bids/<study>_<date>/`	Midgard	dated, immutable BIDS releases; `<study>` points at the current one	nobody; NILS writes them	yes
`/data/source/<study>/`	Midgard	the study’s original and anonymised DICOM (`dcm-original`, `dcm-raw`) and NILS’s records	nobody; read `dcm-raw` if a grant names you	yes

The same paths exist in every container, so a pipeline that runs for one of us runs unchanged for the next.

Private and shared. Your home and your scratch are mounted only into your own containers; nobody else can reach them. /work, /data/results and /data/shared are mounted into everyone’s container and are shared by all staff.

In a shared place, make a folder first. The top level of /work, /data/results and /data/shared is not writable, by design. Make a folder and work inside it:

mkdir /data/results/<a-clear-name>

Everything created inside it belongs to the group staff and is group-writable, so any colleague can read, add and edit it, without asking anyone. Nobody can delete or rename your top-level folder by accident.

Sharing files with a colleague

Choose the place by what the files are:
- they must be kept: /data/results/<folder>
- work in progress you both iterate on: /work/<folder>
- input a pipeline should read fast: /data/shared/<folder>
mkdir the folder and put the files in it.
Send your colleague the path. They can read and edit straight away.

Warning: shared means shared

Anyone on staff can change or delete files inside a shared folder, and the protection on top-level folders stops accidents, not intent. Keep what must not change in git or in /data/results, which has snapshots, and ask before deleting a colleague’s files. Never change or delete a <study>_<date> folder in /data/shared: it is a staged release, and other people’s pipelines are reading it.

Clarification: nobody in ls -l

The top folders show as nobody:staff. That is normal: their owner is the server’s root account, which your container cannot name, so it shows as nobody. The group staff is what lets you write.

Warning: what is not backed up

Your home, your scratch, /work, /data/shared and everything installed inside your container are not backed up. If Asgard’s pool is lost, they are gone. The only safety net is the 48-hour undo on your home: hourly snapshots, kept two days, under /home/<you>/.zfs/snapshot/<time>/. Everything you cannot afford to lose is in git (pushed daily) or in /data/results, which lives on Midgard with its own snapshot history.

GitHub. The group’s code lives in the organization github.com/kineuro (Experimental Neuroradiology Research at Karolinska Institutet). You are invited to it with your GitHub username at onboarding; accept the invitation (e-mail, or github.com/kineuro) and turn on two-factor authentication on your GitHub account, it is required. Members of the staff team can create repositories and push to the group’s. git and gh are installed in every container; once per container:

gh auth login        # GitHub.com, HTTPS, login in the browser
gh auth setup-git    # git uses that login for pushes
git config --global user.name "<your name>"
git config --global user.email "<your e-mail>"

Warning: GitHub holds code only

Nothing from /data, no images, no derivatives, no results, no subject lists, ever. Configuration and code, with secrets kept out of the repository. The old organization NeuroGranberg is a read-only archive.

5. The rules

Your container is yours, and so is everything inside it. The platform does not back it up. If it breaks beyond repair, an administrator rebuilds it from the template in minutes; /home, /work, /scratch and /data are untouched, because they live outside the container.
Code lives in git, pushed daily. See the box above.
Read releases from /data/bids or their staged copy in /data/shared; write results you need to keep to a folder in /data/results. Never write into /data/source or /data/bids; you cannot, and you should not want to. Never change a staged <study>_<date> folder in /data/shared.
Identifying data stays in /data/source/<study>/dcm-original. Never copied anywhere: not to your home, not to results, not to a laptop. If you can see that folder, a study grant says you may look, not that you may take.
/scratch/<you> is temporary. Every Monday morning, files you have not modified for 30 days are listed in SCRATCH-PURGE-NOTICE.txt in your scratch, and deleted at the next Monday’s run unless you move or modify them first. Reading a file does not count. Copies made with rsync -a or cp -p keep their original dates, so data copied in that way can be announced on the first Monday after you copy it. Move anything worth keeping to /data/results or your home.
Your tools go in your home; system packages go in the template. micromamba, uv, pixi, pip --user, the VS Code server: under /home/<you>, they follow you into any container. Anything that needs apt is added to the template on request, so everyone gets it and a rebuilt container keeps it. sudo apt install in your own container works and is lost on a rebuild.
The GPUs are shared, not allocated. Two cards, 96 GB each, visible to everyone. Take what you need while you run, free it when you stop, look at the dashboard before starting something big. Do not leave an idle notebook holding 90 GB overnight.
Nobody types a password to reach the servers, and nobody shares an account. A lost laptop is reported the same day.
Every pipeline run ends with a completion marker (an empty file or a manifest written last). It is how we know a run finished, and the only way a lost tail of writes after an unclean stop of Midgard is ever noticed.
When you leave, your home goes to the archive under the retention policy. Commit, push and hand over before that day.

Data

Read from /data/shared/<study>_<date> and name the release you used in your notes and results. If the release you need is not staged, ask; staging is a copy from /data/bids at several GB/s.
Write results to your folder under /data/results/ directly; it streams at over 2 GB/s. No need to stage on Asgard and copy later.
Intermediate derivatives a team iterates on: a folder under /work. Temporary files: /scratch/<you>.
Never modify a release; NILS makes a new dated one. If you believe identifying data has leaked into dcm-raw, a release or a result, tell an administrator; do not fix it yourself.
Both pools alert at 80 %. Need a lot of space for a while? Say so; quotas are raised for a project, not silently exceeded.

Software

Docker inside your container works out of the box. Keep compose files in git; images and volumes live inside the container unless you put them under /work or /home. If a docker pull stops with lchown ... invalid argument, the image has files owned by very high user numbers. Ask an administrator: widening your container’s ID range is a one-time change and restarts your container.
The NVIDIA driver is shared by every container and upgraded in an announced quarterly window; your environment pins its own CUDA runtime. Never attempt to change the driver.

FreeSurfer (shared, nothing to install)

FreeSurfer 8.2.0 and 7.4.1 are installed once, for everyone, in /data/shared/software/freesurfer, and the lab’s license is already set. Type FreeSurfer commands as usual:

recon-all -s sub01 -i sub-01_T1w.nii.gz -all
freeview            # in your VNC desktop

8.2.0 is the default. To use another version in one terminal:
```
fs-version          # shows the version in this terminal
fs-version 7.4.1    # switches this terminal to 7.4.1
fs-version 8.2.0    # back to the default
```
Other terminals and other people are not affected. To start every terminal on 7.4.1, put fs-version 7.4.1 in your ~/.bashrc.
In scripts and pipelines, name the version with the full path, so a run stays the same when the default changes later: /data/shared/software/freesurfer/7.4.1/bin/recon-all. A script started with bash script.sh does not read your terminal’s settings; the full path works everywhere.
What FreeSurfer sees. It runs inside an Apptainer image, as you. Your home, /scratch, /work, /data and /tmp are there at the same paths, the GPUs too, and what it writes belongs to you. Set SUBJECTS_DIR as usual. Your own tools (conda, micromamba, anything installed in your container) are not visible inside it.
A shell inside FreeSurfer: fs-run 8.2.0 shell (or fs-run 7.4.1 shell), with every FreeSurfer program on its PATH.

Warning: one subject, one version

Never run 7.4.1 and 8.2.0 on the same subject folder. Their outputs differ, and mixing them silently spoils the results. Keep one SUBJECTS_DIR per version, for example /work/<project>/fs741/ and /work/<project>/fs820/.

Sharing the machine

RAM and CPU are ceilings, not reservations. If the host runs short, the container that exceeds its own ceiling is reclaimed, never its neighbours.
Long jobs run under tmux or as a service inside your container; a dropped SSH session must not kill a day of work.
One announced maintenance window a month. Asgard reboots are rare and announced; your container comes back with everything in it.

Never hand a partner a path, a container login or a copy on a disk. Ask an administrator for a Bifrost bridge (bifrost.kineuro.se): the partner gets a link that works in a browser or with a small command line tool, sends or fetches the data with checksums and resume, and the bridge closes on a date. What arrives is quarantined on Midgard until an administrator accepts it into a study; what we give is a staged copy, never the original. Send the token by a channel other than the one you use for the link if the bridge has a passcode.

7. When something breaks

Symptom	What to do
`ssh lxc1` fails	check the block (§1 Step 2) and that the key from §1 is the one on this computer; do not retry more than a few times (ban); then ask an administrator, who sees the gateway log
My container is broken or will not start	ask for a rebuild from the template; it takes minutes; your home, work, scratch and data are untouched; what you installed by hand into the OS is gone
I cannot write in `/work`, `/data/results` or `/data/shared`	make a folder first and write inside it (§4); the top level itself is not writable, by design
`recon-all: command not found`, or FreeSurfer is the wrong version	open a new terminal, or run `fs-version 8.2.0`; in a script, use the full path `/data/shared/software/freesurfer/<version>/bin/<command>` (§6, FreeSurfer)
`freeview` does not open	run it in your VNC desktop; a plain SSH session has no screen
I cannot write in `/scratch`	write in `/scratch/<you>`: that is your scratch, and `/scratch` itself is only the folder it is mounted in. Nobody else can see your scratch, so it cannot be used to share
I deleted or overwrote something in my home	look in `/home/<you>/.zfs/snapshot/`, or ask an administrator within 48 hours
I deleted something in `/work` or `/scratch`	it is gone; regenerate it
I deleted something in `/data/results`	ask an administrator; results have hourly, daily and weekly snapshots on Midgard
Midgard was restarted uncleanly while my pipeline ran	rerun every subject directory that has no completion marker
I think identifying data is somewhere it should not be	tell an administrator immediately; do not copy, move or delete it yourself
Something is slow	say what, where, and when; the dashboards keep the history

Report through the team channel with: your container name, the path, the time, and the exact command or error. Do not report by fixing it yourself on a shared path.

8. Leaving

Commit and push, hand over your folders in /work, /data/results and /data/shared to a colleague, and tell an administrator the date. On that day your logins stop, your containers are destroyed, and your home moves to the archive under the retention policy; nothing else of yours is kept.

9. Acknowledgement

I have read this document and I understand that everything inside my container and under my home, work and scratch is my responsibility, that the platform backs up none of it beyond a 48-hour undo on my home, and that identifying data never leaves /data/source.

Name, date.