Development environment¶
This chapter collects the devenv side of msks: everything a checkout of this repository presets, seeds, and runs for development. The published chapters describe the deployed product; this one describes the developer's tree — the shell, the dev daemon, the client presets, and the dev identity.
The dev daemon¶
A development host (this repository's checkout) runs the daemon
through devenv. msks-dev seeds a per-worktree state dir with
its own port, token, and socket, and execs msksd through
/run/wrappers/bin/msks-caps, the wrapper the host's NixOS
configuration installs; on a host NixOS does not manage the
script finds the setup tool's /usr/local/bin/msks-caps by
itself (DEV_CAPS names another path outright). The tool is
msks-setup-debian-host on a Debian host and
msks-setup-fedora-host on a Fedora host.
The same daemon is the default devenv processes up process:
the manager runs scripts/dev-daemon.sh in the foreground, and
Ctrl-C stops it through the daemon's own graceful path.
Developing on a macOS host¶
The devenv shell carries the same edit→lint→test loop on a Mac
that it carries on Linux: msks-preflight runs every commit gate
(with the suite under the off-Linux coverage floor), the client
talks to any daemon the Mac can reach, and the docs build renders.
The daemon itself is a Linux program — it drives cloud-hypervisor
on /dev/kvm — so the dev daemon, the msksd process, and the
smoke tests run on a Linux host, and the msksd process is
registered on Linux hosts only. On a Mac, devenv processes up
exits at once — No 'processes' option defined, then
No processes defined — and msks-dev refuses with a pointer
back to this chapter; both refusals are the platform split
working as designed, a broken devenv.nix they are not.
Two paths take a Mac to a live daemon.
The Linux dev host, from the Mac¶
The daemon runs under this chapter's dev-daemon workflow on a Linux host (a dedicated dev machine or the same checkout elsewhere), and the Mac's client talks to it through an ssh tunnel: the daemon binds 127.0.0.1 and mints its certificate for that name, so the tunnel keeps verified TLS end to end.
Copy the remote worktree's dev-daemon state trio — port,
bootstrap-token, msks-ca.pem — into the Mac worktree's own
state dir:
mkdir -p .devenv/state/msksd
scp <linux-host>:<remote-tree>/.devenv/state/msksd/{port,bootstrap-token,msks-ca.pem} \
.devenv/state/msksd/
The trio is exactly what The client presets
reads: with the three files in place, every fresh devenv shell on
the Mac exports MSKSC_URL, MSKSC_TOKEN, and MSKSC_CAFILE by
itself, pointing at 127.0.0.1 and the port the port file names.
The tunnel's local side matches that port, so the preset URL
lands on the forward:
A different local port works when 127.0.0.1:8660 is already
spoken for — the Lima path below claims it too — by forwarding
that port instead and re-pointing MSKSC_URL inside the shell:
the state trio's preset wins over a value exported before entry
(the port file gates it), so unset or override the variable
after the shell opens.
msks console reaches a running workspace from the Mac over the
same trio. msks ssh and msks rsync additionally authenticate
with the dev identity that created the workspace — the Linux
tree's .devenv/state/msksc/identity. Copy that file over the
Mac's own minted one, or point MSKSC_IDENTITY_FILE at the same
personal key on both hosts (see The dev
identity).
The product Lima host, from a build in hand¶
msks-setup-macos-host (the installation chapter's Apple Silicon
section) runs msksd inside
its own Lima VM on the Mac and writes the client config — an
end-to-end daemon with no separate Linux host. The tool installs
the PyPI release matching its own version; --wheel installs a
build from this tree instead:
devenv --quiet shell -- uv build # dist/msks-<version>-py3-none-any.whl
devenv --quiet shell -- msks-setup-macos-host --wheel dist/msks-<version>-py3-none-any.whl
The VM runs the built wheel, not the working tree: each iteration rebuilds the wheel and re-runs the tool (every phase is idempotent, and the daemon restarts through its systemd unit). Use this path to verify a change end to end on the Mac itself; the Linux dev host above stays the faster inner loop.
The daemon's config file¶
The dev daemon runs bare: scripts/dev-daemon.sh points
MSKSD_CONFIG_DIR at the devenv root. The daemon's durable
settings therefore live in <repo>/msksd.yaml, generated as
the commented template on first daemon start and untracked, so
local edits stay personal. The environment the dev script
exports (API port, state dir, egress wiring) wins over file
values under the precedence the Configuration
chapter documents.
The client presets¶
The devenv shell presets the client's connection to the
worktree's own dev daemon. Once the dev daemon has served once,
a fresh devenv shell needs no exports: the token and CA come
from the dev-daemon state dir, .devenv/state/msksd/:
msks ls # presets: https://127.0.0.1:8660, the worktree's
# token, and .devenv/state/msksd/msks-ca.pem
# (verified TLS)
The same shell presets MSKSC_CACHE_DIR at the worktree's own
.devenv/state/msksc/cache/, so each checkout's client state
stays its own and one tree's cache entry is never read by
another. A non-empty value exported before entering the shell
survives the preset; an empty one counts as unset.
Targeting a hand-run bare-host daemon instead (no managed
process; msks-dev-ready then msksd from a shell) takes the
explicit pair from its state, with its own CA:
export MSKSC_URL=https://127.0.0.1:8660
export MSKSC_TOKEN=$(cat .devenv/state/msksd/bootstrap-token)
export MSKSC_CAFILE=.devenv/state/msksd/msks-ca.pem
The dev identity¶
A checkout's devenv shell presets MSKSC_IDENTITY_FILE at the
worktree's own dev identity (.devenv/state/msksc/identity),
minted passphrase-less the first time the shell opens, so the
smokes and msks-dev have a key without touching yours. The
preset fills an unset variable: a non-empty
MSKSC_IDENTITY_FILE you exported before entering the shell
survives, and so does the file's identity_file (the
environment preset lands only when you named nothing). Pointing the
dev identity at your own key (a symlink over the minted file)
works, with the identity chapter's rules: the passphrase prompt
appears when the daemon's rows are locked under it and no agent
holds it. See The identity file.
Doctor on the dev host¶
The dev daemon runs bare: scripts/dev-daemon.sh points
MSKSD_CONFIG_DIR at the devenv root. A hand-run msksd doctor
in the shell therefore resolves the same <repo>/msksd.yaml
the daemon serves with. The environment the dev script exports
(API port, state dir, egress wiring) wins over file values
under the precedence the Configuration chapter
documents.
Development¶
Install nix and devenv on any Linux host — a NixOS host is not required. Enter the environment (Python 3.14, uv venv, cloud-hypervisor + ch-remote, qemu, the pytest toolchain, xenon):
All scripted/CI invocations disable dotenv loading:
Run the test suite the way CI runs it (-n auto is never optional —
see AGENTS.md for the coverage story):
Scoped iteration picks only tests whose coverage touches changed lines:
Complexity gate (also runs as a pre-commit hook):
The Python-side pre-commit gates' full offender list in one pass —
all ruff and deferred-import findings, every xenon offender, and the
jscpd report at once, plus, when sources under src/msks/ changed,
every missing coverage line and branch arc for the changed files
after one gated suite run:
Build outputs¶
The release artifacts each build on their own store path, without evaluating or building the others (#537):
devenv build outputs.server # msksd with the full daemon closure
devenv build outputs.client # the msks CLI/TUI with the client closure alone
devenv build outputs.guest-image-debian # the Debian workspace image archive
devenv build outputs.guest-image-nixos # the NixOS workspace image archive
The daemon and client outputs are two dependency closures over the
one wheel (nix/msks-pkg.nix, nix/msks-client-pkg.nix): the
server output runs msksd; the client output runs msks and
msks-term-popup. The same outputs build from the flake with plain
nix — no devenv in the environment — against the same pinned
nixpkgs (flake.lock and devenv.lock pin the same revision, so each
flake attribute is the same store path its devenv build twin
builds):
Wheel installs¶
The msks wheel is pure Python on every platform — nothing
compiles at install time. Egress consent is the one runtime
library need: the daemon's vendored NFQUEUE binding (#542) loads
libnetfilter_queue.so.1 by soname, so a daemon host with
interactive egress carries the runtime package —
libnetfilter-queue1 on apt, libnetfilter_queue on dnf and
pacman, libnetfilter_queue1 on zypper, libnetfilter_queue on
apk. A host without it boots workspaces without egress and
refuses an egress boot by name. msksd doctor names the package
for the detected package manager when the load fails.
VM guest assets¶
The workspace guest is Debian 13 (trixie), straight from Debian's
official genericcloud cloud image (#30, #41): systemd as PID 1, apt,
cloud-init, and Debian's own kernel, initrd, and modules — booted
directly (no BIOS/UEFI) by cloud-hypervisor. The image is pinned by
its dated cloud.debian.org URL and sha512, and the build turns it
into the msks boot contract — vmlinux (Debian's bzImage,
CONFIG_PVH=y), initrd, a pristine ext4 base rootfs, and
guest-manifest.json — with a small overlay of msks systemd units
(vsock console, serial autologin, the /home mount, the cloud-init
dropins that pin NoCloud and keep cloud-init off the guest's
networking). Extraction is fully unprivileged:
qemu-img convert, partition slice, debugfs rdump, mke2fs -d.
Measured boot on bare-metal KVM: the vsock shell prompt at ~2.8s,
the serial login prompt at ~7.5-8.6s (#37 tracks the <5s interactive
goal).
Root writes persist through the per-workspace overlay (#14), and
/home is the workspace's own ext4 volume; apt reaches the
upstream through the workspace's egress NIC — served by the
daemon's consent stack whenever the daemon holds CAP_NET_ADMIN
(#52). The repack runs under
fakeroot so the image is root-owned with sane password-file modes,
and the setuid binaries the workspace user's sudo needs ride the
same fakeroot session as uid-0 inodes (#169).
Stopping a workspace presses the ACPI power button
(vm.power-button): the guest's systemd-logind runs the clean
poweroff that flushes its persistent disks (#14).
Every step of the build runs inside the repo on any Linux host with nix:
The artifacts (plus a guest-manifest.json describing them and the
boot cmdline) land in .devenv/state/guest/ (relocatable with
GUEST_DIR). Boot one interactive VM from them —
— which attaches the guest's serial console to your terminal
(poweroff -f inside the guest or Ctrl-C stops it) and leaves
ch-remote reachable on the printed API socket path.
Boot tests self-provision: when .devenv/state/guest/ holds built
artifacts and
/dev/kvm is usable, the smoke tests find them without any exported
variables (TEST_VMLINUX / TEST_INITRD /
TEST_ROOTFS / TEST_CMDLINE
keep precedence when you do export them). When the
artifacts were never built, or /dev/kvm is missing or not accessible
to your user (add yourself to the kvm group,
users.users.<name>.extraGroups = [ "kvm" ]; on NixOS, then
re-login), the smoke tests skip themselves.
Running the daemon (msksd)¶
devenv --quiet -O dotenv.enable:bool false shell --
MSKSD_STATE_DIR=/tmp/msksd MSKSD_BOOTSTRAP_TOKEN=dev-secret MSKSD_PORT=8660 msksd
- Trust: with no
MSKSD_TLS_CERT/MSKSD_TLS_KEY, msksd generates a self-signed CA + certificate into the state dir on first run and logs the CA fingerprint — pin it on first connect (trust-on-first-use, like SSH). The CA key lives beside the database under the state dir. - First credential:
MSKSD_BOOTSTRAP_TOKENseeds one bearer token, inserted once when absent; it is visible in the process environment to the same user (acceptable for a single-user local daemon — unset it after minting real tokens).--no-tlsserves plain HTTP for development. - Events:
wss://host/api/v1/eventsstreams workspace status transitions. The websocket handshake authenticates with the sameAuthorization: Bearerheader the REST surface uses (#216), so the token never lands in a URL. Uvicorn's access log prints request lines (method, path, query) and not headers, so turning it on (MSKSD_ACCESS_LOG=true) does not persist credentials. A bad token closes the websocket with code 4401. - Rotating the bootstrap token: setting
MSKSD_BOOTSTRAP_TOKENto a new value adds a token; the previous bootstrap credential stays valid until revoked via the API. - Config file: settings also live in a YAML file —
msksd --config /path/to/msksd.yamlreads exactly that file,--config=nonereads env vars only, and a baremsksdresolves$MSKSD_CONFIG_DIR/msksd.yaml(default~/.config/msksd/msksd.yaml), generating a commented template on first run. Env vars override the file; seedocs/config.mdfor the key-by-key reference. - Schema: the SQLite database is created and upgraded by Alembic at
startup (inside the package:
msks/migrations).
The dev daemon (the default processes up, #231)¶
msksd runs FIRST-LEVEL on the dev host — cloud-hypervisor on the
real /dev/kvm, per-VM taps, the egress consent stack in the host
kernel — through the msks-caps capability wrapper the host's
NixOS config installs. devenv processes up runs it attached in
the foreground (Ctrl-C stops it, workspaces get their stop cycle);
msks-dev runs the same script by hand in a terminal. Each
worktree state dir (.devenv/state/msksd, MSKSD_STATE_DIR
relocates) seeds its own bootstrap token, API port, and
port-derived egress subnet, so concurrent worktrees run one daemon
each; a state-dir lock refuses a second daemon on the same catalog.
The client presets below land in every devenv shell. (An older
hand-run flow without KVM — msks-dev-ready — still exists for
API/client-only work:)
devenv --quiet -O dotenv.enable:bool false shell -- msks-dev-ready
export MSKSD_STATE_DIR="$PWD/.devenv/state/msksd" MSKSD_BOOTSTRAP_TOKEN="$(cat .devenv/state/msksd/bootstrap-token)"
msksd & # serves https://127.0.0.1:8660
MSKSC_URL=https://127.0.0.1:8660 MSKSC_CAFILE=$PWD/.devenv/state/msksd/msks-ca.pem \
MSKSC_TOKEN="$(cat .devenv/state/msksd/bootstrap-token)" msks ls
The state lives in .devenv/state/msksd/ (TLS CA, bootstrap token,
sqlite catalog, workspace volumes). The dir honors MSKSD_STATE_DIR
— export it before msks-dev-ready to relocate it, as an absolute
path: the script anchors a relative value below the repo root while
the daemon resolves one against its own CWD, so only an absolute
value moves both to the same place.
Gitignored but NOT disposable-clean — git clean -xfd deletes all
of it, along with the dev daemon's personal msksd.yaml at the
repo root (#262). Egress (and msks ssh, whose forwards ride the egress NIC)
needs CAP_NET_ADMIN (#101): the dev host's wrapper grants it, so
egress workspaces run first-level; on a host without the grant a
workspace created without --no-egress refuses to start — the 503
names MSKSD_EGRESS_ENABLED, and the fix is --no-egress or the
host config in nix/module.nix. Daemon edits restart with Ctrl-C
and re-run (--reload restarts on tree change, #144).
Known quirk worth knowing: cloud-hypervisor v52 rejects writes to
sector 0 on disks without an explicit image_type (a QCOW2
misdetection guard), which breaks any guest writing an ext4
superblock — every disk the daemon creates declares
image_type: Raw.
The image catalog (#40)¶
Images are plural: msksd holds a catalog under
<state_dir>/images/ — every registered archive keyed by content
hash, with the boot files unpacked once per hash (workspace launches
never unpack anything). The whole /api/v1 surface is its own
chapter; these curls are the import and list forms:
curl -sk -H "authorization: Bearer $MSKSC_TOKEN" \
-H 'content-type: application/json' \
-d '{"source": "/nix/store/...-msks-guest/workspace-debian-13.6.tar"}' \
https://127.0.0.1:8660/api/v1/images # import
curl -sk -H "authorization: Bearer $MSKSC_TOKEN" \
https://127.0.0.1:8660/api/v1/images # list (name/version/hash/default)
A workspace create selects an image by reference — "image":
"debian:13.6" (or a bare name for its newest version, or a hash);
with no image and no explicit artifacts the designated default
resolves. The first import becomes the default; MSKSD_DEFAULT_IMAGE
points the daemon at an archive to import on first boot, and the dev
daemon seeds it from its state dir's default-image symlink
(msks-build-guest-archive converges it) — a bare
POST /workspaces works on a fresh daemon with nothing else
imported. Explicit kernel/rootfs fields still win over
the catalog (the shape the tests and dev flows use).
The client CLI (msks ls, msks create, msks start, msks stop, msks rm) (#59, #66)¶
The client also covers the non-interactive half of the operator flow:
devenv --quiet -O dotenv.enable:bool false shell -- msks ls
devenv --quiet -O dotenv.enable:bool false shell -- msks create my-workspace --start
msks ls prints one line per workspace (name, id, status, image
hash, host); --json prints one JSON document for scripting. A
workspace carries two identity fields (#246): the name you choose at
create — the label every command addresses it by — and the id the
daemon mints, immutable and never reused, keying the artifact paths
and the client caches; either reference reaches the same workspace.
msks create
POSTs the same body the API accepts — --image picks a catalog
reference, --cpus/--mem-mib/--root-mib/--home-mib size the VM,
--user-data attaches a first-boot provisioning script (#41, a
cidata seed disk the guest's provisioner runs once), and explicit
--kernel/--rootfs (with optional --initrd,
--cmdline) bypass the catalog. --start boots the workspace right
after creating it, so msks create ws --start then msks console ws
is the two-step path from nothing to a shell; msks start <ws> boots
an existing workspace later, msks stop <ws> powers one off (a
graceful, deadline-bounded shutdown; the data survives), msks rm
<ws>… deletes one or more workspaces together with their persistent
root overlay and /home volume, msks console <ws> boots one itself
when the daemon reports it as not running (a notice prints on
stderr while the boot runs), and msks home export/import <ws>
moves the whole /home volume through the daemon for backup,
migration, and seeding (#80). All commands use the same
MSKSC_URL/MSKSC_TOKEN/MSKSC_CAFILE environment as msks console;
failures (unreachable daemon, timed-out request, bad token, API or
validation errors) print one readable line instead of a traceback.
See docs/cli.md for the full command and environment reference.
Workspace egress networking (#52)¶
Workspaces are networked from creation — msks create ws, or a bare
"id" on the API, boots with egress; msks create ws --no-egress
(or "egress": false) boots NIC-less — and the whole path lives in
the daemon's own net stack:
workspace VM ──virtio-net──► per-VM tap ──► per-VM nftables chain
│ guest → uplink: accept
▼
NAT (masquerade) → host uplink
The daemon is the guest's only DHCP server and resolver: each
egress workspace gets a dedicated /30 from MSKSD_EGRESS_SUBNET, a
DHCP offer naming the tap as gateway and resolver, a small DNS
forwarder on that resolver, and NAT out the host's uplink
(MSKSD_EGRESS_UPLINK). Everything derives deterministically from
the workspace id, so stop/start cycles rebuild the same network;
stop and delete tear the tap, chain, and services down again. The
guest side is just the image's DHCP client (systemd-networkd +
resolved in the overlay); a workspace without egress presents no
NIC, on every backend.
Egress arms while MSKSD_EGRESS_ENABLED=true and the daemon holds
CAP_NET_ADMIN — the dev host's wrapper grants the capability, and
nix/module.nix does the same for a deployment host. A daemon that
cannot arm the plumbing still serves everything else, and an egress
workspace refuses to boot with the cause named (boot those with
--no-egress). Per-flow consent (allow/deny holds on each new
connection) is #69.
See docs/networking.md for the full reference.
The docs site (#591)¶
The documentation builds into a static site (the same build the
Deploy Docs workflow publishes to GitHub Pages on every push to
main and every release tag). Preview it locally — the preview
serves whatever the checkout holds (the current branch's docs,
so edits review against the tree that will ship), and while it
runs, every edit to the chapters or the site config re-renders
the open tab in realtime:
DOCS_SERVE_PORT relocates the port. The chapters live under
docs/; the TUI chapter's screen renderings regenerate with
RENDER_DOCS_SCREENS=1 pytest -q
src/msks/tests/test_render_docs_screens.py.
The workspace LLM proxy (#259)¶
Each workspace's tap carries its own proxy service when the daemon
is configured for it: an OpenAI-shaped proxy (/v1/models,
/v1/chat/completions, streaming included) served by msksd at
MSKSD_LLM_PORT, admitted by that workspace's input chain from
that tap only, and authenticated by a per-workspace credential the
first-boot seed plants (/etc/msks/llm.token plus the
MSKSWS_BASE_URL/MSKSWS_API_KEY exports). Provider credentials
live host-side — MSKSD_LLM_MODELS entries with file:/cmd:
secret indirection, a single * entry for single-upstream
passthrough, any other list through the litellm router. See
docs/llm.md.
Developing msks inside a workspace (#77)¶
The dev-workspace bootstrap seed turns a pristine Debian workspace into an msks development environment over the workspace's own egress NIC. One create with the seed and egress is the whole of the setup — the image catalog serves the pristine Debian base unchanged:
msks create dev --egress --user-data scripts/dev-workspace.sh \
--mem-mib 8192 --root-mib 20480 # the in-guest suite's budget
msks start dev # first boot provisions; later boots resume
Cloud-init runs the seed once per overlay lifetime: it installs
uv (which fetches its own Python 3.14), clones the repo, and runs
uv sync — all into the persistent root overlay, so stop/start
cycles keep it and a factory reset re-provisions from the same
seed. Every step checks before doing, so re-running the script is
a no-op. Progress is guest-observable in
/root/.msks-bootstrap/state (the running step name, then done),
so msks console into a booting workspace shows where setup stands.
The suite runs inside the guest the way the unit-tests task runs
it — the task's exec line, from the venv uv built:
devenv and nix remain an optional developer comfort inside the guest, off the seed's critical path: building them there exercises upstream toolchains for tens of minutes and gigabytes and tests nothing msks owns.
The seed's downloads (PyPI, uv's Python builds, the git remote)
all ride the egress NIC the daemon serves. The end-to-end proof is
the opt-in root smoke test_local_dev_workspace_bootstrap
(TEST_EGRESS=1). A baked dev image — the same substrate the
guest build uses — remains an optional cold-start accelerator on
top of the seed, not a prerequisite.
The workspace's whole /home also moves through the daemon (#80):
msks home export dev streams the volume to the client's machine
(backup, or migration to another daemon), and msks home import
restores or seeds one from an exported image —
— while day-to-day code in and out rides the workspace's own egress
(git remotes, substitutes) and the forward seam (msks ssh, and
msks rsync dev -- -av ./src/ :src/ for file copies, with no
setup beyond the client environment).
Outbound, the push carries its own credentials: logging in through
the forward with -A delivers the operator's ssh agent into the
workspace, so a git push from inside authenticates to any remote
over the egress NIC with nothing stored in the image or the seed
(the loop and its proof, test_local_egress_git_out, are in
docs/networking.md).
See docs/storage.md for the byte-stream endpoints and their
contract.
The workspace console (msks console) (#21)¶
An interactive shell in a running workspace, from any host that can reach the daemon:
A devenv shell presets MSKSC_URL/MSKSC_TOKEN/MSKSC_CAFILE
from the worktree's dev-daemon state; an outside shell exports the
same trio by hand (the URL from the daemon's log line, the token
and CA from .devenv/state/msksd/).
The client speaks the daemon's console websocket
(/api/v1/workspaces/{id}/console): TLS plus bearer token — the same
authentication as the REST surface, with the token on the query
string (like /api/v1/events). Ctrl-] detaches (Ctrl-C and Ctrl-D
reach the guest); pressing Ctrl-] twice quickly sends one literal
Ctrl-] to the guest instead (see docs/cli.md). The session ends
cleanly when either side closes.
Detaching leaves the workspace running; the shell process inside the
guest exits when the stream closes.
Transport (#21), in the preferred vsock-first shape:
- The workspace VM boots with a virtio-vsock device whose host side
is a unix socket cloud-hypervisor listens on
(
<state>/vms/<id>/vsock.sock). The daemon's proxy connects, sendsCONNECT 1023\n, reads theOK <port>\nreply, then pumps raw bytes both ways — no framing, backpressure is websocket/TCP flow control. - The guest loads
vmw_vsock_virtio_transport(systemd-modules-load) and runsmsks-console.service: the msks console helper (/usr/bin/msks-console-helper, a static binary the image builds fromsrc/console-helper), restarted by systemd if it dies. The helper owns the vsock listener, accepts host-originated connections only, and each connection negotiates the identity prelude (#63): the daemon sends the requested user, the client terminal's size and TERM, and the helper answersMSKS OK <user>(or a named refusal) before exec'ing that user's login shell on a fresh pty. The pty is a plain canonical terminal — ISIG, ONLCR, ECHO, and ICANON all on: Ctrl-C generates SIGINT in the guest, output arrives CRLF-terminated, and the line discipline echoes and edits input for programs that read stdin directly. The shell's TERM comes from the client's terminal, so readline engages and provides line editing and history while it is active (#61); a TERM=dumb client gets readline off. The session's user is root by default;msks console --user <name>requests the image's workspace user (users the image does not serve are refused by name). - The guest pty is created at the client terminal's size (#61's 0x0 fixed): the console request carries the geometry at connect, and the pty keeps that size for the session's life. The console stream is a raw byte pipe by design (#108's terminal-layer decision), so a window resized mid-session does not reach it: reconnect for a new size, or use an ssh session through the forward, where ssh's window-change channel resizes the pty live (#108–#112). If console resize ever returns to the queue, it starts with a control-channel decision — a dedicated vsock control port, or another design that keeps the stream raw (#78).
MSKSC_CAFILEpins the daemon certificate for verification when you have it (the dev daemon writesmsks-ca.peminto its state dir on first serve). Without it the client proceeds with certificate verification off and says so on stderr — the CA fingerprint the daemon logs at startup is the cross-check.
The VZ virtiofs hardlink landmine (#669)¶
The macOS host-share design (#667) passes host folders to
workspace guests over virtiofs twice, and the first hop — Apple's
VZ virtiofs server — carried a known instability through macOS 14:
creating a hardlink sometimes renumbered the original file's
synthetic inode, and the Linux guest's dentry cache went stale
(link()/stat() returning ENOENT for files that exist). Docker
Desktop answered with a private fs/fuse patch in its VM; the
patch never mainlined. The decision for msks: the msksd VM keeps
the stock Debian kernel — the instability is no longer
reproducible on macOS 15 per the upstream report and the suite
runs clean on macOS 27, and the unstable releases sit below the
macOS floor the feature needs anyway (Lima refuses nested
virtualization, which workspaces require, on older macOS); no
stock-kernel mount knob exists that approximates the patch; and
UTM shares the same VZ server on its Apple backend while its QEMU
backend's only POSIX-filesystem transport is 9p (directory
sharing there is SPICE WebDAV or 9p) — the transport every msks
Lima template keeps out (mountTypesUnsupported: [9p], a
deliberate msks choice Lima's default does not make; the suite
ships its own template). The evidence, the re-run procedure, and
the escape hatch (the Docker patch carried in a custom kernel,
should a future macOS regress) live in the suite's README under
scripts/vz-hardlink-repro/; re-run the suite on new macOS
releases before shipping rw shares.
The two-hop virtiofs landmine (#670)¶
The second hop of the host-share design (#667) — the msksd VM
exporting a VZ virtiofs mount into workspace VMs through virtiofsd —
is territory no shipping product covers (NFS-serving a virtiofs
mount is broken upstream). The scripts/hop2-virtiofsd-suite/
rig builds the real stack — a nested cloud-hypervisor guest whose
virtiofs share is the outer VM's rw virtiofs mount — and runs the
seven workloads the issue names. All pass on stock Debian
(virtiofsd 1.13.2, cloud-hypervisor v54, macOS 27): listings are
byte-identical across the hop, link/unlink churn is clean, inodes
from hop 1 pass through verbatim, a 40k-file tree leaves virtiofsd
holding 6 fds, a mid-write kill of the daemon and VMM leaves every
landed file complete, and cross-writer file visibility stays in
the milliseconds once a responder is live (the first response after
publishing one runs ~1.5s while the guest consumes its command). Two findings shape the design: cloud-hypervisor's
userspace vhost-user master is what makes hop 2 possible on the
stock kernel (Debian's cloud kernel builds no /dev/vhost-vfs, so
a QEMU-shaped hop 2 would need a custom kernel), and ownership
collapses to root across both hops (macOS uid 501 → 0 → 0), which
the #667 single-writer contract already covers. The suite's README
carries the evidence table, the re-run procedure, and the
marker-file protocol for cross-VM signaling on a share.