Local disks not mounted for `web` service sandboxes
Summary
Apps that define [[services.web.disks]] with provider = "local" in their app.toml do not get the disk mounted in their sandbox. The sandbox is also scheduled as stateful=false despite having a disk definition. This causes apps like uptime-kuma to crash because their data directory doesn't exist.
Affected apps on Garden
- uptime-kuma — crashes with
ENOENT: no such file or directory, mkdir './data/'because/miren/data/local(the disk mount path) doesn't exist in the container - victoriametrics — "crashed" status is a cascade from failing to scrape uptime-kuma; victoriametrics itself runs fine
- socials — also crashed, may be same issue (not investigated)
Observations
-
Disks defined on non-web services work correctly. Tempo's
[[services.tempo.disks]]mounts/miren/data/localas expected — confirmed via/proc/<pid>/mountinfoshowing the ext4 bind mount from/dev/sdb. -
Disks defined on the
webservice do not mount. Tempo'swebservice (Caddy, no disk defined) correctly has no mount. But uptime-kuma'swebservice (disk defined) also has no mount — the disk definition is ignored. -
The sandbox scheduler treats
webservice sandboxes with disks asstateful=false:scheduling stateless sandbox to available node (no runners) -
A fresh
miren deploydoes not fix the issue — the new version's sandboxes also fail to mount the disk. -
miren app runalso does not mount disks (may be expected/separate).
Reproduction
# app.toml — this disk config is ignored at runtime
[[services.web.disks]]
name = "data"
provider = "local"
mount_path = "/miren/data/local"
Deploy any app with the above config; the /miren/data/local path will not exist in the web service sandbox.
Expected behavior
provider = "local" disks on the web service should be mounted the same way they are for other service names. The sandbox should be scheduled as stateful=true when it has disk definitions.
Environment
- Cluster: Garden (miren-garden, us-central1-a, miren-development)
- Disk definitions added in PR #57