Legendary Arena Lab

Provision Droplet

wiki

Provision Droplet

Migration in progress. api.legendary-arena.com and its PostgreSQL are moving off Render onto a self-hosted DigitalOcean Ubuntu droplet fronted by Cloudflare, per docs/PLAN.md . Until decommission, Render stays warm as the rollback target.

Summary

Provision Droplet creates the DigitalOcean Ubuntu 24.04 droplet and brings it to a known first-boot baseline: the non-root operator user, base packages, and the /etc/la anchor for the deploy .env. It is the first concrete step of Phase 1, owned by infra/cloud-init.yaml , and runs exactly once at droplet creation. Size, region, hostname, and SSH-key-only auth are chosen at creation time; DNS for api. stays on Render and is untouched until cutover (docs/PLAN.md ).

Mechanics

Creation-time parameters

Chosen in the DigitalOcean UI or doctl at create time, from the locked decisions (docs/PLAN.md §1 ) and the pre-Phase-1 checklist (docs/PLAN.md §5.1 A ):

Image      Ubuntu 24.04 LTS x64
Region     sfo3  (nearest to current users; DigitalOcean has no Oregon region)
Size       4 vCPU / 8 GB  (start high for migration safety, ~$48/mo; measure and step down to 4 GB ~$24 if idle)
Hostname   api-prod-01  (example)
Auth       SSH keys only — never enable password auth in the DO UI
Tags       project membership + migration tags (e.g. la, api)
User-data  infra/cloud-init.yaml (operator key substituted)
DNS        do NOT point api. yet — cutover is a later phase
Backups    attach DO automated backups only if you will operate them now; otherwise prefer snapshots

The OS image is a plain Ubuntu 24.04 droplet — Cloudflare does not supply the image; it only fronts api. later (Cloudflare TLS ). See Droplet Snapshots for the snapshot-vs-backup tradeoff behind that last line.

SSH keys: the DO picker vs the cloud-init key

Two independent key paths land on the box, and mixing them up is the classic lockout trap:

  • Keys selected in the DigitalOcean SSH-key picker are injected into the image’s default user (root, and the distro default user preserved by the - default entry in the cloud-init users block).
  • The cloud-init users block separately creates operator with its own ssh_authorized_keys, filled from the REPLACE_WITH_OPERATOR_PUBLIC_KEY placeholder.

Injecting the same public key in both places means a problem in the operator block still leaves you a way in via root, and vice-versa. See Edge Cases .

First-boot: what cloud-init does

infra/cloud-init.yaml is a #cloud-config user-data document. Unlike the numbered infra/scripts/*.ps1 provisioners, cloud-init is not idempotent — it is a one-shot bootstrap that fires once on first boot and does not re-run on demand (docs/PLAN.md ).

  1. User creation. Creates operator in the sudo group, shell /bin/bash, ALL=(ALL) NOPASSWD:ALL, with a single key placeholder that must be filled before boot — cloud-init runs once, so an unreplaced placeholder means no operator login.
  2. Package refresh. package_update: true and package_upgrade: true refresh and upgrade apt.
  3. Base packages. Installs curl, git, jq, ufw, fail2ban, and unattended-upgrades — pre-installing the last three so the later hardening scripts only configure them (docs/PLAN.md ).
  4. Locale and timezone. Sets the system locale to en_US.UTF-8 and the timezone to UTC — both set explicitly, per docs/PLAN.md §5.1 B .
  5. runcmd. Runs mkdir -p /etc/la && chown operator:operator /etc/la (the /etc/la anchor where the deploy .env later lives at /etc/la/.env, mode 600), then writes Bootstrap complete to /var/log/la-bootstrap.log as a completion marker.

Interactions

Edge Cases

  • Unreplaced key placeholder locks you out. If REPLACE_WITH_OPERATOR_PUBLIC_KEY is not swapped for a real public key before boot, the operator account has no usable key and — with password auth later disabled by SSH Hardening — there is no way in via operator.
  • DO-picker key vs cloud-init key mismatch. If the DigitalOcean UI key and the operator key differ, you can end up able to reach root but not operator (or the reverse). Inject the same public key in both places.
  • Password/root login stay open until the next scripts run. After this step, PasswordAuthentication and root login are still possible — that is expected. SSH Hardening (05-user-and-ssh.ps1) and UFW Firewall (06-ufw.ps1) close those paths in the very next steps; do not panic that the fresh box is not yet locked down.
  • Cloud-init is one-shot. It only re-runs on a droplet rebuild, not on demand. Fix drift with the numbered scripts, not by “re-provisioning.”
  • /etc/la is the secrets anchor. Later phases place the deploy .env at /etc/la/.env (mode 600). If this directory or its operator ownership is missing, downstream secret placement fails.
  • First-boot upgrade window. package_upgrade: true runs a full upgrade at first boot; the droplet may be busy for several minutes or briefly reboot before it is ready for the hardening scripts (docs/PLAN.md §5.1 B ).

Execute

Cloud-init is supplied as user-data at droplet creation time, not launched by a script. This is the run-list from the pre-Phase-1 checklist (docs/PLAN.md §5.1 A–C ); the operator runbook has the full Control-Panel walkthrough.

  1. Generate or locate the operator Ed25519 public key.
  2. Replace REPLACE_WITH_OPERATOR_PUBLIC_KEY in infra/cloud-init.yaml with that key.
  3. Create the droplet — Control Panel, or doctl (illustrative):
doctl compute droplet create api-prod-01 \
  --image ubuntu-24-04-x64 \
  --region sfo3 \
  --size s-4vcpu-8gb \
  --ssh-keys <operator-key-fingerprint> \
  --user-data-file infra/cloud-init.yaml \
  --tag-names la,api
  1. Wait until the droplet is Active and cloud-init status --long reports done.
  2. Record the public IP, but do not change any DNS for api. — it stays on Render until cutover.
  3. SSH in as operator, run the Verify block (sudo, sizing, packages, /etc/la, bootstrap log).
  4. Take a DigitalOcean snapshot named pre-hardening-baseline (Droplet Snapshots ).
  5. Proceed to SSH Hardening — keep two SSH sessions open through the hardening steps (docs/PLAN.md §5.1 D ).

Verify

Run after first login as operator:

# cloud-init finished cleanly
cloud-init status --long          # expected: status: done

# OS + sizing match the plan
lsb_release -a                    # expected: Ubuntu 24.04 LTS
nproc; free -h; df -h /           # expected: 4 vCPU / ~8 GB / disk per plan

# operator account + sudo
id operator                       # expected: groups include sudo
sudo -v                           # expected: succeeds without password

# base packages present
dpkg -l curl git jq ufw fail2ban unattended-upgrades

# locale + timezone (both set by cloud-init)
localectl status                  # expected: System Locale: LANG=en_US.UTF-8
timedatectl                       # expected: Time zone: UTC

# anchor dir + bootstrap marker
ls -ld /etc/la                    # expected: drwxr-xr-x ... operator operator ... /etc/la
cat /var/log/la-bootstrap.log     # expected: Bootstrap complete

References