Ship it

In this tutorial you take a finished site from a folder to a production host running on your own machine: you create the keys, start the host, build a signed and encrypted Capsule, activate it, see your page served by domain, ship a change and roll it back.

You need the crisptastic executable from the Quickstart, and the standard age and openssl tools. This runs on Linux, as a normal user, on the loopback address only, so nothing is exposed and nothing needs root. Every command below was run as written, and the output shown is what came back.

1. A finished site

Use your own site if you have one. These three files will do otherwise.

mkdir -p launch/site/assets launch/keys launch/capsules && cd launch

From here on you stay in launch/, and every command runs from there. The other folders hold what must stay out of the site folder: everything in that folder gets packaged.

[deployment]
domains = ["launch.example.com"]

This is the one setting deploy requires. It is also the only thing that decides which domains the site answers for.

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Launch</title>
  <link rel="stylesheet" href="/style.css">
</head>
<body>
<h1>Version one</h1>
<p>Served for {{= request.headers["x-forwarded-host"] }}.</p>
</body>
</html>

$ink: #16201e;

body {
  color: $ink;
  font-family: system-ui, sans-serif;
}

The host compiles your pages when a site is activated, so catch mistakes before you ship:

crisptastic check site

ok: 1 routes, 0 partials

2. Three keys

Three keys keep three jobs apart.

Create the first two:

age-keygen -o keys/host.agekey
crisptastic capsule keygen --private keys/release.pk8 --public keys/release.pub

Public key: age1n984kz8h75858jgh5dtskzq0g2kwfhggf2gfv2pxwxy5xcvy0vssy6h086
private keys/release.pk8
public keys/release.pub
signer bd448b6776d0e53e361d045b00dd78f3

The line beginning Public key is the host's public half, called its recipient. You can print it again at any time with age-keygen -y keys/host.agekey.

The third is mutual TLS on the host's management port. Crisptastic does not generate certificates, so use your own PKI or openssl. You need a CA, a server certificate whose name matches the address you will connect to, and a client certificate:

openssl req -x509 -newkey ed25519 -nodes -keyout keys/ca.key -out keys/ca.crt -subj '/CN=management CA' -days 365 2> /dev/null
openssl req -newkey ed25519 -nodes -keyout keys/host.key -out keys/host.csr -subj '/CN=host' 2> /dev/null
openssl x509 -req -in keys/host.csr -CA keys/ca.crt -CAkey keys/ca.key -CAcreateserial -out keys/host.crt -days 365 -extfile <(printf 'subjectAltName=IP:127.0.0.1\nextendedKeyUsage=serverAuth\n') 2> /dev/null
openssl req -newkey ed25519 -nodes -keyout keys/deployer.key -out keys/deployer.csr -subj '/CN=deployer' 2> /dev/null
openssl x509 -req -in keys/deployer.csr -CA keys/ca.crt -CAkey keys/ca.key -CAcreateserial -out keys/deployer.crt -days 365 -extfile <(printf 'extendedKeyUsage=clientAuth\n') 2> /dev/null
ls keys

ca.crt
ca.key
ca.srl
deployer.crt
deployer.csr
deployer.key
host.agekey
host.crt
host.csr
host.key
release.pk8
release.pub

3. Start the host

Open a second terminal, go to launch/, and start the host:

crisptastic host --identity "$PWD/keys/host.agekey" --trusted-signer "$PWD/keys/release.pub" --host-root "$PWD/host-root" --store-dir "$PWD/store" --bind 127.0.0.1:8080 --proxy-mode x-forwarded --trusted-cidr 127.0.0.1/32 --trusted-hops 1 --management-bind 127.0.0.1:8443 --management-cert "$PWD/keys/host.crt" --management-key "$PWD/keys/host.key" --management-client-ca "$PWD/keys/ca.crt" --locked-secret-memory-bytes 65536

management mTLS listening on 127.0.0.1:8443
hiccup token not set (local/dev); GET /health?Hiccup-Offer still returns declaration when domains are live
serving 0 domains from 0 local capsule(s) + 0 store digest(s) on 127.0.0.1:8080 (proxy mode XForwarded)

It is running and serving nothing, which is correct. There are no shortcuts in this command: it is the production process profile, so before opening a socket the host checked that it is not root, took a lock on its root folder and locked memory for secrets.

Back in the first terminal:

curl -s http://127.0.0.1:8080/health; echo
curl -si http://127.0.0.1:8080/ | grep -vE '^(x-|referrer|permissions|date|content)'; echo

{"status":"live"}
HTTP/1.1 400 Bad Request

Invalid trusted forwarding metadata

The health check answers, and an ordinary request is refused. A production host serves a site only when a trusted reverse proxy says the visitor came over HTTPS, and it says which domain they asked for. You have no proxy here, but a proxy is only something that adds headers, so curl can play the part:

proxied() { curl -si -H "X-Forwarded-Host: $1" -H 'X-Forwarded-For: 203.0.113.7' -H 'X-Forwarded-Proto: https' "http://127.0.0.1:8080$2"; }
proxied launch.example.com / | grep -E '^HTTP|isdirected'; echo
curl -si http://127.0.0.1:8080/__crisptastic/ready/launch.example.com | grep -E '^HTTP|ready'; echo

HTTP/1.1 421 Misdirected Request
Misdirected request

HTTP/1.1 503 Service Unavailable
not ready

421 means that no site claims this domain, and the host never falls back to another site. The readiness check agrees. Time to give it a site.

4. Build and publish the Capsule

crisptastic deploy ./site --recipient "$(age-keygen -y keys/host.agekey)" --signing-key keys/release.pk8 --out capsules/v1.age --publish-store-dir store

built encrypted deployment capsules/v1.age (3 files, 0 secure variables, 948 bytes)
sha256 c71407828764a692f9f25a2992c834450e1c9741eed29ab6a54dbbd8eb0c2255
signer bd448b6776d0e53e361d045b00dd78f3
domain launch.example.com
published capsule digest c71407828764a692f9f25a2992c834450e1c9741eed29ab6a54dbbd8eb0c2255
key capsules/sha256/c7/c71407828764a692f9f25a2992c834450e1c9741eed29ab6a54dbbd8eb0c2255.age

deploy packaged the folder, signed it, encrypted it to the host's recipient and copied the result into the store. The sha256 line is the Capsule's digest, and the digest is how every later step refers to this release. Keep it in a variable:

V1=$(sha256sum capsules/v1.age | cut -d' ' -f1)
echo "$V1"

c71407828764a692f9f25a2992c834450e1c9741eed29ab6a54dbbd8eb0c2255

Here the store is a folder that you and the host can both see. It can also be an S3 bucket; see Publish. Publishing changes nothing the visitor can see, and the host has not noticed yet.

5. Activate it

crisptastic notify --peer 127.0.0.1:8443 --digest "$V1" --client-cert keys/deployer.crt --client-key keys/deployer.key --server-ca keys/ca.crt

HTTP/1.1 200 OK
{"ok":true,"capsule_digest":"c71407828764a692f9f25a2992c834450e1c9741eed29ab6a54dbbd8eb0c2255","deployment_id":"568b7d7e-0e60-455a-a66b-5f7572ad9143","domains":["launch.example.com"],"state":"active"}

The host fetched the Capsule by digest, decrypted it, checked the signature against its trusted signers, compiled the pages, and only then switched the domain over.

6. See it served by domain

proxied launch.example.com / | grep -E '^HTTP|<h1>|<p>'
proxied launch.example.com /style.css | grep -E '^HTTP|^content-type|color'
curl -si http://127.0.0.1:8080/__crisptastic/ready/launch.example.com | grep -E '^HTTP|ready'; echo
proxied other.example.com / | grep -E '^HTTP'

HTTP/1.1 200 OK
<h1>Version one</h1>
<p>Served for launch.example.com.</p>
HTTP/1.1 200 OK
content-type: text/css; charset=utf-8
  color: #16201e;
HTTP/1.1 200 OK
ready

HTTP/1.1 421 Misdirected Request

Your page is live for launch.example.com and no other name. The Sass file was compiled and served at /style.css, because the Capsule carries assets/ too.

The proxied function is standing in for a reverse proxy that owns public TLS and sends those three headers. The configuration to copy is in Running a host.

7. Ship a change

Change the page.

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Launch</title>
  <link rel="stylesheet" href="/style.css">
</head>
<body>
<h1>Version two</h1>
<p>Served for {{= request.headers["x-forwarded-host"] }}.</p>
</body>
</html>

The whole release is the same three steps: check, deploy, notify.

crisptastic check site
crisptastic deploy ./site --recipient "$(age-keygen -y keys/host.agekey)" --signing-key keys/release.pk8 --out capsules/v2.age --publish-store-dir store
V2=$(sha256sum capsules/v2.age | cut -d' ' -f1)
crisptastic notify --peer 127.0.0.1:8443 --digest "$V2" --client-cert keys/deployer.crt --client-key keys/deployer.key --server-ca keys/ca.crt
proxied launch.example.com / | grep '<h1>'

ok: 1 routes, 0 partials
built encrypted deployment capsules/v2.age (3 files, 0 secure variables, 966 bytes)
sha256 cea5f508deb52a6043b4fe23eb0f0f19a634f4efdd89f18a81fba12edfad4690
signer bd448b6776d0e53e361d045b00dd78f3
domain launch.example.com
published capsule digest cea5f508deb52a6043b4fe23eb0f0f19a634f4efdd89f18a81fba12edfad4690
key capsules/sha256/ce/cea5f508deb52a6043b4fe23eb0f0f19a634f4efdd89f18a81fba12edfad4690.age
HTTP/1.1 200 OK
{"ok":true,"capsule_digest":"cea5f508deb52a6043b4fe23eb0f0f19a634f4efdd89f18a81fba12edfad4690","deployment_id":"37ecd9f7-b53c-4503-b3a0-c321db64590c","domains":["launch.example.com"],"state":"active","replaced":1}
<h1>Version two</h1>

"replaced":1 says one running version was swapped out. The switch is atomic and happens only after the new version has compiled.

8. Roll back

There is no rollback command, because there is no need for one. An earlier release is still in the store, so activate it:

crisptastic notify --peer 127.0.0.1:8443 --digest "$V1" --client-cert keys/deployer.crt --client-key keys/deployer.key --server-ca keys/ca.crt
proxied launch.example.com / | grep '<h1>'

HTTP/1.1 200 OK
{"ok":true,"capsule_digest":"c71407828764a692f9f25a2992c834450e1c9741eed29ab6a54dbbd8eb0c2255","deployment_id":"6d582271-73fd-4470-afb9-ead2859d27b1","domains":["launch.example.com"],"state":"active","replaced":1}
<h1>Version one</h1>

9. See a bad release refused

Sign a release with a key the host has never heard of:

crisptastic capsule keygen --private keys/stranger.pk8 --public keys/stranger.pub
crisptastic deploy ./site --recipient "$(age-keygen -y keys/host.agekey)" --signing-key keys/stranger.pk8 --out capsules/stranger.age --publish-store-dir store
crisptastic notify --peer 127.0.0.1:8443 --digest "$(sha256sum capsules/stranger.age | cut -d' ' -f1)" --client-cert keys/deployer.crt --client-key keys/deployer.key --server-ca keys/ca.crt
proxied launch.example.com / | grep '<h1>'

private keys/stranger.pk8
public keys/stranger.pub
signer 2b3b4e05a337a460add998e12a1c30f2
built encrypted deployment capsules/stranger.age (3 files, 0 secure variables, 982 bytes)
sha256 4ea4502d66a87618176a895f7e695c18f0e3e8d8e0ea6a1da28fc74e137e026d
signer 2b3b4e05a337a460add998e12a1c30f2
domain launch.example.com
published capsule digest 4ea4502d66a87618176a895f7e695c18f0e3e8d8e0ea6a1da28fc74e137e026d
key capsules/sha256/4e/4ea4502d66a87618176a895f7e695c18f0e3e8d8e0ea6a1da28fc74e137e026d.age
HTTP/1.1 400 Bad Request
{"ok":false,"capsule_digest":"4ea4502d66a87618176a895f7e695c18f0e3e8d8e0ea6a1da28fc74e137e026d","deployment_id":"","domains":[],"state":"failed","error_code":"capsule.signer_unauthorized","error":"Capsule signer 2b3b4e05a337a460add998e12a1c30f2 is not authorized"}
error: notify peer rejected activation: HTTP/1.1 400 Bad Request
<h1>Version one</h1>

The Capsule decrypted, because it was encrypted to the right recipient, and was then rejected, because decryption and authorisation are separate. The site that was serving kept serving. A failure at any step, such as a page that does not compile (deployment.runtime_build_failed), leaves the running version alone in the same way.

The management port insists on its certificate as well:

curl -sS --cacert keys/ca.crt https://127.0.0.1:8443/status 2>&1 | cut -c1-90
curl -s --cacert keys/ca.crt --cert keys/deployer.crt --key keys/deployer.key https://127.0.0.1:8443/status; echo

curl: (56) OpenSSL SSL_read: OpenSSL/3.0.13: error:0A00045C:SSL routines::tlsv13 alert cer
{"schema":1,"mode":"standalone","ready":true,"published_domain_count":1}

The wording of the first line depends on your curl. What matters is that the connection was refused without a client certificate.

10. Restart

A restart loses the routing table on purpose, along with the secrets held in memory. In the second terminal, stop the host (Ctrl+C) and run the same command again:

crisptastic host --identity "$PWD/keys/host.agekey" --trusted-signer "$PWD/keys/release.pub" --host-root "$PWD/host-root" --store-dir "$PWD/store" --bind 127.0.0.1:8080 --proxy-mode x-forwarded --trusted-cidr 127.0.0.1/32 --trusted-hops 1 --management-bind 127.0.0.1:8443 --management-cert "$PWD/keys/host.crt" --management-key "$PWD/keys/host.key" --management-client-ca "$PWD/keys/ca.crt" --locked-secret-memory-bytes 65536

proxied launch.example.com / | grep -E '^HTTP'
crisptastic notify --peer 127.0.0.1:8443 --digest "$V2" --client-cert keys/deployer.crt --client-key keys/deployer.key --server-ca keys/ca.crt
proxied launch.example.com / | grep -E '^HTTP|<h1>'

HTTP/1.1 421 Misdirected Request
HTTP/1.1 200 OK
{"ok":true,"capsule_digest":"cea5f508deb52a6043b4fe23eb0f0f19a634f4efdd89f18a81fba12edfad4690","deployment_id":"6d542c86-c8a8-4a66-8dab-969bb90a80c8","domains":["launch.example.com"],"state":"active"}
HTTP/1.1 200 OK
<h1>Version two</h1>

On a server you would rather not wait for someone to run notify, so add --digest <sha256> to the host command for each site it should activate at start-up. Everything that matters outlives the restart: the store, and under host-root/, the sessions and any embedded stores.

11. Stop

Stop the host (Ctrl+C). There is nothing else to tidy up: everything you made is in launch/. The keys in it are throwaways, so do not reuse them.

What changes on a real server

• A reverse proxy in front, owning TLS for your domains and sending the forwarding headers. Its address goes in --trusted-cidr. See Reverse proxy requirements.
• DNS pointing each domain at the proxy.
• A service manager to run the host, as a dedicated user, with LimitMEMLOCK=64M. See The production process profile.
• Separate machines. The host holds host.agekey, release.pub and the management server certificate. You or your CI hold release.pk8 and the deployer certificate. Add every host that might run the site as a --recipient, up to 32.
• The management port stays private. Reach it through something like an SSH tunnel.
• Back up the Capsule store, the host identity and any embedded stores. See Backup and recovery.

What next

• Secrets, S3 stores and the deployment checklist: Deploying a site.
• Every host option and fixed limit: Running a host.
• Check that your forms survive the production rules: Keep a form honest.