A guestbook with its own store

In this tutorial you build a guestbook that saves entries in an embedded store of its own, lists them newest first, keeps them across restarts and redeploys, and runs on a host as well as on your laptop.

You need the crisptastic executable from the Quickstart, and the standard openssl and age tools for the keys. No database server and no Rust are involved: the store is declared in crisptastic.toml and used from pages. Every file below was served by the real engine while this tutorial was written.

1. Create the folder

mkdir guestbook && cd guestbook
mkdir _partials

Everything below is a file inside guestbook/.

2. Declare the store

crisptastic.toml

[data]
provider = "residiuum-embedded-development"
application_id = "guestbook"
records = ["entries", "counters"]
memory_budget = "128M"

[routes."/sign"]
methods = ["POST"]
body = "form"
max_body_bytes = 2048
capabilities = ["data"]
rate = [{ per = "client", limit = 20, window = "1m" }]

provider has one possible value. Despite the name, it is what a single host uses too.
application_id names the store.
records lists the collections of records you want. Each name gets five operations on the data capability, such as data.entries_list and data.entries_create.
memory_budget is what this store may use. A server can inherit a limit from its service unit, but a laptop has none to inherit, so say it here.

The [routes."/sign"] table describes a page you will write in step 4. The route must exist when the site loads, so write the pages before you start the server. Keep a form honest goes through these settings one at a time.

3. The list page

_partials/_layout-open.html.crisp

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Guestbook</title>
</head>
<body>
<main>

_partials/_layout-close.html.crisp

</main>
</body>
</html>

index.html.crisp

@call page = data.entries_list({ limit: 5, after: request.query.after })
@include("_partials/_layout-open.html.crisp")
<h1>Guestbook</h1>
<form method="post" action="/sign">
  <input type="hidden" name="_csrf" value="{{ context.csrf.token }}">
  <p><label>Name <input name="name" required></label></p>
  <p><label>Message <textarea name="message" required></textarea></label></p>
  <button>Sign</button>
</form>
@if (page.ok) {
  @for (entry in page.value.items) {
    <article><h2>{{ entry.document.name }}</h2><p>{{ entry.document.message }}</p></article>
  }
  @if (page.value.next) {
    <p><a href="/?after={{ page.value.next }}">Older entries</a></p>
  }
} @else {
  @status page.status
  <p>The guestbook cannot be read right now ({{ page.status }}).</p>
}
@include("_partials/_layout-close.html.crisp")

@call binds an envelope, the same idea as @fetch: a failure is a value, so check .ok. A record comes back as { id, document, version, created, updated }, and what you stored is under .document. list answers { items, next }, where next is a cursor for the following page.

4. The page that writes

list returns records in id order, and a page cannot sort: there is no sort function, SDA does not sort, and a page has no clock. So choose ids that sort the way you want. For newest first, keep a count in the counters collection and count down: 9000000 - n gives 8999999, 8999998 and so on, all the same width.

sign.html.crisp

@let name = trim(request.body.name)
@let message = trim(request.body.message)
@if (!empty(name) and !empty(message)) {
  @call counter = data.counters_get({ id: "entries" })
  @let n = num(counter.value.document.n) + 1
  @if (counter.ok) {
    @call bumped = data.counters_replace({ id: "entries", version: counter.value.version, document: { n: n } })
  } @else {
    @call bumped = data.counters_create({ id: "entries", document: { n: n } })
  }
  @if (bumped.ok) {
    @call saved = data.entries_create({ id: str(9000000 - n), document: { name: name, message: message } })
    @if (saved.ok) {
      @redirect "/"
    }
  }
  @status 409
  @include("_partials/_layout-open.html.crisp")
  <h1>Not signed</h1>
  <p>Someone else was signing at the same moment. <a href="/">Try again</a></p>
  @include("_partials/_layout-close.html.crisp")
} @else {
  @status 400
  @include("_partials/_layout-open.html.crisp")
  <h1>Not signed</h1>
  <p>A guestbook entry needs a name and a message. <a href="/">Try again</a></p>
  @include("_partials/_layout-close.html.crisp")
}

Reading it from the top:

counter.value.document.n is a missing path until the counter exists. A missing path is null and num(null) is 0, so the first n is 1.
@call binds its name for the rest of the page, even from inside a block, which is how bumped can be read after the @if.
create takes an id and a document and succeeds only if the id is free.
replace succeeds only at the version the page read. If another visitor moved the counter first, this one gets a failed result instead of overwriting them, and sees the message at the bottom.
@redirect stops the page and sends a 303, so a refresh does not sign twice.

5. Start it, once with a key

A store is sealed. It is created once, with a 32-byte unseal key, and after that it opens without one. Try starting without it, in another terminal inside guestbook/:

crisptastic dev --bind 127.0.0.1:8080

error: failed to open embedded Residiuum: an unseal key is required to create a new Residiuum store

Make a key, outside the site folder because everything in that folder gets deployed:

openssl rand -hex 32 > ../guestbook.key

dev does not take the key as a value. You name a file descriptor, and the shell opens the file on it:

CRISPTASTIC_RESIDIUUM_UNSEAL_KEY_FD=3 crisptastic dev --bind 127.0.0.1:8080 3< ../guestbook.key

The store now exists under .crisptastic/residiuum/guestbook/.

6. Sign it

In the first terminal, sign the book twice and read it back. Use the browser at http://127.0.0.1:8080/ if you prefer.

curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' -d 'name=Ada&message=First!' http://127.0.0.1:8080/sign
curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' -d 'name=Grace&message=<b>Second</b>' http://127.0.0.1:8080/sign
curl -s http://127.0.0.1:8080/ | grep article

303 http://127.0.0.1:8080/
303 http://127.0.0.1:8080/
    <article><h2>Grace</h2><p>&lt;b&gt;Second&lt;/b&gt;</p></article>
    <article><h2>Ada</h2><p>First!</p></article>

The newest is on top, and the message is escaped on the way out, so nobody can put markup into your page. Now check the other answers. An empty form, and a GET, which the [routes."/sign"] table refuses before the page runs:

curl -s -o /dev/null -w '%{http_code}\n' -d 'name=&message=' http://127.0.0.1:8080/sign
curl -si http://127.0.0.1:8080/sign | grep -E '^HTTP|^allow'

400
HTTP/1.1 405 Method Not Allowed
allow: POST

7. See it survive

Stop the server (Ctrl+C) and start it again, this time with no key:

crisptastic dev --bind 127.0.0.1:8080

curl -s http://127.0.0.1:8080/ | grep -c article

The key is only for creating the store. You will want it once more in step 10, for the host's own store, so keep it somewhere safe.

8. More than a page

The list page asks for five at a time. Sign six more and look:

for who in Hedy Katherine Margaret Radia Barbara Frances; do curl -s -o /dev/null -d "name=$who&message=Hello" http://127.0.0.1:8080/sign; done
curl -s http://127.0.0.1:8080/ | grep -E 'article|Older' | cut -c1-60

    <article><h2>Frances</h2><p>Hello</p></article>
    <article><h2>Barbara</h2><p>Hello</p></article>
    <article><h2>Radia</h2><p>Hello</p></article>
    <article><h2>Margaret</h2><p>Hello</p></article>
    <article><h2>Katherine</h2><p>Hello</p></article>
    <p><a href="/?after=525343312e9f902f02a94f2d8f2b6313c611

Follow the link for the next five:

curl -s "http://127.0.0.1:8080/$(curl -s http://127.0.0.1:8080/ | grep -o '?after=[0-9a-f]*')" | grep -E 'article|Older' | cut -c1-60

    <article><h2>Hedy</h2><p>Hello</p></article>
    <article><h2>Grace</h2><p>&lt;b&gt;Second&lt;/b&gt;</p><
    <article><h2>Ada</h2><p>First!</p></article>

A bad cursor is one of the failures the list page checks for, and @status page.status passes on the code that came back:

curl -si 'http://127.0.0.1:8080/?after=nonsense' | grep -E '^HTTP|cannot be read'

HTTP/1.1 400 Bad Request
  <p>The guestbook cannot be read right now (400).</p>

Now have six people sign at the same moment:

seq 1 6 | xargs -P 6 -I{} curl -s -o /dev/null -w '%{http_code}\n' -d 'name=Burst{}&message=Hello' http://127.0.0.1:8080/sign | sort | uniq -c

      1 303
      5 409

Your split will differ. Every 303 is an entry, and every 409 is someone who lost the race for the counter and was asked to try again, so no entry was overwritten. That is the price of the counter: one shared record that every signature has to change. It is no trouble for a guestbook. For something busier, use random ids from secure_token(8) and do without the order.

9. Check it

crisptastic check .
crisptastic routes .

ok: 2 routes, 2 partials
/
/sign

10. On a host

A host keeps its own copy of the store, beneath its own folder, and hands it to each new version of your site. It creates the store the first time it activates the site, so the Capsule has to carry a key for that one occasion. Stop the server (Ctrl+C) and change the configuration:

crisptastic.toml

[deployment]
domains = ["guestbook.example.com"]

[env_from_process]
GUESTBOOK_UNSEAL_KEY = "GUESTBOOK_UNSEAL_KEY"

[data]
provider = "residiuum-embedded-development"
application_id = "guestbook"
records = ["entries", "counters"]
memory_budget = "128M"
unseal_secret = "GUESTBOOK_UNSEAL_KEY"

[routes."/sign"]
methods = ["POST"]
body = "form"
max_body_bytes = 2048
capabilities = ["data"]
rate = [{ per = "client", limit = 20, window = "1m" }]

[env_from_process] reads the variable from your shell when you run deploy, and seals its value inside the encrypted Capsule. unseal_secret says which secure variable holds the key. A side effect: dev also insists on a variable named in [env_from_process], so from now on export GUESTBOOK_UNSEAL_KEY before you run dev here.

Make the host's key and your signing key, and build the Capsule:

age-keygen -o ../host.agekey
crisptastic capsule keygen --private ../release.pk8 --public ../release.pub
GUESTBOOK_UNSEAL_KEY=$(cat ../guestbook.key) crisptastic deploy . --recipient "$(age-keygen -y ../host.agekey)" --signing-key ../release.pk8 --out ../guestbook.age

Public key: age13xrr2x7l5mg4dcksj2ng037rwz0f5294ztud5398v5yklp895yrskauh0m
private ../release.pk8
public ../release.pub
signer 71987214540235c24b0346cc5446c266
built encrypted deployment ../guestbook.age (5 files, 1 secure variables, 3352 bytes)
sha256 fc7dd6df49c0f8c7307f9726ab408d5636950705ecbc3db4c553c646077ea4be
signer 71987214540235c24b0346cc5446c266
domain guestbook.example.com
secure variable GUESTBOOK_UNSEAL_KEY

secure variable lists the name and never the value. Start a local trial host in the other terminal:

crisptastic host ../guestbook.age --identity ../host.agekey --trusted-signer ../release.pub --development-process-profile --bind 127.0.0.1:8080 --locked-secret-memory-bytes 65536 --deployments-root ../trial-host/deployments

The small --locked-secret-memory-bytes is because the default of 64 MiB is more than an ordinary user may usually lock; ulimit -l shows your limit in KiB. --deployments-root puts the host's files outside the site folder. Without it they go in .crisptastic/ under the current folder, and the site folder treats every .crisp file beneath it as a page, hidden folders included. --development-process-profile is what makes this a trial: it skips the hardening checks and the requirement for a reverse proxy. Ship it runs the real thing.

A host chooses the site by domain, so name it in the request:

curl -s -o /dev/null -w '%{http_code}\n' -H 'Host: guestbook.example.com' -d 'name=Mary&message=Hello from the host' http://127.0.0.1:8080/sign
curl -s -H 'Host: guestbook.example.com' http://127.0.0.1:8080/ | grep article

303
    <article><h2>Mary</h2><p>Hello from the host</p></article>

Only Mary is there. The host's store is not the one dev was using: a Capsule carries your pages, never your data. A trial host keeps its state beside its deployments, so this store is in ../trial-host/stores/guestbook/.

Stop the host (Ctrl+C), start it again the same way, and the entry is still there:

crisptastic host ../guestbook.age --identity ../host.agekey --trusted-signer ../release.pub --development-process-profile --bind 127.0.0.1:8080 --locked-secret-memory-bytes 65536 --deployments-root ../trial-host/deployments

curl -s -H 'Host: guestbook.example.com' http://127.0.0.1:8080/ | grep -c article

Before you rely on it

One host. The store lives on one machine, and a Fabric cluster refuses a site that declares [data].
Back it up. A site's records live only in the host's store. See Backup and recovery.
One site, one store. A different site claiming the same application_id is refused. See Sites with an embedded store for the other rules, including what memory_budget does on a host.
Anyone can sign. A form with no session has no CSRF check, so the rate limit is what stands between you and a spam script. See Keep a form honest.

What next

Change and remove records with data.entries_get, data.entries_replace and data.entries_delete: Capabilities and integrations.
Take the site to a production host: Ship it.