Keep a form honest

In this tutorial you build a waitlist form that turns away the wrong requests, a forged post and a flood, with a real 403 and 429, and you find out which of those rules your laptop applies and which only a host does.

You need the crisptastic executable from the Quickstart and, for the last part, the standard age tool. Every request below was really made, and the answers shown are the ones that came back.

1. Create the folder

mkdir waitlist && cd waitlist

Everything below is a file inside waitlist/.

2. A form and a page that handles it

<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><title>Waitlist</title></head>
<body>
<h1>Join the waitlist</h1>
@if (context.session.email) {
  <p>You are on the list as {{ context.session.email }}.</p>
}
<form method="post" action="/join">
  <input type="hidden" name="_csrf" value="{{ context.csrf.token }}">
  <label>Email <input name="email" type="email" required></label>
  <button>Join</button>
</form>
</body>
</html>

@let email = lower(trim(request.body.email))
@if (contains(email, "@")) {
  @session email = email
  @redirect "/"
}
@status 400
<!doctype html>
<p>That does not look like an email address. <a href="/">Back</a></p>

Start the development server in another terminal, inside waitlist/:

crisptastic dev --bind 127.0.0.1:8080

Then, back in the first terminal, send the form as a browser would:

curl -si -d 'email=ada@example.com' http://127.0.0.1:8080/join | grep -E '^HTTP|^location|^set-cookie'

HTTP/1.1 303 See Other
set-cookie: session=XJtPZE8b-GyixdbwOy0erzg7UcUjlEF5AD5vm0ilNqU; Path=/; HttpOnly; SameSite=Lax
location: /

It works, and the server issued a session identifier of its own the moment the page first wrote to the session. Now look at what else the page will put up with:

curl -si http://127.0.0.1:8080/join | grep -E '^HTTP|<p>' | cut -c1-70
curl -si -H 'content-type: application/json' -d '{"email":"bot@example.com"}' http://127.0.0.1:8080/join | grep -E '^HTTP|^location'

HTTP/1.1 400 Bad Request
<p>That does not look like an email address. <a href="/">Back</a></p>
HTTP/1.1 303 See Other
location: /

A GET runs a page that was only ever meant for a form, and a script can post JSON at it. Nothing is broken yet, but the page is doing work it should never have been asked to do.

3. Declare what the route accepts

[routes."/join"]
methods = ["POST"]
body = "form"
max_body_bytes = 1024

The name in quotes must be a route of the site, or the site does not load. Stop the server (Ctrl+C) and start it again, because this file is read only at startup:

crisptastic dev --bind 127.0.0.1:8080

Try the same requests, plus one that is too big:

curl -si http://127.0.0.1:8080/join | grep -E '^HTTP|^allow|accepts'
curl -si -H 'content-type: application/json' -d '{"email":"bot@example.com"}' http://127.0.0.1:8080/join | grep -E '^HTTP|accepts'
head -c 2000 /dev/zero | tr '\0' 'x' | curl -si -d @- http://127.0.0.1:8080/join | grep -E '^HTTP|accepts'
curl -si -d 'email=ada@example.com' http://127.0.0.1:8080/join | grep -E '^HTTP|^location'

HTTP/1.1 405 Method Not Allowed
allow: POST
/join accepts POST
HTTP/1.1 415 Unsupported Media Type
/join accepts application/x-www-form-urlencoded
HTTP/1.1 413 Payload Too Large
/join accepts a body of at most 1024 bytes
HTTP/1.1 303 See Other
location: /

The server gives these answers before the page runs, so a refused request costs you no page and calls no capability. dev applies them exactly as a host does.

4. The CSRF token

A forged post is a form on someone else's site that sends a visitor's browser to /join, with their cookie attached. The defence is a token that only your pages know: once a visitor has a session, the server refuses any POST that does not carry that session's token. That is why the form has this field:

<input type="hidden" name="_csrf" value="{{ context.csrf.token }}">

See it work. Get a session and read the token out of the page:

SESSION=$(curl -si -d 'email=ada@example.com' http://127.0.0.1:8080/join | grep '^set-cookie: session=' | sed 's/.*session=\([^;]*\).*/\1/')
TOKEN=$(curl -s -H "Cookie: session=$SESSION" http://127.0.0.1:8080/ | grep -o 'name="_csrf" value="[0-9a-f]*"' | cut -d'"' -f4)
curl -s -H "Cookie: session=$SESSION" http://127.0.0.1:8080/ | grep 'on the list'
echo "${#TOKEN} characters"

  <p>You are on the list as ada@example.com.</p>
64 characters

Post with the cookie but no token, as a forged form would. Then post with it in the field, and then in the X-CSRF-Token header that browser JavaScript would use:

curl -si -H "Cookie: session=$SESSION" -d 'email=ada@example.com' http://127.0.0.1:8080/join | grep -E '^HTTP|csrf'
curl -si -H "Cookie: session=$SESSION" -d "email=ada@example.com&_csrf=$TOKEN" http://127.0.0.1:8080/join | grep -E '^HTTP'
curl -si -H "Cookie: session=$SESSION" -H "X-CSRF-Token: $TOKEN" -d 'email=ada@example.com' http://127.0.0.1:8080/join | grep -E '^HTTP'

HTTP/1.1 403 Forbidden
{"code":"security.csrf_invalid","detail":"CSRF verification failed","status":403,"title":"Forbidden","type":"urn:crisptastic:security:csrf-invalid"}
HTTP/1.1 303 See Other
HTTP/1.1 303 See Other

The rules, all of them visible above:

• A POST without a session is not checked. There is nothing to protect yet, and context.csrf.token is null, so the field is empty on a first visit.
• A POST with a session must carry its token. An empty field counts as missing, so a tab left open since before the first submit gets a 403 too. Reloading the page gets it the token.
• The check is on every route, whether you declared it or not, and it runs before the page. The answer is a fixed JSON document, not one of your error pages.
• The token is the only defence. There is no Origin or Referer check.

5. Rate limits

Nothing so far stops a script from trying ten thousand addresses. Add limits to the route.

[routes."/join"]
methods = ["POST"]
body = "form"
max_body_bytes = 1024
rate = [
  { per = "client", limit = 5, window = "1m" },
  { per = "body.email", limit = 3, window = "1h" },
]

Five joins a minute from one address, and three an hour for one email address, whoever sends them. Stop the server (Ctrl+C) and start it again:

crisptastic dev --bind 127.0.0.1:8080

Send the same address four times:

for n in 1 2 3 4; do curl -si -d 'email=grace@example.com' http://127.0.0.1:8080/join | grep -E '^HTTP|^retry-after'; done

HTTP/1.1 303 See Other
HTTP/1.1 303 See Other
HTTP/1.1 303 See Other
HTTP/1.1 429 Too Many Requests
retry-after: 3600

There is the 429, from the body.email limit, telling the client to come back in an hour. Now send new addresses:

for n in 1 2 3; do curl -si -d "email=guest$n@example.com" http://127.0.0.1:8080/join | grep -E '^HTTP|^retry-after'; done

HTTP/1.1 303 See Other
HTTP/1.1 303 See Other
HTTP/1.1 429 Too Many Requests
retry-after: 60

The sixth counted request from this address hits the client limit, which has a minute to run at most. Notice that you got two more through, not one: the refused fourth request above spent nothing, because every limit is checked before any is counted. Here is the whole answer:

curl -si -d 'email=hedy@example.com' http://127.0.0.1:8080/join | grep -vE '^(x-|referrer|permissions|connection)'

HTTP/1.1 429 Too Many Requests
content-type: text/plain; charset=utf-8
content-length: 34
retry-after: 60

Too many requests; try again later

Wait out the minute. The client is welcome again, and the address that used up its hour is still not:

sleep 60
curl -si -d 'email=hedy@example.com' http://127.0.0.1:8080/join | grep -E '^HTTP'
curl -si -d 'email=grace@example.com' http://127.0.0.1:8080/join | grep -E '^HTTP|^retry-after'

HTTP/1.1 303 See Other
HTTP/1.1 429 Too Many Requests
retry-after: 3540

What counts, and what does not:

• A request refused for its method, body or CSRF token runs no limit at all, and GET and HEAD are never counted. A limit on a route that accepts only those stops the site loading.
• per = "body.email" counts the field lower-cased and trimmed. A request without the field counts nothing for that limit, so your page still has to decide what a missing field means.
• The 429 is a fixed answer from the server, and your error pages are not used.
• Pages cannot read or change the counters. They live in the server's memory, so a restart forgives everyone. On a host, a redeploy forgives nobody.

Behind a reverse proxy, client means the address the host's --trusted-hops setting selects, and an IPv6 client counts as its /64. See Configuration.

6. What only a host enforces

Everything so far behaved the same in dev and on a host. Some things do not, and they are the ones that catch people out. Add a page that writes to the session during a GET:

@session visited = "yes"
<!doctype html>
<p>Noted.</p>

curl -si http://127.0.0.1:8080/visit | grep -E '^HTTP|^set-cookie|<p>'

HTTP/1.1 200 OK
set-cookie: session=8N4yiYth9y9bYwspbpjzVDFt-eXP_k4Ksns41QbUEK0; Path=/; HttpOnly; SameSite=Lax
<p>Noted.</p>

Fine in dev. To try it on a host you need a Capsule. Give the site a domain:

[deployment]
domains = ["waitlist.example.com"]

[routes."/join"]
methods = ["POST"]
body = "form"
max_body_bytes = 1024
rate = [
  { per = "client", limit = 5, window = "1m" },
  { per = "body.email", limit = 3, window = "1h" },
]

Stop the server (Ctrl+C), then make the keys and build the Capsule:

age-keygen -o ../host.agekey
crisptastic capsule keygen --private ../release.pk8 --public ../release.pub
crisptastic deploy . --recipient "$(age-keygen -y ../host.agekey)" --signing-key ../release.pk8 --out ../waitlist.age

Start a local trial host in the other terminal. --development-process-profile skips the server hardening checks and the need for a reverse proxy, and --locked-secret-memory-bytes asks for less locked memory than the default 64 MiB, which is usually more than an ordinary user may lock. --deployments-root keeps the host's unpacked files out of the site folder, where they would count as pages. It is still the production request path: Ship it runs the same thing without the shortcuts.

crisptastic host ../waitlist.age --identity ../host.agekey --trusted-signer ../release.pub --development-process-profile --bind 127.0.0.1:8080 --locked-secret-memory-bytes 65536 --deployments-root ../trial-host/deployments

A host picks the site by domain, so every request now names it:

curl -si -H 'Host: waitlist.example.com' http://127.0.0.1:8080/visit | grep -E '^HTTP|code'
curl -si -H 'Host: waitlist.example.com' -d 'email=ada@example.com' http://127.0.0.1:8080/join | grep -E '^HTTP|^set-cookie'

HTTP/1.1 500 Internal Server Error
{"code":"security.safe_method_mutation","detail":"safe methods cannot mutate session or cookie state","status":500,"title":"Internal Server Error","type":"urn:crisptastic:security:safe-method-mutation"}
HTTP/1.1 303 See Other
set-cookie: session=-vtuZC22B-rZgmyLKH7enUipshrJmY6sLfu83bAn8js; Path=/; HttpOnly; SameSite=Lax; Secure

Two differences. The page that was fine in dev is now a 500, and the cookie has gained Secure. Because a GET may not change anything on a host, a visitor gets their first session from their first POST, which is what the waitlist does.

A note if you test this by hand: when you set a Host header over plain HTTP, curl will not keep a Secure cookie in its cookie jar, which is why this tutorial copies the value into a header. A browser behind your HTTPS proxy does it for you.

The route's own rules come along unchanged:

SESSION=$(curl -si -H 'Host: waitlist.example.com' -d 'email=katherine@example.com' http://127.0.0.1:8080/join | grep '^set-cookie: session=' | sed 's/.*session=\([^;]*\).*/\1/')
curl -si -H 'Host: waitlist.example.com' -H "Cookie: session=$SESSION" -d 'email=katherine@example.com' http://127.0.0.1:8080/join | grep -E '^HTTP|csrf'
curl -si -H 'Host: waitlist.example.com' http://127.0.0.1:8080/join | grep -E '^HTTP|^allow'
for n in 1 2 3 4; do curl -si -H 'Host: waitlist.example.com' -d 'email=radia@example.com' http://127.0.0.1:8080/join | grep -E '^HTTP|^retry-after'; done

HTTP/1.1 403 Forbidden
{"code":"security.csrf_invalid","detail":"CSRF verification failed","status":403,"title":"Forbidden","type":"urn:crisptastic:security:csrf-invalid"}
HTTP/1.1 405 Method Not Allowed
allow: POST
HTTP/1.1 303 See Other
HTTP/1.1 303 See Other
HTTP/1.1 303 See Other
HTTP/1.1 429 Too Many Requests
retry-after: 3600

Stop the host (Ctrl+C), and delete visit.html.crisp before you go any further.

7. What applies where

Each line of this table was tested for this tutorial.

So the form you test in dev is the form you get, with one habit to keep: do all your writing in a page that answers POST, then @redirect.

What next

• Everything a page can read about a request, and how sessions are stored: Sessions and forms.
• Every key of [routes]: Configuration.
• Put the site on a real host: Ship it.