Cookieless identity, built on a salt that deletes itself

The problem

Pug’s Web SDK defaults to cookieless: nothing is written to the device until there is a basis to write it. That is the right default, and it leaves a question the SDK cannot answer on its own. Events still arrive from these visitors. Without any identifier, they cannot be grouped into sessions, which means no session counts, no duration, no pages-per-session — a large fraction of what makes analytics useful, lost for the majority of traffic that never answers a consent prompt.

So we needed an identity for a visitor we are deliberately not allowed to recognise. Specifically: stable enough to group one visitor’s events within a day, and provably useless for linking them to anything the next day.

The derivation

Every request already carries facts we do not have to ask for. The identifier is a keyed hash of three of them, under a secret that changes daily:

cookieless- + base64url-unpadded(
    HMAC-SHA256(salt_of_day, project ‖ 0x00 ‖ ip ‖ 0x00 ‖ ua)[:16]
)

A few decisions are load-bearing. HMAC rather than a bare hash of salt-plus-input, because HMAC’s construction is the well-analysed one and there is no reason to hand-roll a keyed hash. Null-byte framing between the fields, because concatenating variable-length strings is ambiguous — without a separator, different field splits can produce identical input, and two distinct visitors could collapse to one identifier. The framing is injective here because every IP has been through parsing before it arrives and the project ID is database-issued, so no field before the last can contain a null byte.

Truncated to 16 bytes, which is 22 characters under unpadded base64url — 64 bits short of the full digest, still far beyond collision range for the population involved, and meaningfully cheaper to store on every event row. A reserved cookieless- prefix, so every downstream system — the query builders, the ClickHouse materialised views, the validation patterns — can classify an identifier from its value alone, with no lookup and no join.

The IP and User-Agent are inputs only. They are never written to the event, never returned by any API. They exist inside the hash function for the duration of one call and are gone.

Why a plain hash would be theatre

It is worth being explicit about what the salt is for, because “we hash the IP” is a claim a lot of tools make and it is close to meaningless on its own.

There are about 4.3 billion IPv4 addresses. That is a small number. Anyone holding a set of unsalted IP hashes can compute SHA-256 over the entire address space and build a lookup table in minutes on a laptop — the hashes are then exactly as identifying as the addresses were. A secret salt defeats that precomputation, since the attacker cannot build the table without the key.

But a retained salt only moves the problem: the day someone obtains it, every historical identifier hashed under it becomes reversible in the same way. Rotation alone does not fix that either — if old salts are archived anywhere, the whole history stays re-derivable. The property only becomes durable when the salt is destroyed.

The deletion is the privacy guarantee. Rotation just decides how often it happens. Once the day’s salt is gone, the identifiers minted under it cannot be linked to any IP, to any User-Agent, or to each other — by us, by an attacker with full database access, or by anyone holding the backups.

Anchor the TTL to the day, not to the write

The salt lives in Redis under a TTL, and getting that TTL right took a second pass. The obvious implementation sets a flat duration when the key is written. That is wrong in a way that only shows up under real traffic.

Salts are minted lazily — on the first event attributed to a given day, not at midnight. We accept events for today and yesterday, a two-day window that exists so an SDK holding events through an offline period can flush them without losing identity. Combine those two facts and a day’s salt can first be written almost two days after the day it belongs to. A flat TTL then starts counting from there, so the salt’s real lifetime floats with traffic patterns rather than being a property of the day.

The concrete result: a 72-hour flat TTL kept salts re-derivable as late as D+5 — three days after any code path could still use one — with up to five coexisting instead of the two the design assumed. Nothing failed. No test caught it, because everything still worked; the only symptom was that the deletion guarantee quietly meant less than the documentation said.

The fix is to compute the TTL from the day itself: expire at D+2 00:00 UTC, the exact instant the day leaves the accepted window, never later. That makes the accepted window and the salt’s lifetime the same fact computed once, so they cannot drift apart. It also means a malformed or out-of-window day produces an error from the same function, which turns the boundary check and the lifetime calculation into one thing instead of two that must agree.

The type that caught what the tests could not

This package threads a lot of same-shaped strings side by side: a day, a project ID, a distinct ID, an IP, a User-Agent. All of them are strings. Transposing any two compiles cleanly, runs without error, and returns a confident-looking identifier.

That is not hypothetical. Swapping the day and project arguments at the call site keyed the salt by project instead of by day. The consequence: one salt per project, minted once, never rotating. The entire daily-rotation guarantee — the reason the package exists — was gone, and the full test suite stayed green. Every test asserted on behaviour that still held: identifiers were derived, they were stable within a request, sessions stitched, nothing threw.

// Day is a UTC calendar day in yyyymmdd form — the unit the salt rotates on.
type Day string

A one-line named type makes the transposition a compile error at every call site, including the ones nobody thought to test. That last part is the whole argument. A test can only pin the call sites someone anticipated; a type pins all of them, including the one added next year by someone who never read this post. When a bug class is silent — no exception, no wrong-looking output, just a guarantee that quietly stopped holding — detection has to be structural, because there is nothing for a test to observe.

Sessions with nothing on the device

With a stable within-day identifier, sessions become tractable server-side. Each event looks up the visitor’s current session in Redis and reuses it if the gap from its last activity is inside a 30-minute inactivity window, otherwise mints a fresh one and slides the watermark forward.

The window is evaluated on event time, not arrival time. The Redis TTL on the session key is garbage collection, not session semantics — conflating the two would put buffered events into whatever session was live when they happened to arrive, which for an offline flush could be hours later.

Minting uses SET NX GET so that two pods resolving the same visitor concurrently resolve the race safely: the first writer wins, and the loser observes and adopts the winner’s session rather than overwriting it. A residual last-write-wins overlap can split at most one session, which is bounded and accepted.

Degrade loudly, never fabricate identity

Two classes of failure matter here, and folding them together was a mistake worth undoing.

On the salt path, an unreachable store with a cold cache is transient — the same events succeed on retry. A salt that fails to decode is permanent until the key expires, because the mint only writes when the key reads as absent, so nothing overwrites a corrupt value. Those need opposite operator responses, so they are separate error values mapped to separate drop reasons. Either way the events are dropped rather than given a fallback identity: a fabricated identifier would merge unrelated visitors into one profile and corrupt the data permanently, while a visible gap in ingestion is recoverable and self-describing.

On the session path the returned ID is always usable, so failures are reported as a reason alongside it rather than as an error. The reasons are distinguished because they need different responses — a syntax error from a Redis older than 7.0 rejecting SET NX GET is a permanent deployment fault that repeats on every event; a timeout is transient; a write failure on a read-only replica leaves reads working so nothing else looks wrong. One undifferentiated counter cannot tell an operator which of those is happening.

The subtlest one is a session key holding an unparseable value. That is not a Redis failure — the read succeeded — and it takes the same mint path as an ordinary inactivity expiry, so without its own reason the two are indistinguishable and mass re-minting looks exactly like healthy traffic. Since session rollups are keyed by session ID, those rows are permanent and cannot be reconciled after the fact, which makes “looks healthy” an expensive thing to be wrong about.

Keeping it out of the numbers that count people

Deriving the identity is half the work. The other half is making sure it never reaches a metric it would corrupt, because a rotating identifier counted as a person inflates every user-level number. That split is its own subject, but one implementation detail belongs here: the classification is an exhaustive switch over every aggregation type, not a list of the ones that exclude.

The list form fails open. An aggregation added to the schema and not considered defaults to “include”, admitting rotating identifiers into a metric that may well count people, with nothing raising an error at any layer. The exhaustive form fails the build. A contract test ranges over the enum so that adding a member without deciding where it belongs breaks compilation rather than shipping a number that is wrong until someone happens to check.

That is the same instinct as the Day type, applied one level up. Both bugs are silent by nature, both corrupt data in a direction that looks like growth, and neither has anything a test could observe. When those three things are true, the answer is to make the mistake unrepresentable rather than to write another test.

Everything here is AGPL-3.0 and in the open. For the product-level view, see cookieless analytics, explained; for how the SDK side decides what to store and when, inside the Pug Web SDK.