Files
wiki/dev/specs/notifications.md

54 KiB
Raw Permalink Blame History

Notifications

Status: implemented — phases 1 to 4 of §16; push, digests and the admin categories are still to come. §2 describes what the system was built on, as it stood before. Where building it changed the design, this document was changed with it. Covers: what a user is notified about, how they choose which notifications they get and how, how an event becomes an inbox entry and an email without slowing the request that caused it, and how new kinds of notification are added later.

A wiki already tells you a great deal if you go looking for it: the watch list, the review queue, the Talk tab. What it does not do is come and find you. This document is about that half: being told that a page you watch has changed, that someone answered you, that a suggestion is waiting for your review.

The constraints this is written against:

  • A notification never tells anyone more than they could have found out for themselves. Every recipient is checked against the permission that would let them see the thing they are told about, at the time they are told.
  • The request that caused an event does one write, and nothing else. Working out who to tell, checking each person's access, and talking to an SMTP server all happen in worker threads, however many people end up being notified.
  • A new kind of notification is one file, one emit call and its strings. The admin-facing ones that are coming (registrations, failed logins, a failed git sync) must not need a second system.

1. Goals and non-goals

Goals

  • Seven categories at launch (§4.2), five on by default and two opt-in.
  • Per-user, per-category choice of channel: Email, In-App, both or neither, from Profile → Notifications. One set of preferences per user, not one per site.
  • Scales with the instance: the cost of an event is roughly proportional to the number of people who actually receive it, not to the number of accounts on the wiki.
  • Coalescing: a page saved forty times in an afternoon does not produce forty inbox entries.
  • RFC 8058 one-click unsubscribe on every notification email. This is a hard requirement.
  • A per-site switch that turns notifications off for a site.
  • Polling now, push later, without the push work having to change anything polling relies on.

Non-goals

  • Live push in the first version. §9.3 says how it attaches.
  • Hourly and daily digest emails in the first version. They are expected to follow, and §10.5 is what the first version does to make them additive.
  • Other channels (browser Web Push, Slack DMs, Matrix). The preference model leaves room for them (§5), but none is planned.
  • Third-party comment providers. Disqus, Giscus and the rest keep their discussions in somebody else's service. Only the built-in provider produces comment events.
  • Notifying guests. A notification needs an account to belong to.
  • Notifying anyone of their own actions. The actor of an event is never one of its recipients.

2. What exists today

Piece Where State
Page watching db/schema.ts → pageWatching, models/pageWatching.ts, api/watching.ts Done. One row per (page, user), carrying the site. The comments in that code already expect "whatever delivers the news later" to read it. Rows cascade away when the page is deleted, which matters for §6.3
Watch button and list the bell in the page header, pages/InboxWatching.vue Done
Inbox shell layouts/InboxLayout.vue, /_inbox/messages → pages/InboxMessages.vue Placeholder, reading "Nothing here yet"
Review deep link /_inbox/review/:submissionId? Done. Its route comment anticipates a notification linking to it
Profile entry layouts/ProfileLayout.vue, key notifications Present but disabled: true, and has no route
Header button components/HeaderNav.vue, mdi:inbox-full → /_inbox Present, with no badge
Strings locales/en.json → profile.notifications, inbox.* inbox.title already reads "Inbox & Notifications"

What it is built from:

Piece Where Used for
Webhook emit models/hooks.ts → emit() The contract to copy: called from the models, never throws, and does no delivery inline. Every hooks.emit call site is a candidate notification emit site
Scheduler and workers core/scheduler.ts, worker.ts, tasks/workers/ Fan-out and mail both run as worker tasks. dispatch-webhook.ts is the model for a worker task that imports its own models
Page rules helpers/pageRules.ts → resolvePageRule, models/groups.ts → checkAccess Pure and in memory. The answer depends only on a user's set of groups, which is what makes bulk access checks cheap (§7.2)
Approval rules models/approvals.ts → matchesPage, reviewerGroups Who reviews a page
Mentions models/comments.ts → resolveMentions Turns @handle text into users with one query
Mail models/mail.ts, locales.translator() Templates, two bodies (HTML and text), locale fallback
Import scope storage.importingFrom() (AsyncLocalStorage) Tells an import or a git pull apart from a person's edit (§6.2)
Form bodies @fastify/formbody, registered in index.ts The one-click unsubscribe POST is application/x-www-form-urlencoded

3. Architecture

 request path                  worker: fan-out                  worker: mail drain
 ────────────                  ───────────────                  ──────────────────
 notifications.emit()  ──►  notificationEvents  ──►  notifications            ──►  SMTP
   one INSERT, never          (outbox; claimed       one row per recipient         one mail per user
   throws; a debounced        with SKIP LOCKED)      per coalescing key, with      per drain, single
   job kick                                          in-app and email state        or digest
                                                            │
                                                            └──►  GET /notifications/summary  ◄── polling
                                                                  (push attaches here later, §9.3)

Three tables and two worker tasks. This is deliberately not one scheduler job per recipient, or even one per event. Every job costs a jobs row and a jobHistory row. On a busy wiki one job per page save is thousands of rows a day before anyone has been notified, and one job per recipient is that multiplied by the audience. Instead, the outbox is drained in batches, and the job is only the thing that wakes the drain up.

3.1 Where the code lives

Path What
backend/notifications/index.ts The category registry: NOTIFICATION_CATEGORIES, the NotificationCategory type, lookup by event
backend/notifications/categories/<key>.ts One file per category (§4.1)
backend/models/notifications.ts emit(), preferences, inbox queries, the unsubscribe token, settings
backend/api/notifications.ts Inbox, summary, preferences, unsubscribe, admin settings
backend/tasks/workers/dispatch-notifications.ts Fan-out (§7)
backend/tasks/workers/send-notification-mail.ts Mail drain (§10)
backend/tasks/simple/purge-notifications.ts Retention (§12.4)

notifications/ is a new top-level backend directory rather than a directory under modules/: refreshFromDisk expects every directory there to hold a definition.yml. A category is code, not an installable module, and there is nothing to discover from disk.


4. Categories

4.1 A category is a file

export interface NotificationCategory<E extends NotificationEventKind = NotificationEventKind> {
  key: string                        // 'watchedPage', also the preference key and the i18n key
  events: readonly E[]               // which emitted events it reacts to
  scope: 'site' | 'instance'         // see §11; every launch category is 'site'
  origins: readonly EventOrigin[]    // which origins it fires for, see §6.2
  defaults: { inApp: boolean; email: boolean }
  priority: number                   // higher wins when one event reaches a user twice, §7.4
  visibleTo(actor: AccessActor): boolean      // shown in Profile → Notifications?
  recipients(event, ctx): AsyncIterable<string[]>  // candidate user IDs, in batches
  accessCheck: string | null         // page permission each recipient must hold NOW
  groupKey(event): string            // the coalescing key, §8.3
  variant(event): string             // which message: 'edited', 'moved', 'deleted', ...
  snapshot(event): Record<string, unknown>  // what the entry displays, §8.2
}

NOTIFICATION_CATEGORIES is a closed as const list, the same pattern as AUDIT_ACTIONS, so npm run typecheck refuses a key that does not exist. Each key is also its translation prefix (notifications.categories.<key>.*), so adding a category means adding its strings too.

recipients returns candidates. Removing the actor, checking access, applying preferences and deduplicating are done once by the fan-out for every category (§7), so a category cannot get any of them wrong.

4.2 The launch categories

Key Reacts to Candidates Must hold now Default Coalesces on Origins
watchedPage page:edit, page:rename, page:delete the page's watchers read:pages in-app ✓ email ✓ watchedPage:<pageId> all
reviewRequested submission:new, submission:update members of the reviewer groups of every enabled rule matching the page none (§4.3) ✓ ✓ review:<submissionId> user
watchedPageComment comment:new the page's watchers read:comments ✓ ✓ watchedComment:<pageId> user
commentReply comment:new with a parent the author of the parent comment, if it has an account read:comments ✓ ✓ reply:<parentId> user
mention comment:new, comment:edit users whose handle is written in the comment (on an edit, only handles that were not there before) read:comments ✓ ✓ mention:<commentId> user
pageCreated page:create users who opted in read:pages ✗ ✗ pageCreated:<pageId> user
pageDeleted page:delete users who opted in read:pages, against the page as it was ✗ ✗ pageDeleted:<pageId> user

watchedPage variants. edited, moved (path or locale changed, which is what page:rename is), published, unpublished, scheduled (the publish window was set or changed), and deleted. A publish-state change is a variant of an edit, worked out by comparing the old and new publishState, publishStartDate and publishEndDate in updatePage. A coalesced entry (§8.3) records every variant it has absorbed, so "edited 4 times and moved" can be said.

Email defaults apply only where email can be sent. On an instance where mail.isConfigured is false, every email preference reads as off and the Profile screen says why (§5.2).

4.3 Reviewers

Only members of the groups named in reviewerGroups of the enabled rules that match the page are notified. Holding review:pages at the page, or manage:system, does not make anyone a recipient. On a large wiki, every administrator would otherwise receive every suggestion. An administrator who wants these notifications is placed in a reviewer group like anyone else. (They can still see and answer the whole queue, as getReviewableSubmissions allows today. Only the notification is narrower.)

No page permission is checked on top of that: the rule naming the group is the grant, and the notification says no more than the reviewer's own queue already shows. The submitter is excluded, like any actor. A guest's suggestion notifies just as a signed-in user's does.

submission:update is the author revising a suggestion that is still open (there is one per author per page, by the unique index). It bumps the existing unread entry rather than adding a second one.

approvals.matchesPage has to be callable from a worker thread, so if it reads anything off this beyond its arguments, it moves to a pure helper beside helpers/pageRules.ts.


5. Preferences

5.1 Storage

userNotificationPrefs
  userId    uuid  → users.id, on delete cascade
  category  varchar(64)
  channel   varchar(16)      -- 'inApp' | 'email'
  enabled   boolean
  updatedAt timestamp
  PRIMARY KEY (userId, category, channel)
  INDEX (category, channel) WHERE enabled

A row is stored only when it differs from the category's default. No rows means the defaults, and setting a choice back to its default deletes the row. That keeps the table small, and keeps the defaults in the registry rather than copied into every account.

A table rather than a key in users.prefs, because of the opt-in categories. To fan out pageCreated, the question is "who has turned this on?" across every account on the instance. That is an index lookup on the partial index above, rather than a scan of a JSONB column. The default-on categories ask the other way round, "of these watchers, who has turned it off?", which joins the already-small candidate list against the primary key.

One row per channel, not one column per channel, so a third channel would be a new value rather than a migration.

Preferences are global: one set per user, covering every site they use.

5.2 The Profile screen

pages/ProfileNotifications.vue, at /_profile/notifications (enabling the entry ProfileLayout.vue already has). One row per category the caller can see (visibleTo), each with its title, a one-line description, and two toggles: In-App and Email. Grouped under headings: Pages you watch, Discussions, Reviews, Everything on the wiki. The last group holds the two opt-in categories, with a note that they can be busy.

  • With mail not configured, the Email column is disabled, with a hint that the administrator has not set up outgoing mail.
  • On a site where notifications are switched off (§11), a banner says so. The preferences still apply on the user's other sites.
  • A Stop all email action at the foot of the screen turns off email for every category in one go. It is the same thing the unsubscribe page offers.

GET /_api/users/profile/notifications answers with every visible category, its effective { inApp, email }, its defaults, and emailAvailable. PUT takes the full map back, stores only the rows that differ from the defaults, and audits updateNotificationPrefs (kind profile) with the categories that changed.


6. Emitting

6.1 The call

WIKI.models.notifications.emit('page:edit', {
  siteId, actorId, pageId,
  snapshot: { title, path, locale, tags, publishState, ... },
  variant: 'edited'
})

It writes one notificationEvents row and asks for a fan-out run. Like hooks.emit, it never throws: a notification problem must not fail the action that caused it. It returns at once, and it is called only after the action has succeeded.

It returns without writing anything when:

  • the site has notifications switched off (§11), checked against WIKI.sites, which the request path has and a worker does not, or
  • no category listens for this event from this origin. A git pull's page:create is dropped here rather than written and then thrown away.

The kick is debounced per instance. The first emit arms a timer of about one second, and when it fires, a single dispatchNotifications job is added. A burst of 500 saves becomes a handful of jobs. A * * * * * SYSTEM_SCHEDULE entry runs the same task as a safety net, so an instance that dies with a timer armed loses at most a minute.

Emit sites, beside the existing hooks.emit calls:

Event Where
page:create pages.createPage, and the restore path that re-creates a page
page:edit pages.updatePage, including the save approvals makes when a suggestion is approved (the actor is the reviewer, and the snapshot carries who suggested it)
page:rename pages.movePage
page:delete pages.deletePage, deletePagesByTag, tree.deleteFolder, and git's removals in applyIncoming. Every path that removes a page row has to go through §6.3
submission:new / submission:update where approvals writes a pageEditSubmissions row
comment:new / comment:edit api/comments.ts, beside the webhook emits (comments are only ever created from a route)

6.2 Origin

Every event carries an origin: user, import or bulk.

  • import: anything running inside storage.importingFrom(), which covers Import Everything and a git pull's applyIncoming. emit reads the scope itself, so callers do not pass it.
  • bulk: deletePagesByTag and tree.deleteFolder. These run their per-page work inside a notifications.bulk(work) AsyncLocalStorage scope of the same shape.
  • user: everything else.

A category's origins decides whether it fires for an origin. watchedPage fires for all three, so a watcher hears about a page a git pull rewrote, with the entry saying "via storage sync" where there is no actor. pageCreated and pageDeleted fire for user only, so an import of 5,000 pages, or a tag deletion, does not send 5,000 notifications to everyone who opted in.

6.3 Deletion: snapshot the watchers first

pageWatching rows are deleted with the page, so by the time a fan-out runs, a deleted page has no watchers left to find. The page:delete emit therefore happens before the page row is deleted, and captures the watchers in the same statement:

INSERT INTO "notificationEvents" (…, "recipients")
VALUES (…, ARRAY(SELECT "userId" FROM "pageWatching" WHERE "pageId" = $1))

The IDs never pass through Node, and the row is written whether or not the delete then succeeds. A delete that fails after this point (rare, since the page row is the last thing to go) leaves an event saying a page was deleted that was not. To avoid that, the fan-out checks that the page row is really gone before it sends any deleted variant, and drops the event if the page is still there.

Access for a deleted page is checked against the snapshot's path, locale and tags, the page as it was.

Restoring a deleted page does not bring its watchers back: the cascade has already removed them. That is a property of the existing schema, noted here rather than changed.


7. Fan-out

tasks/workers/dispatch-notifications.ts, in a worker thread.

7.1 Claiming

It claims up to 50 unprocessed events, oldest first, with FOR UPDATE SKIP LOCKED, writing claimedAt and claimedBy. Several instances can drain the outbox at once without two of them taking the same event. A claim older than scheduler.taskTimeout plus a margin counts as abandoned and can be claimed again, the same reasoning as reapStaleJobs. When the batch is done, the task runs again straight away if more events are waiting, and stops otherwise.

7.2 Access is checked per group set, not per user

checkAccess depends only on the actor's groups (plus manage:system). Candidates are read together with their memberships:

SELECT u.id, array_agg(ug."groupId" ORDER BY ug."groupId") AS groups, u.prefs->>'locale' …
FROM users u JOIN "userGroups" ug ON ug."userId" = u.id
WHERE u.id = ANY($candidates) AND u."isActive" AND NOT u."isSystem"
GROUP BY u.id

The check is memoized on the joined group IDs. Ten thousand candidates spread over six group combinations cost six rule evaluations. The worker has no WIKI.models.groups, so it reads groups.rules and groups.permissions directly once per run and calls resolvePageRule itself. What it answers is what checkAccess would answer, from the same rows.

Access is checked when the fan-out runs, not when the event happened: someone whose access was removed in between is not told.

7.3 Candidates, in batches

recipients() yields IDs in batches of 1,000, using keyset pagination on userId:

  • Watchers: pageWatching by pageId, or notificationEvents.recipients for a deletion.
  • Opted in: userNotificationPrefs where category = $1 AND enabled, on its partial index. The users table is never scanned.
  • Reviewers: userGroups where groupId = ANY($reviewerGroups).
  • Mentions and replies: a handful of IDs, resolved from the comment.

7.4 One entry per person per event

A single comment:new can reach the same person three ways: they watch the page, they wrote the parent comment, and they are mentioned in it. They get one entry, from the category with the highest priority among those that would notify them on some channel: mention (40) > commentReply (30) > watchedPageComment (10). Likewise watchedPage (20) beats pageDeleted (5) for a watcher who also opted into deletions everywhere.

The dedup runs after preferences have been applied: someone who turned mentions off but watches the page still gets the watched-page entry.

7.5 Writing

Rows are written 500 at a time, as one multi-row INSERT … ON CONFLICT … DO UPDATE (§8.3). Each row carries inApp from the preferences, and emailState = 'pending' with emailAfter set (§10.2) where email is on, the account is verified and mail is configured. When anything has been written with email pending, the mail drain is kicked.

7.6 Long fan-outs

The task checks signal between batches. When it is about to time out, it writes the keyset cursor it has reached onto the event row (cursor) and leaves the event claimed but unprocessed. The next run picks it up from the cursor. This is the same idea as renderPages handing the rest of its queue to a fresh job, so a pageCreated going out to a very large opted-in audience is spread over several runs instead of running into the timeout.


8. Storage

8.1 notificationEvents (the outbox)

id          uuid PK          -- the event ID, also the idempotency key
kind        varchar(64)      -- 'page:edit', 'comment:new', …
origin      varchar(16)      -- 'user' | 'import' | 'bulk'
siteId      uuid null        -- null for an instance-scoped event, §11
actorId     uuid null        -- no FK: an event outliving its actor is fine
data        jsonb            -- snapshot, variant, IDs of the page / comment / submission
recipients  uuid[] null      -- pre-resolved candidates (deletions)
cursor      jsonb null       -- progress of an interrupted fan-out
claimedAt   timestamp null
claimedBy   varchar null
processedAt timestamp null
createdAt   timestamp
INDEX (createdAt) WHERE processedAt IS NULL

Processed events are kept for a day, which is enough to debug a notification that did not arrive. After that they are purged (§12.4).

8.2 notifications (the inbox and the email queue)

id          uuid PK
userId      uuid → users.id, on delete cascade
siteId      uuid null → sites.id, on delete cascade      -- null: instance-scoped, §11
category    varchar(64)
variant     varchar(32)                                  -- the latest variant
groupKey    varchar(255)
pageId      uuid null → pages.id, on delete set null
commentId   uuid null → comments.id, on delete set null
actorId     uuid null → users.id, on delete set null
data        jsonb      -- snapshot: title, path, locale, actor name, excerpt, variants seen, …
count       integer default 1
lastEventId uuid
inApp       boolean
emailState  varchar(16) default 'none'   -- none | pending | sending | sent | failed | skipped
emailAfter  timestamp null
emailedAt   timestamp null
readAt      timestamp null
createdAt   timestamp
updatedAt   timestamp                    -- bumped on coalescing, the inbox's sort key

INDEX (userId, updatedAt DESC)                                -- the inbox
INDEX (userId) WHERE readAt IS NULL AND inApp                 -- the badge
INDEX (emailAfter) WHERE emailState IN ('pending', 'sending') -- the mail drain
UNIQUE (userId, groupKey) WHERE readAt IS NULL                -- §8.3

The entry carries a snapshot, because what it is about may be renamed, moved or deleted before it is read. The link is built from pageId where the page still exists, so it follows a move, and from the snapshot's path otherwise.

An excerpt is shown only while its comment exists. commentId is set null when the comment is deleted, and the API leaves the excerpt out of any entry whose commentId is null. A moderator deleting spam therefore removes its text from every inbox it reached. An email that has already been sent cannot be recalled.

One row covers both channels. An email-only recipient still gets a row, with inApp = false, and the inbox and the badge never show it. That keeps one source of truth, and gives coalescing (§8.3) and the email cadence (§10.2) one model to work on. An email-only row is closed (readAt = emailedAt) once its email is sent, since there is no inbox in which it could be read.

8.3 Coalescing and idempotency are the same index

Every category gives each entry a groupKey, and a user has at most one unread entry per key. The insert is:

INSERT INTO notifications (…) VALUES (…)
ON CONFLICT ("userId", "groupKey") WHERE "readAt" IS NULL
DO UPDATE SET
  count       = notifications.count + 1,
  variant     = EXCLUDED.variant,
  data        = notifications.data || EXCLUDED.data,   -- variants seen are unioned in code
  actorId     = EXCLUDED."actorId",
  lastEventId = EXCLUDED."lastEventId",
  updatedAt   = now(),
  emailState  = <per §10.2>,
  emailAfter  = <per §10.2>
WHERE notifications."lastEventId" IS DISTINCT FROM EXCLUDED."lastEventId"
  • Coalescing: forty saves to a page you have not looked at yet are one entry, "edited 40 times, last by Bob", which moves to the top of the inbox each time.
  • Idempotency: a fan-out retried after a crash replays the same event ID. The WHERE makes the replay change nothing, so nobody's count is inflated and nobody gets a second row.
  • Reading resets it: once the entry is read it no longer holds the unique slot, and the next event starts a new one.

The keys in §4.2 are chosen so that only things worth merging merge: two comments on a watched page merge, but two mentions in two comments do not (mention:<commentId>).


9. In-app delivery

9.1 API

All routes are the caller's own rows: any logged-in session, no permission, and 401 for a guest. No route declares config.permissions. Each comments No route-level permissions: and filters by userId from the session.

Route What
GET /_api/sites/:siteId/notifications?cursor=&unread= Keyset-paginated on (updatedAt, id), 30 per page. Includes the site's rows plus instance-scoped ones (siteId IS NULL)
GET /_api/sites/:siteId/notifications/summary { unread, latestAt }. unread is counted from SELECT 1 … LIMIT 100 on the badge index and shown as "99+" above 99. Answers with an ETag built from the two values, so a poll that finds nothing new is a 304
PUT /_api/sites/:siteId/notifications/:id/read Marks one entry read
PUT /_api/sites/:siteId/notifications/read Marks everything read, or what matches every filter given: ids, pageId, categories (mark-read-on-view, §10.2). An entry read before its email has gone cancels the email
DELETE /_api/sites/:siteId/notifications/:id Dismisses an entry

Marking read and dismissing are not audited. They are bookkeeping on one's own inbox, and a click per notification would bury the audit log, for the same reason page views are not recorded. Preference changes and the admin settings are audited.

9.2 Polling

A Pinia store, stores/notifications.js, holds unread, latestAt and the loaded page of entries, and has one method that matters: refresh(), which calls summary and loads the list only if latestAt moved. What calls refresh() is kept separate from the store itself:

  • app boot, once the session is known to be signed in;
  • a route change, throttled to once per 15 s;
  • visibilitychange to visible;
  • an interval of 60 s, only while the tab is visible, so a hundred background tabs do not poll.

The badge in HeaderNav.vue reads unread. InboxMessages.vue becomes the list: grouped by day, unread entries marked, each entry's text produced in the browser from notifications.messages.<category>.<variant> plus its snapshot, so an entry is read in the language the interface is drawn in. Each entry links to its target:

  • a page, or the page's Talk tab with the comment anchored;
  • /_inbox/review/:submissionId for a review;
  • for a deleted page, the snapshot's path, which leads to the page-not-found screen and its restore offer where the reader may restore.

9.3 Adding push later

Push is designed in now but not built. When it is added, it only ever says "something changed, call refresh()". The summary endpoint stays the single source of truth, so a pushed update and a polled one leave the store in exactly the same state. Nothing about polling or the endpoint changes.

When it is built:

  • every write that changes what a user's summary would answer (the fan-out's inserts, the read and dismiss routes, the purge) sends NOTIFY notifications with the affected user IDs, through createNotifier and chunked under Postgres's 8 kB payload limit;
  • every instance LISTENs and forwards to its own sockets on a /_notifications websocket, beside the collab and terminal ones;
  • the 60 s interval drops to a 5 minute safety net while the socket is open.

No NOTIFY is sent in the polling version: with nothing listening it would be dead code.


10. Email

10.1 Templates

MailTemplateData gains notification (one entry) and notificationDigest (several), with strings under mail.notification.*. Both render through the existing MailContent → htmlShell / textBody pair. The language is the recipient's prefs.locale, falling back to the site's primary locale, which is what localeFor already does.

What a notification email contains: the title, who did it, what it is about (page title and path), for a comment the excerpt, one action button linking to the target, and a footer with Manage notifications (linking to /_profile/notifications) and Unsubscribe. No page content is ever included.

The mail model runs in a worker. It used to read WIKI.sites[siteId] (title, primary locale) and go through WIKI.models.locales, and a worker has neither. send() now takes a MailSite ({ name, primaryLocale }) from its caller instead of a site id — the existing callers build one with mail.siteFor(siteId), the drain from the sites rows it reads once per run — and baseUrl() accepts a hostname for a caller with nothing to look one up in. The translator is the locales model imported directly. It caches through WIKI.cache, which a worker does not have, so the drain gives each run a fresh one: a worker thread outlives many runs and never hears reloadLocales.

A worker's settings go stale. A worker thread reads the settings table once, when it first opens the database, and does not hear reloadConfig. Both notification tasks therefore re-read it at the start of every run (refreshWorkerConfig), or mail configured after the thread started would never be seen and a changed email delay would be ignored until a restart.

10.2 Cadence

A window, then quiet until read. An entry is emailed once, a short delay after it first appears, and then not again until its recipient has read it:

emailState / emailAfter
A new row pending, emailAfter = now() + emailDelay
A coalesce while pending unchanged. The event merges into the email that is already due
A coalesce after sent stays sent. The inbox entry keeps counting, but no further email is sent
Read the entry gives up its unique slot (§8.3), so the next event starts a new entry and a new email

emailAfter is the only thing the drain looks at. It groups everything due for one user into a single mail: the notification template for one entry, notificationDigest for several.

In practice: Alice watches Runbook, Bob saves it twelve times between 10:00 and 10:40, and at 10:05 Carol mentions Alice on another page. Alice receives two emails, "Runbook was edited" at about 10:03 and the mention at about 10:08, and nothing more about Runbook until she opens it. Her inbox entry reads "edited 12 times". Sending an email per event would have meant thirteen of them, and a window alone about ten, since Bob's pace keeps opening new windows.

This only works if entries get read in the ordinary course of things, or it becomes "one email, ever". So opening a page marks its watchedPage and pageCreated entries read, and opening its Talk tab marks its watchedPageComment, commentReply and mention entries read. The page payload answers viewer.unreadNotifications, the categories the reader has unread entries in about that page, and the browser calls PUT …/notifications/read { pageId, categories } only for the ones it has just shown (markSeen in stores/notifications.js). A page view therefore writes nothing unless there was something to clear.

An email-only row closes when it is emailed (§8.2), because it has no inbox to be read in. Its recipient therefore gets one email per window rather than one ever.

emailAfter is computed in exactly one place, emailAfterFor in notifications/fanout.ts, and both the fan-out insert and the coalesce go through it. Today it answers now + emailDelay. It is the seam where hourly and daily digests attach later (§10.5).

10.3 The drain

tasks/workers/send-notification-mail.ts:

  1. Picks up to notifications.mailBatchSize (default 100) (user, site) pairs with rows due.
  2. For each pair, claims the due rows with one UPDATE … WHERE id IN (SELECT … FOR UPDATE SKIP LOCKED) that marks them sending and makes emailAfter a ten-minute lease. No transaction is held across the send: a worker thread has a single database connection, and a transaction kept open while the mail is rendered leaves none for the queries rendering makes — the run waits on itself. A run that dies mid-send leaves rows sending with a lease that lapses, and the next run takes them again; at least once, rather than at most once.
  3. Renders the claimed rows (capped at 50; above that, the digest says "and N more" and links to the inbox), drops them as skipped if the site has notifications switched off, renders one mail and sends it. One mail per site, not one per user, because a mail names the wiki it comes from and its links use that site's hostname. A user active on two sites gets two digests, each of which reads as coming from its own wiki. Instance-scoped entries (siteId null) go out as coming from the first site (§11).
  4. Marks the rows sent with emailedAt. A send that fails puts them back to pending ten minutes on, and after three attempts marks them failed. A failure on one user does not stop the batch.
  5. Makes one pass and stops, then asks for the next run at the moment the next email comes due. A five-minute schedule is the safety net, for fan-out as well.

Every send goes through the one cached transporter. Turning on nodemailer's pool option for this task is worth measuring once it exists.

Notification mails also carry Auto-Submitted: auto-generated, so out-of-office replies are not sent back to the wiki.

10.4 One-click unsubscribe (RFC 8058)

This is a hard requirement. Every notification mail carries:

List-Unsubscribe: <https://{host}/_api/notifications/unsubscribe?t={token}>
List-Unsubscribe-Post: List-Unsubscribe=One-Click

The token. base64url(payload) . base64url(HMAC-SHA256(payload)), where the payload is { v: 1, u: userId, c: [categories in this mail] }. It is signed with notifications.unsubscribeSecret, a 32-byte value generated on boot wherever it is missing, by the startup checks in core/startupChecks.ts — so an installation from before notifications gets one as well as a new one. It is deliberately not auth.secret: that one is rotated whenever an administrator invalidates every session, which would break every unsubscribe link already sitting in somebody's mailbox (the reason apiKeys.ts gives for keeping the certificate passphrase apart). The token does not expire. A link in an email from last year still has to work, and the only thing it can do is turn email off for one person.

POST (the one-click):

  • publicAccess, no session, no cookie, no CSRF token. The mail client sends none of these.
  • Takes application/x-www-form-urlencoded with List-Unsubscribe=One-Click.
  • Verifies the HMAC with timingSafeEqual, then turns email off for the token's categories, writing enabled = false rows. In-app is left alone: the user only asked to stop receiving mail.
  • Idempotent, and answers 200 with no body. An invalid token gets 400, saying nothing about which part failed.
  • Recorded in the audit log as unsubscribeNotifications (kind profile), with the token's user as the actor. Like the auth events in models/users.ts, this is an action with no session, whose user is identified only by a credential.

GET on the same URL does nothing but redirect to the frontend page /_unsubscribe?t=. Mail scanners fetch every link in a message, so a GET that acted would unsubscribe people who never asked. This is the same principle as the welcome mail's verify link. That page names the categories (from GET /_api/notifications/unsubscribe/info, which says what a token would do and nothing about whose it is), offers Unsubscribe from these and Stop all notification email (the same POST with scope=all), and links to the Profile screen. The footer link in the mail body points to this page.

Deliverability:

  • Gmail and Yahoo require both List-Unsubscribe headers to be covered by the DKIM signature. Nodemailer's default list covers List-Unsubscribe and not List-Unsubscribe-Post, so the transport signs its own list (DKIM_SIGNED_HEADERS in models/mail.ts), which is the default plus the second header.
  • Gmail honours one-click only for an https URL. A site served over plain http still gets the header, which works as an ordinary link elsewhere. Admin → Notifications warns about it (§12.2).

10.5 Leaving room for hourly and daily digests

Not built in the first version, but expected to follow: a user choosing Immediate, Hourly or Daily delivery for their notification email. Nothing built now should have to be reshaped when it comes. What the design already does for it, and what adding it involves:

What is in place from the start

  • The drain only looks at emailAfter. It never asks why a row is due, so a digest is nothing more than a later emailAfter. Claiming, grouping, rendering and marking rows sent stay the same.
  • emailAfterFor is the only place that computes it (§10.2). A schedule is a change to that one function.
  • The digest template already exists. notificationDigest takes any number of entries, grouped by category, with the "and N more" cap. A daily digest is the same mail with more in it.
  • Quiet-until-read already holds back repeats. An entry that went out in a digest stays sent however many more events land on it, so the next digest does not repeat it. It reappears only if it was read and a new entry was started.
  • The unsubscribe token already covers several categories (§10.4), which a digest needs.
  • One mail per (user, site) (§10.3) is the right unit for a digest as well.

What adding it involves

  • A per-user preference, global like the rest (§5): emailSchedule (immediate | hourly | daily) and, for daily, the hour to send. It lives in users.prefs beside timezone rather than in userNotificationPrefs. It belongs to the user, not to a category, and the fan-out already reads each candidate's row for the locale (§7.2), so it costs no extra query. It appears as one Email delivery selector above the grid on Profile → Notifications.
  • emailAfterFor gains the schedule. Hourly is the next top of the hour. Daily is the next occurrence of the chosen hour in the user's prefs.timezone, falling back to the site's, then UTC. That is computed with Temporal.ZonedDateTime so a daylight-saving change does not move the slot by an hour, then converted back to an instant for the column.
  • Some categories skip the digest. A category definition gains digest: boolean, defaulting to true. A mention, a reply or a review request (and any later admin alert) is something a person is waiting on, so the proposal is that those stay immediate even for a daily-digest user. The implementer decides that list when the feature is built.
  • Changing the schedule reschedules what is pending. Saving the preference recomputes emailAfter for that user's pending rows, so switching from daily to immediate does not hold back what has already queued.
  • Spreading the load. Thousands of users on the default daily hour would all fall due in the same minute. emailAfterFor therefore adds a fixed offset per user within the hour, derived from the user ID, so the same user always lands at the same minute and the mail runs see a steady trickle instead of one spike. mailBatchSize and the drain re-running itself while rows remain handle the rest.
  • Retention. A daily digest needs entries to survive at least a day before they are sent. retentionDays is already far longer, but the Admin → Notifications screen should refuse a value under 2 once digests exist.

11. The site switch

features.notifications, on by default, under Admin → General → Features, beside features.comments.

Turned off for a site:

  • emit writes nothing for that site's events (§6.1).
  • Pending emails for that site are marked skipped by the drain rather than sent.
  • The inbox entry, the badge and the bell's "you will be notified" wording are hidden on that site. The Watching list stays, since watching is still a list a user keeps.
  • Existing entries are kept, not deleted. Turning the switch back on shows them again.
  • Preferences are untouched, since they are global (§5.2).

Instance-scoped categories ignore it. None ships at launch, but the admin categories planned (§13) include some that belong to no site (a registration, since users are per instance). Their entries have siteId = null, appear in every site's inbox, and their email links use the first site for now. Nothing records which site a person last used; the first instance-scoped category is the one to decide whether that is worth recording.


12. Administration and housekeeping

12.1 Instance settings

Settings an administrator sets once for the whole instance. They are not user preferences and not the per-site switch (§11).

Key Default What Where
notifications.retentionDays 60 Entries older than this are purged, read or not Settings blob, edited on Admin → Notifications
notifications.emailDelay 3m The window of §10.2 Same
notifications.mailBatchSize 100 Users per mail run. Raise it for a mail server that can take more, lower it for one that throttles. The API refuses anything outside 1–1000 Same
notifications.sections email for each of watching, discussions, reviews, everything What each section of Profile → Notifications may deliver: off, inApp or email. A master switch over the users' choices, never written into them (§12.3) Same
notifications.unsubscribeSecret generated on boot when missing Signs unsubscribe tokens (§10.4) Settings blob, never returned by the API

12.2 Admin → Notifications

pages/AdminNotifications.vue at /_admin/notifications, listed in the System section of AdminLayout.vue's sidebar. It goes in alphabetical order between Metrics and Rendering, behind manage:system like its neighbours, with fluent-topic-push-notification.svg from _assets/icons/ as its icon. Its strings go under admin.notifications.*.

GET / PUT /_api/system/notifications, behind manage:system. A save is audited (kind admin, action updateNotificationSettings) with the fields that changed, as the other configuration routes are. The screen has two cards:

  • Settings: retention, the email delay and the mail batch size.
  • Categories: one row per section, each Off / In-App Only / In-App + Email (§12.3).
  • Status, read-only and answered by the same GET:
    • the outbox backlog (unprocessed events and the age of the oldest one), which is what shows that fan-out is falling behind;
    • emails pending, sent and failed over the last 24 hours;
    • whether outgoing mail is configured, linking to Admin → Mail when it is not;
    • a warning when a site is served over plain http, where Gmail ignores one-click unsubscribe (§10.4).

This is also where any later instance-wide setting goes, for example turning a category off for the whole instance.

12.3 What a section may deliver

notifications.sections lets an administrator narrow each section of Profile → Notifications for the whole instance: off, inApp or email (the default, base.yml). It is a master switch laid over each person's choices, never written into them — nothing in userNotificationPrefs changes, so a section turned off and back on finds everybody's choices where they left them, defaults included. deliveryFor(section) in notifications/index.ts is the one reading of it, and every place that acts on a choice asks it too:

  • The Profile screen. A category in an off section is not offered (visibleCategories); one in an inApp section is offered with emailAllowed: false, and its Email switch is hidden. setPrefs ignores an email choice it is sent for such a category, so the stored one survives a save. Turn off all emails is the exception, and deliberately: it sends stopAllEmail, which turns email off for every category there is — hidden or not offered included — so that somebody who asked for no email gets none when a section is turned back on.
  • emit() writes nothing for an event whose every category is in an off section.
  • The fan-out skips a category in an off section — leaving the person to whichever lower category of the same event reaches them, as their own choice would — and writes no email for one in an inApp section.
  • The mail drain treats a category whose section is not email as email turned off, so an email already waiting when the switch moved is skipped rather than sent.

Entries already in somebody's inbox are left there; they are purged with everything else.

12.4 Purging

tasks/simple/purge-notifications.ts runs daily off SYSTEM_SCHEDULE. It deletes entries past retention and processed outbox events older than a day, in batches of 10,000, checking signal between batches.


13. Adding a category

Worked example: telling the people who look after storage that a git sync failed.

  1. notifications/categories/storageSyncFailed.ts: scope: 'site', origins: ['user', 'import', 'bulk'] (the event comes from a scheduled job, which counts as whatever origin it ran under, and all are fine here), defaults in-app ✓ email ✓, accessCheck: null, visibleTo: (actor) => actor.permissions.includes('manage:storage'), recipients: users in a group whose permissions contain manage:storage, and groupKey: storageSyncFailed:<targetId>, so a target failing every five minutes is one entry until somebody looks at it.
  2. Add its key to NOTIFICATION_CATEGORIES.
  3. Add WIKI.models.notifications.emit('storage:syncFailed', …) in storage.recordState, only on a transition from healthy to error rather than on every failure.
  4. Add notifications.categories.storageSyncFailed.*, notifications.messages.storageSyncFailed.* and its mail strings to en.json.

Nothing else changes: the Profile screen, fan-out, dedup, coalescing, mail and unsubscribe all come from the registry.

An event anyone can trigger must coalesce, or anyone can fill an administrator's mailbox. A failed-login category is the clear case: the login endpoint is open to the world, which is why failed logins are not audited. It would need a groupKey per account, so repeated failures bump a single entry, which the quiet-until-read cadence (§10.2) then holds to a single email.


14. Behaviour worth pinning down

  • The actor is never a recipient, including when they watch the page they just edited.
  • Inactive and system accounts are skipped, and so is the guest account. Unverified accounts get in-app entries but no email, because the address has not been confirmed.
  • A comment edited to add a mention notifies only the handles that were not in the previous version. Re-saving a comment does not mention everyone in it again.
  • Replies go to the parent's author only, not to everyone else in the thread. Someone who wants to follow a whole discussion watches the page.
  • A scheduled page going live at its publishStartDate is not an event. Visibility is worked out when the page is read, and no job flips anything at that time. Setting or changing the window is a scheduled variant; the moment it takes effect is silent.
  • pageCreated fires on creation whatever the publish state, with the state in the snapshot ("created as a draft"). That matches what the recipient can see: any signed-in session sees unpublished pages where it has read:pages. It does not fire again when the page is later published.
  • Password-locked pages notify like any other, by title and path, which is what the Watching list already shows. No content is ever part of an entry.
  • A user deleted with entries pending takes them with them (on delete cascade).

15. Performance

Where Cost Why it holds
The request One INSERT (with a sub-select for a deletion) No recipient is resolved, no access is checked, no SMTP
Jobs About one dispatchNotifications per second per instance under load, and one per minute otherwise Debounced kick (§6.1)
Candidates Proportional to the audience Watchers, reviewer groups and mentions are indexed lookups. Opt-ins come from a partial index (§5.1). The users table is never scanned
Access One rule evaluation per distinct group set §7.2
Writes Multi-row inserts of 500 Coalescing keeps a busy page at one row per watcher
Badge Index-only, capped at 100 rows, 304 when unchanged, no polling from hidden tabs §9.1, §9.2
Mail One mail per user and site per drain Digesting is part of the drain, not an extra step
Timeouts None to hit Fan-out resumes from a cursor (§7.6), and the purge works in batches
HA Any instance drains SKIP LOCKED on both queues. No instance-local state except the debounce timer

16. Suggested order

  1. Foundation: the three tables and their migration (npm run db-generate -- --name=notifications), the registry with the seven categories, userNotificationPrefs with its API, ProfileNotifications.vue, features.notifications in General → Features, and the notifications settings blob, with unsubscribeSecret generated by a startup check.
  2. In-app: emit() with origins, the debounce and the emit sites, the deletion snapshot, dispatch-notifications.ts, the inbox API, stores/notifications.js with polling, the HeaderNav badge, and InboxMessages.vue. Usable on its own: an instance with no mail configured is complete at this point.
  3. Email: the MailSite refactor, both templates and their strings, send-notification-mail.ts with the quiet-until-read cadence and mark-read-on-view, the unsubscribe token, routes and /_unsubscribe page, the DKIM check, and Admin → Notifications (§12.2).
  4. Housekeeping: purge-notifications.ts and its SYSTEM_SCHEDULE entries.
  5. Later: push (§9.3), hourly and daily digests (§10.5), and the admin categories (§13).

Verification for each phase: npm run typecheck and npx oxlint in backend/, npm run build in frontend/. Phases 2 and 3 are worth a throwaway instance. Mailpit is the mail server for development: it receives every message without delivering any, which shows the digest, both bodies and the List-Unsubscribe headers. Script two accounts, one watching a page the other edits ten times, and check that one entry with a count of ten comes out. Then curl the one-click POST with the token from the mail.