Spartan/wiki
1
0 Fork 0
mirror of https://github.com/requarks/wiki.git synced 2026-10-12 05:54:49 -05:00
Code Issues Packages Projects Releases Wiki Activity

fix: encode paths to hex in ltrees to support non-latin chars

Browse Source
This commit is contained in:
NGPixel committed 2026-10-08 00:39:14 -04:00
1 parent f4c60a2e1a
commit b868bf7b7f
38 files changed
+18525 -211
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

CLAUDE.md
+44 -7
View File
@@ -3,13 +3,20 @@
Next-generation open source wiki. This is the **3.x development branch** — incomplete, unstable, and
with no upgrade path from 2.x. AGPL-3.0.
**Nothing here has to stay compatible with an existing installation.** Nobody is expected to be
running an earlier state of this branch, so do not write migration shims, legacy-value fallbacks,
deprecated aliases or "old data may still contain X" handling. Change the shape, change the callers,
and delete the old path — a fallback for a case that cannot occur is dead code that still has to be
read, tested and reasoned about. This applies to db columns, API payloads, stored settings and
config keys alike; only real migrations under `backend/db/migrations/` are exempt, because Drizzle
needs the history to get a live dev database to the current schema.
**A database created by committed code has to be brought forward by a migration.** Anything that
has been committed may be running somewhere, so when a change alters what is stored — a column, the
format of a value, the meaning of a setting — an installation on an earlier commit must reach the new
shape through a migration under `backend/db/migrations/`. That includes rewriting data, not only
schema: a format change gets a migration that converts the existing rows. Converting the data is the
point — the code is not to carry fallbacks for the old format alongside the new one; once the
migration has run there is one shape, and the code reads only that.
**Nothing has to stay compatible with code that was never committed.** A shape introduced in
uncommitted work, which only a developer's own dev database can hold, can be changed freely: change
the shape, change the callers, and delete the old path, with no migration shim, legacy-value
fallback, deprecated alias or "old data may still contain X" handling — a fallback for a case that
cannot occur is dead code that still has to be read, tested and reasoned about. This applies to db
columns, API payloads, stored settings and config keys alike.
The one sanctioned exception is a value an existing installation **cannot** have and no static
default can stand in for — a secret generated per installation, say. That goes in a **startup check**
@@ -692,6 +699,36 @@ Consequences worth knowing:
[Utilities and dates](#utilities-and-dates); the `lodash-es` and `luxon` still present in older
files are on their way out.
### Page paths and the tree
A path segment — a folder name, a page's own name, an asset's file name — may be written in any
script: letters, digits and combining marks of any language, plus the hyphen, which may separate
them but never start or end a segment (`isValidPathSegment` in `helpers/common.ts`, mirrored in the
frontend's `helpers/pagePaths.js`), at most
`MAX_PATH_SEGMENT_BYTES` (240) UTF-8 bytes. Always lowercase and NFC-normalized. Nothing that means
something in a URL, a file system or a git pathspec is allowed, which is why a path can be
concatenated into a link or a file path without escaping.
- **Written and looked up are two different normalizations.** `normalizeNewPagePath` is for a path
being written (create, move, folder create or rename): it turns every run of spaces, underscores
and hyphens into a single hyphen, then drops any hyphen left at either end of a segment.
`normalizePagePath` is for a lookup and turns only spaces into hyphens, because pages written before
the underscore rule keep their `_` and must stay reachable. A move or rename that asks for a page's
or folder's current name again gets exactly that name back (`normalizePath(input, current)` in
`models/pages.ts`, `pathSegmentToWrite` in the frontend), so editing only a title never moves
anything.
- **ltree labels are hex, never the name.** `tree.folderPath` is an ltree whose every label is
`encodeTreeLabel(name)`, the hex of the name's UTF-8 bytes, while `fileName` holds the name itself.
This is because ltree accepts in a label only what the database's locale calls alphanumeric (ASCII
on a `C` database, and never a combining mark). `encodeTreePath` and `decodeTreePath` convert a
whole slash path. Neither is idempotent, so encode a human path once and decode a value read off a
row once. A query that builds a child's path from a row's own `fileName` in SQL wraps it in
`treeLabelSql`, never the bare column. `childPathOf` and `splitPath` in `models/tree.ts` are the
JavaScript side of the same rule.
- **A URL path is percent-encoded and a page path is not.** Decode with `decodeUrlPath` (backend and
frontend) before normalizing or hashing anything that came from a request line, an `href` or
`route.path`. The router never decodes a path for you.
### Drawings (the Excalidraw editor)
A page whose whole body is one drawing, written with `excalidraw` and stored as the `.excalidraw`
backend/api/assets.ts
+3 -6
View File
@@ -1,7 +1,7 @@
import type { FastifyInstance, FastifyRequest } from 'fastify'
import { audit } from '../helpers/audit.ts'
import { decodeTreePath, normalizeFolderPath } from '../helpers/common.ts'
import { attachmentDisposition, decodeTreePath, normalizeFolderPath } from '../helpers/common.ts'
import { INLINE_EXTS } from '../models/assets.ts'
const assetIdParam = {
@@ -159,7 +159,7 @@ async function routes(app: FastifyInstance) {
? await WIKI.models.tree.getFolderById(req.query.folderId)
: null
const folderPath = req.query.folderId ? null : normalizeFolderPath(req.query.folderPath)
const parentPath = folder ? (decodeTreePath(folder.folderPath ?? '') ?? '') : ''
const parentPath = folder ? decodeTreePath(folder.folderPath) : ''
const destination = folder
? [parentPath, folder.fileName].filter(Boolean).join('/')
: (folderPath ?? '')
@@ -294,10 +294,7 @@ async function routes(app: FastifyInstance) {
}
if (download) {
reply.header(
'Content-Disposition',
`attachment; filename="${encodeURIComponent(asset.fileName)}"`
)
reply.header('Content-Disposition', attachmentDisposition(asset.fileName))
}
// -> The bytes came from a user, so the browser must take the type at its word rather than
// looking for something more interesting in them
backend/api/pages.ts
+1 -1
View File
@@ -1201,7 +1201,7 @@ async function routes(app: FastifyInstance) {
path: {
type: 'string',
maxLength: 255,
pattern: '^/?[a-zA-Z0-9-_/]*$'
pattern: '^/?[\\p{L}\\p{M}\\p{N}\\s_/-]*$'
},
locale: {
type: 'string',
backend/api/schemas/page.ts
+3 -2
View File
@@ -27,8 +27,9 @@ export async function registerSchemas(app: FastifyInstance): Promise<void> {
path: {
type: 'string',
maxLength: 255,
pattern: '^/?[a-zA-Z0-9-_/]*$',
description: 'Where the page lives, without a leading slash. Lowercased when stored.'
pattern: '^/?[\\p{L}\\p{M}\\p{N}\\s_/-]*$',
description:
'Where the page lives, without a leading slash. Lowercased when stored, with spaces and underscores turned into hyphens.'
},
title: {
type: 'string',
backend/api/schemas/tree.ts
+3 -2
View File
@@ -164,8 +164,9 @@ export async function registerSchemas(app: FastifyInstance): Promise<void> {
type: 'string',
minLength: 1,
maxLength: 255,
pattern: '^[a-z0-9-]+$',
description: "The folder's own path segment, as it appears in a URL."
pattern: '^[\\p{L}\\p{M}\\p{N}\\s_-]+$',
description:
"The folder's own path segment, as it appears in a URL. Lowercased when stored, with spaces and underscores turned into hyphens."
},
title: {
type: 'string',
backend/api/tree.ts
+6 -5
View File
@@ -1,7 +1,7 @@
import type { FastifyInstance, FastifyRequest } from 'fastify'
import { TREE_ORDER_BY, type TreeItemType, type TreeOrderBy, type TreeRow } from '../models/tree.ts'
import { audit } from '../helpers/audit.ts'
import { decodeTreePath, normalizeFolderPath } from '../helpers/common.ts'
import { decodeTreePath, normalizeFolderPath, normalizeNewPagePath } from '../helpers/common.ts'
import { actorFrom, unpublishedFor } from './pages.ts'
interface TreeQuery {
@@ -128,7 +128,7 @@ function visibleTreeItems<T extends { type?: string; folderPath?: string; fileNa
function toFolderResponse(folder: TreeRow) {
return {
...folder,
folderPath: decodeTreePath(folder.folderPath ?? '') ?? '',
folderPath: decodeTreePath(folder.folderPath),
childrenCount: folder.meta?.children ?? 0,
// -> Absent rather than zero on a folder nobody has coloured; see `setFolderColor`
...(folder.meta?.hue ? { hue: folder.meta.hue } : {})
@@ -136,7 +136,7 @@ function toFolderResponse(folder: TreeRow) {
}
function folderPathOf(folder: { folderPath?: string | null; fileName: string }): string {
const parent = decodeTreePath(folder.folderPath ?? '') ?? ''
const parent = decodeTreePath(folder.folderPath)
return parent ? `${parent}/${folder.fileName}` : folder.fileName
}
@@ -626,7 +626,8 @@ async function routes(app: FastifyInstance) {
const parent = await WIKI.models.tree.getFolderById(req.body.parentId)
parentPath = parent ? folderPathOf(parent) : parentPath
}
const target = [parentPath, req.body.pathName].filter(Boolean).join('/')
// -> Under the name it will be created as, which is what a rule written for it names
const target = [parentPath, normalizeNewPagePath(req.body.pathName)].filter(Boolean).join('/')
const locale = req.body.locale ?? defaultLocale(req.params.siteId)
if (!mayOnFolder(req, 'manage:pages', target, locale)) {
return reply.forbidden('You are not allowed to create a folder here.')
@@ -903,7 +904,7 @@ async function routes(app: FastifyInstance) {
!mayOnFolder(
req,
'manage:pages',
[destination, req.body.pathName].filter(Boolean).join('/'),
[destination, normalizeNewPagePath(req.body.pathName)].filter(Boolean).join('/'),
destinationLocale
)
) {
backend/controllers/files.ts
+2 -4
View File
@@ -1,3 +1,4 @@
import { attachmentDisposition } from '../helpers/common.ts'
import { INLINE_EXTS } from '../models/assets.ts'
import type { FastifyInstance } from 'fastify'
@@ -107,10 +108,7 @@ async function routes(app: FastifyInstance) {
}
if (download) {
reply.header(
'Content-Disposition',
`attachment; filename="${encodeURIComponent(asset.fileName)}"`
)
reply.header('Content-Disposition', attachmentDisposition(asset.fileName))
}
// -> Set by hand because the body may be a stream, which Fastify would otherwise send chunked —
// and a download with no length is a download with no progress bar
backend/db/migrations/20261008040447_tree-folderpath-hash/migration.sql
+2
View File
@@ -0,0 +1,2 @@
DROP INDEX "tree_folderpath_idx";--> statement-breakpoint
CREATE INDEX "tree_folderpath_idx" ON "tree" USING hash ("folderPath");
backend/db/migrations/20261008040447_tree-folderpath-hash/snapshot.json
+8930
View File
File diff suppressed because it is too large. Load diff
backend/db/migrations/20261008040453_tree-hex-labels/migration.sql
+10
View File
@@ -0,0 +1,10 @@
-- Every ltree label in the tree becomes the hex of the UTF-8 bytes of the folder name it used to be
-- written as (`encodeTreeLabel` in helpers/common.ts), so that a folder name may hold characters an
-- ltree label cannot. Each label is converted on its own: the path keeps its depth and its order, and
-- only how each level is spelled changes. The root's empty path has no labels and is left alone.
UPDATE "tree"
SET "folderPath" = (
SELECT string_agg(encode(convert_to("label", 'UTF8'), 'hex'), '.' ORDER BY "position")
FROM unnest(string_to_array("tree"."folderPath"::text, '.')) WITH ORDINALITY AS "labels"("label", "position")
)::ltree
WHERE nlevel("folderPath") > 0;
backend/db/migrations/20261008040453_tree-hex-labels/snapshot.json
+8930
View File
File diff suppressed because it is too large. Load diff
backend/db/schema.ts
+6 -2
View File
@@ -1444,7 +1444,9 @@ export const tree = pgTable(
{
id: uuid().primaryKey().defaultRandom(),
// -> Genuinely hierarchical, and queried as such with `<@`, `@>` and lquery: this is what ltree is
// for. The locale beside it is not, and is a plain string.
// for. The locale beside it is not, and is a plain string. Each label is a folder name ENCODED
// (`encodeTreeLabel`), never the name itself — what ltree accepts in a label depends on the
// database's locale, and a path segment's characters are no business of the database's.
folderPath: ltree('folderPath'),
fileName: varchar({ length: 255 }).notNull(),
hash: varchar({ length: 255 }).notNull(),
@@ -1465,7 +1467,9 @@ export const tree = pgTable(
.references(() => sites.id)
},
(table) => [
index('tree_folderpath_idx').on(table.folderPath),
// -> Hash rather than btree for the equality lookups a folder listing makes: a btree entry may not
// exceed a third of a page, and a deep path of encoded non-Latin names gets there
index('tree_folderpath_idx').using('hash', table.folderPath),
index('tree_folderpath_gist_idx').using('gist', table.folderPath),
index('tree_fileName_idx').on(table.fileName),
index('tree_hash_idx').on(table.hash),
backend/helpers/appShell.ts
+9 -2
View File
@@ -2,7 +2,14 @@ import * as cheerio from 'cheerio'
import type { FastifyRequest } from 'fastify'
import type { AnalyticsInjections } from '../models/analytics.ts'
import type { PageDescription } from '../models/pages.ts'
import { htmlEscape, isPageUrl, normalizePagePath, originOf, splitLocalePath } from './common.ts'
import {
decodeUrlPath,
htmlEscape,
isPageUrl,
normalizePagePath,
originOf,
splitLocalePath
} from './common.ts'
/**
* What the app shell is enriched with before it is handed to a client that will not run it.
@@ -198,7 +205,7 @@ function resolvePagePath(
return {
locale: split?.locale ?? locales?.primary ?? 'en',
// -> The site root is the `home` page rather than an empty path, the same as everywhere else
path: normalizePagePath(below) || HOME_PATH,
path: normalizePagePath(decodeUrlPath(below)) || HOME_PATH,
isRoot: ROOT_PATHS.has(below)
}
}
backend/helpers/common.ts
+153 -16
View File
@@ -5,6 +5,7 @@ import mime from 'mime'
import fs from 'node:fs'
import path from 'node:path'
import type { FastifyReply, FastifyRequest } from 'fastify'
import { sql, type SQL, type SQLWrapper } from 'drizzle-orm'
export interface Deferred<T = void> {
resolve: (value: T) => void
@@ -141,35 +142,128 @@ export function htmlEscape(value: string): string {
}
/**
* Decode a tree path
* One path segment as the ltree label the tree stores it under: the hex of its UTF-8 bytes.
*
* @param str String to decode
* @returns Decoded tree path
* An ltree label may only hold what the database's locale calls alphanumeric, plus `_` and `-` — on a
* `C` database that is ASCII, on any other it is whatever its C library says, and even a UTF-8 locale
* refuses combining marks (Hindi, Thai, decomposed kana). A path segment is none of those things'
* business, so the label is an encoding of it that every database accepts and every database reads
* back the same. Hex rather than an escape scheme because postgres can produce it too —
* `treeLabelSql` is the same function for a column, which queries that build a child's path from
* its parent's row depend on — and because it never contains anything lquery would read as syntax.
*
* The segment is taken as given: callers hand in a name already lowercased and NFC-normalized, which
* is the form `fileName` is stored in and what `treeLabelSql` encodes.
*/
export function decodeTreePath(str?: string | null): string | undefined {
return str?.replaceAll('.', '/')
export function encodeTreeLabel(segment: string): string {
return Buffer.from(segment, 'utf8').toString('hex')
}
/**
* Encode a tree path
* The path segment an ltree label encodes. The exact inverse of `encodeTreeLabel`.
*/
export function decodeTreeLabel(label: string): string {
return Buffer.from(label, 'hex').toString('utf8')
}
/**
* `encodeTreeLabel` in SQL, for a query that builds a path from a row's own `fileName`.
*
* @param str String to encode
* @returns Encoded tree path
* `convert_to` is what makes it byte-for-byte the same as `Buffer.from(…, 'utf8')`: the database's
* encoding is not assumed to be UTF-8, the conversion says so explicitly.
*/
export function treeLabelSql(fileName: SQLWrapper): SQL {
return sql`encode(convert_to(${fileName}, 'UTF8'), 'hex')`
}
/**
* A slash-separated folder path as the ltree the tree stores it under.
*
* Each segment is lowercased and NFC-normalized before it is encoded, which is the form every folder
* name is stored in — so a lookup does not care how the path it was handed was spelled. Empty
* segments are dropped rather than encoded as labels ltree would refuse.
*
* Never hand it a path that is already encoded: hex has no slashes, so the whole dotted path would be
* taken as one segment and encoded a second time.
*/
export function encodeTreePath(str?: string | null): string {
return str?.toLowerCase()?.replaceAll('/', '.') || ''
return (str ?? '')
.split('/')
.filter(Boolean)
.map((segment) => encodeTreeLabel(segment.toLowerCase().normalize('NFC')))
.join('.')
}
/**
* Reduce a page path to the single form it is stored, addressed and looked up under.
* An ltree path from the tree as the slash-separated path it encodes. Empty for the site root.
*
* A path is a URL, and a URL that differs only in casing or in how a space was encoded is the same
* page as far as anyone reading the wiki is concerned — so there is one spelling, and everything
* that takes a path from a human or from page content passes it through here first. Wrapping slashes
* go, runs of whitespace become a single hyphen, and what is left is lowercased.
* Only ever for a value read off a row, or built by `encodeTreePath` / `encodeTreeLabel` — decoding a
* path that is already slash-separated reads it as hex and returns garbage.
*/
export function decodeTreePath(str?: string | null): string {
return str ? str.split('.').map(decodeTreeLabel).join('/') : ''
}
/**
* A `Content-Disposition` that downloads a file under its own name, whatever script it is written in.
*
* What it does not do is decide whether the result is *allowed*: the characters a path may contain
* are the page model's rule to enforce, on the normalized form.
* RFC 6266: `filename*` carries the name as UTF-8, which every current browser reads, and `filename`
* is the fallback for anything that does not. That one is held to printable ASCII, with everything
* else replaced rather than percent-encoded — a browser reading `filename` shows a `%E6` literally.
*/
export function attachmentDisposition(fileName: string): string {
const fallback = fileName.replaceAll(/[^\x20-\x7e]|["\\]/gu, '_')
// -> `encodeURIComponent` leaves `'()*` alone, and RFC 5987 does not allow them unencoded
const encoded = encodeURIComponent(fileName).replaceAll(
/['()*]/g,
(char) => `%${char.charCodeAt(0).toString(16).toUpperCase()}`
)
return `attachment; filename="${fallback}"; filename*=UTF-8''${encoded}`
}
/**
* Longest a single path segment may be, in UTF-8 bytes — a folder name, a page's own name, an asset's
* file name.
*
* Bytes rather than characters, because each limit it stands in front of counts bytes: a file system
* name is 255 of them on ext4 and most others, and a page is stored as its name plus an extension of
* up to five; and an ltree label is 1000 characters, which hex spends two of per byte. 240 clears both
* — about 80 characters of Japanese, 240 of ASCII.
*/
export const MAX_PATH_SEGMENT_BYTES = 240
/**
* What one segment of a page or folder path may be made of: letters and digits of any script, the
* combining marks some scripts cannot be written without, and the hyphen — between them, never at
* either end.
*
* Nothing that means something in a URL, a file system or a git pathspec — no `/`, `\`, `.`, `%`,
* `?`, `#`, `*` or whitespace — which is what lets a path be concatenated into a link, a file path or
* an object key without escaping it first. The hyphen is kept off the ends because a leading one is
* an option to every command-line tool a path is handed to, and a trailing one is a separator with
* nothing after it. Checked on the normalized form, so uppercase letters never reach it.
*/
const PATH_SEGMENT = /^[\p{L}\p{M}\p{N}](?:[\p{L}\p{M}\p{N}-]*[\p{L}\p{M}\p{N}])?$/u
/**
* Whether a normalized segment is one a path may be written with. See `PATH_SEGMENT`.
*/
export function isValidPathSegment(segment: string): boolean {
return PATH_SEGMENT.test(segment) && Buffer.byteLength(segment, 'utf8') <= MAX_PATH_SEGMENT_BYTES
}
/**
* Reduce a page path to the single form it is looked up under.
*
* A path is a URL, and a URL that differs only in casing, in how a space was encoded, or in how an
* accented letter was composed is the same page as far as anyone reading the wiki is concerned — so
* there is one spelling, and everything that takes a path from a human or from page content passes it
* through here first. Wrapping slashes go, runs of whitespace become a single hyphen, and what is left
* is lowercased and NFC-normalized (macOS hands out decomposed names, which would otherwise be a
* different path to the same eye).
*
* A path being WRITTEN goes through `normalizeNewPagePath` instead, which is this and then some: the
* two differ in `_`, which a path written today never has and a path written before may still.
*/
export function normalizePagePath(input?: string | null): string {
return (input ?? '')
@@ -178,6 +272,49 @@ export function normalizePagePath(input?: string | null): string {
.replace(/\/+$/, '')
.replaceAll(/\s+/g, '-')
.toLowerCase()
.normalize('NFC')
}
/**
* A URL path as the page path it spells: each segment percent-decoded.
*
* What a request line or an `href` carries is encoded — a browser percent-encodes every character of
* `/にほんご` before sending it — while a page path is stored as the characters themselves. So a URL
* path is decoded before it is normalized and hashed, or it names a page that is not there. A segment
* that will not decode is not one a page path could have produced, and is left as it is. Mirrored by
* `decodeUrlPath` in the frontend's `helpers/pagePaths.js`.
*/
export function decodeUrlPath(urlPath: string): string {
return urlPath
.split('/')
.map((segment) => {
try {
return decodeURIComponent(segment)
} catch {
return segment
}
})
.join('/')
}
/**
* Reduce a path that is about to be written — a page created or moved, a folder created or renamed —
* to the form it is stored under.
*
* `normalizePagePath`, with underscores turned into hyphens as whitespace already is: a path uses ONE
* hyphen to separate words, whichever of the three the author typed and however many of them, so
* `part 1 - intro` is `part-1-intro`. And since a hyphen only ever separates, one left at either end
* of a segment — from `_draft`, from `notes /` — is dropped. Only for a path being written, because
* pages written before those rules kept their `_` (or `--`), and looking one of them up through this
* would look for a page that is not there.
*
* What it does not do is decide whether the result is *allowed*: see `isValidPathSegment`.
*/
export function normalizeNewPagePath(input?: string | null): string {
return normalizePagePath((input ?? '').trim().replaceAll(/[\s_-]+/g, '-'))
.split('/')
.map((segment) => segment.replaceAll(/^-+|-+$/g, ''))
.join('/')
}
/**
backend/helpers/pageLinks.ts
+9 -2
View File
@@ -1,6 +1,12 @@
import path from 'node:path'
import * as cheerio from 'cheerio'
import { isPageUrl, normalizePagePath, splitLocalePath, stripPageExtension } from './common.ts'
import {
decodeUrlPath,
isPageUrl,
normalizePagePath,
splitLocalePath,
stripPageExtension
} from './common.ts'
/**
* Reading a link the way the reader's browser will.
@@ -241,7 +247,8 @@ function resolvePage(href: string, urlPath: string, targetSiteId: string): Resol
targetLocale: split?.locale ?? locales?.primary ?? 'en',
// -> The site root is the page stored at the empty path, which is what `normalizePagePath` makes
// of `/` -- and of `/fr` once its prefix has come off above
targetPath: normalizePagePath(split?.path ?? withoutExtension),
// -> Decoded, since an `href` written by markdown percent-encodes every character outside ASCII
targetPath: normalizePagePath(decodeUrlPath(split?.path ?? withoutExtension)),
targetRef: null
}
}
backend/locales/en.json
+3 -3
View File
@@ -2802,8 +2802,8 @@
"fileman.folderDuplicating": "Duplicating folder...",
"fileman.folderDuplicatingHint": "Every page and file under it is being copied. This may take a moment for a large folder.",
"fileman.folderFileName": "Path Name",
"fileman.folderFileNameHint": "URL friendly version of the folder name. Must consist of lowercase alphanumerical or hypen characters only.",
"fileman.folderFileNameInvalid": "Invalid Characters in Folder Path Name. Lowercase alphanumerical and hyphen characters only.",
"fileman.folderFileNameHint": "URL friendly version of the folder name. Letters, numbers and hyphens only. Spaces and underscores become hyphens.",
"fileman.folderFileNameInvalid": "Invalid folder path name. Use letters, numbers and hyphens only, up to 240 bytes.",
"fileman.folderFileNameMissing": "Missing Folder Path Name",
"fileman.folderMove": "Move Folder To...",
"fileman.folderMoving": "Moving folder...",
@@ -3281,7 +3281,7 @@
"pageSaveDialog.loadFailed": "Failed to load folder tree.",
"pageSaveDialog.localeHint": "Which locale the page belongs to.",
"pageSaveDialog.pageTitle": "Page Title",
"pageSaveDialog.pathInvalid": "Invalid Characters in Page Path Name. Lowercase alphanumerical and hyphen characters only.",
"pageSaveDialog.pathInvalid": "Invalid page path name. Use letters, numbers and hyphens only, up to 240 bytes.",
"pageSaveDialog.pathName": "Path Name",
"pageSaveDialog.title": "Save As...",
"pageSaveDialog.titleMissing": "Missing Page Title",
backend/models/assets.ts
+91 -23
View File
@@ -7,6 +7,7 @@ import {
CustomError,
decodeTreePath,
encodeTreePath,
MAX_PATH_SEGMENT_BYTES,
normalizeFolderPath
} from '../helpers/common.ts'
import {
@@ -132,8 +133,10 @@ export interface AssetAtPath extends Asset {}
* Reduce whatever a client called the file to something safe to store, address and serve.
*
* Any directory part is dropped — the folder comes from the request, never from the name — and what
* is left is lowercased down to the characters that survive a URL untouched, which is the same bar
* folder path names are held to.
* is left is held to the bar a path segment is (`isValidPathSegment`), plus the dots an extension
* needs: lowercased and NFC-normalized, whitespace and underscores turned into hyphens and runs of
* them into one, and anything that is not a letter, a digit or a combining mark of some script
* dropped.
*
* Applied to every upload, with nothing to turn it off: a stored name is a URL, and a path is looked
* up lowercased, so a name that skipped this would be one the site could not serve back.
@@ -142,22 +145,69 @@ export function sanitizeFileName(input: string): string {
const base = path.basename(input.trim().replaceAll('\\', '/'))
const cleaned = base
.toLowerCase()
.replaceAll(/\s+/g, '-')
.replaceAll(/[^a-z0-9._-]/g, '')
// -> A leading dot would make it a hidden file, and a run of them can walk out of the folder
.replace(/^\.+/, '')
.normalize('NFC')
.replaceAll(/[\s_]+/g, '-')
.replaceAll(/[^\p{L}\p{M}\p{N}.-]/gu, '')
// -> One hyphen between words, as a path has: after the removal above, since `a ? b` is `a---b`
// by then
.replaceAll(/-{2,}/g, '-')
// -> A run of dots can walk out of the folder
.replaceAll(/\.{2,}/g, '.')
return cleaned.slice(0, 255)
// -> A hyphen only ever separates two words, so none against a dot -- `photo-.png` -- nor at
// either end, where a leading one is an option to every tool a path is handed to. A leading dot
// would make it a hidden file.
.replaceAll(/-*\.-*/g, '.')
.replace(/^[.-]+/, '')
.replace(/-+$/, '')
return truncateFileName(cleaned)
}
/**
* Cut a file name down to `MAX_PATH_SEGMENT_BYTES`, from the end of its stem so that the extension —
* which decides how it is served — survives. Whole characters only, since the limit is in bytes and a
* character may be four of them.
*/
function truncateFileName(name: string): string {
if (Buffer.byteLength(name, 'utf8') <= MAX_PATH_SEGMENT_BYTES) {
return name
}
const dot = name.lastIndexOf('.')
const ext = dot > 0 ? name.slice(dot) : ''
// -> An "extension" that alone outgrows the limit is not one worth keeping
const suffix = Buffer.byteLength(ext, 'utf8') < MAX_PATH_SEGMENT_BYTES ? ext : ''
const stem = suffix ? name.slice(0, dot) : name
const budget = MAX_PATH_SEGMENT_BYTES - Buffer.byteLength(suffix, 'utf8')
let kept = ''
let used = 0
for (const char of stem) {
used += Buffer.byteLength(char, 'utf8')
if (used > budget) {
break
}
kept += char
}
// -> The cut may land just after a hyphen, which would then end the stem
return `${kept.replace(/-+$/, '')}${suffix}`
}
/**
* A file name as a path looks it up: lowercased and NFC-normalized, nothing more. Not
* `sanitizeFileName`, which is what a name is WRITTEN as — a file uploaded before underscores became
* hyphens is still found under its own.
*/
function lookupFileName(name: string): string {
return name.toLowerCase().normalize('NFC')
}
/**
* The form a file path is cached under.
*
* Matches what the lookup does with it — empty segments dropped, lowercased — so that the spellings
* of a path that reach the same asset share one cache entry instead of each getting their own.
* Matches what the lookup does with it — empty segments dropped, lowercased, NFC — so that the
* spellings of a path that reach the same asset share one cache entry instead of each getting their
* own.
*/
function normalizePath(filePath: string): string {
return filePath.split('/').filter(Boolean).join('/').toLowerCase()
return lookupFileName(filePath.split('/').filter(Boolean).join('/'))
}
/**
@@ -411,7 +461,7 @@ class Assets {
id: occupant.id,
siteId,
locale,
folderPath: decodeTreePath(occupant.folderPath ?? '') ?? '',
folderPath: decodeTreePath(occupant.folderPath),
fileName: occupant.fileName,
title: occupant.title,
fileExt,
@@ -444,7 +494,7 @@ class Assets {
const storedName = entry.fileName
// -> Read off the row rather than from the request: the folder may have just been created, and a
// name that was taken took the next free one
const storedFolderPath = decodeTreePath(entry.folderPath ?? '') ?? ''
const storedFolderPath = decodeTreePath(entry.folderPath)
try {
// -> The metadata row goes in before the bytes, since the database target writes them into it
@@ -751,7 +801,7 @@ class Assets {
return {
...rest,
fileSize: row.fileSize ?? 0,
folderPath: decodeTreePath(row.folderPath ?? '') ?? '',
folderPath: decodeTreePath(row.folderPath),
hasPreview: Boolean(row.hasPreview),
...dimensionMeta(dimensionsOf(meta as Record<string, any>))
} as Asset
@@ -763,14 +813,15 @@ class Assets {
*
* The path lives on the tree row rather than on the asset — the two share an ID — so the lookup
* splits it into the folder and the file the way the tree stores them, the folder as an ltree.
* Both are lowercased, because that is what an upload stored them as.
* Both are lowercased and NFC-normalized, because that is what an upload stored them as.
*
* A path can exist once per locale and the URL carries none, so the site's primary locale wins
* where more than one has a file there. That is also the only one the file manager uploads into.
*/
async getAssetByPath(siteId: string, filePath: string): Promise<AssetAtPath | null> {
const segments = filePath.split('/').filter(Boolean)
const fileName = segments.pop()?.toLowerCase()
const last = segments.pop()
const fileName = last && lookupFileName(last)
if (!fileName) {
return null
}
@@ -815,7 +866,7 @@ class Assets {
return {
...rest,
fileSize: row.fileSize ?? 0,
folderPath: decodeTreePath(row.folderPath ?? '') ?? '',
folderPath: decodeTreePath(row.folderPath),
hasPreview: Boolean(row.hasPreview),
...dimensionMeta(dimensionsOf(meta as Record<string, any>))
} as AssetAtPath
@@ -855,7 +906,7 @@ class Assets {
id: row.id,
siteId: row.siteId,
locale: row.locale,
folderPath: decodeTreePath(row.folderPath ?? '') ?? '',
folderPath: decodeTreePath(row.folderPath),
fileName: row.fileName,
kind: row.kind,
fileSize: row.fileSize ?? 0
@@ -894,7 +945,7 @@ class Assets {
siteId,
actorId,
locale: row.locale,
folderPath: decodeTreePath(row.folderPath ?? '') ?? '',
folderPath: decodeTreePath(row.folderPath),
fileName: row.fileName,
kind: row.kind,
fileSize: row.fileSize ?? 0
@@ -936,7 +987,7 @@ class Assets {
id: row.id,
siteId,
locale: row.locale,
folderPath: decodeTreePath(row.folderPath ?? '') ?? '',
folderPath: decodeTreePath(row.folderPath),
fileName: row.fileName,
kind: row.kind,
fileSize: row.fileSize ?? 0
@@ -1019,7 +1070,20 @@ class Assets {
overwrite?: boolean
dispatch?: boolean
}): Promise<Asset | null> {
const safeName = sanitizeFileName(fileName)
// -> A file uploaded before underscores became hyphens is found under its own name, and one
// uploaded since under the name this file would be written as — so that a tree the wiki
// exported itself comes back in onto the files it came from, rather than beside them
const asWritten = lookupFileName(fileName)
const sanitized = sanitizeFileName(fileName)
const keepsOwnName =
asWritten !== sanitized &&
(await WIKI.models.tree.getEntryAt({
siteId,
locale,
parentPath: folderPath,
fileName: asWritten
})) !== null
const safeName = keepsOwnName ? asWritten : sanitized
if (!safeName) {
return null
}
@@ -1063,7 +1127,7 @@ class Assets {
id: occupant.id,
siteId,
locale,
folderPath: decodeTreePath(occupant.folderPath ?? '') ?? '',
folderPath: decodeTreePath(occupant.folderPath),
// -> The names it is already filed under, not the ones off the file: they only differ by
// sanitization, and the entry is the authority on what it is actually called
fileName: occupant.fileName,
@@ -1088,7 +1152,7 @@ class Assets {
})
// -> Read off the row rather than from the caller: the folder may have just been created, and a
// name that was taken took the next free one
const importedFolderPath = decodeTreePath(entry.folderPath ?? '') ?? ''
const importedFolderPath = decodeTreePath(entry.folderPath)
try {
// -> The metadata row goes in before the bytes, since the database target writes them into it
@@ -1451,7 +1515,11 @@ class Assets {
if (!asset || !entry) {
return null
}
const safeName = fileName === undefined ? asset.fileName : sanitizeFileName(fileName)
// -> Its own name back is no rename, even one written before underscores became hyphens
const safeName =
fileName === undefined || lookupFileName(fileName) === asset.fileName
? asset.fileName
: sanitizeFileName(fileName)
if (!safeName) {
throw new CustomError('assetInvalidFileName', 'This file name cannot be used.')
}
backend/models/blogs.ts
+21 -7
View File
@@ -1,6 +1,11 @@
import { and, eq, inArray, sql, type SQL } from 'drizzle-orm'
import { pages as pagesTable, tree as treeTable, users as usersTable } from '../db/schema.ts'
import { decodeTreePath, encodeTreePath } from '../helpers/common.ts'
import {
decodeTreeLabel,
decodeTreePath,
encodeTreePath,
normalizeNewPagePath
} from '../helpers/common.ts'
import { isBodylessEditor, parseBlogContent, type BlogContent } from './pages.ts'
import type { AccessActor } from './groups.ts'
import type { RulePageRef } from '../helpers/pageRules.ts'
@@ -217,9 +222,9 @@ class Blogs {
*
* **It never fails the caller.** The folder is a convenience and the blog is complete without it,
* so the two ways this can legitimately not work are logged and stepped over rather than raised:
* a folder name is held to `[a-z0-9-]` while a page path may also carry underscores (a blog at
* `my_blog` has no legal folder name), and an ASSET already sitting at that name blocks a folder
* where it would not have blocked the page.
* a page written before underscores became hyphens may still carry one, which a folder name may
* not (a blog at `my_blog` has no legal folder name), and an ASSET already sitting at that name
* blocks a folder where it would not have blocked the page.
*
* @returns Whether a folder was created. False when one was already there, and when one could not be.
*/
@@ -236,7 +241,7 @@ class Blogs {
}): Promise<boolean> {
const encoded = encodeTreePath(path)
const parts = encoded.split('.')
const fileName = parts.at(-1)!
const fileName = decodeTreeLabel(parts.at(-1)!)
const folderPath = parts.slice(0, -1).join('.')
/*
@@ -262,9 +267,18 @@ class Blogs {
return false
}
// -> Refused here rather than by `createFolder`, which would write the name as a new path is
// written — `my-blog`, a folder beside the blog rather than the one its posts sit in
if (normalizeNewPagePath(fileName) !== fileName) {
WIKI.logger.warn(
`Could not create the folder for the blog at /${path}: "${fileName}" is not a valid folder name.`
)
return false
}
try {
await WIKI.models.tree.createFolder({
parentPath: decodeTreePath(folderPath) ?? '',
parentPath: decodeTreePath(folderPath),
pathName: fileName,
title,
locale,
@@ -362,7 +376,7 @@ class Blogs {
const truncated = rows.length > MAX_POSTS
const candidates: Candidate[] = rows.slice(0, MAX_POSTS).map((row) => {
const folderPath = decodeTreePath(row.folderPath ?? '') ?? ''
const folderPath = decodeTreePath(row.folderPath)
return {
id: row.id,
path: folderPath ? `${folderPath}/${row.fileName}` : row.fileName,
backend/models/import.ts
+21 -7
View File
@@ -18,7 +18,12 @@ import {
userGroups as userGroupsTable,
users as usersTable
} from '../db/schema.ts'
import { CustomError, dataPathRoot } from '../helpers/common.ts'
import {
CustomError,
dataPathRoot,
normalizeFolderPath,
normalizeNewPagePath
} from '../helpers/common.ts'
import { GLOBAL_PERMISSIONS, PAGE_PERMISSIONS } from './groups.ts'
import type { GroupRule, GroupRuleMatch } from './groups.ts'
import type { NavigationItem } from './navigation.ts'
@@ -334,6 +339,19 @@ function stringOf(value: unknown, fallback = ''): string {
return typeof value === 'string' ? value : fallback
}
/**
* A 2.x folder path as this wiki writes one: the segments `normalizeFolderPath` keeps, each written
* the way a page path is — which is what puts a folder where the pages imported into it land, since
* those go through the same normalization on their own way in.
*/
function folderPathOf(value: unknown): string {
return normalizeFolderPath(stringOf(value))
.split('/')
.filter(Boolean)
.map(normalizeNewPagePath)
.join('/')
}
/**
* A date off a record, or undefined.
*
@@ -1470,9 +1488,7 @@ class Import {
let skipped = 0
for (const record of records) {
const folderPath = stringOf(record?.path)
.trim()
.replace(/^\/+|\/+$/g, '')
const folderPath = folderPathOf(record?.path)
const locale = localeOf(stringOf(record?.localeCode).trim())
if (!folderPath || !locale) {
warnings.push(
@@ -2119,9 +2135,7 @@ class Import {
const asset = await WIKI.models.assets.adoptStoredFile({
siteId: target.siteId,
locale,
folderPath: stringOf(record?.folderPath)
.trim()
.replace(/^\/+|\/+$/g, ''),
folderPath: folderPathOf(record?.folderPath),
fileName,
data,
authorId: await this.#authorFor(session, record?.authorId),
backend/models/navigation.ts
+10 -8
View File
@@ -1,6 +1,6 @@
import { and, eq, inArray, sql } from 'drizzle-orm'
import { navigation as navigationTable, tree as treeTable } from '../db/schema.ts'
import { CustomError, decodeTreePath } from '../helpers/common.ts'
import { CustomError, decodeTreePath, encodeTreeLabel, treeLabelSql } from '../helpers/common.ts'
export const NAVIGATION_MODES = [
'inherit',
@@ -169,9 +169,9 @@ class Navigation {
FROM tree
WHERE "siteId" = ${siteId}
AND "locale" = ${locale}
AND ("folderPath" || "fileName") @> ${folderPath}::ltree
AND ("folderPath" || ${treeLabelSql(sql`"fileName"`)}) @> ${folderPath}::ltree
AND "navigationMode" IN ('override', 'hide')
ORDER BY nlevel("folderPath" || "fileName") DESC
ORDER BY nlevel("folderPath") DESC
LIMIT 1
`)
const rows = (result.rows ?? result) as any[]
@@ -225,7 +225,7 @@ class Navigation {
and always asking about the page lets them rewrite one handed down from above.
*/
if (mode !== 'inherit') {
const own = decodeTreePath(folderPath) ?? ''
const own = decodeTreePath(folderPath)
return {
navigationId: entry.id,
path: own ? `${own}/${entry.fileName}` : entry.fileName,
@@ -314,7 +314,8 @@ class Navigation {
// which is what makes it the menu every other page in that locale inherits
const isSiteRoot = folderPath === '' && entry.fileName === 'home'
const ownNavId = isSiteRoot ? await this.siteNavId(siteId, entry.locale) : entry.id
const fullPath = folderPath ? `${folderPath}.${entry.fileName}` : entry.fileName
const label = encodeTreeLabel(entry.fileName)
const fullPath = folderPath ? `${folderPath}.${label}` : label
const ancestorId = await this.ancestorNavId(siteId, entry.locale, folderPath)
@@ -404,7 +405,7 @@ class Navigation {
AND tc."locale" = ${entry.locale}
AND tc.tree IN ('page', 'folder')
AND tc."folderPath" <@ ${fullPath}::ltree
AND (tc."folderPath" || tc."fileName") @> tt."folderPath"
AND (tc."folderPath" || ${treeLabelSql(sql`tc."fileName"`)}) @> tt."folderPath"
AND tc."navigationMode" IN ('override', 'hide')
)
`)
@@ -446,7 +447,8 @@ class Navigation {
fileName: string
}): Promise<void> {
const inherited = await this.ancestorNavId(siteId, locale, folderPath)
const fullPath = folderPath ? `${folderPath}.${fileName}` : fileName
const label = encodeTreeLabel(fileName)
const fullPath = folderPath ? `${folderPath}.${label}` : label
// -> The folder itself first: it is not under its own path, so the cascade below passes it over
await WIKI.db
@@ -470,7 +472,7 @@ class Navigation {
AND tc."locale" = ${locale}
AND tc.tree IN ('page', 'folder')
AND tc."folderPath" <@ ${fullPath}::ltree
AND (tc."folderPath" || tc."fileName") @> tt."folderPath"
AND (tc."folderPath" || ${treeLabelSql(sql`tc."fileName"`)}) @> tt."folderPath"
AND tc."navigationMode" IN ('override', 'hide')
)
`)
backend/models/pageGraph.ts
+1 -1
View File
@@ -388,7 +388,7 @@ class PageGraphModel {
const folders: GraphFolder[] = []
for (const row of rows) {
const parent = decodeTreePath(row.folderPath ?? '') ?? ''
const parent = decodeTreePath(row.folderPath)
const path = parent ? `${parent}/${row.fileName}` : row.fileName
if (!wanted.has(path)) {
continue
backend/models/pages.ts
+27 -13
View File
@@ -8,6 +8,9 @@ import {
import {
CustomError,
generatePathHash,
isValidPathSegment,
MAX_PATH_SEGMENT_BYTES,
normalizeNewPagePath,
normalizePagePath,
timingSafeCompare
} from '../helpers/common.ts'
@@ -213,8 +216,6 @@ function sitemapCacheKey(siteId: string): string {
*/
const sitemapBuilds = new Map<string, Promise<SitemapPage[]>>()
/** A page path is what ends up in a URL, so it is held to what reads and routes cleanly. */
const rePagePath = /^[a-zA-Z0-9-_/]*$/
const reAlias = /^[a-zA-Z0-9-_]*$/
/** Fields kept in the `config` blob rather than as columns, and flattened again on the way out. */
@@ -534,15 +535,23 @@ function hasPermission(actor: PageActor, permission: string): boolean {
/**
* Normalize a path to the form that gets stored, and refuse it if what is left is not addressable.
*
* Casing and spaces are corrected rather than rejected — `My Page` is a path someone meant, and it
* means `my-page`. Anything else outside the allowed characters is not something to guess at.
* Casing, spaces and underscores are corrected rather than rejected — `My_Page` is a path someone
* meant, and it means `my-page`. Anything else outside what a segment may hold
* (`isValidPathSegment`) is not something to guess at.
*
* @param current The path the page already has, when this is a move or a restore. Asked for the same
* path back, it gets exactly that path: one written before underscores became hyphens keeps its
* `_`, rather than a rename that only changes the title moving the page somewhere new.
*/
function normalizePath(input: string): string {
const path = normalizePagePath(input)
if (!rePagePath.test(path)) {
function normalizePath(input: string, current?: string): string {
if (current !== undefined && normalizePagePath(input) === current) {
return current
}
const path = normalizeNewPagePath(input)
if (!path.split('/').every(isValidPathSegment)) {
throw new CustomError(
'pageInvalidPath',
'A page path may only contain alphanumeric, hyphen, underscore and slash characters.'
`A page path may only contain letters, numbers, hyphens and slashes, and each part of it be at most ${MAX_PATH_SEGMENT_BYTES} bytes long.`
)
}
return path
@@ -2290,7 +2299,7 @@ class Pages {
return null
}
const existingContent = page.content
const newPath = normalizePath(path)
const newPath = normalizePath(path, page.path)
/*
A move may cross locales — the same page, translated, is the same page moved — so the
destination is a locale AND a path, and everything below asks about the pair rather than about
@@ -2778,7 +2787,7 @@ class Pages {
const meta = deletion.meta
const title = input.title?.trim() || deletion.title
const path = normalizePath(input.path ?? deletion.path)
const path = normalizePath(input.path ?? deletion.path, deletion.path)
const locale = input.locale || deletion.locale
const editor = meta.editor || 'markdown'
const contentType = meta.contentType || EDITOR_CONTENT_TYPES[editor] || 'text'
@@ -3298,21 +3307,26 @@ class Pages {
authorId: string
overwrite?: boolean
}): Promise<Page | null> {
const normalized = normalizePath(path)
// -> A page written before underscores became hyphens is found under its own path, and one written
// since under the path this file would be written as — so that a tree the wiki exported itself
// comes back in onto the pages it came from, rather than beside them
const asWritten = normalizePagePath(path)
const existing = await WIKI.db
.select({ id: pagesTable.id })
.select({ id: pagesTable.id, path: pagesTable.path })
.from(pagesTable)
.where(
and(
eq(pagesTable.siteId, siteId),
eq(pagesTable.locale, locale),
eq(pagesTable.path, normalized)
inArray(pagesTable.path, [asWritten, normalizeNewPagePath(path)])
)
)
.orderBy(desc(sql`${pagesTable.path} = ${asWritten}`))
.limit(1)
if (existing.length > 0 && !overwrite) {
return null
}
const normalized = existing[0]?.path ?? normalizePath(path)
/*
Imported content is rendered with NO script or style permission, whoever ran the import.
backend/models/tree.ts
+72 -46
View File
@@ -5,12 +5,18 @@ import type { RulePageRef } from '../helpers/pageRules.ts'
import { pages as pagesTable, tree as treeTable } from '../db/schema.ts'
import {
CustomError,
decodeTreeLabel,
decodeTreePath,
encodeTreeLabel,
encodeTreePath,
generateHash,
generatePathHash,
isValidPathSegment,
MAX_PATH_SEGMENT_BYTES,
normalizeFolderPath,
normalizePagePath
normalizeNewPagePath,
normalizePagePath,
treeLabelSql
} from '../helpers/common.ts'
import type { PageActor } from './pages.ts'
@@ -146,7 +152,6 @@ export interface TreeRow {
}
/** Folders are addressed by URL, so their file name is restricted to what reads well in one. */
const rePathName = /^[a-z0-9-]+$/
const reTitle = /^[^<>"]+$/
/** Ceiling on how many entries one listing returns, and how deep it may recurse. */
@@ -161,22 +166,43 @@ const MAX_NAME_ATTEMPTS = 100
/**
* The ltree path of a folder's *contents*, i.e. the value its children carry in `folderPath`.
*
* The folder's own name is a `fileName`, which is stored as written, so it is encoded on the way onto
* the end of its parent's path — the two are never the same string, see `encodeTreeLabel`.
*/
function childPathOf(folder: { folderPath?: string | null; fileName: string }): string {
return folder.folderPath ? `${folder.folderPath}.${folder.fileName}` : folder.fileName
const label = encodeTreeLabel(folder.fileName)
return folder.folderPath ? `${folder.folderPath}.${label}` : label
}
/**
* Split an ltree path into the (folderPath, fileName) pair that addresses the entry itself.
* Split an ltree path into the (folderPath, fileName) pair that addresses the entry itself: the
* parent's path still encoded, as `folderPath` is compared, and the last label decoded, as `fileName`
* is.
*/
function splitPath(path: string): { folderPath: string; fileName: string } {
const parts = path.split('.')
return {
folderPath: parts.slice(0, -1).join('.'),
fileName: parts.at(-1) ?? ''
fileName: decodeTreeLabel(parts.at(-1) ?? '')
}
}
/**
* The name a folder is created or renamed to, normalized as a page path is written, or an error if
* what is left cannot be one.
*/
function folderNameFrom(pathName: string): string {
const name = normalizeNewPagePath(pathName)
if (!isValidPathSegment(name)) {
throw new CustomError(
'treeInvalidPath',
`A folder path name may only contain letters, numbers and hyphens, and be at most ${MAX_PATH_SEGMENT_BYTES} bytes long.`
)
}
return name
}
/**
* Turn a row into the shape the API returns.
*/
@@ -186,7 +212,7 @@ function toTreeItem(row: TreeRow, depth: number, parentPath: string): TreeItem {
id: row.id,
type: row.type,
depth,
folderPath: decodeTreePath(folderPath) ?? '',
folderPath: decodeTreePath(folderPath),
fileName: row.fileName,
title: row.title,
tags: row.tags ?? [],
@@ -339,7 +365,7 @@ class Tree {
locations.push(
and(
eq(treeTable.folderPath, parts.slice(0, parts.length - 1 - i).join('.')),
eq(treeTable.fileName, parts[parts.length - 1 - i]),
eq(treeTable.fileName, decodeTreeLabel(parts[parts.length - 1 - i])),
eq(treeTable.type, 'folder')
)!
)
@@ -449,7 +475,7 @@ class Tree {
.limit(limit)
return rows.flatMap((row) => {
const folderPath = decodeTreePath(row.folderPath ?? '') ?? ''
const folderPath = decodeTreePath(row.folderPath)
const page = {
id: row.id,
path: folderPath ? `${folderPath}/${row.fileName}` : row.fileName,
@@ -494,7 +520,7 @@ class Tree {
unpublished?: Unpublished
}): Promise<BrowseLevel | null> {
const encodedPath = encodeTreePath(path)
const basePath = decodeTreePath(encodedPath) ?? ''
const basePath = decodeTreePath(encodedPath)
// -> What the level is called. The root is not a folder, so it has no row and no title of its own
// — and a path that is not a folder is nothing this can list.
@@ -523,7 +549,7 @@ class Tree {
const descendant = alias(treeTable, 'descendantTree')
const descendantPage = alias(pagesTable, 'descendantPage')
// -> Text rather than an ltree operator, so that the child path can be built from a bound prefix
// and the row's own name: `foo.bar.` + `baz`
// and the row's own name, encoded in SQL as `childPathOf` encodes it: `foo.bar.` + `baz`
const childPathPrefix = encodedPath ? `${encodedPath}.` : ''
/*
@@ -547,7 +573,7 @@ class Tree {
eq(descendant.siteId, treeTable.siteId),
eq(descendant.locale, treeTable.locale),
eq(descendant.type, 'page'),
sql`${descendant.folderPath} <@ (${childPathPrefix}::text || ${treeTable.fileName})::ltree`,
sql`${descendant.folderPath} <@ (${childPathPrefix}::text || ${treeLabelSql(treeTable.fileName)})::ltree`,
...pageIsVisible(descendantPage, unpublished)
)
)
@@ -753,7 +779,7 @@ class Tree {
throw new CustomError('treeInvalidFolder', 'This folder does not exist.', 404)
}
return this.createFolder({
parentPath: folderPath,
parentPath: decodeTreePath(folderPath),
pathName: fileName,
title: fileName,
locale: locale!,
@@ -786,13 +812,7 @@ class Tree {
}): Promise<TreeRow> {
// -> A folder name is a segment of every page path under it, so it is normalized the same way a
// page path is before it is held to what a segment may contain
const name = normalizePagePath(pathName)
if (!rePathName.test(name)) {
throw new CustomError(
'treeInvalidPath',
'A folder path name may only contain lowercase alphanumeric and hyphen characters.'
)
}
const name = folderNameFrom(pathName)
if (!reTitle.test(title)) {
throw new CustomError('treeInvalidTitle', 'The folder title contains invalid characters.')
}
@@ -840,9 +860,9 @@ class Tree {
// a rename that left a gap — so every level above the new folder is filled in first
if (path) {
const parts = path.split('.')
const expected = parts.map((_, i) => ({
const expected = parts.map((label, i) => ({
folderPath: parts.slice(0, i).join('.'),
fileName: parts[i]
fileName: decodeTreeLabel(label)
}))
const found = await WIKI.db
.select({ folderPath: treeTable.folderPath, fileName: treeTable.fileName })
@@ -869,6 +889,15 @@ class Tree {
(row.folderPath ?? '') === ancestor.folderPath && row.fileName === ancestor.fileName
)
)
// -> Created under the name the path asked for, so it is held to what a folder created on its own
// would be: a path is only checked where it is written, and this is where these are. Checked
// before any is created, so that a bad segment deep in the path leaves nothing behind.
if (missing.some((ancestor) => !isValidPathSegment(ancestor.fileName))) {
throw new CustomError(
'treeInvalidPath',
'A folder path name may only contain letters, numbers and hyphens.'
)
}
// -> Shallowest first, so that each one's own parent is already there to be counted against
for (const ancestor of missing) {
WIKI.logger.debug(
@@ -935,14 +964,10 @@ class Tree {
throw new CustomError('treeInvalidFolder', 'This folder does not exist.', 404)
}
// -> Normalized as it is on the way in, since this renames the segment every page path under the
// folder is built from
const name = normalizePagePath(pathName)
if (!rePathName.test(name)) {
throw new CustomError(
'treeInvalidPath',
'A folder path name may only contain lowercase alphanumeric and hyphen characters.'
)
}
// folder is built from. Unless it is the name the folder already has: one created before
// underscores became hyphens keeps its own, rather than a title change moving every page in it.
const name =
normalizePagePath(pathName) === folder.fileName ? folder.fileName : folderNameFrom(pathName)
if (!reTitle.test(title)) {
throw new CustomError('treeInvalidTitle', 'The folder title contains invalid characters.')
}
@@ -982,7 +1007,7 @@ class Tree {
}
const oldPath = childPathOf(folder)
const newPath = folder.folderPath ? `${folder.folderPath}.${name}` : name
const newPath = childPathOf({ folderPath: folder.folderPath, fileName: name })
WIKI.logger.debug(`Renaming folder ${folder.id} from ${oldPath} to ${newPath}...`)
@@ -1084,9 +1109,9 @@ class Tree {
previous: {
locale: row.locale,
// -> Where it was: the same place it is now, with the renamed segment put back
folderPath: (decodeTreePath(row.folderPath ?? '') ?? '').replace(
decodeTreePath(newPath)!,
decodeTreePath(oldPath)!
folderPath: decodeTreePath(row.folderPath).replace(
decodeTreePath(newPath),
decodeTreePath(oldPath)
),
fileName: row.fileName
}
@@ -1151,7 +1176,7 @@ class Tree {
const destinationLocale = locale || source.locale
const requested = normalizeFolderPath(folderPath)
const sourcePath = childPathOf(source)
const ownPath = decodeTreePath(sourcePath) ?? ''
const ownPath = decodeTreePath(sourcePath)
// -> On the paths, before anything is created: a folder copied into its own subtree would be
// copying into what it is still reading from
@@ -1206,10 +1231,10 @@ class Tree {
}
const copyPath = childPathOf(copy)
const newPrefix = decodeTreePath(copyPath) ?? ''
const newPrefix = decodeTreePath(copyPath)
/** Where an entry under the source sits under the copy, as a slash-separated folder path. */
const mapFolder = (path: string | null) =>
`${newPrefix}${(decodeTreePath(path ?? '') ?? '').slice(ownPath.length)}`
`${newPrefix}${decodeTreePath(path).slice(ownPath.length)}`
WIKI.logger.debug(
`Copying folder ${source.id} and ${descendants.length} descendant(s) to ${destinationLocale}:${copyPath}...`
@@ -1383,7 +1408,7 @@ class Tree {
const oldParentPath = folder.folderPath ?? ''
const oldPath = childPathOf(folder)
const requested = normalizeFolderPath(folderPath)
const ownPath = decodeTreePath(oldPath) ?? ''
const ownPath = decodeTreePath(oldPath)
// -> Decided on the paths, before the destination is resolved: resolving it would create it, and
// a folder moved inside itself would take its own subtree out of the tree entirely
@@ -1437,7 +1462,7 @@ class Tree {
)
}
const newPath = newParentPath ? `${newParentPath}.${folder.fileName}` : folder.fileName
const newPath = childPathOf({ folderPath: newParentPath, fileName: folder.fileName })
const isLocaleChange = destinationLocale !== folder.locale
WIKI.logger.debug(
@@ -1557,8 +1582,8 @@ class Tree {
sql`${treeTable.folderPath} <@ ${newPath}::ltree`
)
)
const newPrefix = decodeTreePath(newPath)!
const oldPrefix = decodeTreePath(oldPath)!
const newPrefix = decodeTreePath(newPath)
const oldPrefix = decodeTreePath(oldPath)
await WIKI.models.assets.relocateAssets(
siteId,
movedAssets.map((row) => ({
@@ -1567,7 +1592,7 @@ class Tree {
locale: folder.locale,
// -> Where it was: the same place it is now, with the moved folder's path put back. Sliced
// rather than replaced, since the segment that moved can occur again further down.
folderPath: `${oldPrefix}${(decodeTreePath(row.folderPath ?? '') ?? '').slice(newPrefix.length)}`,
folderPath: `${oldPrefix}${decodeTreePath(row.folderPath).slice(newPrefix.length)}`,
fileName: row.fileName
}
})),
@@ -1649,7 +1674,7 @@ class Tree {
const movedPages = []
for (const row of rows) {
const folderPath = decodeTreePath(row.folderPath ?? '')
const folderPath = decodeTreePath(row.folderPath)
const fullPath = folderPath ? `${folderPath}/${row.fileName}` : row.fileName
await WIKI.db
.update(treeTable)
@@ -1728,7 +1753,7 @@ class Tree {
const asEntry = (row: (typeof deleted)[number]): DeletedEntry => ({
id: row.id,
folderPath: decodeTreePath(row.folderPath ?? '') ?? '',
folderPath: decodeTreePath(row.folderPath),
fileName: row.fileName,
locale: row.locale
})
@@ -1764,7 +1789,8 @@ class Tree {
an empty one is what the next pass is for; anything else and the branch stays.
The folder's own path is built from its row, as `browse` does above: `foo.bar` + `.` + `baz`,
and `baz` alone at the root, where `folderPath` is the empty path rather than an absent one.
and `baz` alone at the root, where `folderPath` is the empty path rather than an absent one —
the name encoded in SQL as `childPathOf` encodes it.
*/
const holdsNothing = not(
exists(
@@ -1775,7 +1801,7 @@ class Tree {
and(
eq(child.siteId, treeTable.siteId),
eq(child.locale, treeTable.locale),
sql`${child.folderPath} <@ (COALESCE(NULLIF(${treeTable.folderPath}::text, '') || '.', '') || ${treeTable.fileName})::ltree`
sql`${child.folderPath} <@ (COALESCE(NULLIF(${treeTable.folderPath}::text, '') || '.', '') || ${treeLabelSql(treeTable.fileName)})::ltree`
)
)
)
@@ -1976,7 +2002,7 @@ class Tree {
}
}
const folderPath = decodeTreePath(entry.folderPath ?? '')
const folderPath = decodeTreePath(entry.folderPath)
const fullPath = folderPath ? `${folderPath}/${fileName}` : fileName
const updated = await WIKI.db
.update(treeTable)
backend/modules/storage/azure/storage.ts
+2 -3
View File
@@ -5,6 +5,7 @@ import {
generateBlobSASQueryParameters
} from '@azure/storage-blob'
import { DefaultAzureCredential } from '@azure/identity'
import { attachmentDisposition } from '../../../helpers/common.ts'
import { objectStorageModule, signingBaseUrl } from '../../../helpers/storageObjects.ts'
import type { ContainerClient } from '@azure/storage-blob'
import type { ObjectStoreClient } from '../../../helpers/storageObjects.ts'
@@ -137,9 +138,7 @@ const azureClient: ObjectStoreClient = {
startsOn: new Date(now - 60_000),
expiresOn: new Date(now + expiresInSeconds * 1000),
contentType,
...(downloadAs
? { contentDisposition: `attachment; filename="${encodeURIComponent(downloadAs)}"` }
: {})
...(downloadAs ? { contentDisposition: attachmentDisposition(downloadAs) } : {})
}
const sas = accountKey
backend/modules/storage/git/storage.ts
+13 -3
View File
@@ -457,7 +457,8 @@ async function reattach(repo: Repo, branch: string): Promise<string> {
/** Whether the repository's own ignore rules exclude this path. */
async function isIgnored(repo: Repo, relPath: string): Promise<boolean> {
try {
return (await repo.git.checkIgnore([relPath])).length > 0
// -> Raw, for the `--`: a path is never read as an option, whatever it starts with
return (await repo.git.raw(['check-ignore', '--', relPath])).trim().length > 0
} catch {
// -> `check-ignore` exits non-zero when nothing matches, which simple-git raises
return false
@@ -504,7 +505,16 @@ async function commitPaths(
if (!staged.trim()) {
return false
}
await repo.git.commit(message, paths, { '--author': await commitAuthor(target, actorId) })
// -> Raw, for the `--` that keeps a path from being read as an option -- `commit()` puts the paths
// straight after the message with nothing in between
await repo.git.raw([
'commit',
'-m',
message,
`--author=${await commitAuthor(target, actorId)}`,
'--',
...paths
])
return true
}
@@ -519,7 +529,7 @@ async function stageAndCommit(
if (await isIgnored(repo, relPath)) {
return
}
await repo.git.add(relPath)
await repo.git.raw(['add', '--', relPath])
await commitPaths(repo, target, [relPath], message, actorId)
}
backend/modules/storage/s3/storage.ts
+2 -5
View File
@@ -6,6 +6,7 @@ import {
S3Client
} from '@aws-sdk/client-s3'
import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
import { attachmentDisposition } from '../../../helpers/common.ts'
import { objectStorageModule, signingBaseUrl } from '../../../helpers/storageObjects.ts'
import type { ObjectStoreClient } from '../../../helpers/storageObjects.ts'
import type { StorageTarget } from '../../../models/storage.ts'
@@ -182,11 +183,7 @@ const s3Client: ObjectStoreClient = {
Bucket: bucket,
Key: key,
ResponseContentType: contentType,
...(downloadAs
? {
ResponseContentDisposition: `attachment; filename="${encodeURIComponent(downloadAs)}"`
}
: {})
...(downloadAs ? { ResponseContentDisposition: attachmentDisposition(downloadAs) } : {})
}),
{ expiresIn: expiresInSeconds }
)
frontend/package-lock.json
Generated
-8
View File
@@ -64,7 +64,6 @@
"react": "19.3.0",
"react-dom": "19.3.0",
"semver": "7.8.5",
"slugify": "1.6.9",
"sortablejs": "1.15.7",
"sortablejs-vue3": "1.3.0",
"tailwindcss": "4.3.3",
@@ -7028,13 +7027,6 @@
"deprecated": "Unsupported",
"license": "MIT"
},
"node_modules/slugify": {
"version": "1.6.9",
"license": "MIT",
"engines": {
"node": ">=8.0.0"
}
},
"node_modules/sortablejs": {
"version": "1.15.7",
"license": "MIT"
frontend/package.json
-1
View File
@@ -73,7 +73,6 @@
"react": "19.3.0",
"react-dom": "19.3.0",
"semver": "7.8.5",
"slugify": "1.6.9",
"sortablejs": "1.15.7",
"sortablejs-vue3": "1.3.0",
"tailwindcss": "4.3.3",
frontend/src/components/FolderCreateDialog.vue
+4 -5
View File
@@ -71,11 +71,10 @@ import { useI18n } from 'vue-i18n'
import { dialogComponentEmits, useDialogComponent } from '@/composables/dialog'
import { notify } from '@/composables/notify'
import { reactive, ref, watch } from 'vue'
import slugify from 'slugify'
import { useSiteStore } from '@/stores/site'
import { apiErrorMessage } from '@/helpers/apiError'
import { normalizePagePath } from '@/helpers/pagePaths'
import { isValidPathSegment, normalizeNewPagePath, pathSegmentFromTitle } from '@/helpers/pagePaths'
// PROPS
@@ -134,7 +133,7 @@ const titleValidation = [
const pathValidation = [
(val) => val.length > 0 || t('fileman.folderFileNameMissing'),
(val) => /^[a-z0-9-]+$/.test(val) || t('fileman.folderFileNameInvalid')
(val) => isValidPathSegment(val) || t('fileman.folderFileNameInvalid')
]
// WATCHERS
@@ -146,7 +145,7 @@ watch(
state.pathDirty = false
}
if (!state.pathDirty) {
state.path = slugify(newValue, { lower: true, strict: true })
state.path = pathSegmentFromTitle(newValue)
}
}
)
@@ -157,7 +156,7 @@ async function create() {
state.loading++
try {
// -> The name is a segment of every page path under the folder, and is corrected the way one is
state.path = normalizePagePath(state.path)
state.path = normalizeNewPagePath(state.path)
const isFormValid = await newFolderForm.value.validate(true)
if (!isFormValid) {
throw new Error(t('fileman.createFolderInvalidData'))
frontend/src/components/FolderRenameDialog.vue
+8 -5
View File
@@ -66,11 +66,10 @@ import { useI18n } from 'vue-i18n'
import { dialogComponentEmits, useDialogComponent } from '@/composables/dialog'
import { notify } from '@/composables/notify'
import { onMounted, reactive, ref, watch } from 'vue'
import slugify from 'slugify'
import { useSiteStore } from '@/stores/site'
import { apiErrorMessage } from '@/helpers/apiError'
import { normalizePagePath } from '@/helpers/pagePaths'
import { isValidPathSegment, pathSegmentFromTitle, pathSegmentToWrite } from '@/helpers/pagePaths'
// PROPS
@@ -103,6 +102,8 @@ const { t } = useI18n()
const state = reactive({
path: '',
// -> The name it has now, which is kept as it is when asked for again -- see `pathSegmentToWrite`
currentPath: null,
title: '',
pathDirty: false,
loading: false
@@ -122,7 +123,8 @@ const titleValidation = [
const pathValidation = [
(val) => val.length > 0 || t('fileman.folderFileNameMissing'),
(val) => /^[a-z0-9-]+$/.test(val) || t('fileman.folderFileNameInvalid')
(val) =>
val === state.currentPath || isValidPathSegment(val) || t('fileman.folderFileNameInvalid')
]
// WATCHERS
@@ -134,7 +136,7 @@ watch(
state.pathDirty = false
}
if (!state.pathDirty) {
state.path = slugify(newValue, { lower: true, strict: true })
state.path = pathSegmentFromTitle(newValue)
}
}
)
@@ -145,7 +147,7 @@ async function rename() {
state.loading++
try {
// -> The name is a segment of every page path under the folder, and is corrected the way one is
state.path = normalizePagePath(state.path)
state.path = pathSegmentToWrite(state.path, state.currentPath)
const isFormValid = await renameFolderForm.value.validate(true)
if (!isFormValid) {
throw new Error(t('fileman.renameFolderInvalidData'))
@@ -187,6 +189,7 @@ onMounted(async () => {
throw new Error('Failed to fetch folder data.')
}
state.path = folder.fileName
state.currentPath = folder.fileName
state.title = folder.title
state.pathDirty = true
} catch (err) {
frontend/src/components/LinkPickerDialog.vue
+2 -2
View File
@@ -150,7 +150,7 @@ import { notify } from '@/composables/notify'
import { apiErrorMessage } from '@/helpers/apiError'
import fileTypes from '@/helpers/fileTypes'
import { folderIconStyle } from '@/helpers/folderColors'
import { splitLocalePath } from '@/helpers/pagePaths'
import { decodeUrlPath, splitLocalePath } from '@/helpers/pagePaths'
import LocaleSelectorMenu from '@/components/LocaleSelectorMenu.vue'
import Tree from '@/components/TreeNav.vue'
@@ -468,7 +468,7 @@ onMounted(async () => {
if (split) {
state.locale = split.locale
}
state.path = (split?.path ?? props.initialHref).replace(/^\/+/, '')
state.path = decodeUrlPath(split?.path ?? props.initialHref).replace(/^\/+/, '')
}
}
frontend/src/components/LocaleSelectorMenu.vue
+4 -2
View File
@@ -48,7 +48,7 @@ import { useI18n } from 'vue-i18n'
import { computed } from 'vue'
import { useRoute, useRouter } from 'vue-router'
import { splitLocalePath } from '@/helpers/pagePaths'
import { decodeUrlPath, splitLocalePath } from '@/helpers/pagePaths'
import { useCommonStore } from '@/stores/common'
import { usePageStore } from '@/stores/page'
@@ -119,7 +119,9 @@ const currentLocale = computed(() => props.selected ?? commonStore.locale)
*/
const onCurrentPage = computed(() => {
const current = splitLocalePath(route.path, siteStore.localePrefixes)
const path = (current?.path ?? route.path).replace(/^\/+/, '').replace(/\/+$/, '')
const path = decodeUrlPath(current?.path ?? route.path)
.replace(/^\/+/, '')
.replace(/\/+$/, '')
return (path || 'home') === (pageStore.path || 'home')
})
frontend/src/components/NavSidebar.vue
+3 -1
View File
@@ -55,6 +55,7 @@ import { computed, onMounted, reactive, ref, watch } from 'vue'
import { useRouter, useRoute } from 'vue-router'
import { useI18n } from 'vue-i18n'
import { decodeUrlPath } from '@/helpers/pagePaths'
import { routableHref } from '@/helpers/renderedContent'
import { usePageStore } from '@/stores/page'
@@ -143,7 +144,8 @@ function destination(item) {
*/
function isCurrent(item) {
const { to } = destination(item)
return Boolean(to) && router.resolve(to).path === route.path
// -> Decoded on both sides: either may be percent-encoded, depending on how it was arrived at
return Boolean(to) && decodeUrlPath(router.resolve(to).path) === decodeUrlPath(route.path)
}
/** ...and whether one of a group's children is, which is what opens the group. */
frontend/src/components/PageRedirect.vue
+2 -1
View File
@@ -71,6 +71,7 @@ import { useI18n } from 'vue-i18n'
import { loading } from '@/composables/loading'
import { decodeUrlPath } from '@/helpers/pagePaths'
import { isFollowable, parseRedirect, REDIRECT_INTERSTITIAL_MS } from '@/helpers/pageRedirect'
import { usePageStore } from '@/stores/page'
@@ -228,7 +229,7 @@ function clear() {
/** Whether a page target is the page holding it, however the two are spelled. */
function isSelf(target) {
const stored = `/${pageStore.path}`.replace(/\/+$/, '')
const to = target.replace(/\/+$/, '').toLowerCase()
const to = decodeUrlPath(target).replace(/\/+$/, '').toLowerCase()
return to === stored.toLowerCase() || (stored === '/home' && to === '')
}
frontend/src/components/TreeBrowserDialog.vue
+17 -10
View File
@@ -208,8 +208,6 @@ import { dialog, dialogComponentEmits, useDialogComponent } from '@/composables/
import { notify } from '@/composables/notify'
import { computed, onMounted, reactive, ref, watch } from 'vue'
import slugify from 'slugify'
import fileTypes from '../helpers/fileTypes'
import { folderIconStyle } from '@/helpers/folderColors'
@@ -219,7 +217,12 @@ import Tree from '@/components/TreeNav.vue'
import { useSiteStore } from '@/stores/site'
import { apiErrorMessage } from '@/helpers/apiError'
import { normalizePagePath } from '@/helpers/pagePaths'
import {
isValidPathSegment,
normalizeNewPagePath,
pathSegmentFromTitle,
pathSegmentToWrite
} from '@/helpers/pagePaths'
// PROPS
@@ -290,6 +293,9 @@ const state = reactive({
fileList: [],
title: '',
path: '',
// -> The page's own name in the modes that start from one, kept as it is when asked for again --
// see `pathSegmentToWrite`
currentPath: null,
typesToFetch: [],
pathDirty: false,
updateLinks: true
@@ -402,7 +408,7 @@ watch(
state.pathDirty = false
}
if (!state.pathDirty) {
state.path = slugify(newValue, { lower: true, strict: true })
state.path = pathSegmentFromTitle(newValue)
}
}
)
@@ -435,8 +441,8 @@ async function save() {
notify({ type: 'negative', message: t('fileman.folderTitleMissing') })
return
}
state.path = normalizePagePath(state.path)
if (!/^[a-z0-9-]+$/.test(state.path)) {
state.path = normalizeNewPagePath(state.path)
if (!isValidPathSegment(state.path)) {
notify({ type: 'negative', message: t('fileman.folderFileNameInvalid') })
return
}
@@ -477,10 +483,10 @@ async function save() {
})
return
}
// -> A path is a URL: casing and spaces are corrected rather than refused, the way the server does
// it, and the field is left showing what will actually be saved
state.path = normalizePagePath(state.path)
if (!/^[a-z0-9-]+$/.test(state.path)) {
// -> A path is a URL: casing, spaces and underscores are corrected rather than refused, the way the
// server does it, and the field is left showing what will actually be saved
state.path = pathSegmentToWrite(state.path, state.currentPath)
if (state.path !== state.currentPath && !isValidPathSegment(state.path)) {
notify({
type: 'negative',
message: t('pageSaveDialog.pathInvalid')
@@ -717,6 +723,7 @@ onMounted(async () => {
case 'restorePage': {
state.typesToFetch = ['folder', 'page']
state.pathDirty = true
state.currentPath = fName || null
break
}
/*
frontend/src/helpers/pagePaths.js
+105 -5
View File
@@ -1,10 +1,10 @@
/**
* The one spelling a page path has.
* The one spelling a page path is looked up under.
*
* Mirrors `normalizePagePath` in the backend's `helpers/common.ts`, so that a path typed into a
* dialog is corrected in front of the person typing it rather than silently changed by the server
* after they hit save. Whether what comes out is *allowed* is still each field's own rule — this only
* settles casing and spaces.
* Mirrors `normalizePagePath` in the backend's `helpers/common.ts`: wrapping slashes dropped,
* whitespace to hyphens, lowercased and NFC-normalized. A path being WRITTEN goes through
* `normalizeNewPagePath` instead, which also turns underscores into hyphens — a page written before
* that rule may still have one, and looking it up through that would miss it.
*/
export function normalizePagePath(input) {
return (input ?? '')
@@ -13,6 +13,106 @@ export function normalizePagePath(input) {
.replace(/\/+$/, '')
.replaceAll(/\s+/g, '-')
.toLowerCase()
.normalize('NFC')
}
/**
* The spelling a page or folder path is written under: `normalizePagePath`, with every run of spaces,
* underscores and hyphens turned into a single hyphen, and none left at either end of a segment.
*
* Mirrors `normalizeNewPagePath` in the backend's `helpers/common.ts`, so that a path typed into a
* dialog is corrected in front of the person typing it rather than silently changed by the server
* after they hit save. Whether what comes out is *allowed* is `isValidPathSegment`'s question.
*/
export function normalizeNewPagePath(input) {
return (
normalizePagePath((input ?? '').trim().replaceAll(/[\s_-]+/g, '-'))
.split('/')
// -> A hyphen only separates, so one left at either end of a segment is dropped
.map((segment) => segment.replaceAll(/^-+|-+$/g, ''))
.join('/')
)
}
/**
* Longest a path segment may be, in UTF-8 bytes. Mirrors `MAX_PATH_SEGMENT_BYTES` in the backend's
* `helpers/common.ts`, which says where the number comes from.
*/
const MAX_PATH_SEGMENT_BYTES = 240
/** Letters and digits of any script, combining marks, and the hyphen -- never at either end. */
const PATH_SEGMENT = /^[\p{L}\p{M}\p{N}](?:[\p{L}\p{M}\p{N}-]*[\p{L}\p{M}\p{N}])?$/u
const utf8 = new TextEncoder()
/**
* Whether a normalized segment is one a path may be written with. Mirrors `isValidPathSegment` in the
* backend's `helpers/common.ts`.
*/
export function isValidPathSegment(segment) {
return PATH_SEGMENT.test(segment) && utf8.encode(segment).length <= MAX_PATH_SEGMENT_BYTES
}
/**
* A path segment as a dialog sends it: what `normalizeNewPagePath` makes of what was typed — unless
* it is the name the page or folder already has, asked for again, which comes back exactly as it is.
* The server does the same, so that renaming only the title of a page written before underscores
* became hyphens does not move it.
*
* @param current The name it has now, when there is one
*/
export function pathSegmentToWrite(input, current = null) {
return current !== null && normalizePagePath(input) === current
? current
: normalizeNewPagePath(input)
}
/**
* The path segment a title suggests, for a dialog that fills one in as the title is typed.
*
* Anything that cannot be in a segment becomes a hyphen rather than disappearing — `C++ Guide` is
* `c-guide`, and `日本語・ガイド` is `日本語-ガイド` — and the result is cut to the byte limit on a whole
* character.
*/
export function pathSegmentFromTitle(title) {
const slug = (title ?? '')
.toLowerCase()
.normalize('NFC')
.replaceAll(/[^\p{L}\p{M}\p{N}]+/gu, '-')
.replaceAll(/^-+|-+$/g, '')
let kept = ''
let used = 0
for (const char of slug) {
used += utf8.encode(char).length
if (used > MAX_PATH_SEGMENT_BYTES) {
break
}
kept += char
}
return kept.replace(/-+$/, '')
}
/**
* A URL path as the page path it spells: each segment percent-decoded.
*
* `route.path` is whatever the location was written as, because the router never decodes a path:
* raw after the app pushes a raw string, percent-encoded after a reload or a click on a link in a
* page, since the browser encodes what it puts in the address bar. So anything that compares a URL
* with a page path, or hashes one, decodes it first. A segment that will not decode is not one a page
* path could have produced, and is left as it is. Mirrors `decodeUrlPath` in the backend's
* `helpers/common.ts`.
*/
export function decodeUrlPath(urlPath) {
return (urlPath ?? '')
.split('/')
.map((segment) => {
try {
return decodeURIComponent(segment)
} catch {
return segment
}
})
.join('/')
}
/**
frontend/src/pages/Index.vue
+4 -2
View File
@@ -631,7 +631,7 @@ import { withViewTransition } from '@/composables/viewTransition'
import { loading } from '@/composables/loading'
import { scrollToAnchor, scrollToAnchorWhenReady } from '@/helpers/anchors'
import { clearHighlights, indexText, locateAnchor } from '@/helpers/annotations'
import { isPagePath, splitLocalePath } from '@/helpers/pagePaths'
import { decodeUrlPath, isPagePath, splitLocalePath } from '@/helpers/pagePaths'
import {
enhanceRenderedContent,
resolveContentClick,
@@ -1418,7 +1418,9 @@ watch(
No prefix means the site's primary locale, which is what the API assumes when none is given.
*/
const localePath = splitLocalePath(newValue, siteStore.localePrefixes)
const pagePath = localePath?.path ?? newValue
// -> Decoded: a reload or a link click hands the path over percent-encoded, which would be hashed
// as the name of some other page
const pagePath = decodeUrlPath(localePath?.path ?? newValue)
const pageLocale = localePath?.locale
/*
frontend/src/stores/page.js
+2 -1
View File
@@ -1218,7 +1218,8 @@ export function pagePathHash(path) {
* root is the `home` page rather than an empty path.
*/
function normalizePath(path) {
const clean = (path ?? '').replace(/^\/+/, '').replace(/\/+$/, '').toLowerCase()
// -> NFC as the server's `normalizePagePath` does, or a decomposed spelling hashes as another page
const clean = (path ?? '').replace(/^\/+/, '').replace(/\/+$/, '').toLowerCase().normalize('NFC')
return clean || 'home'
}