From 9bb45d0777324fefa827b1ce31d7974ad1bb323c Mon Sep 17 00:00:00 2001 From: Jonathan Shook Date: Fri, 4 Sep 2026 22:50:06 +0000 Subject: [PATCH] docs(vectordata): describe record binding in the contract MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit State the binding contract and the bind-type asymmetry, replace the note that binding forms are out of scope with the three deviations that remain — field views as objects, form extras as node values, and the raw predicate scanner not ported — and show the cycle loop. --- nb-vectordata/README.md | 35 ++++++++++++++++++++ nb-vectordata/RUST_COMPATIBILITY.md | 51 +++++++++++++++++++++++++---- 2 files changed, 80 insertions(+), 6 deletions(-) diff --git a/nb-vectordata/README.md b/nb-vectordata/README.md index 2b56830b3..6f73499b1 100644 --- a/nb-vectordata/README.md +++ b/nb-vectordata/README.md @@ -90,6 +90,41 @@ and `display`); a `PNode`'s display form reads back with `PNodeDisplay`. Reading one record from a remote facet fetches the page that holds it, not the file. A series of slabs is one facet of records in shard order. +## Binding records to parameters + +Rendering produces text for a human; binding produces named, typed values +for a driver, with the statement prepared once and only values moving per +cycle. The layout is learned once, a binder is compiled against it, and +the per-record path never looks at a field name: + +```java +RecordFacet facet = view.openFacetRecords("metadata_content"); +Layout layout = Layout.discover(facet); // once: names, and what each binds as +Binder binder = Binder.select(layout, "id", "tag") // template order + .withOverrides(Map.of("id", "pk")); // parameter names, not field names +List parameters = binder.parameters(); // [pk, tag] — prepare the statement +List types = binder.types(); // [int64, text] + +for (long ordinal : cycle) { // cycle → ordinal is the caller's policy + binder.bindEach(facet.recordBytes(ordinal), (slot, field) -> { + if (slot == 0) statement.setLong(slot, field.longValue()); + else statement.setString(slot, field.stringValue()); + }); +} + +List
offered = Forms.of(facet); // declared in the `forms` namespace, or one implicit form +Binder row = Forms.byName(facet, "row").binder(layout); + +PredicateBinder where = PredicateBinder.compile(template, layout); // shape once, typed from the fields +where.bindEach(predicates.recordBytes(ordinal), (condition, comparands) -> { /* condition.parameter(), condition.bindType() */ }); +``` + +A `Field` is a view over the record's own bytes; its primitive accessors +copy nothing, and `field.value()` materializes the AST view for a caller +that wants structure. A `Half` binds as a float, a `DateTime` as text, and +a container's element type as undetermined — the bind types answer what +a value *is*, which the schema vernaculars, right about DDL, do not. + ## Facets spread across several files A facet may be a series of files forming one dense ordinal space, in either diff --git a/nb-vectordata/RUST_COMPATIBILITY.md b/nb-vectordata/RUST_COMPATIBILITY.md index 756ee486b..6f118078c 100644 --- a/nb-vectordata/RUST_COMPATIBILITY.md +++ b/nb-vectordata/RUST_COMPATIBILITY.md @@ -147,6 +147,32 @@ implementation's current behavior and its format tests are normative. inference (quoted strings, bare integers, decimal-point floats, `true`/`false`, each language's null); the schema, value, and display renderings refuse to, naming themselves. +- Binding: the reference's `binding` module, for driving load rather + than rendering text. A `Layout` is learned **once** from a facet's + first record — names in wire order, each with the `BindType` its tag + implies — and a `Binder` compiled against it (`all`, or `select` in + template order, with parameter renames applied at compile time) binds + every record after by position: the walk in `Scan.fields` skips names + and reaches each value as a `Field` view over the record's own bytes, + read through primitive accessors typed by the wire form, or through + `value()` for the AST view. A field the facet lacks is refused when the + binder is built, naming what it has; a record whose layout differs + from the compiled one is refused, not bound short. `BindType` is its + own exhaustive mapping from the tag, distinct from the schema + vernaculars': a `Half` binds as `FLOAT16`, `DateTime` as + `TIMESTAMP_TEXT`, `Null` as no type, and containers with their element + types left undetermined rather than guessed as text. Forms are read + from the facet's `forms` namespace — one JSON record per form, unknown + keys preserved, unreadable records skipped — and a facet without one + offers exactly one implicit form named `default`, which is every + dataset written before forms existed; an unknown form is refused + naming what is offered, the implicit one not listed as a choice. A + `PredicateBinder` compiles a flat conjunction against a sample + predicate **and** the layout, typing each `Condition` from the field's + tag rather than the comparand's variant, and binds a record's + comparands only after its leader byte says PNode and its fingerprint + matches the compiled shape; a disjunctive or nested template is + refused at compile time. - Facet shape: every extension the spec names belongs to a `FacetFormat` with a `FacetShape` — element runs, or opaque records — and `facetShape` answers it so a caller handling both branches on the fact rather than @@ -236,12 +262,25 @@ Remaining representation differences: relies on its JSON library's map, whose order is that library's choice. Numbers follow the reference's inference: no point or exponent and fits a long is an integer, anything else numeric is a float. -- **Record binding forms are not ported.** The reference binds decoded - records to operation parameters and walks MNode fields without - allocating; both belong to workload generation, not to the access API, - and are out of this module's scope. Slab facets take part in everything - else — windows, shards, residency, planning, prefetch, the - whole-profile prebuffer, and now records by ordinal. +- **A bound field is a small view object.** The reference hands out a + borrowed `Field` by value; here each field the walk meets is one + object holding offsets into the record's bytes, and its primitive + accessors (`longValue`, `doubleValue`, …) copy nothing. What the + contract forbids — resolving or copying a field *name* per record — is + honored: names are read only when a layout is discovered. A Java + caller wanting the allocation-free loop uses `bindEach` with the + primitive accessors and never calls `name()` or `value()` per cycle. +- **A form's unknown keys are kept as node values.** The reference keeps + them as its JSON library's values; here `Form.extra()` holds the + `MValue`s the JSON vernacular parses them to, typed as that parse + types them. +- **The raw predicate scanner is not ported.** The reference's + `mnode::scan` also compiles predicates to positions and evaluates them + against raw bytes for its predicate-index pipeline; that is workload + generation, not the access API. The walk, the schema discovery, and + the predicate flattening it shares with binding are here; predicate + evaluation over a record goes through `PredicateEvaluator` on the + decoded node. - **A windowed reader's `prebuffer` fetches its window.** The reference's windowed reader inherits a no-op `precache`; here the reader a view hands back for a windowed facet warms the same bytes the whole-profile prebuffer