The predicate's structure is reified through the template: the
predicate bind point takes a fragment, the adapter prepares one
statement per predicate shape keyed by its fingerprint, and the
comparands bind as parameters typed from the metadata fields. The
rendered form stays available through RecordText for a driver that
cannot prepare.
When a bind point of a prepared op yields a fragment, its text takes
the bind point's place in the statement and its values take the place
of that bind value. The space holds one prepared statement per op and
form key, prepared the first time the form is seen; a cycle without a
fragment takes the single-statement path unchanged. A template with one
fragment is keyed by the fragment's own key, so a cycle allocates no
key.
Every dataset-shaped parameter is derived from the profile: the table's
metadata columns from the record layout, dimensions and k from the
facets, the similarity from the attributes. Rampup binds each base
vector with its metadata record field by field through a prepared
insert; search runs each query under the predicate published for it,
rendered as a WHERE clause since the predicates differ in shape, with
SAI indexes on every column they reference, and verifies against the
pre-filtered ground truth, padding a short result set the way the
ground truth pads a short match set.
Asking for a prefetch with progress meant asking for prefetch=false and
then a binding mode — two knobs that read as contradicting each other,
because the background expr function was the one prefetch form that
never got a meter, so the only way to see a background download was to
route it through a binding.
prefetchBackground now announces its plan and watches the handle like
every other form, and prefetchCyclesBackground takes the ordinal range
directly. With the API reachable at the surface, the demo asks for what
it wants in one word: prefetch=load blocks on prefetchCycles before the
run, prefetch=background hands back a metered handle that fetches while
the cycles run, prefetch=none leaves reads to demand-page. The binding
mode knob is gone.
The documented 'nb5 run vectordata_demo' does not work: run is itself a
command, so the workload name is parsed as a second one and resolves to
no service. The scenario form drops the run — 'nb5 vectordata_demo
dataset=<name>:<profile>' — and the long form names the full bundled
path, 'nb5 run workload=activities/examples/vectordata_demo.yaml'.
Both are now shown in the workload and the guide.
Verified against the 5.25.15-release binary: the workload is bundled
and listed, both corrected forms load it, and a real dataset run meters
the download and reads base, query, and neighbor records.
The demo only worked from a checkout. It now ships in the stdout
adapter's activities, so 'nb5 run vectordata_demo dataset=<name>:<profile>'
works from the distribution alone, with --cat and --copy for reading or
editing it.
Every catalog-touching expression is guarded on the dataset parameter
being given. Bundled workloads are parsed through the expression
pipeline by --list-workloads and --list-scenarios, so a workload that
opened a catalog unconditionally would fail listing for every user
without one configured; with no dataset named it now renders usage text
and touches nothing.
A prefetch driven from a workload was silent and awkward to aim:
- every expr prefetch now reports through the same meter the bindings
use, so a download started from a workload announces its cost, emits
progress while the bytes move, and confirms completion
- prefetchProfile(spec) warms every facet the profile declares, each to
the window it declares, in one call — the form that cannot
accidentally pull a whole shared base file, since an empty window
means the entire facet by design
- prefetchCycles(spec, facet, start, end) takes the ordinal range
directly, which is how an activity already describes its work
The docs record the three things that bite: expressions evaluate at
load time rather than when a tagged block runs, an empty window is a
request for everything, and assignment sigils render their value where
they are written.
cql_vector_dataset.yaml takes one dataset input —
dataset='<dataset>:<profile>' — and derives everything dataset-shaped
through the expr system at load time: the schema's vector dimensions
and similarity function, rampup cycles from the base count, search
cycles from the query count, and the ANN LIMIT and recall k from the
ground-truth width. Bindings read through the vectordata mappers, so
the profile's base window warms eagerly with download progress before
the first cycle; the plan's cost is echoed into the rendered
description. Operational knobs (keyspace, table, rf, cl) stay
conventional TEMPLATE parameters.
Verified through the real load-path transforms (TemplateRewriter +
ExprPreprocessor) against a published remote dataset: dimensions 384,
train 100k, test 10k, k 100, dot_product, all spliced correctly.
- Persisted min/max/mean/stddev in a new sample_statistics SQLite table; added StatsCommand and made InstantCommand/StatsCommand use per-metric latest timestamp.
- Fixed lost activity metrics on short runs: ActivityExecutor now triggers a bypass-bucket snapshot before op dispensers detach; SQLite reporter is silent under post-close ticks.
- New stress-report bundled app + dummy_session.yaml workload with two new diag op-tasks (cycledelay, errormodulo); c-stress-shaped per-op output, A/B equivalence mode, comparability warning at the top.
- removed NBCLI's foundational-label re-injection.
Added 9 new benchmark classes in nb-engine-core/.../sysperf/engineflow/ for measuring engine flow performance
(stride context, op flow, space mapping, motor operations, etc.)
JMH documentation: Added devdocs/devguide/jmh_engineflow.md
Testing tags:
POM updates: Added test profiles
CONVENTIONS.md updates around test tags
Test annotations: Added 2-3 lines to ~280 test files
Split tests classes to allow one tag set per class (no mixed methods)
- expose encodable histogram snapshots, reuse prepared statements, and Base64-encode histogram payloads directly
- propagate histograms flag through CLI/session wiring and document usage in local/sqlite_metrics.md
- expand integration tests to cover histogram capture and ensure SQLite reporter behaviour is exercised
Jonathan Shook
dependabot[bot]
Jonathan Shook
Stefano Lottini
yabinmengandYabin Meng
Mark Wolters