Asking for a prefetch with progress meant asking for prefetch=false and
then a binding mode — two knobs that read as contradicting each other,
because the background expr function was the one prefetch form that
never got a meter, so the only way to see a background download was to
route it through a binding.
prefetchBackground now announces its plan and watches the handle like
every other form, and prefetchCyclesBackground takes the ordinal range
directly. With the API reachable at the surface, the demo asks for what
it wants in one word: prefetch=load blocks on prefetchCycles before the
run, prefetch=background hands back a metered handle that fetches while
the cycles run, prefetch=none leaves reads to demand-page. The binding
mode knob is gone.
The documented 'nb5 run vectordata_demo' does not work: run is itself a
command, so the workload name is parsed as a second one and resolves to
no service. The scenario form drops the run — 'nb5 vectordata_demo
dataset=<name>:<profile>' — and the long form names the full bundled
path, 'nb5 run workload=activities/examples/vectordata_demo.yaml'.
Both are now shown in the workload and the guide.
Verified against the 5.25.15-release binary: the workload is bundled
and listed, both corrected forms load it, and a real dataset run meters
the download and reads base, query, and neighbor records.
The demo only worked from a checkout. It now ships in the stdout
adapter's activities, so 'nb5 run vectordata_demo dataset=<name>:<profile>'
works from the distribution alone, with --cat and --copy for reading or
editing it.
Every catalog-touching expression is guarded on the dataset parameter
being given. Bundled workloads are parsed through the expression
pipeline by --list-workloads and --list-scenarios, so a workload that
opened a catalog unconditionally would fail listing for every user
without one configured; with no dataset named it now renders usage text
and touches nothing.
A prefetch driven from a workload was silent and awkward to aim:
- every expr prefetch now reports through the same meter the bindings
use, so a download started from a workload announces its cost, emits
progress while the bytes move, and confirms completion
- prefetchProfile(spec) warms every facet the profile declares, each to
the window it declares, in one call — the form that cannot
accidentally pull a whole shared base file, since an empty window
means the entire facet by design
- prefetchCycles(spec, facet, start, end) takes the ordinal range
directly, which is how an activity already describes its work
The docs record the three things that bite: expressions evaluate at
load time rather than when a tagged block runs, an empty window is a
request for everything, and assignment sigils render their value where
they are written.
cql_vector_dataset.yaml takes one dataset input —
dataset='<dataset>:<profile>' — and derives everything dataset-shaped
through the expr system at load time: the schema's vector dimensions
and similarity function, rampup cycles from the base count, search
cycles from the query count, and the ANN LIMIT and recall k from the
ground-truth width. Bindings read through the vectordata mappers, so
the profile's base window warms eagerly with download progress before
the first cycle; the plan's cost is echoed into the rendered
description. Operational knobs (keyspace, table, rf, cl) stay
conventional TEMPLATE parameters.
Verified through the real load-path transforms (TemplateRewriter +
ExprPreprocessor) against a published remote dataset: dimensions 384,
train 100k, test 10k, k 100, dot_product, all spliced correctly.
- Persisted min/max/mean/stddev in a new sample_statistics SQLite table; added StatsCommand and made InstantCommand/StatsCommand use per-metric latest timestamp.
- Fixed lost activity metrics on short runs: ActivityExecutor now triggers a bypass-bucket snapshot before op dispensers detach; SQLite reporter is silent under post-close ticks.
- New stress-report bundled app + dummy_session.yaml workload with two new diag op-tasks (cycledelay, errormodulo); c-stress-shaped per-op output, A/B equivalence mode, comparability warning at the top.
- removed NBCLI's foundational-label re-injection.
Added 9 new benchmark classes in nb-engine-core/.../sysperf/engineflow/ for measuring engine flow performance
(stride context, op flow, space mapping, motor operations, etc.)
JMH documentation: Added devdocs/devguide/jmh_engineflow.md
Testing tags:
POM updates: Added test profiles
CONVENTIONS.md updates around test tags
Test annotations: Added 2-3 lines to ~280 test files
Split tests classes to allow one tag set per class (no mixed methods)
- expose encodable histogram snapshots, reuse prepared statements, and Base64-encode histogram payloads directly
- propagate histograms flag through CLI/session wiring and document usage in local/sqlite_metrics.md
- expand integration tests to cover histogram capture and ensure SQLite reporter behaviour is exercised
Jonathan Shook
dependabot[bot]
Jonathan Shook
Stefano Lottini
yabinmengandYabin Meng
Mark Wolters