Commit Graph
130 Commits
Author SHA1 Message Date
Jonathan Shook 5a82d60305 fix(expr-api): render datetime seconds even on a whole minute
The ISO forms of LocalTime and LocalDateTime omit the seconds field
when the second is 00, so currentTime(), currentDateTime(),
currentTimeInZone() and millisToDateTime() each returned a shorter
string once a minute — HH:mm instead of HH:mm:ss. That failed
testCurrentTime in CI roughly one run in sixty, and silently handed
workloads a differently-shaped timestamp the rest of the time.

Each now formats with an explicit pattern rather than toString(), which
also makes the truncatedTo(SECONDS) calls redundant.

The new test pins the boundary through millisToDateTime, whose instant
comes from the caller: the wall-clock functions can only be asserted
for shape, which catches this one second in sixty. It builds a
timestamp on a whole minute, requires the seconds to be there, and
repeats at second 30 so a fix in the other direction is caught too.
2026-09-02 20:19:36 +00:00
Youki Shiraishi 654574784c Fix metrics scheduler lifecycle across rebases 2026-08-21 16:50:06 +09:00
J S c7d9d87f31 Merge branch 'main' into dependabot/maven/nb-apis/nb-mql-api/org.apache.logging.log4j-log4j-api-2.25.5 2026-08-20 16:11:00 -05:00
Youki Shiraishi ecf34b56b2 Flush pending metric snapshots at activity completion 2026-08-19 17:32:22 +09:00
dependabot[bot] 743197ad3c build(deps): bump org.apache.logging.log4j:log4j-api
Bumps org.apache.logging.log4j:log4j-api from 2.25.2 to 2.25.5.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-api
  dependency-version: 2.25.5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-13 19:47:04 +00:00
Youki Shiraishi 72e57b3508 Fix timezone-sensitive NBEnvironment test 2026-08-13 16:09:37 +09:00
Jonathan Shook 2c88373c28 add runapp option 2026-06-01 06:23:39 +00:00
Jonathan Shook 500c7b9609 - Diagnosed & fixed CsvReporter label-overlap crash when --report-csv-to was used; added JSONL manifest mapping fully-qualified labels → CSV file.
- Persisted min/max/mean/stddev in a new sample_statistics SQLite table; added StatsCommand and made InstantCommand/StatsCommand use per-metric latest timestamp.
  - Fixed lost activity metrics on short runs: ActivityExecutor now triggers a bypass-bucket snapshot before op dispensers detach; SQLite reporter is silent under post-close ticks.
  - New stress-report bundled app + dummy_session.yaml workload with two new diag op-tasks (cycledelay, errormodulo); c-stress-shaped per-op output, A/B equivalence mode, comparability warning at the top.
  - removed NBCLI's foundational-label re-injection.
2026-05-26 15:52:08 +00:00
Jonathan Shook fe8d9d0fba fix doc resolver and virtdata-lib-io inclusion 2026-05-21 07:13:17 +00:00
dependabot[bot] 8ea0d0f4c2 build(deps): bump org.apache.logging.log4j:log4j-core
Bumps org.apache.logging.log4j:log4j-core from 2.25.3 to 2.25.4.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-core
  dependency-version: 2.25.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-10 23:47:25 +00:00
Jonathan Shook f531ee9524 bugfix for --list-workloads 2026-02-19 14:23:35 +00:00
Jonathan Shook 545564ea9a release fix for groovy libs 2026-02-18 01:54:10 +00:00
Jonathan Shook f95fbfc049 fix param mutability bug, defer dryrun till full activity rendering 2026-02-09 22:59:00 +00:00
Jonathan Shook 95e119d759 improve options readout 2026-02-09 22:59:00 +00:00
Jonathan Shook b1f5cd8cd2 Fixed and added comprehensive coverage of historic TEMPLATE nesting patterns 2026-02-05 18:54:03 +00:00
Jonathan Shook 49d1a7c2fd fix: apply capture point parsing to all op fields except reserved fields 2026-02-04 19:06:23 +00:00
Jonathan Shook 8e9aafcfa0 Merge pull request #2240 from A-k-A-s-H-1/fix/tls1.3-default
Default TLS version updated to TLSv1.3 and add TLSv1.3 test
2026-02-04 13:01:43 -06:00
Jonathan Shook d9bf7e49c7 Fine-tune the TEMPLATE substitution rules in the expr layer:
1. Provided parameter - always takes precedence
  2. Current TEMPLATE's default - used if present (even if a previous TEMPLATE set a different default)
  3. Previous TEMPLATE's default - only used when current TEMPLATE has no default (e.g., TEMPLATE(var) or ${var})
2026-01-21 17:46:59 +00:00
Jonathan Shook 925383765d Merge branch 'main' into stacked_scopes 2026-01-20 11:43:18 -06:00
Jonathan Shook d308cddf21 New JMH benchmarking framework:
Added 9 new benchmark classes in nb-engine-core/.../sysperf/engineflow/ for measuring engine flow performance
  (stride context, op flow, space mapping, motor operations, etc.)
JMH documentation: Added devdocs/devguide/jmh_engineflow.md
Testing tags:
 POM updates: Added test profiles
 CONVENTIONS.md updates around test tags
 Test annotations: Added 2-3 lines to ~280 test files
 Split tests classes to allow one tag set per class (no mixed methods)
2026-01-20 17:23:18 +00:00
Jonathan Shook 6e934879e3 Merge branch 'main' into fix/summary-rate-units 2026-01-15 13:56:35 -06:00
dependabot[bot] d655966040 build(deps): bump org.apache.logging.log4j:log4j-core
Bumps org.apache.logging.log4j:log4j-core from 2.25.2 to 2.25.3.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-core
  dependency-version: 2.25.3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-02 09:47:20 -05:00
Jonathan Shook 62a99401a6 bugfix 2025-12-16 17:18:26 +00:00
Jonathan Shook fbd232dfec sqlite histos 2025-12-16 17:18:20 +00:00
A-k-A-s-H-1 0868f1e0da Fix console rate units and restore duration unit labels 2025-12-16 14:59:00 +05:30
Jonathan Shook bbbf2bc084 checkpoint before hdr reservoir updates 2025-12-12 21:48:09 +00:00
Jonathan Shook 655aaef093 • - New stride-local flow context primitives (OpFlowContext, OpFlowState).
- Ops can accept flow context via FlowContextAwareOp.
  - StandardAction manages per-stride StrideContext and passes flow context to ops.
  - CapturingOp now writes captured values into the flow context.
  - Dispensers/adapters expose cycle→space index/name for binding.
  - Space mapping logic cleaned up; BaseSpace retains original names.
  - Sync motor path replaced by StrideAction plus StrideMotor/dispenser.
  - CoreMotor/SyncAction removed; loaders/tests moved to stride motor.
  - CycleResultSegmentBuffer adds reset().
  - StringCompositor skips local/global refs when deduping specs.
  - Planning docs lightly edited.
2025-12-11 22:54:14 +00:00
A-k-A-s-H-1 fe70792f53 Default TLS version updated to TLSv1.3 and add TLSv1.3 test 2025-12-10 16:48:38 +05:30
Jonathan Shook bebc861b21 fix JVM-variant ordering glitch 2025-12-04 22:48:13 +00:00
Jonathan Shook ecc48e85eb fix space warning 2025-11-02 06:48:11 +00:00
Jonathan Shook 59a543110c ● Fix query command and scenario handling bugs
- Fix SQL syntax error in SelectorTransformer: Move LEFT JOINs before WHERE clause
  - Support nested {{...}} expressions in TEMPLATE() defaults
  - Fix CMD_await to correctly use activity= parameter instead of alias
  - Add available activities list to missing activity error messages
2025-11-02 06:25:07 +00:00
Jonathan Shook 4707b54175 Add unit of measurement property to metrics
- Add getUnit() method to NBMetric interface
  - Update all NBMetric implementations (Counter, Timer, Histogram, Meter, Gauge variants) with unit field
  - Update MetricsView to use metric.getUnit() instead of extracting from labels
  - Fix InstantCommand SQL join order bug (use joinAllLabelsWithSampleName())
  - Update NBCreators with default units (nanoseconds, operations, empty)
  - Fix test compilation errors in nb-api, nb-mql-api, and nb-engine-core (12 test files)
  - Regenerate all test databases with updated metric constructors
2025-10-31 23:31:40 +00:00
Jonathan Shook 292a21a332 Reorganize ANTLR packages and update documentation to match project conventions
- Move grammar files from src/main/antlr4 to src/main/java/.../grammars following conventions in adapter-cqld4 and
   virtdata-lang modules
  - Update ANTLR generated output to src/main/java/.../generated with explicit package configuration (-package
  argument)
  - Add imports for generated MetricsQLLexer/Parser classes in 11 parser and test files; all 285 tests passing
  - Replace FEATURE_TRADEOFF_ANALYSIS.md with user-focused METRICSQL_DIFFERENCES.md guide (~40% shorter, removes
  internal decision-making)
  - Add session metadata (nb.version, nb.commandline, nb.hardware) to all 5 test database generators
  - Update TEST_DATABASES.md with metadata documentation and correct regeneration instructions from project root
2025-10-31 21:51:16 +00:00
Jonathan Shook b613170f91 - Add metrics command to command list and quick start examples
- Document tree display options: --no-condense, --group-by, --keep-labels
  - Expand --explain flag documentation with examples across query types
  - Document two-layer tree architecture (LabelSetTree → DisplayTree with condensation algorithms)
  - Add Discovery & Navigation section with metrics command usage examples
2025-10-30 17:55:01 +00:00
Jonathan Shook 6e6d0d9518 Fix metric grouping: Implement two-layer tree architecture for label set display
- Add LabelSetTree (canonical layer): Algebraic tree maintaining strict label set containment relationships with support for merging trees
  - Add DisplayTree (display layer): Derived tree with three-pass reduction algorithm:
    - Differential label rendering (show only labels differing from parent)
    - Sibling condensation (merge siblings differing by single label with comma-separated values)
    - Identical label set consolidation (combine nodes with same labels)
  - Update MetricsCommand: Use canonical→display tree pipeline for --group-by labelset output
  - Fix SummaryCommand: Build single shared canonical tree for all metrics instead of per-metric trees, enabling proper consolidation of metrics with identical label sets
  - Add LabelSetTreeTest: 10 tests verifying containment relationships, merging, and rendering
  - Result: Metrics with identical label sets now properly consolidated under single parent node (e.g., 20+ separate session=oDChioC entries → 1 node with 20+ child metrics)
2025-10-28 21:55:26 +00:00
Jonathan Shook 30c8a9d015 - New nb-mql-api module with 11 query commands (instant, range, rate, increase, quantile, agg, ratio, topk, session, summary, sql)
- BundledApp integration (@Service selector="mql") accessible via nb5 CLI
- 6 output formats (table, csv, tsv, json, jsonl, markdown) with enum-based architecture
- 107 passing tests with self-validating documentation and 5 test databases
- Add sqlite-jdbc to dependency management and nb-mql-api to nb-engine-all
- Add shutdown hook to update symlink mtimes on session end
2025-10-27 19:48:37 +00:00
Jonathan Shook a1f5c57017 Metrics pipeline docs tightened: scheduler cadences + MetricsView snapshot role clarified (scheduler & view Javadocs).
Metric filters upgraded: include/exclude clauses, deny-by-default option, cached lookups; Log4J reporter imports simplified.
Help topics fixed end-to-end: classpath scanning handles jars/denied paths, nb help topics prints list, markdown finder hardened.
AppImage launcher opts into --enable-native-access=ALL-UNNAMED to silence JNA warnings.
New metrics_reporting.md gives a concise CLI reporter/filter cheat-sheet.
2025-10-22 23:18:28 +00:00
Jonathan Shook 5e896cb912 - add optional histogram persistence to SQLite reporter and extend schema with sample_histogram
- expose encodable histogram snapshots, reuse prepared statements, and Base64-encode histogram payloads directly
  - propagate histograms flag through CLI/session wiring and document usage in local/sqlite_metrics.md
  - expand integration tests to cover histogram capture and ensure SQLite reporter behaviour is exercised
2025-10-22 15:54:46 +00:00
Jonathan Shook 5182075188 fix session duration calculation 2025-10-20 16:17:37 +00:00
Jonathan Shook d0560f2801 templates as exprs (#2228)
* improve exprs dryrun output and error messages

* modularize user-extensible workload exprs

* make TEMPLATE just exprs

* square up template variable syntax

* banish double angle brackets and make Dryrun enum pervasive

* template replacement cleanup

* clean up logger levels, support copy target
2025-10-17 12:10:50 -05:00
Jonathan Shook 859d39e301 modularize user-extensible workload exprs 2025-10-15 00:45:05 +00:00
Jonathan Shook aa7d687b40 improve exprs dryrun output and error messages 2025-10-14 17:56:38 +00:00
Jonathan Shook 0312a70bae add better version command 2025-10-13 17:00:12 +00:00
Jonathan Shook 7b8b2d2de1 Jshook vectors (#2222)
* Update nodewalk parsing/rendering for vectordata 0.1.10
  - swap nbvectors predicate imports for vectordata.spec.predicates and make the renderers Function<PNode<?>, String> wrappers with explicit fallback handling
  - drive NodewalkParser through PNode.fromBuffer and case-insensitive renderer selection so both CQL and JSON views work with the new API
  - refresh predicate-focused tests to use the vectordata types and confirm with targeted Maven test runs

* update to groovy 5 for exprs

* groovy 5 expr support checkpoint
* basic expression libraries and lvar support

* add preprocedssor expr layer to support autoconfig of vector params

* added lvar expr integrated tests

* fixed eventing leak
2025-10-10 16:05:28 -05:00
Jonathan Shook f8a07eb12c Revert "Jshook/factoring2 - logger updates (#2220)"
This reverts commit 901ca55eb5.
2025-10-02 22:46:43 +00:00
Jonathan Shook 901ca55eb5 Jshook/factoring2 - logger updates (#2220)
* shade not assembly
* fix default log init
* logger robustification
2025-10-02 01:49:04 -05:00
Jonathan Shook a83d83bbe8 branch reconciliation: Flatten Activity API (#2217)
update and plan true-up: Flatten Activity API
2025-09-29 13:25:37 -05:00
Jonathan Shook ee41890a22 provide error context 2025-06-26 20:39:43 -05:00
Jonathan Shook e93da31261 modify error message when space is variant 2025-06-26 20:39:26 -05:00
Jonathan Shook ea303291e1 preserve non-numeric space names 2025-06-26 20:39:17 -05:00