Commit Graph

252 Commits

Author SHA1 Message Date
Jonathan Shook
a1445927b9 minor fix for sampling window test stability 2023-11-01 11:24:50 -05:00
Jonathan Shook
1cc2f5352c stability detection in progress 2023-11-01 11:09:16 -05:00
Jonathan Shook
222fbbbafd renaming workloads part1 2023-10-23 19:54:17 -05:00
Jonathan Shook
921c6e17e3 cleanups and minor docs 2023-10-23 08:42:19 -05:00
Jonathan Shook
aaf58984d6 findmax in the house 2023-10-23 08:42:19 -05:00
Jonathan Shook
cc8d4be719 minor update on findmax 2023-10-23 08:42:19 -05:00
Jonathan Shook
137ce31136 findmax 80% ready 2023-10-23 08:42:19 -05:00
Jonathan Shook
bff86b5525 bursty cycle rate test online 2023-10-23 08:42:19 -05:00
Jonathan Shook
9978d50d7e improving optimo sampler and rates 2023-10-23 08:42:19 -05:00
Jonathan Shook
e64de7476e partial working optimo, something is still broken in the search behavior 2023-10-23 08:42:19 -05:00
Jonathan Shook
6b85515715 fix example metric 2023-10-23 08:42:19 -05:00
Jonathan Shook
9e91a6201d incremental progress 2023-10-23 08:42:19 -05:00
Jonathan Shook
9fa711b7ab checkpoint 2023-10-23 08:42:19 -05:00
Jonathan Shook
61b923cfed indirect loading on nbr working 2023-10-23 08:42:00 -05:00
Jonathan Shook
9daf06cd62 rehome all metrics construction patterns, remove ActivityMetrics 2023-10-23 08:41:40 -05:00
Jonathan Shook
57db140a94 removed all ActivityMetrics constructions 2023-10-23 08:41:40 -05:00
Jonathan Shook
17052eaadf rework scenario invocation for better tests, unit test running only 2023-10-23 08:41:17 -05:00
Jonathan Shook
d3edd71344
Merge pull request #1573 from yabinmeng/rm-docker-metrics
Remove "--docker-metrics" functionality.
2023-09-28 15:39:37 -05:00
yabinmeng
9a61a1f2af Remove "--docker-metrics" functionality. Update documents and scripts accordingly. 2023-09-28 14:30:18 -05:00
Jonathan Shook
3ba9c50bbc stack timeouts so that errors in integrated tests are not lost 2023-09-28 10:48:16 -05:00
Jonathan Shook
c561695685 make jdk21 appimage scripts use ZGC and preview 2023-09-28 10:48:16 -05:00
Jonathan Shook
b9b33f1a9d base project on jdk21 toolchain and runtime 2023-09-28 10:48:16 -05:00
Jonathan Shook
112ad332cd fixups and package structure updates 2023-09-11 13:20:52 -05:00
Jonathan Shook
2d1f23a8d8 remove duplicitous doc files 2023-09-10 17:37:49 -05:00
Jonathan Shook
497432137e recall computation 2023-08-17 18:07:04 -05:00
Jonathan Shook
23aba88ee0 engine and runtime updates for vector branch 2023-07-25 11:50:28 -05:00
Jonathan Shook
b324c59c11 remove package element from log4j2 configs 2023-06-06 09:03:13 -05:00
Sean McCarthy
ef667e356c introduce filterScriptParams to remove params meant to control scenario script from the activity context 2023-05-24 12:02:35 -07:00
Jeff Banks
02ff160b3c
Metrics Publishing (#1234)
Included the following with core changes to allow labeled metrics for Prometheus exposition format publishing.
* javadoc updates
* remove extra types
* use NBLabeledElement instead of NBNamedElement
* contextualize NBLabeledElement for graphite/metrics
* externalize labeled ScriptContext to API
* add labels to NicerTimer
* remove partial packaging
* more progress on formatting for prom exposition format
* added metrics diagram
* resolve build issues with label params
* resolve build issues with label params
* prometheus export services
* added PromExpoFormat Tests for NBMetricMeter(Counting+Sampling) and NBMetricTimer(Counting)
* added test for Gauge Formatting
* added Gauge Formatting as well as Sampling values (count, stdev ...)
* added sketch for metrics labeling contexts
* add NBLabeledElement in all the places, retool calling paths to use it
* synchronize antlr versions after partial snyk change
* unbreak static initializer block after IntelliJ "fixed" it.
* engine-api - adapt to NBLabeledElement
* adapters-api - adapt to NBLabeledElement
* nb-api - adapt to NBLabeledElement
* engine-core - adapt to NBLabeledElement
* misc-adapters - adapt to NBLabeledElement
* streaming-adapters - adapt to NBLabeledElement
* add missing test
* initial implementation of a prom push reporter
* Resolve build issue with parseGlobalOptions
* replaced with PromPushReporter
* cleanup unused deps
* dependency removal for micrometer
* allow empty labels for tests
* change space.getName to space.getSpaceName
* cleanup poms
* graphite linearization now includes space element
* http adapter should only depend on adapters API
* http space does not create its own metric names
* import cleanups
* improved javadocs
* introduce component concepts

---------

Co-authored-by: Jonathan Shook <jshook@gmail.com>
Co-authored-by: Mike Yaacoub <mike.yaacoub@datastax.com>
2023-05-09 09:52:42 -05:00
Jeff Banks
388ece75f7
MongoDB Adapter fix and updates for scenarios (#1196)
* Updates for mongodb adapter; fix connection usage on cli

* Reduce the cycles for test w/ sporadic failures

* Removed redundant '-> string'

* Removed state tracking and cleanup

* Space cache capture and connection management

* Mongo client management w/ connection

* Remove unused getData()

* Removed debug entry no longer required
2023-04-04 16:09:00 -05:00
Jonathan Shook
f5c4843e6c
Merge pull request #1082 from nosqlbench/nosqlbench-1081-nbrbuild
fix nbr build scripts
2023-02-10 13:36:04 -06:00
Jonathan Shook
02479749b9 nbr build updates 2023-02-10 11:46:47 -06:00
Jonathan Shook
70408621dd nosqlbench-1071 NBIO should favor finding exactly specified file path as a short circuit 2023-02-10 10:20:45 -06:00
Jonathan Shook
da866eae6c make bindings-utilities more useful 2023-02-07 19:16:07 -06:00
Jonathan Shook
262ac17b98 correct binding regexes 2023-02-07 19:16:05 -06:00
Jonathan Shook
0d955b2158 remove incompatible example from bindings-bytebuffers 2023-02-07 18:19:53 -06:00
Jonathan Shook
47c1f2db5a update bindings-bigdecimal to work with J17 enums 2023-02-07 18:19:03 -06:00
Jonathan Shook
90a1ff3df6 fix for #1007 2023-02-06 18:12:51 -06:00
Jonathan Shook
0366235f71 Merge branch 'main' into cleanup-includes 2023-02-06 15:27:37 -06:00
jeffbanks
cc989c8a72 nb5 revamp for scenarios 2023-02-06 14:58:04 -06:00
Jonathan Shook
172ef0847d include examples and fix error message 2023-02-06 01:04:55 -06:00
Jonathan Shook
f12fd6f0c4 partial work towards mapping resources correctly 2023-02-06 00:19:57 -06:00
Jonathan Shook
1eead1add8 centralize jacoco template for reconfiguration 2023-01-26 23:32:15 -06:00
Jonathan Shook
1f5b133f99 version fixes 2023-01-23 15:53:56 -06:00
Jonathan Shook
362e3c6cd3
Merge pull request #942 from nosqlbench/nosqlbench-937
cleanup duplicitous activity parameters during initialization (yaml,type)
2023-01-23 14:19:18 -06:00
Jonathan Shook
a886ff2f82 restore grafana annotator 2023-01-23 13:02:19 -06:00
Jonathan Shook
f63c7aeadb replace CI+CD friendly revision 2023-01-19 16:54:06 -06:00
Jonathan Shook
2847543aa5 update app docs 2023-01-17 19:30:03 -06:00
Jonathan Shook
1ea5c8a1e5 set weight levels and ranges for exported markdown 2023-01-17 19:30:00 -06:00
nb-droid
a49a9e9380 [maven-release-plugin] prepare for next development iteration 2023-01-13 19:51:57 +00:00
nb-droid
67aeba7a34 [maven-release-plugin] prepare release nosqlbench-5.17.0 2023-01-13 19:51:55 +00:00
Jonathan Shook
e04eef814c temporary work-around for release plugin incompatibility 2023-01-13 13:19:35 -06:00
Jonathan Shook
ff188d4860
actions-for-docs (#919)
* updated docs action with version state

* script updates

* misc doc system updates

* Update build.yml

remove versions logic, since zip contains version info now.

* Update build.yml

corrected git add command

* update version format

* updated docs site url
2023-01-13 11:57:08 -06:00
Jonathan Shook
31478ae591
Nosqlbench 904 version5 (#908)
* Ignore .flattened-pom.xml files from the CI-friendly flattening

* Allow newer versions to squelch version warning

* * switch to CI-friendly revisions in Maven
* add flattener
* refactor version tags
* remove unecesary indirection in poms which break version intellisense

* * switch to CI-friendly revisions in Maven
* add flattener
* refactor version tags
* remove unnecessary indirection in poms which break version intellisense

* deleting commented code
* version dep updates
* streamline LUT trimming options
* use a more explicit regex substituion technique
* specify updated okhttp version
* update resources plugin to bugfixed version
* exclude filtering for problematic files
2023-01-12 12:48:50 -06:00
Jeff Banks
9c25ad9336
Code coverage integration (#897)
* Jacoco introduced for full reporting of test coverage.
* All modules included in scope.
* Ability to scope to module OR overall reporting.
* Ability to run in dev environment and bypass as needed w/out GHA.
2023-01-12 11:25:40 -06:00
Jeff Banks
4484292cb6
md5 deprecation (#909)
md5 deprecation per code scan recommendations.
2023-01-12 10:04:19 -06:00
nb-droid
6e93468632 [maven-release-plugin] prepare for next development iteration 2023-01-09 19:36:28 +00:00
nb-droid
d1b9c91e44 [maven-release-plugin] prepare release nosqlbench-4.17.32 2023-01-09 19:36:26 +00:00
Jonathan Shook
f4aa80f9e2 remove previous "nb" module, which will be replaced by the nb5 module soon 2023-01-05 17:26:01 -05:00
Jonathan Shook
1e7d676979 deactivate rest module with old dependencies 2023-01-05 17:26:01 -05:00
Mike Yaacoub
f38a635d71 Modified HDR Digits to 3 to save space on MetricsIntegrationTest. 2023-01-05 15:09:52 -05:00
Jonathan Shook
4f4b4982c8 templatize test logging level for logger setup and root logger 2022-12-20 23:11:55 -06:00
Jonathan Shook
d80632773b merge fixes for: improve concurrency patterns for activity execution 2022-12-20 20:05:17 -06:00
Jonathan Shook
0de80887b1 move log4j.xml to log4j2.xml 2022-12-20 20:03:33 -06:00
Jonathan Shook
1816381f97 make integrated tests more robust 2022-12-20 20:03:12 -06:00
Jonathan Shook
31dc3ce8f7 package housekeeping 2022-12-20 20:02:16 -06:00
Mike Yaacoub
8fd13fcddf added explicit cast to explicitly narrow precision for security purposes 2022-12-19 17:39:30 -05:00
Mike Yaacoub
74bfbc691d Merge branch 'main' into my-NB5-I446 2022-12-19 16:58:50 -05:00
Mike Yaacoub
a06e1556f0 merged my-NB5-I446 with main 2022-12-19 15:57:58 -05:00
Mike Yaacoub
7eeb1d2888 fixed unused dependency failure 2022-12-19 14:05:06 -05:00
Mike Yaacoub
ec8ed024ff Changed build workflow to re-add verify and run on non-release builds, removed unused dependency from nbr 2022-12-19 13:54:39 -05:00
Mike Yaacoub
c149684d09 moved from commonmark to inhouse MutableMarkdown for verification 2022-12-19 13:24:40 -05:00
Jonathan Shook
2b39f3c350 Merge branch 'main' into nosqlbench-797-callable 2022-12-12 15:04:39 -06:00
Jonathan Shook
df534fd83a
Merge branch 'main' into snyk-fix-b8424b271d02d50c5baa47f9e8745c03 2022-12-12 12:33:40 -06:00
MikeYaacoubStax
e7705d13de NB5-I446 Modified export-docs command to export organized driver and binding docs, also removed gendocs command (VirtDataGenDocsApp now used for generating bindings in StringBuilders to pass to the zipstream)
added the export-docs build workflow job
Updated build.yml to fix path to nb5 in workflow
uploading/downloading docs artifacts in build workflow
Attempt to correct the path to the exported docs to upload them
Corrected exported-docs path
Corrected yml build workflow error
corrected naming of exported docs
2022-12-08 12:20:11 -05:00
nb-droid
02522a468e [maven-release-plugin] prepare for next development iteration 2022-12-07 23:30:05 +00:00
nb-droid
fb131e9b09 [maven-release-plugin] prepare release nosqlbench-4.17.31 2022-12-07 23:30:02 +00:00
Jonathan Shook
86c4dfd966 partial fixes for nosqlbench-797 Race condition between exceptional activity shutdown and normal scenario shutdown. 2022-11-30 11:17:10 -06:00
Jonathan Shook
875e2d8f99 temporarily disable test for close exception handler 2022-11-22 23:04:49 -06:00
Jonathan Shook
901034875a fix for second noisy shutdown test 2022-11-22 22:50:53 -06:00
Jonathan Shook
18ba2fdaf9
Merge branch 'main' into shutdown_hooks 2022-11-22 22:24:03 -06:00
Jonathan Shook
28f561aedd direct work-around for flaky test, simple improvements to logging, some improvements to consistent view of status, misc cleanups 2022-11-22 10:49:12 -06:00
Jonathan Shook
16417f37b6 additional diagnostics for spurious test 2022-11-21 10:24:28 -06:00
jeffbanks
dcf7baf867 Dead code utest resurrection and timing w/ gh actions
Cyclerate=10 to test github actions part2
Debug and exception handling
Detailed error handler logging
System out diagnostics
Capture step included
Try-catch diagnostics.
sysout cleanup; general cleanup
2022-11-18 23:52:29 -06:00
Jonathan Shook
ebb6f2c58d include space usage in diag for testing 2022-11-18 23:18:48 -06:00
jeffbanks
2250beb892 Timing change for github actions; warnings cleanup/refactor 2022-11-14 19:25:26 -06:00
Jonathan Shook
33e330dc35
[Snyk] Security upgrade com.amazonaws:aws-java-sdk-s3 from 1.12.325 to 1.12.330 (#768)
* fix: nb-api/pom.xml to reduce vulnerabilities

The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-3038424

* Timing change for github actions; warnings cleanup/refactor

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
Co-authored-by: jeffbanks <jeff.banks@datastax.com>
2022-11-14 18:32:52 -06:00
jeffbanks
33fe4d0a7e Timing change for github actions; warnings cleanup/refactor 2022-11-14 18:10:42 -06:00
nb-droid
d9acf85597 [maven-release-plugin] prepare for next development iteration 2022-09-30 15:58:45 +00:00
nb-droid
098455410d [maven-release-plugin] prepare release nosqlbench-4.17.30 2022-09-30 15:58:43 +00:00
nb-droid
19faf3c0ae [maven-release-plugin] prepare for next development iteration 2022-09-29 17:29:14 +00:00
nb-droid
20a5ab4234 [maven-release-plugin] prepare release nosqlbench-4.17.29 2022-09-29 17:29:12 +00:00
nb-droid
5f72b6c0d8 [maven-release-plugin] prepare for next development iteration 2022-09-21 19:27:40 +00:00
nb-droid
b52e6a4774 [maven-release-plugin] prepare release nosqlbench-4.17.28 2022-09-21 19:27:38 +00:00
Jonathan Shook
3304b1d8d8 improve findmax, stepup analysis methods 2022-09-16 16:18:19 -05:00
nb-droid
60b8ccc200 [maven-release-plugin] prepare for next development iteration 2022-08-25 21:55:33 +00:00
nb-droid
6e8ee26cf9 [maven-release-plugin] prepare release nosqlbench-4.17.27 2022-08-25 21:55:31 +00:00
nb-droid
32ca3657ca [maven-release-plugin] prepare for next development iteration 2022-08-24 22:04:47 +00:00
nb-droid
27686ef5d2 [maven-release-plugin] prepare release nosqlbench-4.17.26 2022-08-24 22:04:44 +00:00
nb-droid
1397cd1b88 [maven-release-plugin] prepare for next development iteration 2022-08-18 21:28:25 +00:00