Commit Graph

65 Commits

Author SHA1 Message Date
Jonathan Shook
3ec511f61f partial progress to moving instrumentation 2023-10-23 08:41:17 -05:00
Jonathan Shook
139ac879c5 cleanup dependencies 2023-10-23 08:41:15 -05:00
dependabot[bot]
e0dec53ba0
build(deps): bump org.apache.avro:avro in /adapter-pulsar
Bumps org.apache.avro:avro from 1.11.2 to 1.11.3.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-29 22:10:02 +00:00
yabinmeng
9a61a1f2af Remove "--docker-metrics" functionality. Update documents and scripts accordingly. 2023-09-28 14:30:18 -05:00
Jonathan Shook
5d0a890e28 doc link fixes - made compatible with the site generator by explicitly qualifying anchor names 2023-09-10 14:54:24 -05:00
snyk-bot
31f639fe4e
fix: upgrade org.apache.pulsar:pulsar-client-admin from 3.0.1 to 3.1.0
Snyk has created this PR to upgrade org.apache.pulsar:pulsar-client-admin from 3.0.1 to 3.1.0.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.pulsar/pulsar-client-admin/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/829dafa3-4836-462a-b1f4-15be99fe372a?utm_source=github&utm_medium=referral&page=upgrade-pr
2023-08-25 17:24:16 +00:00
Jonathan Shook
34dcc7bc1b merging main into vector-mergup 2023-08-16 16:19:49 -05:00
snyk-bot
3456485a3d
fix: upgrade org.apache.commons:commons-lang3 from 3.12.0 to 3.13.0
Snyk has created this PR to upgrade org.apache.commons:commons-lang3 from 3.12.0 to 3.13.0.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.commons/commons-lang3/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/829dafa3-4836-462a-b1f4-15be99fe372a?utm_source=github&utm_medium=referral&page=upgrade-pr
2023-08-16 00:31:32 +00:00
Jeff Banks
dd0f942a27
fix: adapter-pulsar/pom.xml to reduce vulnerabilities (#1432)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-COMGOOGLEGUAVA-5710356

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2023-08-04 16:12:29 -05:00
Jonathan Shook
76b740f8ab add cqlvector examples 2023-07-25 11:50:51 -05:00
Jeff Banks
acbd1b0524
fix: adapter-pulsar/pom.xml to reduce vulnerabilities (#1377)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-3038424
- https://snyk.io/vuln/SNYK-JAVA-COMFASTERXMLJACKSONCORE-3038426

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2023-07-03 14:24:28 -05:00
yabinmeng
f95ad23d0a Minor update of NB Pulsar and Kafka readme 2023-06-28 17:15:35 -05:00
yabinmeng
624720b481 1) Address NB issue #1283 (overlapping label name \"name\" in both parent and child labels when adding Pulsar adapter metrics).
2) Add named scenarios for NB Pulsar and NB Kafka adapters.
2023-06-28 16:33:33 -05:00
yabinmeng
f7d7fd56c7 1) Address NB issue #1283 (overlapping label name \"name\" in both parent and child labels when adding Pulsar adapter metrics).
2) Add named scenarios for NB Pulsar and NB Kafka adapters.
2023-06-28 16:32:55 -05:00
yabinmeng
2730b0aa24 Update NBS4J version to 4.0.1 (which is based on Pulsar client version 3.0.0) and NB Pulsar version to 3.0.0 2023-06-23 21:47:19 -05:00
yabinmeng
046c0fc1c6 Update NBS4J version to 4.0.1 (which is based on Pulsar client version 3.0.0) and NB Pulsar version to 3.0.0 2023-06-23 21:44:30 -05:00
Jeff Banks
a9fb0d7e63 Revert "fix: upgrade org.apache.pulsar:pulsar-client from 2.10.4 to 2.11.1 (#1282)"
This reverts commit 4de2aa67d6.
2023-05-19 16:14:38 -05:00
Jonathan Shook
4de2aa67d6
fix: upgrade org.apache.pulsar:pulsar-client from 2.10.4 to 2.11.1 (#1282)
Snyk has created this PR to upgrade org.apache.pulsar:pulsar-client from 2.10.4 to 2.11.1.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.pulsar/pulsar-client/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/829dafa3-4836-462a-b1f4-15be99fe372a?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2023-05-19 08:51:40 -05:00
yabinmeng
7a5831b4ff 1) NB Kafka and NB S4J adaptor code efficiency improvement (following PR 10115 by Lari on Pulsar adaptor)
2) Downgrade NB Pulsar version from 2.11.1 to 2.10.4 to avoid NB S4J execution failure (runtime version conflict)
3) Fix S4J Adaptor metrics labeling conflict
4) Change the space shutdown logic for NB Pulsar, Kafka, and S4J such that to simply log an error message instead of throwing out a run-time exception (NB is shutting down anyway).
5) Minor update of NB Kafka and NB S4J properties file and add settings for connecting to Astra Streaming
2023-05-16 19:59:39 -05:00
Jeff Banks
02ff160b3c
Metrics Publishing (#1234)
Included the following with core changes to allow labeled metrics for Prometheus exposition format publishing.
* javadoc updates
* remove extra types
* use NBLabeledElement instead of NBNamedElement
* contextualize NBLabeledElement for graphite/metrics
* externalize labeled ScriptContext to API
* add labels to NicerTimer
* remove partial packaging
* more progress on formatting for prom exposition format
* added metrics diagram
* resolve build issues with label params
* resolve build issues with label params
* prometheus export services
* added PromExpoFormat Tests for NBMetricMeter(Counting+Sampling) and NBMetricTimer(Counting)
* added test for Gauge Formatting
* added Gauge Formatting as well as Sampling values (count, stdev ...)
* added sketch for metrics labeling contexts
* add NBLabeledElement in all the places, retool calling paths to use it
* synchronize antlr versions after partial snyk change
* unbreak static initializer block after IntelliJ "fixed" it.
* engine-api - adapt to NBLabeledElement
* adapters-api - adapt to NBLabeledElement
* nb-api - adapt to NBLabeledElement
* engine-core - adapt to NBLabeledElement
* misc-adapters - adapt to NBLabeledElement
* streaming-adapters - adapt to NBLabeledElement
* add missing test
* initial implementation of a prom push reporter
* Resolve build issue with parseGlobalOptions
* replaced with PromPushReporter
* cleanup unused deps
* dependency removal for micrometer
* allow empty labels for tests
* change space.getName to space.getSpaceName
* cleanup poms
* graphite linearization now includes space element
* http adapter should only depend on adapters API
* http space does not create its own metric names
* import cleanups
* improved javadocs
* introduce component concepts

---------

Co-authored-by: Jonathan Shook <jshook@gmail.com>
Co-authored-by: Mike Yaacoub <mike.yaacoub@datastax.com>
2023-05-09 09:52:42 -05:00
Jonathan Shook
ed6593af06
fix: upgrade org.apache.pulsar:pulsar-client from 2.11.0 to 2.11.1 (#1249)
Snyk has created this PR to upgrade org.apache.pulsar:pulsar-client from 2.11.0 to 2.11.1.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.pulsar/pulsar-client/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/829dafa3-4836-462a-b1f4-15be99fe372a?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2023-05-05 09:12:35 -05:00
Jonathan Shook
9295adf03f
Merge pull request #1143 from MMirelli/mm-ls797-continue-add-e2e-error-metrics-to-kafka-adapter
[kafka-adapter] Add e2e error metrics -- to be continued
2023-04-25 09:10:32 -05:00
snyk-bot
265dcce899
fix: upgrade org.apache.commons:commons-configuration2 from 2.8.0 to 2.9.0
Snyk has created this PR to upgrade org.apache.commons:commons-configuration2 from 2.8.0 to 2.9.0.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.commons/commons-configuration2/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/829dafa3-4836-462a-b1f4-15be99fe372a?utm_source=github&utm_medium=referral&page=upgrade-pr
2023-04-18 02:15:34 +00:00
Massimiliano Mirelli
ef92631595 Add flag to run tests for adapters-api and kafka / pulsar adapters 2023-03-07 15:13:47 +02:00
Massimiliano Mirelli
0a2ab1dcca Add scripts to build pulsar and kafka adapters and manually test against local pulsar cluster 2023-03-07 14:52:21 +02:00
Massimiliano Mirelli
8eb9498fa9 Add examples of simple testing pulsar workload 2023-03-07 14:51:23 +02:00
Massimiliano Mirelli
7d5ecf2f32 Enhance pulsar and kafka adapters documentation 2023-03-07 14:15:21 +02:00
Massimiliano Mirelli
971c5d3c6f Add sequence tracking number when seqTracking is enabled in producer 2023-03-07 14:09:34 +02:00
Massimiliano Mirelli
b333898249 Decouple e2e error metrics from adapter modules
Add classes implementing e2e error metrics to adapters-api module in `import io.nosqlbench.engine.api.metrics.ReceivedMessageSequenceTracker`
2023-03-07 14:06:15 +02:00
Massimiliano Mirelli
70d66719db Edit copyright year 2023-02-24 10:44:14 +02:00
Jonathan Shook
42bc77f0f2
Merge pull request #1017 from lhotari/lh-readd-removed-tests
Re-add tests for MessageSequenceNumberSendingHandler and ReceivedMessageSequenceTracker
2023-02-07 13:52:29 -06:00
Lari Hotari
f9fe79818a Re-add tests for MessageSequenceNumberSendingHandler and ReceivedMessageSequenceTracker
- migrate tests from NB4 to NB5
2023-02-07 08:45:53 +02:00
Lari Hotari
5491790751 Optimize cache keys and fix races with caches in PulsarSpace 2023-02-07 08:10:45 +02:00
Lari Hotari
10da1699ed Optimize enum mappings and cache Avro schemas in PulsarAdapterUtil 2023-02-07 08:10:45 +02:00
Jonathan Shook
1eead1add8 centralize jacoco template for reconfiguration 2023-01-26 23:32:15 -06:00
Jonathan Shook
696576a317 Merge branch 'main' of github.com:nosqlbench/nosqlbench 2023-01-23 13:44:11 -06:00
snyk-bot
87074249c1
fix: upgrade org.apache.pulsar:pulsar-client from 2.10.3 to 2.11.0
Snyk has created this PR to upgrade org.apache.pulsar:pulsar-client from 2.10.3 to 2.11.0.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.pulsar/pulsar-client/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/829dafa3-4836-462a-b1f4-15be99fe372a?utm_source=github&utm_medium=referral&page=upgrade-pr
2023-01-22 13:42:54 +00:00
Jonathan Shook
f63c7aeadb replace CI+CD friendly revision 2023-01-19 16:54:06 -06:00
snyk-bot
218eb5f44c
fix: upgrade org.apache.pulsar:pulsar-client-admin from 2.10.1 to 2.10.3
Snyk has created this PR to upgrade org.apache.pulsar:pulsar-client-admin from 2.10.1 to 2.10.3.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.pulsar/pulsar-client-admin/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/829dafa3-4836-462a-b1f4-15be99fe372a?utm_source=github&utm_medium=referral&page=upgrade-pr
2023-01-16 23:37:35 +00:00
nb-droid
a49a9e9380 [maven-release-plugin] prepare for next development iteration 2023-01-13 19:51:57 +00:00
nb-droid
67aeba7a34 [maven-release-plugin] prepare release nosqlbench-5.17.0 2023-01-13 19:51:55 +00:00
Jonathan Shook
e04eef814c temporary work-around for release plugin incompatibility 2023-01-13 13:19:35 -06:00
Jonathan Shook
ff188d4860
actions-for-docs (#919)
* updated docs action with version state

* script updates

* misc doc system updates

* Update build.yml

remove versions logic, since zip contains version info now.

* Update build.yml

corrected git add command

* update version format

* updated docs site url
2023-01-13 11:57:08 -06:00
Jonathan Shook
31478ae591
Nosqlbench 904 version5 (#908)
* Ignore .flattened-pom.xml files from the CI-friendly flattening

* Allow newer versions to squelch version warning

* * switch to CI-friendly revisions in Maven
* add flattener
* refactor version tags
* remove unecesary indirection in poms which break version intellisense

* * switch to CI-friendly revisions in Maven
* add flattener
* refactor version tags
* remove unnecessary indirection in poms which break version intellisense

* deleting commented code
* version dep updates
* streamline LUT trimming options
* use a more explicit regex substituion technique
* specify updated okhttp version
* update resources plugin to bugfixed version
* exclude filtering for problematic files
2023-01-12 12:48:50 -06:00
Jeff Banks
9c25ad9336
Code coverage integration (#897)
* Jacoco introduced for full reporting of test coverage.
* All modules included in scope.
* Ability to scope to module OR overall reporting.
* Ability to run in dev environment and bypass as needed w/out GHA.
2023-01-12 11:25:40 -06:00
nb-droid
6e93468632 [maven-release-plugin] prepare for next development iteration 2023-01-09 19:36:28 +00:00
nb-droid
d1b9c91e44 [maven-release-plugin] prepare release nosqlbench-4.17.32 2023-01-09 19:36:26 +00:00
Jonathan Shook
08fb6ca905 make anchor links zola compatible 2022-12-22 23:54:57 -06:00
Jonathan Shook
972a6af654 remove duplicitous doc file 2022-12-22 23:54:36 -06:00
Mike Yaacoub
74bfbc691d Merge branch 'main' into my-NB5-I446 2022-12-19 16:58:50 -05:00