IntenseFinance/nosqlbench
3
0
Fork 0
mirror of https://github.com/nosqlbench/nosqlbench.git synced 2024-12-24 16:00:09 -06:00
Code Issues Packages Projects Releases Wiki Activity
5,818 Commits 42 Branches 33 Tags 93 MiB
8cb3051af2
Commit Graph

61 Commits

Author SHA1 Message Date
Jonathan Shook
5ae22195ba package factoring 2023-12-09 17:16:44 -06:00
Jonathan Shook
ae19037bce remainder of changes for scenario rework that need to be itemized 2023-12-06 14:26:53 -06:00
Jonathan Shook
f099fa0840 fixup license files 2023-10-23 08:42:19 -05:00
Jonathan Shook
92377b636c all adapters loading indirectly 2023-10-23 08:42:16 -05:00
Jonathan Shook
61b923cfed indirect loading on nbr working 2023-10-23 08:42:00 -05:00
Jonathan Shook
9daf06cd62 rehome all metrics construction patterns, remove ActivityMetrics 2023-10-23 08:41:40 -05:00
Jonathan Shook
57db140a94 removed all ActivityMetrics constructions 2023-10-23 08:41:40 -05:00
Jonathan Shook
4efd428de3 partial switchover of metrics from ActivityMetrics 2023-10-23 08:41:38 -05:00
Jonathan Shook
139ac879c5 cleanup dependencies 2023-10-23 08:41:15 -05:00
snyk-bot
358b9f5e56
fix: adapter-kafka/pom.xml to reduce vulnerabilities 
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-ORGXERIALSNAPPY-5918282
 2023-10-04 17:26:19 +00:00
yabinmeng
9a61a1f2af Remove "--docker-metrics" functionality. Update documents and scripts accordingly. 2023-09-28 14:30:18 -05:00
Jonathan Shook
112ad332cd fixups and package structure updates 2023-09-11 13:20:52 -05:00
Jonathan Shook
bf3e9636c0 remove confusing type and dimensions labeling methods 2023-09-09 20:31:04 -05:00
Jonathan Shook
6c4833d48d differentiate dimension and instance labels 2023-08-28 16:15:10 -05:00
Jonathan Shook
34dcc7bc1b merging main into vector-mergup 2023-08-16 16:19:49 -05:00
snyk-bot
0c577e5b6f
fix: upgrade org.apache.commons:commons-lang3 from 3.12.0 to 3.13.0 
Snyk has created this PR to upgrade org.apache.commons:commons-lang3 from 3.12.0 to 3.13.0.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.commons/commons-lang3/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/bb64938c-e68b-4c38-9e2f-d9d8336c07ee?utm_source=github&utm_medium=referral&page=upgrade-pr
 2023-08-15 02:08:38 +00:00
Jeff Banks
d700979b74
fix: upgrade org.apache.kafka:kafka-clients from 3.5.0 to 3.5.1 (#1434) 
Snyk has created this PR to upgrade org.apache.kafka:kafka-clients from 3.5.0 to 3.5.1.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/bb64938c-e68b-4c38-9e2f-d9d8336c07ee?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
 2023-08-08 10:13:40 -05:00
Jonathan Shook
d1725da0cb merge fixups 2023-07-25 12:06:25 -05:00
Jonathan Shook
76b740f8ab add cqlvector examples 2023-07-25 11:50:51 -05:00
Jonathan Shook
841257dbfd
fix: upgrade org.apache.kafka:kafka-clients from 3.4.0 to 3.5.0 (#1381) 
Snyk has created this PR to upgrade org.apache.kafka:kafka-clients from 3.4.0 to 3.5.0.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/bb64938c-e68b-4c38-9e2f-d9d8336c07ee?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
 2023-07-06 08:01:05 -05:00
yabinmeng
f95ad23d0a Minor update of NB Pulsar and Kafka readme 2023-06-28 17:15:35 -05:00
yabinmeng
f7d7fd56c7 1) Address NB issue #1283 (overlapping label name \"name\" in both parent and child labels when adding Pulsar adapter metrics). 
2) Add named scenarios for NB Pulsar and NB Kafka adapters.
 2023-06-28 16:32:55 -05:00
yabinmeng
b1c06661ce Initial draft of NB5 S4R 2023-05-31 18:21:52 -05:00
yabinmeng
7a5831b4ff 1) NB Kafka and NB S4J adaptor code efficiency improvement (following PR 10115 by Lari on Pulsar adaptor) 
2) Downgrade NB Pulsar version from 2.11.1 to 2.10.4 to avoid NB S4J execution failure (runtime version conflict)
3) Fix S4J Adaptor metrics labeling conflict
4) Change the space shutdown logic for NB Pulsar, Kafka, and S4J such that to simply log an error message instead of throwing out a run-time exception (NB is shutting down anyway).
5) Minor update of NB Kafka and NB S4J properties file and add settings for connecting to Astra Streaming
 2023-05-16 19:59:39 -05:00
Jeff Banks
02ff160b3c
Metrics Publishing (#1234) 
Included the following with core changes to allow labeled metrics for Prometheus exposition format publishing.
* javadoc updates
* remove extra types
* use NBLabeledElement instead of NBNamedElement
* contextualize NBLabeledElement for graphite/metrics
* externalize labeled ScriptContext to API
* add labels to NicerTimer
* remove partial packaging
* more progress on formatting for prom exposition format
* added metrics diagram
* resolve build issues with label params
* resolve build issues with label params
* prometheus export services
* added PromExpoFormat Tests for NBMetricMeter(Counting+Sampling) and NBMetricTimer(Counting)
* added test for Gauge Formatting
* added Gauge Formatting as well as Sampling values (count, stdev ...)
* added sketch for metrics labeling contexts
* add NBLabeledElement in all the places, retool calling paths to use it
* synchronize antlr versions after partial snyk change
* unbreak static initializer block after IntelliJ "fixed" it.
* engine-api - adapt to NBLabeledElement
* adapters-api - adapt to NBLabeledElement
* nb-api - adapt to NBLabeledElement
* engine-core - adapt to NBLabeledElement
* misc-adapters - adapt to NBLabeledElement
* streaming-adapters - adapt to NBLabeledElement
* add missing test
* initial implementation of a prom push reporter
* Resolve build issue with parseGlobalOptions
* replaced with PromPushReporter
* cleanup unused deps
* dependency removal for micrometer
* allow empty labels for tests
* change space.getName to space.getSpaceName
* cleanup poms
* graphite linearization now includes space element
* http adapter should only depend on adapters API
* http space does not create its own metric names
* import cleanups
* improved javadocs
* introduce component concepts

---------

Co-authored-by: Jonathan Shook <jshook@gmail.com>
Co-authored-by: Mike Yaacoub <mike.yaacoub@datastax.com>
 2023-05-09 09:52:42 -05:00
Jonathan Shook
9295adf03f
Merge pull request #1143 from MMirelli/mm-ls797-continue-add-e2e-error-metrics-to-kafka-adapter 
[kafka-adapter] Add e2e error metrics -- to be continued
 2023-04-25 09:10:32 -05:00
Jonathan Shook
7289f400cd
fix: upgrade org.apache.commons:commons-configuration2 from 2.8.0 to 2.9.0 (#1212) 
Snyk has created this PR to upgrade org.apache.commons:commons-configuration2 from 2.8.0 to 2.9.0.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.commons/commons-configuration2/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/bb64938c-e68b-4c38-9e2f-d9d8336c07ee?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
 2023-04-17 16:39:43 -05:00
Massimiliano Mirelli
ef92631595 Add flag to run tests for adapters-api and kafka / pulsar adapters 2023-03-07 15:13:47 +02:00
Massimiliano Mirelli
0a2ab1dcca Add scripts to build pulsar and kafka adapters and manually test against local pulsar cluster 2023-03-07 14:52:21 +02:00
Massimiliano Mirelli
88d291818e Add property instructing kafka client to create ts on record generation 2023-03-07 14:17:45 +02:00
Massimiliano Mirelli
7d5ecf2f32 Enhance pulsar and kafka adapters documentation 2023-03-07 14:15:21 +02:00
Massimiliano Mirelli
e7730ae281 Fix typos 2023-03-07 14:12:28 +02:00
Massimiliano Mirelli
7da1973f52 Make ThreadLocal instance final 2023-03-07 14:10:44 +02:00
Massimiliano Mirelli
b333898249 Decouple e2e error metrics from adapter modules 
Add classes implementing e2e error metrics to adapters-api module in `import io.nosqlbench.engine.api.metrics.ReceivedMessageSequenceTracker`
 2023-03-07 14:06:15 +02:00
Massimiliano Mirelli
c6e7829a2a Simplify handling NPE thrown on empty message sequence number header 2023-03-02 10:17:22 +02:00
Massimiliano Mirelli
97d2a01675 Handle NPE thrown on empty message sequence number header 2023-03-02 10:09:28 +02:00
Massimiliano Mirelli
31fe98b402 Add comment explaining why pulsar-adapter dep is needed 2023-03-01 18:07:06 +02:00
Massimiliano Mirelli
4b5776d720 Fix kafka-adapter pom project groupId and version 2023-03-01 18:07:05 +02:00
Massimiliano Mirelli
7d8c84c567 Add support for end-2-end error metrics on the consumer 2023-03-01 16:47:58 +02:00
Massimiliano Mirelli
e6c39e6f5b Add adapter-pulsar to kafka adapter pom as dependency 
This is probably not the best solution. Encapsulating all the common utilities containing all the E2E error detection tooling would be better.
 2023-03-01 13:40:30 +02:00
Massimiliano Mirelli
375af41131 Add sequence number to produced kafka records 2023-03-01 12:53:57 +02:00
Massimiliano Mirelli
a85ef30a76 Add empty e2e error metrics 2023-02-24 15:44:29 +02:00
Massimiliano Mirelli
e832e5a589 Fix naming inconsistencies 2023-02-24 10:51:58 +02:00
Massimiliano Mirelli
e70952e501 Add e2e message publish metrics to kafka consumer 
The metrics are computed using the kafka client API record.timestamp()ˆ
 2023-02-23 17:31:36 +02:00
Massimiliano Mirelli
c516baaedd Log e2e latency at debugging levelˆ 2023-02-23 14:47:55 +02:00
Massimiliano Mirelli
675f1c99df ˆ 2023-02-22 12:37:47 +02:00
snyk-bot
3e28aab3ef
fix: adapter-kafka/pom.xml to reduce vulnerabilities 
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-ORGAPACHEKAFKA-3317161
 2023-02-09 18:59:08 +00:00
Jonathan Shook
e3a0dedadd
Merge pull request #961 from nosqlbench/nosqlbench-955-jacoco2 
This consolidates our jacoco into mvn-default
 2023-01-28 00:32:46 -06:00
snyk-bot
5803aa73b8
fix: upgrade org.apache.kafka:kafka-clients from 3.3.1 to 3.3.2 
Snyk has created this PR to upgrade org.apache.kafka:kafka-clients from 3.3.1 to 3.3.2.

See this package in Maven Repository:
https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients/

See this project in Snyk:
https://app.snyk.io/org/nosqlbench/project/bb64938c-e68b-4c38-9e2f-d9d8336c07ee?utm_source=github&utm_medium=referral&page=upgrade-pr
 2023-01-27 18:28:30 +00:00
Jonathan Shook
1eead1add8 centralize jacoco template for reconfiguration 2023-01-26 23:32:15 -06:00