IntenseFinance/nosqlbench
3
0 Fork 0
mirror of https://github.com/nosqlbench/nosqlbench.git synced 2026-10-12 17:32:01 -05:00
Code Issues Packages Projects Releases Wiki Activity
5,419 Commits 54 Branches 72 Tags
9a61a1f2af0d39d07566a1272e23e79d401e59d2
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
yabinmeng 9a61a1f2af Remove "--docker-metrics" functionality. Update documents and scripts accordingly. 2023-09-28 14:30:18 -05:00
yabinmeng e3a8640192 Rename adapter-s4r to adapter-amqp Part 2: rename variables and string literals from s4r to amqp 2023-06-30 10:49:05 -05:00
yabinmeng 1880dfa9a0 Rename adapter-s4r to adapter-amqp since this is a generic AMQP-0-9-1 implementation, which includes Pulsar+S4R plugin. 2023-06-30 10:03:57 -05:00
yabinmeng cd0a62f339 Merge pull request #1372 from yabinmeng/main 
Minor NB Pulsar and NB Kafka README update
 2023-06-29 09:58:26 -05:00
yabinmeng 41173c0acf Merge branch 'nosqlbench:main' into main 2023-06-28 17:16:36 -05:00
yabinmeng f95ad23d0a Minor update of NB Pulsar and Kafka readme 2023-06-28 17:15:35 -05:00
yabinmeng 624720b481 1) Address NB issue #1283 (overlapping label name \"name\" in both parent and child labels when adding Pulsar adapter metrics). 
2) Add named scenarios for NB Pulsar and NB Kafka adapters.
 2023-06-28 16:33:33 -05:00
yabinmeng f7d7fd56c7 1) Address NB issue #1283 (overlapping label name \"name\" in both parent and child labels when adding Pulsar adapter metrics). 
2) Add named scenarios for NB Pulsar and NB Kafka adapters.
 2023-06-28 16:32:55 -05:00
yabinmeng 07939cfd3e Add heading block in S4R doc 2023-06-28 09:03:57 -05:00
yabinmeng a26bdb2d38 Change to named steps within the named scenarios for S4J and S4R. Minor doc update based on the reviewer comment. 2023-06-27 17:21:10 -05:00
yabinmeng 46ef2ba96d Merge branch 'nosqlbench:main' into main 2023-06-27 15:20:35 -05:00
yabinmeng 5e7451fa49 Add named scenarios for S4J and S4R. Update the README doc accordingly. 2023-06-27 15:00:38 -05:00
yabinmeng 022cab2f6f Delete sanity validation bash scripts for S4J and S4R 2023-06-26 12:34:23 -05:00
yabinmeng e3009671b6 Update NB S4R, add README doc and testing scripts 2023-06-26 12:09:42 -05:00
yabinmeng ccb1e15d34 Update NB S4R, add README doc and testing scripts 2023-06-26 12:05:02 -05:00
yabinmeng b672c057bc Merge branch 'nosqlbench:main' into main 2023-06-26 10:00:57 -05:00
yabinmeng 2730b0aa24 Update NBS4J version to 4.0.1 (which is based on Pulsar client version 3.0.0) and NB Pulsar version to 3.0.0 2023-06-23 21:47:19 -05:00
yabinmeng 2d1164d014 Merge branch 'nosqlbench:main' into main 2023-06-23 21:45:15 -05:00
yabinmeng 046c0fc1c6 Update NBS4J version to 4.0.1 (which is based on Pulsar client version 3.0.0) and NB Pulsar version to 3.0.0 2023-06-23 21:44:30 -05:00
yabinmeng e559c4628e Address review comment for PR1308 2023-06-05 09:44:10 -05:00
yabinmeng 0639cd4ffd Merge branch 'nosqlbench:main' into main 2023-06-05 09:15:36 -05:00
yabinmeng ba30e80ce8 Initial draft of NB5 S4R 2023-05-31 19:15:21 -05:00
yabinmeng 330c24f6d0 Initial draft of NB5 S4R 2023-05-31 18:41:09 -05:00
yabinmeng b1c06661ce Initial draft of NB5 S4R 2023-05-31 18:21:52 -05:00
yabinmeng 7a5831b4ff 1) NB Kafka and NB S4J adaptor code efficiency improvement (following PR 10115 by Lari on Pulsar adaptor) 
2) Downgrade NB Pulsar version from 2.11.1 to 2.10.4 to avoid NB S4J execution failure (runtime version conflict)
3) Fix S4J Adaptor metrics labeling conflict
4) Change the space shutdown logic for NB Pulsar, Kafka, and S4J such that to simply log an error message instead of throwing out a run-time exception (NB is shutting down anyway).
5) Minor update of NB Kafka and NB S4J properties file and add settings for connecting to Astra Streaming
 2023-05-16 19:59:39 -05:00
yabinmeng 3b6277dbcb Merge pull request #1135 from MMirelli/mm-ls797-add-e2e-error-metrics-to-kafka-adapter 
[kafka-adapter] Add e2e error metrics
 2023-02-24 16:14:48 -06:00
yabinmeng c9be060d50 Merge pull request #1127 from nosqlbench/mm-ls750-add-e2e-latency-metrics-to-kafka-adapter 
[kafka-adapter] Add e2e latency metrics
 2023-02-24 16:12:59 -06:00
yabinmeng 2aeeea125c Separate ActivityExecutor.forceStop() method out from the stop() method. Removing the "forcing" flag in the stop() method. 2023-01-11 17:34:48 -06:00
yabinmeng c172a140eb Add 'forceStop' NB CLI option in NB5 (port over from similar functionally in NB4 - PR# 675) 2023-01-11 16:49:23 -06:00
yabinmeng 37f889359e Fix Kafka producer transaction issue and add support for subscribing from multiple topics. 
NB yaml files and README update
 2022-12-13 21:57:05 -06:00
yabinmeng 1850a7c16f Add NB5 Kafka driver 2022-12-12 21:55:52 -06:00
yabinmeng c08f8b6b3d 1) Add NB5 S4J example yaml files 
2) Update NB5 S4J readme file
3) Minor code adjustments
 2022-12-07 22:14:03 -06:00
yabinmeng 9a00af0806 Merge branch 'nosqlbench:main' into main 2022-12-07 21:58:35 -06:00
yabinmeng b3d3f59d40 Migrate NBS4J from NB4 to NB5 2022-12-06 17:24:52 -06:00
yabinmeng c077218e8a Fix example yaml file producer and consumer parameter names 2022-11-21 21:05:26 -06:00
yabinmeng f7c9e50f35 Add README file and adjust sample yaml files 2022-11-19 18:53:58 -06:00
yabinmeng bd8aff9081 Added support for 1) subscription initial position and 2) regex subscription mode 
Also fixed a consumer configuration loading issue with dlq/negAck/ackTimeout policy using Pulsar client's loadConf() API.

Addressed the review comments in PR #791; reverted the changes in ActivityMetrics and BasedOpDispenser
 2022-11-18 18:22:04 -06:00
yabinmeng 0de7102884 Add support for 
1) producer message compression
2) consumer DLT/negAck/ackTimeout policy

Add sample NB scenario yaml files
 2022-11-17 18:47:37 -06:00
yabinmeng 12f8697e0e Complete the first draft of NB5 Pulsar adapter code, except the per-thread rate limiter. 2022-11-17 10:39:37 -06:00
yabinmeng e128d2867a Merge branch 'nosqlbench:main' into main 2022-11-17 10:27:28 -06:00
yabinmeng 0c71696b15 Completed NB4 Pulsar driver code migration to NB5. But there are still a few lingering issues due to the difference of how NB4 vs NB5 driver/adapter interacts with the NB engine. 2022-11-16 15:37:26 -06:00
yabinmeng ca6f2b052b Add PulsarAdapter specific metrics 2022-11-14 15:27:57 -06:00
yabinmeng c309d200ac - Delete unsed code "driver-jms" 
- NB5 Pulsar driver Admin functionality in place - create/delete tenant, namespace, and topic
 2022-11-11 18:36:06 -06:00
yabinmeng 0d85dc02b4 1) Remove NB Pulsar batch processing. By default, Pulsar client API batches message sending with configurable parameters. Update README file accordingly. 
2) Remove "driver-jms" module (which is practically useless). Starlight for JMS (S4J) driver that is currently available in "nb4-maintenance" branch will be used for JMS oriented testing against Apache Pulsar.
 2022-08-07 13:33:54 -05:00
yabinmeng 7b0c6d07e5 1) Remove NB Pulsar batch processing. By default, Pulsar client API batches message sending with configurable parameters. Update README file accordingly. 
2) Remove "driver-jms" module (which is practically useless). Starlight for JMS (S4J) driver that is currently available in "nb4-maintenance" branch will be used for JMS oriented testing against Apache Pulsar.
 2022-08-07 13:33:44 -05:00
Yabin Meng 4076f79b63 README update 2021-11-13 21:28:54 -06:00
Yabin Meng 591f21b35a README and code cleanup 2021-11-05 15:48:17 -05:00
Yabin Meng ed4ba2725e Convert message error runtime exception to counter metrics 2021-10-13 11:27:54 -05:00
Yabin Meng ac2fdf2a2e Convert message error runtime exception to counter metrics 2021-10-13 10:11:20 -05:00
Yabin Meng 686af3d9ca Convert message error runtime exception to counter metrics 2021-10-13 10:10:48 -05:00
Yabin Meng 2876e102a6 Remove namespace and topic level dedup setting check for each run, which is too much overhead. 2021-10-09 12:29:22 -05:00
Yabin Meng 11073cf0aa Address Shooky's comment in PR#364. 2021-10-07 14:56:51 -05:00
yabinmeng cdfb7c3224 Merge branch 'nosqlbench:main' into main 2021-10-07 14:53:49 -05:00
Yabin Meng 265760d184 Address Lari's comment in PR#364. 2021-10-06 11:41:50 -05:00
Yabin Meng eb5f42f46d Fix NullPointerException of checking namespace and topic deduplication status. 2021-10-06 10:55:32 -05:00
Yabin Meng 647bd7d430 Merge branch 'main' of https://github.com/yabinmeng/nosqlbench 2021-10-06 10:36:41 -05:00
Yabin Meng 48a6e9ae5d Merge remote-tracking branch 'upstream/main' 2021-10-02 16:23:41 -05:00
yabinmeng bf98d6440f Merge branch 'nosqlbench:main' into main 2021-10-02 16:23:29 -05:00
Yabin Meng 793af965af Ignore abnormal message processing error for Shared and Key_Shared subscription type. 2021-10-02 16:23:12 -05:00
Yabin Meng d781e1ac22 README update about customized Pular driver exception 2021-09-29 11:37:07 -05:00
Yabin Meng 949790b797 Add customized Pulsar driver exceptions 2021-09-29 11:27:24 -05:00
Yabin Meng f72bdb6b95 Readme TOC update 2021-09-28 23:19:49 -05:00
Yabin Meng e9e7e95113 Add support for message duplication check (if message de-duplication is enabled, either at broker-level, namespace-level, or topic-level) 2021-09-28 23:04:46 -05:00
Yabin Meng 3ed1e7f6da Merge branch 'main' of https://github.com/yabinmeng/nosqlbench 2021-09-24 09:47:30 -05:00
Yabin Meng d8bc8ce4fd README update 2021-09-24 09:46:45 -05:00
yabinmeng aff249fee8 Merge branch 'nosqlbench:main' into main 2021-09-24 09:35:12 -05:00
Yabin Meng da7fd8948e Merge remote-tracking branch 'upstream/main' 2021-09-24 09:28:25 -05:00
Yabin Meng 6903a23049 README update 2021-09-24 09:27:43 -05:00
Yabin Meng 3ef807172b NB Pulsar driver enhancement (milestone 11: https://github.com/nosqlbench/nosqlbench/milestone/11) 
- add support for message properties
- add support for end-to-end latency measurement (latency histogram)
- add support for message out-of-order and loss detect
- add async api for consumer
- split single-topic consumer and multi-topic consumer
- SSL/TLS bug fix
- Code cleanup
 2021-09-22 17:17:53 -05:00
Yabin Meng c175411cdb Update Pulsar API to 2.8.0 2021-07-20 13:31:40 -05:00
yabinmeng-gitee adfbcacb2c Maven central pom update 2021-05-11 16:35:37 -05:00
yabinmeng-gitee 87385c15d7 Maven central pom update 2021-05-06 16:44:39 -05:00
yabinmeng-gitee e6491433ae Add Pulsar client, producer, and consumer configuration support 2021-05-05 20:18:26 -05:00
yabinmeng-gitee b1fe01e591 Add support for JMS consumer 2021-05-04 15:51:20 -05:00
yabinmeng-gitee 94cbd9c22c Add support for JMS header fields and message properties 2021-05-03 22:50:15 -05:00
yabinmeng-gitee 0e33401453 Merge remote-tracking branch 'upstream/main' into main 
# Conflicts:
#	driver-jms/pom.xml
#	driver-jms/src/main/java/io/nosqlbench/driver/jms/JmsAction.java
#	driver-jms/src/main/java/io/nosqlbench/driver/jms/JmsActivity.java
#	driver-jms/src/main/java/io/nosqlbench/driver/jms/JmsActivityType.java
#	driver-jms/src/main/java/io/nosqlbench/driver/jms/ReadyJmsOp.java
#	driver-jms/src/main/java/io/nosqlbench/driver/jms/ops/JmsMsgSendMapper.java
#	driver-jms/src/main/java/io/nosqlbench/driver/jms/ops/JmsMsgSendOp.java
#	driver-jms/src/main/java/io/nosqlbench/driver/jms/ops/JmsOp.java
#	driver-jms/src/main/java/io/nosqlbench/driver/jms/ops/JmsTimeTrackOp.java
#	driver-jms/src/main/java/io/nosqlbench/driver/jms/util/PulsarJmsActivityUtil.java
#	pom.xml
 2021-05-03 13:36:27 -05:00
yabinmeng-gitee f4aafb70c3 Pulsar JMS driver type name change and package name typo fix 2021-05-03 13:33:59 -05:00
yabinmeng 09615afb0b Revert "Add NB support for Pulsar JMS" 2021-05-03 11:51:21 -05:00
yabinmeng fd2d5c8b61 Revert "Pulsar JMS driver type name change and package name typo fix" 2021-05-01 09:03:07 -05:00
yabinmeng-gitee 136c920e24 Pulsar JMS driver type name change and package name typo fix 2021-04-30 18:42:40 -05:00
yabinmeng-gitee f75ccc1b09 Add Pulsar JMS driver 2021-04-30 17:33:57 -05:00
yabinmeng-gitee 6c9b79993f Add Pulsar JMS driver 2021-04-30 16:40:15 -05:00
yabinmeng-gitee 0aef0e0da3 Add Pulsar JMS driver 2021-04-30 16:37:56 -05:00
yabinmeng-gitee 657d5b80b6 Add Producer config parameters in config.properties: 
- blockIfQueueFull
- maxPendingMessages
- batchingMaxMessages
 2021-04-09 19:14:21 -05:00
yabinmeng-gitee 7fdd6af284 Bug fix of Pulsar Async API 2021-04-09 18:10:14 -05:00
Yabin Meng 49ea75f61b Yaml file minor update 2021-04-04 12:58:20 -05:00
Yabin Meng 2d4fb33ab7 - Admin API for deletion (of tenants, namespaces, and topics) 
- Separate Admin API for tenants and namespaces
- Code standardization
- README update
- Separate the big pulsar.yaml into files for different scenarios
 2021-04-03 23:38:30 -05:00
Yabin Meng d5cfdf13da - AuthN and AuthR for Client API 
- Pulsar client caching fix
- Other minor fix
 2021-04-01 21:49:19 -05:00
Yabin Meng a130ffb8dd Admin API update - check tenant/namespace/topic existence before creating it. 2021-03-31 20:18:00 -05:00
Yabin Meng 098708e004 Address merge conflict 2021-03-30 16:32:53 -05:00
Yabin Meng 4b46c7ef83 Minor merge conflict 2021-03-29 22:04:24 -05:00
Yabin Meng c97b57a488 Add Pulsar Admin API support for creating topics (partitioned or not) 2021-03-29 21:01:26 -05:00
Yabin Meng bc7dce5b04 Document update to reflect Admin API change 2021-03-29 10:11:28 -05:00
Yabin Meng e77635437c Address merge conflict 2021-03-25 16:38:00 -05:00
Yabin Meng 5fd0992e30 Add support for Pulsar Admin API, currently only supporting: 
- create tenant
- create namespace
 2021-03-25 16:08:58 -05:00
Yabin Meng a725bcc11d Merge remote-tracking branch 'nosqlbench/main' into main 2021-03-16 15:02:48 -05:00
Yabin Meng 5d6155ae3b Minor tweak of pom.xml 2021-03-16 15:02:43 -05:00
Yabin Meng 635e2a243c Typo fix in README 2021-03-10 17:32:27 -06:00
Yabin Meng 8b1ff8bbcc Yaml file minor change 2021-03-10 17:30:42 -06:00
Yabin Meng b29713730a Merge remote-tracking branch 'nosqlbench/main' into main 2021-03-10 17:20:35 -06:00