IntenseFinance/nosqlbench
3
0
Fork 0
mirror of https://github.com/nosqlbench/nosqlbench.git synced 2025-02-25 18:55:28 -06:00
Code Issues Packages Projects Releases Wiki Activity

doc link fixes - made compatible with the site generator by explicitly qualifying anchor names

Browse Source
This commit is contained in:
Jonathan Shook
2023-09-10 14:54:24 -05:00
parent 186e87ac1b
commit 5d0a890e28
3 changed files with 61 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
adapter-amqp/src/main/resources/amqp.md
View File

@@ -13,7 +13,7 @@ title: AMQP
--- ---
# 1. Overview # 1. Overview {#1-overview}
The NB AMQP adapter allows sending messages to or receiving messages from The NB AMQP adapter allows sending messages to or receiving messages from
* an AMQP 0-9-1 based server (e.g. RabbitMQ), or * an AMQP 0-9-1 based server (e.g. RabbitMQ), or
@@ -30,9 +30,9 @@ At high level, this adapter supports the following AMQP 0-9-1 functionalities
* Supports message-receive based on binding keys * Supports message-receive based on binding keys
* Receiving messages from AMQP queues with async. consumer acks * Receiving messages from AMQP queues with async. consumer acks
# 2. NB AMQP Usage # 2. NB AMQP Usage {#2-nb-amqp-usage}
## 2.1. Workload Definition ## 2.1. Workload Definition {#21-workload-definition}
There are two main types of workloads supported by this adapter: There are two main types of workloads supported by this adapter:
* Message sender workload (see [amqp_msg_sender.yaml](scenarios/amqp_msg_sender.yaml)) * Message sender workload (see [amqp_msg_sender.yaml](scenarios/amqp_msg_sender.yaml))
@@ -53,7 +53,7 @@ $ <nb_cmd> run driver=amqp -vv cycles=200 strict_msg_error_handling=0 \
config=/path/to/amqp_config.properties config=/path/to/amqp_config.properties
``` ```
### 2.1.1. Named Scenarios ### 2.1.1. Named Scenarios {#211-named-scenarios}
For workload execution convenience, NB engine has the concept of **named scenario** ([doc](https://docs.nosqlbench.io/workloads-101/11-named-scenarios/)). For workload execution convenience, NB engine has the concept of **named scenario** ([doc](https://docs.nosqlbench.io/workloads-101/11-named-scenarios/)).
@@ -68,7 +68,7 @@ $ <nb_cmd> nbamqp_msg_proc_named msg_send
$ <nb_cmd> nbamqp_msg_proc_named msg_recv $ <nb_cmd> nbamqp_msg_proc_named msg_recv
``` ```
## 2.2. CLI parameters ## 2.2. CLI parameters {#22-cli-parameters}
The following CLI parameters are unique to this adapter: The following CLI parameters are unique to this adapter:
@@ -80,9 +80,9 @@ The following CLI parameters are unique to this adapter:
* for message sender workload, it is the number of message publishers for each exchange * for message sender workload, it is the number of message publishers for each exchange
* for message receiver workload, it is the number of message consumers for each queue * for message receiver workload, it is the number of message consumers for each queue
## 2.3. Configuration Properties ## 2.3. Configuration Properties {#23-configuration-properties}
### 2.3.1. Global Properties File ### 2.3.1. Global Properties File {#231-global-properties-file}
A global AMQP properties file can be specified via the `config` CLI parameter. It includes the following required properties: A global AMQP properties file can be specified via the `config` CLI parameter. It includes the following required properties:
* `amqpSrvHost`: AMQP server host (e.g. An Astra Streaming cluster with S4R enabled) * `amqpSrvHost`: AMQP server host (e.g. An Astra Streaming cluster with S4R enabled)
@@ -95,7 +95,7 @@ A global AMQP properties file can be specified via the `config` CLI parameter. I
An example of this file can be found from: [amqp_config.properties](conf/amqp_config.properties) An example of this file can be found from: [amqp_config.properties](conf/amqp_config.properties)
### 2.3.2. Scenario Document Level Properties ### 2.3.2. Scenario Document Level Properties {#232-scenario-document-level-properties}
For message sender workload, the following Document level configuration parameters are supported in the YAML file: For message sender workload, the following Document level configuration parameters are supported in the YAML file:
* `publisher_confirm`: whether to use publisher confirms * `publisher_confirm`: whether to use publisher confirms

20
adapter-pulsar/src/main/resources/pulsar.md
View File

@@ -35,11 +35,11 @@ The NB Pulsar driver allows you to simulate and run different types of workloads
* Negative acknowledgement and acknowledgement timeout redelivery backoff policy * Negative acknowledgement and acknowledgement timeout redelivery backoff policy
## 1.1. Issues Tracker ## 1.1. Issues Tracker {#11-issues-tracker}
If you have issues or new requirements for this driver, please add them at the [pulsar issues tracker](https://github.com/nosqlbench/nosqlbench/issues/new?labels=pulsar). If you have issues or new requirements for this driver, please add them at the [pulsar issues tracker](https://github.com/nosqlbench/nosqlbench/issues/new?labels=pulsar).
# 2. Execute the NB Pulsar Driver Workload # 2. Execute the NB Pulsar Driver Workload {#2-execute-the-nb-pulsar-driver-workload}
In order to run a NB Pulsar driver workload, it follows similar command as other NB driver types. But it does have its unique execution parameters. The general command has the following format: In order to run a NB Pulsar driver workload, it follows similar command as other NB driver types. But it does have its unique execution parameters. The general command has the following format:
@@ -52,7 +52,7 @@ In the above command, make sure the driver type is **pulsar** and provide the fo
* ***service_url***: Pulsar native protocol service url and default to "pulsar://localhost:6650" * ***service_url***: Pulsar native protocol service url and default to "pulsar://localhost:6650"
* ***config***: Pulsar schema/client/producer/consumer related configuration (as a property file) * ***config***: Pulsar schema/client/producer/consumer related configuration (as a property file)
## 2.1. NB Pulsar Driver Yaml File High Level Structure ## 2.1. NB Pulsar Driver Yaml File High Level Structure {#21-nb-pulsar-driver-yaml-file-high-level-structure}
Just like other NB driver types, the actual NB Pulsar workload is defined in a YAML file with the following high level structure: Just like other NB driver types, the actual NB Pulsar workload is defined in a YAML file with the following high level structure:
@@ -84,7 +84,7 @@ blocks:
* ***params***: This section defines **Document level** configuration parameters that apply to all OpTemplate blocks. * ***params***: This section defines **Document level** configuration parameters that apply to all OpTemplate blocks.
* ***blocks***: This section defines the OpTemplate blocks that are needed to execute Pulsar specific workloads. Each OpTemplate block may contain multiple OpTemplates. * ***blocks***: This section defines the OpTemplate blocks that are needed to execute Pulsar specific workloads. Each OpTemplate block may contain multiple OpTemplates.
## 2.2. NB Pulsar Driver Configuration Parameters ## 2.2. NB Pulsar Driver Configuration Parameters {#22-nb-pulsar-driver-configuration-parameters}
The NB Pulsar driver configuration parameters can be set at 3 different levels: The NB Pulsar driver configuration parameters can be set at 3 different levels:
* Global level * Global level
@@ -95,7 +95,7 @@ The NB Pulsar driver configuration parameters can be set at 3 different levels:
Please **NOTE** that when a parameter is specified at multiple levels, the one at the lowest level takes precedence. Please **NOTE** that when a parameter is specified at multiple levels, the one at the lowest level takes precedence.
### 2.2.1. Global Level Parameters ### 2.2.1. Global Level Parameters {#221-global-level-parameters}
The parameters at this level are those listed in the command line config properties file. The parameters at this level are those listed in the command line config properties file.
@@ -152,7 +152,7 @@ categories of the configuration parameters:
* This section defines all configuration parameters that are related with defining a Pulsar Consumer object. * This section defines all configuration parameters that are related with defining a Pulsar Consumer object.
* See [Pulsar Doc Reference](http://pulsar.apache.org/docs/en/client-libraries-java/#configure-consumer) * See [Pulsar Doc Reference](http://pulsar.apache.org/docs/en/client-libraries-java/#configure-consumer)
### 2.2.2. Document Level Parameters ### 2.2.2. Document Level Parameters {#222-document-level-parameters}
For the Pulsar NB driver, Document level parameters can only be statically bound; and currently, the following Document level configuration parameters are supported: For the Pulsar NB driver, Document level parameters can only be statically bound; and currently, the following Document level configuration parameters are supported:
@@ -180,7 +180,7 @@ For the Pulsar NB driver, Document level parameters can only be statically bound
* `message_event_time` : uses the message event timestamp as the starting time * `message_event_time` : uses the message event timestamp as the starting time
* `message_property_e2e_starting_time` : uses a message property `e2e_starting_time` as the starting time. * `message_property_e2e_starting_time` : uses a message property `e2e_starting_time` as the starting time.
# 3. NB Pulsar Driver OpTemplates # 3. NB Pulsar Driver OpTemplates {#3-nb-pulsar-driver-optemplates}
For the NB Pulsar driver, each OpTemplate has the following format: For the NB Pulsar driver, each OpTemplate has the following format:
```yaml ```yaml
@@ -214,9 +214,9 @@ Its value is mandatory and depending on the actual identifier, its value can be
Each Pulsar `OpType` may have optional Op specific parameters. Please refer to [here](yaml_examples) for the example NB Pulsar YAML files for each OpType Each Pulsar `OpType` may have optional Op specific parameters. Please refer to [here](yaml_examples) for the example NB Pulsar YAML files for each OpType
# 4. Message Generation and Schema Support # 4. Message Generation and Schema Support {#4-message-generation-and-schema-support}
## 4.1. Message Generation ## 4.1. Message Generation {#41-message-generation}
A Pulsar message has three main components: message key, message properties, and message payload. Among them, message payload is mandatory when creating a message. A Pulsar message has three main components: message key, message properties, and message payload. Among them, message payload is mandatory when creating a message.
@@ -244,7 +244,7 @@ For `msg_value`, its value can be either
* a plain simple text, or * a plain simple text, or
* a JSON string that follows the specified "value" Avro schema (when Avro schema or KeyValue schema is used) * a JSON string that follows the specified "value" Avro schema (when Avro schema or KeyValue schema is used)
## 4.2. Schema Support ## 4.2. Schema Support {#42-schema-support}
The NB Pulsar driver supports the following Pulsar schema types: The NB Pulsar driver supports the following Pulsar schema types:
* Primitive schema types * Primitive schema types

18
adapter-s4j/src/main/resources/s4j.md
View File

@@ -14,13 +14,13 @@ title: S4J
--- ---
# 1. Overview # 1. Overview {#1-overview}
This driver is similar to [NB Pulsar driver](../../../../adapter-pulsar/src/main/resources/pulsar.md) that allows NB based workload generation and performance testing against a Pulsar cluster. It also follows a similar pattern to configure and connect to the Pulsar cluster for workload execution. This driver is similar to [NB Pulsar driver](../../../../adapter-pulsar/src/main/resources/pulsar.md) that allows NB based workload generation and performance testing against a Pulsar cluster. It also follows a similar pattern to configure and connect to the Pulsar cluster for workload execution.
However, the major difference is instead of simulating native Pulsar client workloads, the NB S4J driver allows simulating JMS oriented workloads (that follows JMS spec 2.0 and 1.1) to be executed on the Pulsar cluster. Under the hood, this is achieved through DataStax's [Starlight for JMS API] (https://github.com/datastax/pulsar-jms). However, the major difference is instead of simulating native Pulsar client workloads, the NB S4J driver allows simulating JMS oriented workloads (that follows JMS spec 2.0 and 1.1) to be executed on the Pulsar cluster. Under the hood, this is achieved through DataStax's [Starlight for JMS API] (https://github.com/datastax/pulsar-jms).
# 2. Execute NB S4J Workload # 2. Execute NB S4J Workload {#2-execute-nb-s4j-workload}
The following is an example of executing a NB S4J workload (defined as *pulsar_s4j.yaml*) The following is an example of executing a NB S4J workload (defined as *pulsar_s4j.yaml*)
@@ -49,7 +49,7 @@ Other NB engine parameters are straight forward:
* yamL: the NB S4J scenario definition yaml file * yamL: the NB S4J scenario definition yaml file
* config: specify the file that contains the connection parameters used by the S4J API * config: specify the file that contains the connection parameters used by the S4J API
# 3. NB S4J Driver Configuration Parameter File # 3. NB S4J Driver Configuration Parameter File {#3-nb-s4j-driver-configuration-parameter-file}
The S4J API has a list of configuration options that can be found here: https://docs.datastax.com/en/streaming/starlight-for-jms/latest/reference/pulsar-jms-reference.html. The S4J API has a list of configuration options that can be found here: https://docs.datastax.com/en/streaming/starlight-for-jms/latest/reference/pulsar-jms-reference.html.
@@ -104,7 +104,7 @@ producer.blockIfQueueFull=true
#... #...
``` ```
# 4. NB S4J Scenario Definition File # 4. NB S4J Scenario Definition File {#4-nb-s4j-scenario-definition-file}
Like any NB scenario yaml file, the NB S4J yaml file is composed of 3 major components: Like any NB scenario yaml file, the NB S4J yaml file is composed of 3 major components:
* bindings: define NB bindings * bindings: define NB bindings
@@ -120,7 +120,7 @@ blocks:
... ... ... ...
``` ```
## 4.1. Document Level Parameters ## 4.1. Document Level Parameters {#41-document-level-parameters}
The parameters defined in this section will be applicable to all statement blocks. An example of some common parameters that can be set at the document level is listed below: The parameters defined in this section will be applicable to all statement blocks. An example of some common parameters that can be set at the document level is listed below:
* temporary_dest: whether JMS workload is dealing with a temporary destination * temporary_dest: whether JMS workload is dealing with a temporary destination
@@ -139,7 +139,7 @@ params:
Please **NOTE** that the above parameters won't necessarily be specified at the document level. If they're specified at the statement level, they will only impact the statement within which they're specified. Please **NOTE** that the above parameters won't necessarily be specified at the document level. If they're specified at the statement level, they will only impact the statement within which they're specified.
## 4.2. NB S4J Workload Types ## 4.2. NB S4J Workload Types {#42-nb-s4j-workload-types}
The NB S4J driver supports 2 types of JMS operations: The NB S4J driver supports 2 types of JMS operations:
* One for message producing/sending/publishing * One for message producing/sending/publishing
@@ -147,7 +147,7 @@ The NB S4J driver supports 2 types of JMS operations:
* One for message consuming/receiving/subscribing * One for message consuming/receiving/subscribing
* this is identified by NB Op identifier ***MessageConsume*** * this is identified by NB Op identifier ***MessageConsume***
### 4.2.1. Publish Messages to a JMS Destination, Queue or Topic ### 4.2.1. Publish Messages to a JMS Destination, Queue or Topic {#421-publish-messages-to-a-jms-destination-queue-or-topic}
***NOTE**: Please see [pulsar_s4j_producer.yaml](scenarios/pulsar_s4j_producer.yaml) as the complete example.* ***NOTE**: Please see [pulsar_s4j_producer.yaml](scenarios/pulsar_s4j_producer.yaml) as the complete example.*
@@ -184,7 +184,7 @@ blocks:
msg_body: "{mytext_val}" msg_body: "{mytext_val}"
``` ```
### 4.2.2. Receiving Messages from a JMS Destination, Queue or Topic ### 4.2.2. Receiving Messages from a JMS Destination, Queue or Topic {#422-receiving-messages-from-a-jms-destination-queue-or-topic}
***NOTE**: Please see [pulsar_s4j_consumer.yaml](scenarios/pulsar_s4j_consumer.yaml) as the complete example.* ***NOTE**: Please see [pulsar_s4j_consumer.yaml](scenarios/pulsar_s4j_consumer.yaml) as the complete example.*
@@ -254,7 +254,7 @@ blocks:
} }
``` ```
## 4.3. S4J Named Scenario ## 4.3. S4J Named Scenario {#43-s4j-named-scenario}
For workload execution convenience, NB engine has the concept of **named scenario** ([doc](https://docs.nosqlbench.io/workloads-101/11-named-scenarios/)). For workload execution convenience, NB engine has the concept of **named scenario** ([doc](https://docs.nosqlbench.io/workloads-101/11-named-scenarios/)).