IntenseFinance/nosqlbench
3
0
Fork 0
mirror of https://github.com/nosqlbench/nosqlbench.git synced 2025-02-25 18:55:28 -06:00
Code Issues Packages Projects Releases Wiki Activity

yaml -> workload & type -> driver synonyms + docs

Browse Source
This commit is contained in:
phact
2020-03-20 14:53:38 -04:00
parent 0622f31344
commit efe77b220f
37 changed files with 191 additions and 191 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
engine-docs/src/main/resources/docs-for-ebhistoric/dev-guide/activity_internals.md
View File

@@ -38,7 +38,7 @@ common parameters.
On the command line, you can specify parameters for an activity in the form:
~~~
type=cql alias=activity1 yaml=inserts_cql.yaml cycles=0..1000 threads=10
driver=cql alias=activity1 workload=inserts_cql.yaml cycles=0..1000 threads=10
~~~
Other convenient forms are available when needed -- a JSON map for example.
@@ -53,7 +53,7 @@ for a couple of reasons:
1. The client behavior can emulate typical threading dynamics of real
applications more accurately than a task-and-queue-only abstraction.
2. The synthetic thread ID can be borrowed and used to directly
2. The synthetic thread ID can be borrowed and used to directly
map some measure of concurrency of data flow.
3. It is a familiar concurrency primitive that is used in many other testing tools.

2
engine-docs/src/main/resources/docs-for-nb/01_showcase/scripting_environment.md
View File

@@ -45,7 +45,7 @@ with their own.
The command line has the form of basic test commands and parameters.
These command get converted directly into scenario control script
in the order they appear. The user can choose whether to stay in
high level executive mode, with simple commands like "run yaml=...",
high level executive mode, with simple commands like "run workload=...",
or to drop down directly into script design. They can look at the
equivalent script for any command line by running --show-script.
If you take the script that is dumped to console and run it, it should

13
engine-docs/src/main/resources/docs-for-nb/02_getting_started/01_example_commands.md
View File

@@ -12,7 +12,7 @@ From your command line, go ahead and execute the following command,
replacing the `host=<dse-host-or-ip>` with that of one of your database nodes.
```
./nb run type=cql yaml=cql-keyvalue tags=phase:schema host=<dse-host-or-ip>
./nb run driver=cql workload=cql-keyvalue tags=phase:schema host=<dse-host-or-ip>
```
This command is creating the following schema in your database:
@@ -30,10 +30,9 @@ Let's break down each of those command line options.
`start` tells nosqlbench to start an activity.
`type=...` is used to specify the activity type. In this case we are using `cql`, which tells nosqlbench to use the DataStax Java Driver and execute CQL statements against a database.
`driver=...` is used to specify the activity type (driver). In this case we are using `cql`, which tells nosqlbench to use the DataStax Java Driver and execute CQL statements against a database.
`yaml=...` is used to specify the yaml file that defines the activity.
All activities require a yaml in which you configure things such as data bindings and CQL statements, but don't worry about those details right now.
`workload=...` is used to specify the workload definition file that defines the activity.
In this example, we use `cql-keyvalue` which is a pre-built workload that is packaged with nosqlbench.
@@ -53,7 +52,7 @@ Before sending our test writes to the database, we will use the `stdout` activit
Go ahead and execute the following command:
./nb start type=stdout yaml=cql-keyvalue tags=phase:rampup cycles=10
./nb start driver=stdout workload=cql-keyvalue tags=phase:rampup cycles=10
You should see 10 of the following statements in your console
@@ -74,7 +73,7 @@ One thing to know is that nosqlbench deterministically generates data, so the ge
Now we are ready to write some data to our database. Go ahead and execute the following from your command line:
./nb start type=cql yaml=cql-keyvalue tags=phase:rampup host=<dse-host-or-ip> cycles=100k --progress console:1s
./nb start driver=cql workload=cql-keyvalue tags=phase:rampup host=<dse-host-or-ip> cycles=100k --progress console:1s
Note the differences between this and the command that we used to generate the schema.
@@ -104,7 +103,7 @@ cql-keyvalue: 100.00%/Finished (details: min=0 cycle=100000 max=100000)
Now that we have a base dataset of 100k rows in the database, we will now run a mixed read / write workload, by default this runs a 50% read / 50% write workload.
./nb start type=cql yaml=cql-keyvalue tags=phase:main host=<dse-host-or-ip> cycles=100k cyclerate=5000 threads=50 --progress console:1s
./nb start driver=cql workload=cql-keyvalue tags=phase:main host=<dse-host-or-ip> cycles=100k cyclerate=5000 threads=50 --progress console:1s
You should see output that looks like this:
```

2
engine-docs/src/main/resources/docs-for-nb/03_basics/01_command_line.md
View File

@@ -27,7 +27,7 @@ named argument to the previous command. The following example
is a commandline with a command *start*, and two named arguments
to that command.
./nb start type=diag alias=example
./nb start driver=diag alias=example
### Discovery options ###

4
engine-docs/src/main/resources/docs-for-nb/04_designing_workloads/02_data_bindings.md
View File

@@ -30,7 +30,7 @@ The above bindings block is also a valid activity YAML, at least for the _stdout
delta: WeightedStrings('one:1;six:6;three:3;')
# EOF (control-D in your terminal)
[test]$ ./nb run type=stdout yaml=stdout-test cycles=10
[test]$ ./nb run driver=stdout workload=stdout-test cycles=10
0,zero,00A_pro,six
1,one,00B_pro,six
2,two,00C_pro,three
@@ -66,7 +66,7 @@ bindings:
delta: WeightedStrings('one:1;six:6;three:3;')
# EOF (control-D in your terminal)
[test]$ ./nb run type=stdout yaml=stdout-test cycles=10
[test]$ ./nb run driver=stdout workload=stdout-test cycles=10
This is a statement, and the file format doesn't
know how statements will be used!
submit job 1 on queue one with options 00B_pro;

18
engine-docs/src/main/resources/docs-for-nb/04_designing_workloads/04_stmt_tags.md
View File

@@ -40,39 +40,39 @@ statements:
# EOF (control-D in your terminal)
# no tag filter matches any
[test]$ ./nb run type=stdout yaml=stdout-test
[test]$ ./nb run driver=stdout workload=stdout-test
I'm alive!
# tag name assertion matches
[test]$ ./nb run type=stdout yaml=stdout-test tags=name
[test]$ ./nb run driver=stdout workload=stdout-test tags=name
I'm alive!
# tag name assertion does not match
[test]$ ./nb run type=stdout yaml=stdout-test tags=name2
[test]$ ./nb run driver=stdout workload=stdout-test tags=name2
02:25:28.158 [scenarios:001] ERROR i.e.activities.stdout.StdoutActivity - Unable to create a stdout statement if you have no active statements or bindings configured.
# tag value assertion does not match
[test]$ ./nb run type=stdout yaml=stdout-test tags=name:bravo
[test]$ ./nb run driver=stdout workload=stdout-test tags=name:bravo
02:25:42.584 [scenarios:001] ERROR i.e.activities.stdout.StdoutActivity - Unable to create a stdout statement if you have no active statements or bindings configured.
# tag value assertion matches
[test]$ ./nb run type=stdout yaml=stdout-test tags=name:foxtrot
[test]$ ./nb run driver=stdout workload=stdout-test tags=name:foxtrot
I'm alive!
# tag pattern assertion matches
[test]$ ./nb run type=stdout yaml=stdout-test tags=name:'fox.*'
[test]$ ./nb run driver=stdout workload=stdout-test tags=name:'fox.*'
I'm alive!
# tag pattern assertion does not match
[test]$ ./nb run type=stdout yaml=stdout-test tags=name:'tango.*'
[test]$ ./nb run driver=stdout workload=stdout-test tags=name:'tango.*'
02:26:05.149 [scenarios:001] ERROR i.e.activities.stdout.StdoutActivity - Unable to create a stdout statement if you have no active statements or bindings configured.
# compound tag predicate matches every assertion
[test]$ ./nb run type=stdout yaml=stdout-test tags='name=fox.*',unit=bravo
[test]$ ./nb run driver=stdout workload=stdout-test tags='name=fox.*',unit=bravo
I'm alive!
# compound tag predicate does not fully match
[test]$ ./nb run type=stdout yaml=stdout-test tags='name=fox.*',unit=delta
[test]$ ./nb run driver=stdout workload=stdout-test tags='name=fox.*',unit=delta
11:02:53.490 [scenarios:001] ERROR i.e.activities.stdout.StdoutActivity - Unable to create a stdout statement if you have no active statements or bindings configured.

2
engine-docs/src/main/resources/docs-for-nb/04_designing_workloads/05_stmt_blocks.md
View File

@@ -25,7 +25,7 @@ blocks:
beta: Combinations('b;l;o;c;k;2;-;COMBINATIONS;')
# EOF (control-D in your terminal)
[test]$ ./nb run type=stdout yaml=stdout-test cycles=10
[test]$ ./nb run driver=stdout workload=stdout-test cycles=10
0,block1-C
1,block2-O
2,block1-M

2
engine-docs/src/main/resources/docs-for-nb/04_designing_workloads/07_multi_docs.md
View File

@@ -29,7 +29,7 @@ bindings:
statements:
- "doc2.number {numname}\n"
# EOF (control-D in your terminal)
[test]$ ./nb run type=stdout yaml=stdout-test cycles=10
[test]$ ./nb run driver=stdout workload=stdout-test cycles=10
doc1.form1 doc1.1
doc1.form2 doc1.2
doc2.number two

4
engine-docs/src/main/resources/docs-for-nb/04_designing_workloads/08_template_params.md
View File

@@ -21,10 +21,10 @@ statements:
- "<<linetoprint:MISSING>>\n"
# EOF (control-D in your terminal)
[test]$ ./nb run type=stdout yaml=stdout-test cycles=1
[test]$ ./nb run driver=stdout workload=stdout-test cycles=1
MISSING
[test]$ ./nb run type=stdout yaml=stdout-test cycles=1 linetoprint="THIS IS IT"
[test]$ ./nb run driver=stdout workload=stdout-test cycles=1 linetoprint="THIS IS IT"
THIS IS IT
```

52
engine-docs/src/main/resources/docs-for-nb/05_activitytypes/06_cql_at.md
View File

@@ -9,7 +9,7 @@ This is the same documentation that you get when you run
./nb help cql
To select this activity type, pass `type=cql` to a run or start command.
To select this activity type, pass `driver=cql` to a run or start command.
---------
@@ -25,22 +25,22 @@ metrics about op binding, op submission, and waiting for a result.
Run a cql activity named 'cql1', with definitions from activities/cqldefs.yaml
~~~
... type=cql alias=cql1 yaml=cqldefs
... driver=cql alias=cql1 workload=cqldefs
~~~
Run a cql activity defined by cqldefs.yaml, but with shortcut naming
~~~
... type=cql yaml=cqldefs
... driver=cql workload=cqldefs
~~~
Only run statement groups which match a tag regex
~~~
... type=cql yaml=cqldefs tags=group:'ddl.*'
... driver=cql workload=cqldefs tags=group:'ddl.*'
~~~
Run the matching 'dml' statements, with 100 cycles, from [1000..1100)
~~~
... type=cql yaml=cqldefs tags=group:'dml.*' cycles=1000..1100
... driver=cql workload=cqldefs tags=group:'dml.*' cycles=1000..1100
~~~
This last example shows that the cycle range is [inclusive..exclusive),
to allow for stacking test intervals. This is standard across all
@@ -48,17 +48,15 @@ activity types.
### CQL ActivityType Parameters
- **driver** - default: dse - The type of driver to use, either dse, or
oss. If you need DSE-specific features, use the dse driver. If you are
connecting to an OSS Apache Cassandra cluster, you must use the oss
driver. The oss driver option is only available in nosqlbench.
- **host** - The host or hosts to use for connection points to
the cluster. If you specify multiple values here, use commas
with no spaces.
Examples:
- `host=192.168.1.25`
- `host=`192.168.1.25,testhost42`
- **yaml** - The file which holds the schema and statement defs.
- **workload** - Workload definition which is also the name of the yaml file or path to the yaml file
which holds the schema and statement defs.
see workload yaml locations below.
(no default, required)
- **port** - The port to connect with
- **cl** - An override to consistency levels for the activity. If
@@ -181,7 +179,7 @@ activity types.
- `jmxreporting=true`
- `jmxreporting=false` (the default)
- **alias** - this is a standard engineblock parameter, however
the cql type will use the yaml value also as the alias value
the cql type will use the workload value also as the alias value
when not specified.
- **errors** - error handler configuration.
(default errors=stop,retryable->retry,unverified->stop)
@@ -215,7 +213,7 @@ activity types.
(default: bucket)
(options: concat | bucket | interval)
The concat sequencer repeats each statement in order until the ratio
is achieved.
is achieved.
The bucket sequencer uses simple round-robin distribution to plan
statement ratios, a simple but unbalanced form of interleaving.
The interval sequencer apportions statements over time and then by
@@ -274,36 +272,36 @@ now **they are limited to a YAML params block**:
params:
ratio: 1
# Sets the statement ratio within the operation sequencer
# Sets the statement ratio within the operation sequencer
# scheme. Integers only.
# When preparing the operation order (AKA sequencing),
# When preparing the operation order (AKA sequencing),
# frequency of the associated statements.
cl: ONE
# Sets the consistency level, using any of the standard
# identifiers from com.datastax.driver.core.ConsistencyLevel,
# any one of:
# LOCAL_QUORUM, ANY, ONE, TWO, THREE, QUORUM, ALL,
# identifiers from com.datastax.driver.core.ConsistencyLevel,
# any one of:
# LOCAL_QUORUM, ANY, ONE, TWO, THREE, QUORUM, ALL,
# EACH_QUORUM, SERIAL, LOCAL_SERIAL, LOCAL_ONE
prepared: true
# By default, all statements are prepared. If you are
# By default, all statements are prepared. If you are
# creating schema, set this to false.
idempotent: false
# For statements that are known to be idempotent, set this
# For statements that are known to be idempotent, set this
# to true
instrument: false
# If a statement has instrument set to true, then
# If a statement has instrument set to true, then
# individual Timer metrics will be tracked for
# that statement for both successes and errors,
# that statement for both successes and errors,
# using the given statement name.
logresultcsv: true
OR
logresultcsv: myfilename.csv
# If a statement has logresultcsv set to true,
# If a statement has logresultcsv set to true,
# then individual operations will be logged to a CSV file.
# In this case the CSV file will be named as
# <statement-name>--results.csv.
@@ -375,7 +373,7 @@ no operation had to be retried.
As for a normal single page read result, both the execute and result timers
are included within the code block wrapped by the pages metric.
### YAML Format
### Workload YAML Format
The YAML file for a CQL activity has the following structure:
1. One or more document sections, separated with '---' and a newline.
@@ -409,14 +407,14 @@ additional parameters to the activity. There are two forms,
form contains a default value. In any case, if one of these parameters is
encountered and a qualifying value is not found, an error will be thrown.
### YAML Location
### Workload YAML Location
The YAML file referenced in the yaml= parameter will be searched for in the following places, in this order:
The YAML file referenced in the workload= parameter will be searched for in the following places, in this order:
1. A URL, if it starts with 'http:' or 'https:'
2. The local filesystem, if it exists there
3. The internal classpath and assets in the jar.
The '.yaml' suffix is not required in the yaml= parameter, however it is
The '.yaml' suffix is not required in the workload= parameter, however it is
required on the actual file. As well, the logical search path "activities/"
will be used if necessary to locate the file, both on the filesystem and in
the classpath.

8
engine-docs/src/main/resources/docs-for-nb/05_activitytypes/06_stdout_at.md
View File

@@ -9,7 +9,7 @@ This is the same documentation that you get when you run
./nb help stdout
To select this activity type, pass `type=stdout` to a run or start command.
To select this activity type, pass `driver=stdout` to a run or start command.
---------
@@ -24,17 +24,17 @@ that uses the curly brace token form in statements.
Run a stdout activity named 'stdout-test', with definitions from activities/stdout-test.yaml
~~~
... type=stdout yaml=stdout-test
... driver=stdout workload=stdout-test
~~~
Only run statement groups which match a tag regex
~~~
... type=stdout yaml=stdout-test tags=group:'ddl.*'
... driver=stdout workload=stdout-test tags=group:'ddl.*'
~~~
Run the matching 'dml' statements, with 100 cycles, from [1000..1100)
~~~
... type=stdout yaml=stdout-test tags=group:'dml.*' cycles=1000..11000 filename=test.csv
... driver=stdout workload=stdout-test tags=group:'dml.*' cycles=1000..11000 filename=test.csv
~~~
This last example shows that the cycle range is [inclusive..exclusive),

10
engine-docs/src/main/resources/docs-for-nb/05_activitytypes/index.md
View File

@@ -5,6 +5,12 @@ weight: 50
Each nosqlbench scenario is comprised of one or more activities of a specific type. The types of activities available are provided by the version of nosqlbench.
Additional activity types will be added in future releases. This section is a reference section that shows the help you would get with a command like:
Additional activity types will be added in future releases. There are command line help topics for each activity type (driver).
./nb help <activity type>
To get a list of topics run:
./nb help topics
To get the help for a particular topic run:
./nb help <topic>

12
engine-docs/src/main/resources/docs-for-nb/09_reference/02_cli_scripting.md
View File

@@ -25,7 +25,7 @@ are non-scripting options.
All activities that run during a scenario run under the control of, but
independently from the scenario script. This means that you can have a number of activities running while the scenario script is doing its own thing. The scenario only completes when both the scenario script and the activities are finished.
### `start type=<activity type> alias=<alias> ...`
### `start driver=<activity type> alias=<alias> ...`
You can start an activity with this command. At the time this command is
evaluated, the activity is started, and the script continues without blocking. This is an asynchronous start of an activity. If you start multiple activities in this way, they will run concurrently.
@@ -41,11 +41,11 @@ scenario to pause until the activity is stopped. This means that all threads for
Await the normal completion of an activity with the given alias. This causes the scenario script to pause while it waits for the named activity to finish. This does not tell the activity to stop. It simply puts the scenario script into a paused state until the named activity is complete.
### `run type=<activity type> alias=<alias> ...`
### `run driver=<activity type> alias=<alias> ...`
Run an activity to completion, waiting until it is complete before continuing with the scenario script. It is effectively the same as
start type=<activity type> ... alias=<alias>
start driver=<activity type> ... alias=<alias>
await <alias>
### `waitmillis <milliseconds>`
@@ -63,9 +63,9 @@ Add the contents of the next argument to the scenario script buffer.
# An example CLI script
~~~
./run-eb \
start type=stdout alias=a cycles=100K yaml=cql-iot tags=phase:main\
start type=stdout alias=b cycles=200K yaml=cql-iot tags=phase:main\
./nb \
start driver=stdout alias=a cycles=100K workload=cql-iot tags=phase:main\
start driver=stdout alias=b cycles=200K workload=cql-iot tags=phase:main\
waitmillis 10000 \
await one \
stop two

6
engine-docs/src/main/resources/docs-for-nb/09_reference/03_scenario_scripting.md
View File

@@ -12,7 +12,7 @@ The EngineBlock runtime is a combination of a scripting sandbox and a workload e
All of the heavy lifting is left to Java and the core nosqlbench runtime. This includes the iterative workloads that are meant to test the target system. This is combined with a control layer which is provided by Nashorn and eventually GraalVM. This division of responsibility allows the high-level test logic to be "script" and the low-level activity logic to be "machinery". While the scenario script has the most control, it also is the least busy relative to activity workloads. The net effect is that you have the efficiency of the iterative test loads in conjunction with the open design palette of a first-class scripting language.
Essentially, the ActivityType drivers are meant to handle the workload-specific machinery. They also provide dynamic control points and parameters which special to that activity type. This exposes a full feedback loop between a running scenario script and the activities that it runs. The scenario is free to read the performance metrics from a running activity and make changes to it on the fly.
Essentially, the ActivityType drivers are meant to handle the workload-specific machinery. They also provide dynamic control points and parameters which special to that activity type (driver). This exposes a full feedback loop between a running scenario script and the activities that it runs. The scenario is free to read the performance metrics from a running activity and make changes to it on the fly.
## Scripting Environment
@@ -44,7 +44,7 @@ Interaction with the nosqlbench runtime and the activities therein is made easy
by the above variables and objects. When an assignment is made to any of these variables, the changes are propagated to internal listeners. For changes to _threads_, the thread pool responsible for the affected activity adjusts the number of active threads (AKA slots). Other changes are further propagated directly to the thread harnesses and components which implement the ActivityType.
:::warning
Assignment to the _type_ and _alias_ activity parameters has no special effect, as you can't change an activity to a different type once it has been created.
Assignment to the _workload_ and _alias_ activity parameters has no special effect, as you can't change an activity to a different driver once it has been created.
:::
You can make use of more extensive Java or Javascript libraries as needed,
@@ -80,7 +80,7 @@ When a script is run, it has absolute control over the scenario runtime while it
## Strategies
You can use nosqlbench in the classic form with `run type=<type> param=value ...` command line syntax. There are reasons, however, that you will sometimes want customize and modify your scripts directly, such as:
You can use nosqlbench in the classic form with `run driver=<activity_type> param=value ...` command line syntax. There are reasons, however, that you will sometimes want customize and modify your scripts directly, such as:
- Permute test variables to cover many sub-conditions in a test.
- Automatically adjust load factors to identify the nominal capacity of a system.

2
engine-docs/src/main/resources/docs-for-nb/09_reference/04_standard_metrics.md
View File

@@ -4,7 +4,7 @@ title: Standard Metrics
# Standard Metrics
nosqlbench comes with a set of standard metrics that will be part of every activity type. Each activity type enhances the metrics available by adding their own metrics with the nosqlbench APIs. This section explains what the standard metrics are, and how to interpret them.
nosqlbench comes with a set of standard metrics that will be part of every activity type (driver). Each activity type (driver) enhances the metrics available by adding their own metrics with the nosqlbench APIs. This section explains what the standard metrics are, and how to interpret them.
## read-input