IntenseFinance/nosqlbench
3
0
Fork 0
mirror of https://github.com/nosqlbench/nosqlbench.git synced 2025-02-25 18:55:28 -06:00
Code Issues Packages Projects Releases Wiki Activity

yaml -> workload & type -> driver synonyms + docs

Browse Source
This commit is contained in:
phact 2020-03-20 14:53:38 -04:00
parent 0622f31344
commit efe77b220f
37 changed files with 191 additions and 191 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
activitytype-cql/src/main/java/io/nosqlbench/activitytype/cql/core/CqlActivity.java
View File

@ -275,10 +275,8 @@ public class CqlActivity extends SimpleActivity implements Activity, ActivityDef
private StmtsDocList loadStmtsYaml() {
StmtsDocList doclist = null;
String yaml_loc = activityDef.getParams().getOptionalString("yaml").orElse("default");
if (yaml_loc.equals("default")){
yaml_loc = activityDef.getParams().getOptionalString("workload").orElse("default");
}
String yaml_loc = activityDef.getParams().getOptionalString("yaml", "workload").orElse("default");
StrInterpolator interp = new StrInterpolator(activityDef);

6
activitytype-cql/src/main/java/io/nosqlbench/activitytype/cql/core/CqlActivityType.java
View File

@ -17,10 +17,8 @@ public class CqlActivityType implements ActivityType<CqlActivity> {
@Override
public CqlActivity getActivity(ActivityDef activityDef) {
Optional<String> yaml = activityDef.getParams().getOptionalString("yaml");
if (!yaml.isPresent()) {
yaml = activityDef.getParams().getOptionalString("workload");
}
Optional<String> yaml = activityDef.getParams().getOptionalString("yaml", "workload");
// sanity check that we have a yaml parameter, which contains our statements and bindings
if (yaml.isEmpty()) {

0
activitytype-cql/src/main/resources/cqlerrors.md → activitytype-cql/src/main/resources/cql-errors.md
View File

0
activitytype-cql/src/main/resources/cqlexceptionlist.md → activitytype-cql/src/main/resources/cql-exception-list.md
View File

93
activitytype-cql/src/main/resources/cql.md
View File

@ -1,30 +1,30 @@
# cql activity type
This is an activity type which allows for the execution of CQL statements.
This particular activity type is wired synchronously within each client
thread, however the async API is used in order to expose fine-grain
This particular activity type is wired synchronously within each client
thread, however the async API is used in order to expose fine-grain
metrics about op binding, op submission, and waiting for a result.
### Example activity definitions
Run a cql activity named 'cql1', with definitions from activities/cqldefs.yaml
~~~
... type=cql alias=cql1 yaml=cqldefs
... driver=cql alias=cql1 workload=cqldefs
~~~
Run a cql activity defined by cqldefs.yaml, but with shortcut naming
~~~
... type=cql yaml=cqldefs
... driver=cql workload=cqldefs
~~~
Only run statement groups which match a tag regex
~~~
... type=cql yaml=cqldefs tags=group:'ddl.*'
... driver=cql workload=cqldefs tags=group:'ddl.*'
~~~
Run the matching 'dml' statements, with 100 cycles, from [1000..1100)
~~~
... type=cql yaml=cqldefs tags=group:'dml.*' cycles=1000..1100
... driver=cql workload=cqldefs tags=group:'dml.*' cycles=1000..1100
~~~
This last example shows that the cycle range is [inclusive..exclusive),
to allow for stacking test intervals. This is standard across all
@ -42,7 +42,8 @@ activity types.
Examples:
- `host=192.168.1.25`
- `host=`192.168.1.25,testhost42`
- **yaml** - The file which holds the schema and statement defs.
- **workload** - The workload definition which holds the schema and statement defs.
see workload yaml location for additional details
(no default, required)
- **port** - The port to connect with
- **cl** - An override to consistency levels for the activity. If
@ -51,7 +52,7 @@ activity types.
the difference with respect to the yaml will be emitted.
This is not a dynamic parameter. It will only be applied at
activity start.
- **cbopts** - default: none - this is how you customize the cluster
- **cbopts** - default: none - this is how you customize the cluster
settings for the client, including policies, compression, etc. This
is a string of *Java*-like method calls just as you would use them
in the Cluster.Builder fluent API. They are evaluated inline
@ -75,7 +76,7 @@ activity types.
- `pooling=4:10,2:5`
keep 4-10 connections to LOCAL hosts and 2-5 to REMOTE
- `pooling=4:10:2000`
keep between 4-10 connections to LOCAL hosts with
keep between 4-10 connections to LOCAL hosts with
up to 2000 requests per connection
- `pooling=5:10:2000,2:4:1000` keep between 5-10 connections to
LOCAL hosts with up to 2000 requests per connection, and 2-4
@ -112,7 +113,7 @@ activity types.
Examples:
- tokens=1:10000,100000:1000000
- tokens=1:123456
- **maxtries** - default: 10 - how many times an operation may be
- **maxtries** - default: 10 - how many times an operation may be
attempted before it is disregarded
- **maxpages** - default: 1 - how many pages can be read from a query which
is larger than the fetchsize. If more than this number of pages
@ -121,14 +122,14 @@ activity types.
- **fetchsize** - controls the driver parameter of the same name.
Suffixed units can be used here, such as "50K". If this parameter
is not present, then the driver option is not set.
- **cycles** - standard, however the cql activity type will default
this to however many statements are included in the current
- **cycles** - standard, however the cql activity type will default
this to however many statements are included in the current
activity, after tag filtering, etc.
- **username** - the user to authenticate as. This option requires
- **username** - the user to authenticate as. This option requires
that one of **password** or **passfile** also be defined.
- **password** - the password to authenticate with. This will be
- **password** - the password to authenticate with. This will be
ignored if passfile is also present.
- **passfile** - the file to read the password from. The first
- **passfile** - the file to read the password from. The first
line of this file is used as the password.
- **ssl** - specifies the type of the SSL implementation.
Disabled by default, possible values are `jdk`, and `openssl`.
@ -164,8 +165,8 @@ activity types.
Examples:
- `jmxreporting=true`
- `jmxreporting=false` (the default)
- **alias** - this is a standard engineblock parameter, however
the cql type will use the yaml value also as the alias value
- **alias** - this is a standard engineblock parameter, however
the cql type will use the workload value also as the alias value
when not specified.
- **errors** - error handler configuration.
(default errors=stop,retryable->retry,unverified->stop)
@ -173,8 +174,8 @@ activity types.
- errors=stop,WriteTimeoutException=histogram
- errors=count
- errors=warn,retryable=count
See the separate help on 'cqlerrors' for detailed
configuration options.
See the separate help on 'cqlerrors' for detailed
configuration options.
- **defaultidempotence** - sets default idempotence on the
driver options, but only if it has a value.
(default unset, valid values: true or false)
@ -199,8 +200,8 @@ activity types.
(default: bucket)
(options: concat | bucket | interval)
The concat sequencer repeats each statement in order until the ratio
is achieved.
The bucket sequencer uses simple round-robin distribution to plan
is achieved.
The bucket sequencer uses simple round-robin distribution to plan
statement ratios, a simple but unbalanced form of interleaving.
The interval sequencer apportions statements over time and then by
order of appearance for ties. This has the effect of interleaving
@ -209,7 +210,7 @@ activity types.
All of the sequencers create deterministic schedules which use an internal
lookup table for indexing into a list of possible statements.
- **trace** - enables a trace on a subset of operations. This is disabled
by default.
by default.
Examples:
`trace=modulo:100,filename:trace.log`
The above traces every 100th cycle to a file named trace.log.
@ -258,36 +259,36 @@ now **they are limited to a YAML params block**:
params:
ratio: 1
# Sets the statement ratio within the operation sequencer
# Sets the statement ratio within the operation sequencer
# scheme. Integers only.
# When preparing the operation order (AKA sequencing),
# When preparing the operation order (AKA sequencing),
# frequency of the associated statements.
cl: ONE
# Sets the consistency level, using any of the standard
# identifiers from com.datastax.driver.core.ConsistencyLevel,
# any one of:
# LOCAL_QUORUM, ANY, ONE, TWO, THREE, QUORUM, ALL,
# identifiers from com.datastax.driver.core.ConsistencyLevel,
# any one of:
# LOCAL_QUORUM, ANY, ONE, TWO, THREE, QUORUM, ALL,
# EACH_QUORUM, SERIAL, LOCAL_SERIAL, LOCAL_ONE
prepared: true
# By default, all statements are prepared. If you are
# By default, all statements are prepared. If you are
# creating schema, set this to false.
idempotent: false
# For statements that are known to be idempotent, set this
# For statements that are known to be idempotent, set this
# to true
instrument: false
# If a statement has instrument set to true, then
# If a statement has instrument set to true, then
# individual Timer metrics will be tracked for
# that statement for both successes and errors,
# that statement for both successes and errors,
# using the given statement name.
logresultcsv: true
OR
logresultcsv: myfilename.csv
# If a statement has logresultcsv set to true,
# If a statement has logresultcsv set to true,
# then individual operations will be logged to a CSV file.
# In this case the CSV file will be named as
# <statement-name>--results.csv.
@ -334,7 +335,7 @@ now **they are limited to a YAML params block**:
- alias.tries - A histogram of how many tries were required to get a
completed operation
- alias.pages - A timer which tracks the performance of paging, specific
to more than 1-page query results. i.e., if all reads return within 1
to more than 1-page query results. i.e., if all reads return within 1
page, this metric will not have any data.
- alias.strides - A timer around each stride of operations within a thread
- alias.skipped-tokens - A histogram that records the count and cycle values
@ -344,7 +345,7 @@ now **they are limited to a YAML params block**:
set that it returns, across all pages. This metric is only counted
for non-exceptional results, while the result metric above includes
all operations.
##### Metrics Details
The cycles metric captures data on the outside of each operation, but it also
@ -366,17 +367,17 @@ The YAML file for a CQL activity has the following structure:
1. An optional tag map
2. One or more statements
1. a descriptive name
2. prepared: false, if you want to modify the default (prepared:true)
2. prepared: false, if you want to modify the default (prepared:true)
3. statement CQL
4. statement data bindings
Each section is a separate yaml document internally to the yaml file. The
tags that are provided allow for subgroups of statements to be activated.
All statements in a matching document (when filtered by tags) are included
tags that are provided allow for subgroups of statements to be activated.
All statements in a matching document (when filtered by tags) are included
in the statement rotation.
If no tags are provided in a document section, then it will be matched by
all possible tag filters. Conversely, if no tag filter is applied in
If no tags are provided in a document section, then it will be matched by
all possible tag filters. Conversely, if no tag filter is applied in
the activity definition, all tagged documents will match.
Data bindings specify how values are generated to plug into each operation. More
@ -384,23 +385,23 @@ details on data bindings are available in the activity usage guide.
### Parameter Templating
Double angle brackets may be used to drop parameters into the YAML
Double angle brackets may be used to drop parameters into the YAML
arbitrarily. When the YAML file is loaded, and only then, these parameters
are interpolated from activity parameters like those above. This allows you
to create activity templates that can be customized simply by providing
additional parameters to the activity. There are two forms,
additional parameters to the activity. There are two forms,
\<\<some_var_name:default_value\>\> and \<\<some_var_name\>\>. The first
form contains a default value. In any case, if one of these parameters is
encountered and a qualifying value is not found, an error will be thrown.
### YAML Location
### Workload YAML Location
The YAML file referenced in the yaml= parameter will be searched for in the following places, in this order:
The YAML file referenced in the workload= parameter will be searched for in the following places, in this order:
1. A URL, if it starts with 'http:' or 'https:'
2. The local filesystem, if it exists there
3. The internal classpath and assets in the jar.
The '.yaml' suffix is not required in the yaml= parameter, however it is
The '.yaml' suffix is not required in the workload= parameter, however it is
required on the actual file. As well, the logical search path "activities/"
will be used if necessary to locate the file, both on the filesystem and in
the classpath.

4
activitytype-cql/src/main/resources/topics.md
View File

@ -1,4 +1,4 @@
# cql help topics
- cql
- cqlerrors
- cqlexceptionlist
- cql-errors
- cql-exception-list

24
activitytype-cqlverify/src/main/resources/cqlverify.md
View File

@ -40,7 +40,7 @@ mapping functions containing the word "Random", as these are the ones that have
historically used internal RNG state. Instead, swap in their replacements that
start with "Hashed". There is a hashed equivalent to all of the original random
functions. The rng-based functions will be deprecated in a future release.
In a typical cql activity, you are allowed to name the bindings however you
like, so long as the binding names match the anchor names in your statement
template. Because we need to match reference field data to actual row data
@ -53,9 +53,9 @@ The simplest way to do this is to follow this recipe:
1. Make the binding names the same as the field names that you use in
in your write statements.
2. When you configure your read statement for the cqlverify activity,
simply include the same bindings as-is, using the partition and
simply include the same bindings as-is, using the partition and
clustering fields in the appropriate where clauses.
*note*: It used to be an error to have bindings names in excess of what anchor
names would match. Now, it is only an error if an anchor is not qualified with
a matching binding name. This allows you to simply copy your bindings as-is
@ -77,7 +77,7 @@ as well.
If more than one statement is active for a cqlverify activity, then an error is
thrown. This may change in the future, but for now it is a requirement.
### Handling Verification Errors
The cqlverify activity extends on the error handling stack mechanism that is
@ -86,7 +86,7 @@ used by the cql activity type, by introducing a new error category:
unverified=stop
However, the usual options, including "stop", "warn", "retry", "histogram",
However, the usual options, including "stop", "warn", "retry", "histogram",
"count", and "ignore" are also allowed.
Care should be taken to set the other error handling categories to be strict
@ -101,7 +101,7 @@ be thrown and the client will exit by default. If you wish for something less
dramatic, then set it to
errors=...,unverified->count
or
errors=...,unverified->warn
@ -118,12 +118,12 @@ A future release may allow for paged reads for quicker verification.
Write 100K cycles of telemetry data
... run type=cql alias=writesome yaml=telemetry tags=group:write cycles=100000 host=...
... run driver=cql alias=writesome workload=cql-iot tags=group:write cycles=100000 host=...
Verify the the same 100K cycles of telemetry data
... run type=cqlverify alias=verify yaml=telemetry tags=group:verify cycles=100000 host=...
... run driver=cqlverify alias=verify workload=cql-iot tags=group:verify cycles=100000 host=...
To see how these examples work, consult the telemetry.yaml file in the nosqlbench.jar.
### CQLVerify ActivityType Parameters
@ -157,10 +157,10 @@ To see how these examples work, consult the telemetry.yaml file in the nosqlbenc
then the field is added to the list of fields to verify.
This parameter is useful if you have a set of default bindings and want
to specify which subset of them of them will be used just for this statement.
If any of the added fields is in the form "f->b", then it is taken as a mapping
from the field name _f_ in the schema to a binding _b_.
### Metrics
The cqlverify activity type adds some verification-specific metrics:

2
activitytype-cqlverify/src/main/resources/topics.md
View File

@ -1,2 +1,2 @@
# cql-verify help topics
# cqlverify help topics
- cqlverify

6
activitytype-diag/src/main/resources/diag.md
View File

@ -4,7 +4,7 @@ This is a diagnostic activity type. Its action simply reports the cycle number a
## example activitydef
alias=testdiag;type=diag;threads=50;interval=2000;
alias=testdiag;driver=diag;threads=50;interval=2000;
## controls
@ -17,8 +17,8 @@ active motor.
( default: 1000ms )
- modulo - The cycle rate at which to generate a single output status line.
( default: modulo=10000000 )
- errormodulo - The cycle rate at which to generate a non-zero status code
- errormodulo - The cycle rate at which to generate a non-zero status code
(1=every cycle, 10=every 10th cycle)
( default: errormodule=1000 )
- phases - The number of phases to run.
( default: phases=1 )
( default: phases=1 )

6
activitytype-http/src/main/java/io/nosqlbench/activitytype/http/HttpActivity.java
View File

@ -54,12 +54,10 @@ public class HttpActivity extends SimpleActivity implements Activity, ActivityDe
public HttpActivity(ActivityDef activityDef) {
super(activityDef);
this.activityDef = activityDef;
String yaml_loc = activityDef.getParams()
.getOptionalString("yaml")
.getOptionalString("yaml", "workload")
.orElse("default");
if (yaml_loc.equals("default")) {
yaml_loc = activityDef.getParams().getOptionalString("workload").orElse("default");
}
stmtsDocList = StatementsLoader.load(logger,yaml_loc, "activities");
}

14
activitytype-http/src/main/resources/http.md
View File

@ -7,7 +7,7 @@ As of this release, only GET requests are supported.
Run an http activity named 'http-test', with definitions from activities/http-google.yaml:
~~~
... type=http yaml=http-google
... driver=http workload=http-google
~~~
This last example shows that the cycle range is [inclusive..exclusive),
@ -19,19 +19,19 @@ activity types.
- **host** - The hosts to send requests to. The hosts are selected in
round-robin fashion.
(default: localhost)
- **yaml** - The file which holds the schema and statement defs.
- **workload** - The workload definition file which holds the schema and statement defs.
(no default, required)
- **cycles** - standard, however the activity type will default
this to however many statements are included in the current
- **cycles** - standard, however the activity type will default
this to however many statements are included in the current
activity, after tag filtering, etc.
(default: 0)
- **alias** - this is a standard nosqlbench parameter
(default: derived from the yaml name)
(default: derived from the workload name)
## Configuration
This activity type uses the uniform yaml configuration format.
For more details on this format, please refer to the
For more details on this format, please refer to the
[Standard YAML Format](http://docs.nosqlbench.io/user-guide/standard_yaml/)
## Configuration Parameters
@ -42,7 +42,7 @@ For more details on this format, please refer to the
parameter takes precedence.
- **seq** - The statement sequencer scheme.
(default: bucket)
## Statement Format
The statement format for this activity type is a simple string. Tokens between

2
activitytype-http/src/main/resources/topics.md Normal file
View File

@ -0,0 +1,2 @@
# http help topics
- http

5
activitytype-stdout/src/main/java/io/nosqlbench/activitytype/stdout/StdoutActivity.java
View File

@ -72,10 +72,7 @@ public class StdoutActivity extends SimpleActivity implements ActivityDefObserve
public StdoutActivity(ActivityDef activityDef) {
super(activityDef);
StrInterpolator interp = new StrInterpolator(activityDef);
String yaml_loc = activityDef.getParams().getOptionalString("yaml").orElse("default");
if (yaml_loc.equals("default")) {
yaml_loc = activityDef.getParams().getOptionalString("workload").orElse("default");
}
String yaml_loc = activityDef.getParams().getOptionalString("yaml", "workload").orElse("default");
this.showstmts = activityDef.getParams().getOptionalBoolean("showstatements").orElse(false);
this.fileName = activityDef.getParams().getOptionalString("filename").orElse("stdout");

5
activitytype-stdout/src/main/java/io/nosqlbench/activitytype/stdout/StdoutActivityType.java
View File

@ -18,10 +18,7 @@ public class StdoutActivityType implements ActivityType<StdoutActivity> {
@Override
public StdoutActivity getActivity(ActivityDef activityDef) {
Optional<String> yaml = activityDef.getParams().getOptionalString("yaml");
if (!yaml.isPresent()) {
yaml= activityDef.getParams().getOptionalString("workload");
}
Optional<String> yaml = activityDef.getParams().getOptionalString("yaml", "workload");
// sanity check that we have a yaml parameter, which contains our statements and bindings

6
activitytype-stdout/src/main/resources/stdout.md
View File

@ -9,17 +9,17 @@ that uses the curly brace token form in statements.
Run a stdout activity named 'stdout-test', with definitions from activities/stdout-test.yaml
~~~
... type=stdout yaml=stdout-test
... driver=stdout workload=stdout-test
~~~
Only run statement groups which match a tag regex
~~~
... type=stdout yaml=stdout-test tags=group:'ddl.*'
... driver=stdout workload=stdout-test tags=group:'ddl.*'
~~~
Run the matching 'dml' statements, with 100 cycles, from [1000..1100)
~~~
... type=stdout yaml=stdout-test tags=group:'dml.*' cycles=1000..11000 filename=test.csv
... driver=stdout workload=stdout-test tags=group:'dml.*' cycles=1000..11000 filename=test.csv
~~~
This last example shows that the cycle range is [inclusive..exclusive),

2
engine-api/src/main/java/io/nosqlbench/engine/api/activityimpl/ActivityDef.java
View File

@ -95,7 +95,7 @@ public class ActivityDef {
}
public String getActivityType() {
return parameterMap.getOptionalString("type").orElse(DEFAULT_ATYPE);
return parameterMap.getOptionalString("type", "driver").orElse(DEFAULT_ATYPE);
}
/**

7
engine-cli/src/main/java/io/nosqlbench/engine/cli/NBCLI.java
View File

@ -56,12 +56,7 @@ public class NBCLI {
NBCLIOptions options = new NBCLIOptions(args);
if (options.wantsBasicHelp()) {
System.out.println(loadHelpFile("commandline.md"));
System.exit(0);
}
if (options.wantsAdvancedHelp()) {
System.out.println(loadHelpFile("cli_scripting.md"));
System.out.println(loadHelpFile("basic.md"));
System.exit(0);
}

10
engine-cli/src/main/java/io/nosqlbench/engine/cli/NBCLIOptions.java
View File

@ -31,7 +31,6 @@ public class NBCLIOptions {
// Discovery
private static final String HELP = "--help";
private static final String ADVANCED_HELP = "--advanced-help";
private static final String METRICS = "--list-metrics";
private static final String ACTIVITY_TYPES = "--list-activity-types";
private static final String WORKLOADS = "--list-workloads";
@ -98,7 +97,6 @@ public class NBCLIOptions {
private int reportInterval = 10;
private String metricsPrefix = "nosqlbench.";
private String wantsMetricsForActivity;
private boolean wantsAdvancedHelp = false;
private String sessionName = "";
private boolean showScript = false;
private Level consoleLevel = Level.WARN;
@ -222,10 +220,6 @@ public class NBCLIOptions {
arglist.removeFirst();
wantsVersionCoords = true;
break;
case ADVANCED_HELP:
arglist.removeFirst();
wantsAdvancedHelp = true;
break;
case ENABLE_CHART:
arglist.removeFirst();
enableChart = true;
@ -470,10 +464,6 @@ public class NBCLIOptions {
return wantsBasicHelp;
}
public boolean wantsAdvancedHelp() {
return wantsAdvancedHelp;
}
public boolean wantsEnableChart() {
return enableChart;
}

22
engine-cli/src/main/resources/basic.md Normal file
View File

@ -0,0 +1,22 @@
# for specific help topics run
./nb help <topic>
# for a list of available topics run
./nb help topics
# to see the web based docs run
./nb docserver
# To run a simple built-in workload run:
./nb cql-iot
# To get a list of available workloads run:
./nb --list-workloads
# Note: this will include built-in workloads, shipped with nb and workloads in your local directory.
# To provide your own contact points (comma separated), add the hosts= parameter
./nb cql-iot hosts=host1,host2
# Additionally, if you have docker installed on your local system, and your user has permissions to use it, you can use --docker-metrics to stand up a live metrics dashboard at port 3000.
./nb cql-iot --docker-metrics

10
engine-cli/src/main/resources/cli_scripting.md → engine-cli/src/main/resources/cli-scripting.md
View File

@ -30,10 +30,10 @@ Anywhere you need to specify a time, you can use standard unit suffixes,
like "1 day", "1m", etc. Both long names and short names work in any
case. The valid time suffixes are ns, us, ms, s, m, h, d, w, y, for
nanoseconds, microseconds, milliseconds, seconds, minutes, hours,
days, weeks, or years, respectively.
days, weeks, or years, respectively.
As well, when specifying sizes, standard SI and IEC units apply for suffixes like
KB, KiB. For more details, see
KB, KiB. For more details, see
[The Wikipedia Page](https://en.wikipedia.org/wiki/Binary_prefix).
For Scenario Designers
@ -43,7 +43,7 @@ You can build up a complex scenario by combining scripts and activities.
If this scenario needs to have some cross-shared logic, that is up to you,
the scenario designer.
## Script Parameters
## Script Parameters
Any arguments following a script in name=value form will be used to parameterize
the script. Script parameters are simply macro tokens in the form &lt;&lt;NAME:default&gt;&gt;.
@ -67,7 +67,7 @@ to make a list that you can copy metric names from for scripting. If you provide
an example activity alias that matches one of your scripts, you can use it exactly
as it appears.
~~~
PROG --list-metrics type=diag alias=anexample
PROG --list-metrics driver=diag alias=anexample
~~~
This will dump a list of metric names in the shortened format that is most suitable
for scenario script development. This format is required for the --list-metrics
@ -111,7 +111,7 @@ fragment '<ecmascript>...'
These can all be used on the command line in any order. The scenario script is assembled
from them before it is executed. If you want to see the resulting script, use the
--show-script option to dump the script to the console instead of running it.
By combining these activity commands on the command line, you can construct a non-trivial
scenario from other snippets, control activity sequencing and concurrency, etc. This does
not replace what is possible for direct scripting, but it does allow for many custom

2
engine-cli/src/main/resources/topics.md
View File

@ -1,3 +1,3 @@
# global help topics
- commandline
- cli_scripting
- cli-scripting

9
engine-core/src/main/java/io/nosqlbench/engine/core/ScenarioController.java
View File

@ -42,7 +42,7 @@ public class ScenarioController {
* Start an activity, given the activity definition for it. The activity will be known in the scenario
* by the alias parameter.
*
* @param activityDef string in alias=value1;type=value2;... format
* @param activityDef string in alias=value1;driver=value2;... format
*/
public synchronized void start(ActivityDef activityDef) {
getActivityExecutor(activityDef, true).startActivity();
@ -240,8 +240,7 @@ public class ScenarioController {
List<String> matching = knownTypes.stream().filter(
n ->
activityDef.getParams().getOptionalString("alias").orElse("").contains(n)
|| activityDef.getParams().getOptionalString("yaml").orElse("").contains(n)
|| activityDef.getParams().getOptionalString("workload").orElse("").contains(n)
|| activityDef.getParams().getOptionalString("yaml", "workload").orElse("").contains(n)
).collect(Collectors.toList());
if (matching.size()==1) {
activityTypeName=matching.get(0);
@ -250,8 +249,8 @@ public class ScenarioController {
}
if (activityTypeName==null) {
String errmsg = "You must provide a type=<activity type> parameter. Valid examples are:\n" +
knownTypes.stream().map(t -> " type="+t+"\n").collect(Collectors.joining());
String errmsg = "You must provide a driver=<activity type> parameter. Valid examples are:\n" +
knownTypes.stream().map(t -> " driver="+t+"\n").collect(Collectors.joining());
throw new BasicError(errmsg);
}

4
engine-docs/src/main/resources/docs-for-ebhistoric/dev-guide/activity_internals.md
View File

@ -38,7 +38,7 @@ common parameters.
On the command line, you can specify parameters for an activity in the form:
~~~
type=cql alias=activity1 yaml=inserts_cql.yaml cycles=0..1000 threads=10
driver=cql alias=activity1 workload=inserts_cql.yaml cycles=0..1000 threads=10
~~~
Other convenient forms are available when needed -- a JSON map for example.
@ -53,7 +53,7 @@ for a couple of reasons:
1. The client behavior can emulate typical threading dynamics of real
applications more accurately than a task-and-queue-only abstraction.
2. The synthetic thread ID can be borrowed and used to directly
2. The synthetic thread ID can be borrowed and used to directly
map some measure of concurrency of data flow.
3. It is a familiar concurrency primitive that is used in many other testing tools.

2
engine-docs/src/main/resources/docs-for-nb/01_showcase/scripting_environment.md
View File

@ -45,7 +45,7 @@ with their own.
The command line has the form of basic test commands and parameters.
These command get converted directly into scenario control script
in the order they appear. The user can choose whether to stay in
high level executive mode, with simple commands like "run yaml=...",
high level executive mode, with simple commands like "run workload=...",
or to drop down directly into script design. They can look at the
equivalent script for any command line by running --show-script.
If you take the script that is dumped to console and run it, it should

13
engine-docs/src/main/resources/docs-for-nb/02_getting_started/01_example_commands.md
View File

@ -12,7 +12,7 @@ From your command line, go ahead and execute the following command,
replacing the `host=<dse-host-or-ip>` with that of one of your database nodes.
```
./nb run type=cql yaml=cql-keyvalue tags=phase:schema host=<dse-host-or-ip>
./nb run driver=cql workload=cql-keyvalue tags=phase:schema host=<dse-host-or-ip>
```
This command is creating the following schema in your database:
@ -30,10 +30,9 @@ Let's break down each of those command line options.
`start` tells nosqlbench to start an activity.
`type=...` is used to specify the activity type. In this case we are using `cql`, which tells nosqlbench to use the DataStax Java Driver and execute CQL statements against a database.
`driver=...` is used to specify the activity type (driver). In this case we are using `cql`, which tells nosqlbench to use the DataStax Java Driver and execute CQL statements against a database.
`yaml=...` is used to specify the yaml file that defines the activity.
All activities require a yaml in which you configure things such as data bindings and CQL statements, but don't worry about those details right now.
`workload=...` is used to specify the workload definition file that defines the activity.
In this example, we use `cql-keyvalue` which is a pre-built workload that is packaged with nosqlbench.
@ -53,7 +52,7 @@ Before sending our test writes to the database, we will use the `stdout` activit
Go ahead and execute the following command:
./nb start type=stdout yaml=cql-keyvalue tags=phase:rampup cycles=10
./nb start driver=stdout workload=cql-keyvalue tags=phase:rampup cycles=10
You should see 10 of the following statements in your console
@ -74,7 +73,7 @@ One thing to know is that nosqlbench deterministically generates data, so the ge
Now we are ready to write some data to our database. Go ahead and execute the following from your command line:
./nb start type=cql yaml=cql-keyvalue tags=phase:rampup host=<dse-host-or-ip> cycles=100k --progress console:1s
./nb start driver=cql workload=cql-keyvalue tags=phase:rampup host=<dse-host-or-ip> cycles=100k --progress console:1s
Note the differences between this and the command that we used to generate the schema.
@ -104,7 +103,7 @@ cql-keyvalue: 100.00%/Finished (details: min=0 cycle=100000 max=100000)
Now that we have a base dataset of 100k rows in the database, we will now run a mixed read / write workload, by default this runs a 50% read / 50% write workload.
./nb start type=cql yaml=cql-keyvalue tags=phase:main host=<dse-host-or-ip> cycles=100k cyclerate=5000 threads=50 --progress console:1s
./nb start driver=cql workload=cql-keyvalue tags=phase:main host=<dse-host-or-ip> cycles=100k cyclerate=5000 threads=50 --progress console:1s
You should see output that looks like this:
```

2
engine-docs/src/main/resources/docs-for-nb/03_basics/01_command_line.md
View File

@ -27,7 +27,7 @@ named argument to the previous command. The following example
is a commandline with a command *start*, and two named arguments
to that command.
./nb start type=diag alias=example
./nb start driver=diag alias=example
### Discovery options ###

4
engine-docs/src/main/resources/docs-for-nb/04_designing_workloads/02_data_bindings.md
View File

@ -30,7 +30,7 @@ The above bindings block is also a valid activity YAML, at least for the _stdout
delta: WeightedStrings('one:1;six:6;three:3;')
# EOF (control-D in your terminal)
[test]$ ./nb run type=stdout yaml=stdout-test cycles=10
[test]$ ./nb run driver=stdout workload=stdout-test cycles=10
0,zero,00A_pro,six
1,one,00B_pro,six
2,two,00C_pro,three
@ -66,7 +66,7 @@ bindings:
delta: WeightedStrings('one:1;six:6;three:3;')
# EOF (control-D in your terminal)
[test]$ ./nb run type=stdout yaml=stdout-test cycles=10
[test]$ ./nb run driver=stdout workload=stdout-test cycles=10
This is a statement, and the file format doesn't
know how statements will be used!
submit job 1 on queue one with options 00B_pro;

18
engine-docs/src/main/resources/docs-for-nb/04_designing_workloads/04_stmt_tags.md
View File

@ -40,39 +40,39 @@ statements:
# EOF (control-D in your terminal)
# no tag filter matches any
[test]$ ./nb run type=stdout yaml=stdout-test
[test]$ ./nb run driver=stdout workload=stdout-test
I'm alive!
# tag name assertion matches
[test]$ ./nb run type=stdout yaml=stdout-test tags=name
[test]$ ./nb run driver=stdout workload=stdout-test tags=name
I'm alive!
# tag name assertion does not match
[test]$ ./nb run type=stdout yaml=stdout-test tags=name2
[test]$ ./nb run driver=stdout workload=stdout-test tags=name2
02:25:28.158 [scenarios:001] ERROR i.e.activities.stdout.StdoutActivity - Unable to create a stdout statement if you have no active statements or bindings configured.
# tag value assertion does not match
[test]$ ./nb run type=stdout yaml=stdout-test tags=name:bravo
[test]$ ./nb run driver=stdout workload=stdout-test tags=name:bravo
02:25:42.584 [scenarios:001] ERROR i.e.activities.stdout.StdoutActivity - Unable to create a stdout statement if you have no active statements or bindings configured.
# tag value assertion matches
[test]$ ./nb run type=stdout yaml=stdout-test tags=name:foxtrot
[test]$ ./nb run driver=stdout workload=stdout-test tags=name:foxtrot
I'm alive!
# tag pattern assertion matches
[test]$ ./nb run type=stdout yaml=stdout-test tags=name:'fox.*'
[test]$ ./nb run driver=stdout workload=stdout-test tags=name:'fox.*'
I'm alive!
# tag pattern assertion does not match
[test]$ ./nb run type=stdout yaml=stdout-test tags=name:'tango.*'
[test]$ ./nb run driver=stdout workload=stdout-test tags=name:'tango.*'
02:26:05.149 [scenarios:001] ERROR i.e.activities.stdout.StdoutActivity - Unable to create a stdout statement if you have no active statements or bindings configured.
# compound tag predicate matches every assertion
[test]$ ./nb run type=stdout yaml=stdout-test tags='name=fox.*',unit=bravo
[test]$ ./nb run driver=stdout workload=stdout-test tags='name=fox.*',unit=bravo
I'm alive!
# compound tag predicate does not fully match
[test]$ ./nb run type=stdout yaml=stdout-test tags='name=fox.*',unit=delta
[test]$ ./nb run driver=stdout workload=stdout-test tags='name=fox.*',unit=delta
11:02:53.490 [scenarios:001] ERROR i.e.activities.stdout.StdoutActivity - Unable to create a stdout statement if you have no active statements or bindings configured.

2
engine-docs/src/main/resources/docs-for-nb/04_designing_workloads/05_stmt_blocks.md
View File

@ -25,7 +25,7 @@ blocks:
beta: Combinations('b;l;o;c;k;2;-;COMBINATIONS;')
# EOF (control-D in your terminal)
[test]$ ./nb run type=stdout yaml=stdout-test cycles=10
[test]$ ./nb run driver=stdout workload=stdout-test cycles=10
0,block1-C
1,block2-O
2,block1-M

2
engine-docs/src/main/resources/docs-for-nb/04_designing_workloads/07_multi_docs.md
View File

@ -29,7 +29,7 @@ bindings:
statements:
- "doc2.number {numname}\n"
# EOF (control-D in your terminal)
[test]$ ./nb run type=stdout yaml=stdout-test cycles=10
[test]$ ./nb run driver=stdout workload=stdout-test cycles=10
doc1.form1 doc1.1
doc1.form2 doc1.2
doc2.number two

4
engine-docs/src/main/resources/docs-for-nb/04_designing_workloads/08_template_params.md
View File

@ -21,10 +21,10 @@ statements:
- "<<linetoprint:MISSING>>\n"
# EOF (control-D in your terminal)
[test]$ ./nb run type=stdout yaml=stdout-test cycles=1
[test]$ ./nb run driver=stdout workload=stdout-test cycles=1
MISSING
[test]$ ./nb run type=stdout yaml=stdout-test cycles=1 linetoprint="THIS IS IT"
[test]$ ./nb run driver=stdout workload=stdout-test cycles=1 linetoprint="THIS IS IT"
THIS IS IT
```

52
engine-docs/src/main/resources/docs-for-nb/05_activitytypes/06_cql_at.md
View File

@ -9,7 +9,7 @@ This is the same documentation that you get when you run
./nb help cql
To select this activity type, pass `type=cql` to a run or start command.
To select this activity type, pass `driver=cql` to a run or start command.
---------
@ -25,22 +25,22 @@ metrics about op binding, op submission, and waiting for a result.
Run a cql activity named 'cql1', with definitions from activities/cqldefs.yaml
~~~
... type=cql alias=cql1 yaml=cqldefs
... driver=cql alias=cql1 workload=cqldefs
~~~
Run a cql activity defined by cqldefs.yaml, but with shortcut naming
~~~
... type=cql yaml=cqldefs
... driver=cql workload=cqldefs
~~~
Only run statement groups which match a tag regex
~~~
... type=cql yaml=cqldefs tags=group:'ddl.*'
... driver=cql workload=cqldefs tags=group:'ddl.*'
~~~
Run the matching 'dml' statements, with 100 cycles, from [1000..1100)
~~~
... type=cql yaml=cqldefs tags=group:'dml.*' cycles=1000..1100
... driver=cql workload=cqldefs tags=group:'dml.*' cycles=1000..1100
~~~
This last example shows that the cycle range is [inclusive..exclusive),
to allow for stacking test intervals. This is standard across all
@ -48,17 +48,15 @@ activity types.
### CQL ActivityType Parameters
- **driver** - default: dse - The type of driver to use, either dse, or
oss. If you need DSE-specific features, use the dse driver. If you are
connecting to an OSS Apache Cassandra cluster, you must use the oss
driver. The oss driver option is only available in nosqlbench.
- **host** - The host or hosts to use for connection points to
the cluster. If you specify multiple values here, use commas
with no spaces.
Examples:
- `host=192.168.1.25`
- `host=`192.168.1.25,testhost42`
- **yaml** - The file which holds the schema and statement defs.
- **workload** - Workload definition which is also the name of the yaml file or path to the yaml file
which holds the schema and statement defs.
see workload yaml locations below.
(no default, required)
- **port** - The port to connect with
- **cl** - An override to consistency levels for the activity. If
@ -181,7 +179,7 @@ activity types.
- `jmxreporting=true`
- `jmxreporting=false` (the default)
- **alias** - this is a standard engineblock parameter, however
the cql type will use the yaml value also as the alias value
the cql type will use the workload value also as the alias value
when not specified.
- **errors** - error handler configuration.
(default errors=stop,retryable->retry,unverified->stop)
@ -215,7 +213,7 @@ activity types.
(default: bucket)
(options: concat | bucket | interval)
The concat sequencer repeats each statement in order until the ratio
is achieved.
is achieved.
The bucket sequencer uses simple round-robin distribution to plan
statement ratios, a simple but unbalanced form of interleaving.
The interval sequencer apportions statements over time and then by
@ -274,36 +272,36 @@ now **they are limited to a YAML params block**:
params:
ratio: 1
# Sets the statement ratio within the operation sequencer
# Sets the statement ratio within the operation sequencer
# scheme. Integers only.
# When preparing the operation order (AKA sequencing),
# When preparing the operation order (AKA sequencing),
# frequency of the associated statements.
cl: ONE
# Sets the consistency level, using any of the standard
# identifiers from com.datastax.driver.core.ConsistencyLevel,
# any one of:
# LOCAL_QUORUM, ANY, ONE, TWO, THREE, QUORUM, ALL,
# identifiers from com.datastax.driver.core.ConsistencyLevel,
# any one of:
# LOCAL_QUORUM, ANY, ONE, TWO, THREE, QUORUM, ALL,
# EACH_QUORUM, SERIAL, LOCAL_SERIAL, LOCAL_ONE
prepared: true
# By default, all statements are prepared. If you are
# By default, all statements are prepared. If you are
# creating schema, set this to false.
idempotent: false
# For statements that are known to be idempotent, set this
# For statements that are known to be idempotent, set this
# to true
instrument: false
# If a statement has instrument set to true, then
# If a statement has instrument set to true, then
# individual Timer metrics will be tracked for
# that statement for both successes and errors,
# that statement for both successes and errors,
# using the given statement name.
logresultcsv: true
OR
logresultcsv: myfilename.csv
# If a statement has logresultcsv set to true,
# If a statement has logresultcsv set to true,
# then individual operations will be logged to a CSV file.
# In this case the CSV file will be named as
# <statement-name>--results.csv.
@ -375,7 +373,7 @@ no operation had to be retried.
As for a normal single page read result, both the execute and result timers
are included within the code block wrapped by the pages metric.
### YAML Format
### Workload YAML Format
The YAML file for a CQL activity has the following structure:
1. One or more document sections, separated with '---' and a newline.
@ -409,14 +407,14 @@ additional parameters to the activity. There are two forms,
form contains a default value. In any case, if one of these parameters is
encountered and a qualifying value is not found, an error will be thrown.
### YAML Location
### Workload YAML Location
The YAML file referenced in the yaml= parameter will be searched for in the following places, in this order:
The YAML file referenced in the workload= parameter will be searched for in the following places, in this order:
1. A URL, if it starts with 'http:' or 'https:'
2. The local filesystem, if it exists there
3. The internal classpath and assets in the jar.
The '.yaml' suffix is not required in the yaml= parameter, however it is
The '.yaml' suffix is not required in the workload= parameter, however it is
required on the actual file. As well, the logical search path "activities/"
will be used if necessary to locate the file, both on the filesystem and in
the classpath.

8
engine-docs/src/main/resources/docs-for-nb/05_activitytypes/06_stdout_at.md
View File

@ -9,7 +9,7 @@ This is the same documentation that you get when you run
./nb help stdout
To select this activity type, pass `type=stdout` to a run or start command.
To select this activity type, pass `driver=stdout` to a run or start command.
---------
@ -24,17 +24,17 @@ that uses the curly brace token form in statements.
Run a stdout activity named 'stdout-test', with definitions from activities/stdout-test.yaml
~~~
... type=stdout yaml=stdout-test
... driver=stdout workload=stdout-test
~~~
Only run statement groups which match a tag regex
~~~
... type=stdout yaml=stdout-test tags=group:'ddl.*'
... driver=stdout workload=stdout-test tags=group:'ddl.*'
~~~
Run the matching 'dml' statements, with 100 cycles, from [1000..1100)
~~~
... type=stdout yaml=stdout-test tags=group:'dml.*' cycles=1000..11000 filename=test.csv
... driver=stdout workload=stdout-test tags=group:'dml.*' cycles=1000..11000 filename=test.csv
~~~
This last example shows that the cycle range is [inclusive..exclusive),

10
engine-docs/src/main/resources/docs-for-nb/05_activitytypes/index.md
View File

@ -5,6 +5,12 @@ weight: 50
Each nosqlbench scenario is comprised of one or more activities of a specific type. The types of activities available are provided by the version of nosqlbench.
Additional activity types will be added in future releases. This section is a reference section that shows the help you would get with a command like:
Additional activity types will be added in future releases. There are command line help topics for each activity type (driver).
./nb help <activity type>
To get a list of topics run:
./nb help topics
To get the help for a particular topic run:
./nb help <topic>

12
engine-docs/src/main/resources/docs-for-nb/09_reference/02_cli_scripting.md
View File

@ -25,7 +25,7 @@ are non-scripting options.
All activities that run during a scenario run under the control of, but
independently from the scenario script. This means that you can have a number of activities running while the scenario script is doing its own thing. The scenario only completes when both the scenario script and the activities are finished.
### `start type=<activity type> alias=<alias> ...`
### `start driver=<activity type> alias=<alias> ...`
You can start an activity with this command. At the time this command is
evaluated, the activity is started, and the script continues without blocking. This is an asynchronous start of an activity. If you start multiple activities in this way, they will run concurrently.
@ -41,11 +41,11 @@ scenario to pause until the activity is stopped. This means that all threads for
Await the normal completion of an activity with the given alias. This causes the scenario script to pause while it waits for the named activity to finish. This does not tell the activity to stop. It simply puts the scenario script into a paused state until the named activity is complete.
### `run type=<activity type> alias=<alias> ...`
### `run driver=<activity type> alias=<alias> ...`
Run an activity to completion, waiting until it is complete before continuing with the scenario script. It is effectively the same as
start type=<activity type> ... alias=<alias>
start driver=<activity type> ... alias=<alias>
await <alias>
### `waitmillis <milliseconds>`
@ -63,9 +63,9 @@ Add the contents of the next argument to the scenario script buffer.
# An example CLI script
~~~
./run-eb \
start type=stdout alias=a cycles=100K yaml=cql-iot tags=phase:main\
start type=stdout alias=b cycles=200K yaml=cql-iot tags=phase:main\
./nb \
start driver=stdout alias=a cycles=100K workload=cql-iot tags=phase:main\
start driver=stdout alias=b cycles=200K workload=cql-iot tags=phase:main\
waitmillis 10000 \
await one \
stop two

6
engine-docs/src/main/resources/docs-for-nb/09_reference/03_scenario_scripting.md
View File

@ -12,7 +12,7 @@ The EngineBlock runtime is a combination of a scripting sandbox and a workload e
All of the heavy lifting is left to Java and the core nosqlbench runtime. This includes the iterative workloads that are meant to test the target system. This is combined with a control layer which is provided by Nashorn and eventually GraalVM. This division of responsibility allows the high-level test logic to be "script" and the low-level activity logic to be "machinery". While the scenario script has the most control, it also is the least busy relative to activity workloads. The net effect is that you have the efficiency of the iterative test loads in conjunction with the open design palette of a first-class scripting language.
Essentially, the ActivityType drivers are meant to handle the workload-specific machinery. They also provide dynamic control points and parameters which special to that activity type. This exposes a full feedback loop between a running scenario script and the activities that it runs. The scenario is free to read the performance metrics from a running activity and make changes to it on the fly.
Essentially, the ActivityType drivers are meant to handle the workload-specific machinery. They also provide dynamic control points and parameters which special to that activity type (driver). This exposes a full feedback loop between a running scenario script and the activities that it runs. The scenario is free to read the performance metrics from a running activity and make changes to it on the fly.
## Scripting Environment
@ -44,7 +44,7 @@ Interaction with the nosqlbench runtime and the activities therein is made easy
by the above variables and objects. When an assignment is made to any of these variables, the changes are propagated to internal listeners. For changes to _threads_, the thread pool responsible for the affected activity adjusts the number of active threads (AKA slots). Other changes are further propagated directly to the thread harnesses and components which implement the ActivityType.
:::warning
Assignment to the _type_ and _alias_ activity parameters has no special effect, as you can't change an activity to a different type once it has been created.
Assignment to the _workload_ and _alias_ activity parameters has no special effect, as you can't change an activity to a different driver once it has been created.
:::
You can make use of more extensive Java or Javascript libraries as needed,
@ -80,7 +80,7 @@ When a script is run, it has absolute control over the scenario runtime while it
## Strategies
You can use nosqlbench in the classic form with `run type=<type> param=value ...` command line syntax. There are reasons, however, that you will sometimes want customize and modify your scripts directly, such as:
You can use nosqlbench in the classic form with `run driver=<activity_type> param=value ...` command line syntax. There are reasons, however, that you will sometimes want customize and modify your scripts directly, such as:
- Permute test variables to cover many sub-conditions in a test.
- Automatically adjust load factors to identify the nominal capacity of a system.

2
engine-docs/src/main/resources/docs-for-nb/09_reference/04_standard_metrics.md
View File

@ -4,7 +4,7 @@ title: Standard Metrics
# Standard Metrics
nosqlbench comes with a set of standard metrics that will be part of every activity type. Each activity type enhances the metrics available by adding their own metrics with the nosqlbench APIs. This section explains what the standard metrics are, and how to interpret them.
nosqlbench comes with a set of standard metrics that will be part of every activity type (driver). Each activity type (driver) enhances the metrics available by adding their own metrics with the nosqlbench APIs. This section explains what the standard metrics are, and how to interpret them.
## read-input