IntenseFinance/nosqlbench
3
0
Fork 0
mirror of https://github.com/nosqlbench/nosqlbench.git synced 2025-02-03 19:30:34 -06:00
Code Issues Packages Projects Releases Wiki Activity

update wip devguide

Browse Source
This commit is contained in:
Jonathan Shook 2021-02-04 17:47:13 -06:00
parent 516aee2ef5
commit 9526208af8
23 changed files with 802 additions and 206 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
devdocs/INDEX.md
View File

@ -1,6 +1,10 @@
# Developer Docs and RFCs
This directory is where new ideas or developer guides will be organized for now. They may eventually find their way into
a developer-focused documentation section.
This directory is where new ideas or developer guides will be organized
for now. They may eventually find their way into a developer-focused
documentation section.
You may be intererested in:
- [The NoSQLBench Dev Guide](devguide/README.md)

58
devdocs/devguide/README.md Normal file
View File

@ -0,0 +1,58 @@
# Developing with NoSQLBench
This is an overview of how to develop within the NoSQLBench ecosystem.
This guide
## Welcome All
The NoSQLBench project welcomes all contributors, so long as you are
willing to abide by the code
of [CODE OF CONDUCT](../../CODE_OF_CONDUCT.md). Further, you must be
willing to license your contributions under the
[APLv2](../../LICENSE.txt).
## Basics
Herein, and in other docs, you may see NoSQLBench abbreviated as NB
(the project) or nb (the command).
This dev guide covers detailed topics which can help you get bootstrapped
as a contributor. It does not try to cover all the topics which are
included in the main documentation. What you see in this guide wil assume
you have some familiarity with NoSQLBench as a user.
## Topics
- [Design Principles](design_principles.md) - This is a short treatise
that explains the architectural principles behind NoSQLBench.
- [Project Structure](project_structure.md) - An explanation of the module
naming and dependency structure.
- [Dynamic Endpoints](apis/dynamic_endpoints.md) - An introduction to the
appserver mode and how to write new webservices for it.
- [Driver How-To](drivers/README.md) - An introduction for new driver
designers, including API guides and driver standards.
- [NBUI Development](nbui/README.md) - An introduction to the UI
subsystem (NBUI), and tricks for effective development.
- [Java versions](java_versions.md) - A map of where java versions are
configured in the project.
- [Scripting Extensions](scripting_extensions.md) - A guide on what
scripting extensions are, how they are used, and how to build one.
- [Documentation Sources](nb_docs.md) - Where docs are located
- [Adding Scenarios](adding_scenarios.md) - How to add built-in scenarios
### API Naming
In general, internal APIs which are suggested for use throughout
NoSQLBench will start with the *NB* prefix, such as NBIO, or NBErrors.
This makes these APIs easier to find and reference when building new
drivers or core functionality.
New contributors should familiarize themselves with these APIs so that
they can employ them when appropriate. These APIs should also have
sufficient documenation here and in Javadoc to explain their usage.
### API Guides
- [NBIO - File and System IO](nbio_api.md) - A standard way to get files
and other resources from a running NB process.

14
devdocs/devguide/adding_scenarios.md Normal file
View File

@ -0,0 +1,14 @@
# Adding Scenarios
All of the named scenarios that are bundled with NoSQLBench are simply
named yaml files with scenarios included.
If you are a developer and want to add one of these, you can submit a pull
request to the main project. However, to be accepted, any named scenario
should follow all the conventions described in the main
_Designing Workloads_ documentation, including naming, tagging, and
default values.
If you are not a developer, and you have come up with a workload that
you'd like to make part of the standard NoSQLBench distribution, you can
add it to an issue on the main issue tracker.

33
devdocs/devguide/apis/dynamic_endpoints.md Normal file
View File

@ -0,0 +1,33 @@
# Dynamic Endpoints
NoSQLBench is more than just a test client. It is a test scenario
execution machine with many pluggable behaviors. It also has a daemon mode
which can be used to enable different types of persistent services:
- Documentation Server
- [NBUI](../nbui/README.md) - The fledgling NoSQLBench UI
- Workload Services
These are all enabled by a set of web services backed by jetty and powered
by jax-rs and related libraries.
## Service Discovery
When NoSQLBench is invoked in appserver mode (classically called
`docserver` mode), it starts up with a set of web services. Like many
other runtime elements of NB, these services are discovered internally as
long as they implement the WebServiceObject tagging interface and are
registered as a service with an annotation like
@Service(WebServiceObject.class)
Any such class is processed as a jax-rs annotated class and automatically
added to the active services accordingly.
This is all you have to do to add endpoints to NoSQLBench!
## Useful Links
* [JAX-RS 2.0 spec](http://download.oracle.com/otn-pub/jcp/jaxrs-2_0-fr-eval-spec/jsr339-jaxrs-2.0-final-spec.pdf)

121
devdocs/devguide/design_principles.md Normal file
View File

@ -0,0 +1,121 @@
# NoSQLBench Design Principles
This is a basic readout on the design principles that inform the core
NoSQLBench design. They can be taken as a set of non-feature requirements
for a serious testing instrument. These principles do not represent the
actual state of NoSQLBench. In fact, some of them may appear more
aspirational than descriptive. Nonetheless, they describe a set of guiding
principles that any contributor should strive to keep in mind as they
build on NoSQLBench.
Ideally, any feature or component will have the best of all of these
attributes. When there is tension between them, designs should always
defer to what a reasonable user would want. This list may adapt and change
over time with more contributions and discussion.
The rest of this guide is rather detailed and delves into design
philosphy.
## Concept Reuse
NoSQLBench aims to be a power tool that is safe for casual users while
also being a trusted tool by serious users. As such, it needs to provide a
set of useful concepts which apply in an axiomatic way across all
NoSQLBench capabilities. This means that once you know what a `cycle` is,
you know that the purpose and usage of it will be consistent no matter
where, when, or how you find it. As such, it is a material part of the
user experience. Concepts like this are often disregarded in tool design.
This puts users as a disadvantage when they want to do anything
non-trivial. Designing robust concepts that are tangible and composable is
one of the hardest aspects of design, and one that is often kicked down
the road. Not attacking this problem directly is one of the deepest forms
of accidental complexity and one that is difficult for a desing to recover
from.
In NoSQLBench, users should take for granted that an understanding of
concepts like _activity_, _cycle_, and _binding_
is directly empowering, and worthy of their attention. Such an
understanding gives them access to additional layers of capability and
expression, and allows all users to speak the same language to each other
with little concern for ambiguity.
## Modularity
Modularity is used throughout NoSQLBench. The reasons are many, but the
practical effect is that once you understand the mechanisms used to
achieve modularity, you can add your own modules of a given type to the
runtime with little fanfare.
Modularity in a project is also a form of applying the Liskov substitution
principle at a higher level. It encourages healthy design boundaries at
key points of re-use. This means that both service providers and service
consumers can depend on the notions of responsibility based design as well
as design-by-contract.
Many runtime elements of NoSQLBench are provided as bundled plugins. Any
time you see a `@Service` annotation, it describes a service entry for
something that is defined by an interface.
## Efficiency
A serious testing tool must be an effective measurement instrument. In the
end all testing tools are merely measurement tools of some sort. With a
load-generating tool, the business of driving a workload can not interfere
with the need to measure accurately. There are multiple paths to achieving
an effective separation of these concerns, but they are all non-trivial. (
load bank orchestration, local resource marshaling, etc.)
One of the greatest simplifications we can provide in a testing tool is to
allow it to do meaningful work without requiring the same level of
orchestration, coordination, and aggregation that many other tools require
for basic workloads. Optimization for performance is absolutely critical
in a testing tool. It doesn't matter what your testing system can do if it
is always the slowest part of the composed system. Thus, many constructs
found within NoSQLBench are purpose built for efficiency. It is "maturely
optimized" because that is what makes NoSQLBench suited for its purpose.
## Flexibility
A testing system which can only do one thing or be used in on particular
way will not be suitable for many common testing tasks. This lack of fit
for many simple purposes would make a system less reusable overall. In the
world of systems testing, requirements are extremely varied, and
circumstances are always changing. Thus, it is uncommon for users to
invest much time in one trick ponies. Such tools have their purposes, and
serve a need at times. However, they do not generally reward users with
incremental knowledge, nor help them to adapt their investment of testing
effort to new or more nuanced needs. As such, they are often used and then
disregarded for further consideration. Even more costly is that these
tools often over-simplify the testing surface area to such a degree that
users are left without even basic answers to the questions they are
asking, or worse, wrong or inaccurate answers to specific questions.
NoSQLBench should provide the building blocks that it has with clarity and
purposes. It should allow the user to recombine and reconfigure these
building blocks as needed for whatever level of testing they require.
### Scalable Experience
For common tasks, or simple usage patterns, it should be possible to use
NoSQLBench quickly. A user should be able to ask a simple question and get
a simple answer without having to debate or wrestle with the system to get
it running. These capabilities should simply be packaged in a pre-baked
form. Each time a user asks more from the testing tool, it should be able
to
### Discoverability
Another element of flexibility, in practical terms, is discoverability. A
tool which can do both simple and sophisticated things is wasted if users
are unable to find and understand the incremental levels of capability it
offers. While the initial surface area of the tool may be intentionally
simplified, there must be some obvious way for a user to opt-in to more
knowledge, more tooling, and more sophistication if and when they want to.
## Interoperability
Where possible, common standard and modern conventions should be used for
interfacing with external systems. This means that most basic tooling in
the modern development ecosystem should be easy to integrate with
NoSQLBench when needed. This includes aspects such as configuration
formats, logging tools, web endpoints, and so on.

24
devdocs/devguide/drivers/README.md Normal file
View File

@ -0,0 +1,24 @@
# Driver Development
This section should have everything you need to know to build a successful
driver in NoSQLBench.
If you are new to NoSQLBench concepts, you may want to
read [about drivers](drivers_overview.md).
You'll want to be generally familiar with the current NoSQLBench
[driver standards](driver_standards.md). This document explains what a
well-behaved driver can do. Much of what is requested in driver standards
is directly supported by a set of supporting APIs which are provided to
all driver implementations. It may seem like a high bar to request this of
developers, but without such guidelines, a bar is set nonetheless. The aim
of this is to help define and clarify how to make a
_good_ driver.
## Driver APIs
- [NBOpTemplate](optemplate_api.md) - Op Templating - This is the
recommended way to map user-level semantics to driver-specific
operations.
- [NBErrors](nberrors_api.md) - Uniform error handling - A modular and
configurable error handler which new drivers should use by default.

283
devdocs/devguide/driver_standards.md → devdocs/devguide/drivers/driver_standards.md
View File

@ -1,34 +1,17 @@
# NoSQLBench Driver Standards
This is a work in progress...
This is the document to read if you want to know if your NoSQLBench driver
is complete. Within this document, the phrase `conformant` will be taken
to mean that a driver or feature is implemented according to the design
intent and standards of the NoSQLBench driver API.
While it may be possible to partially implement a driver for basic use,
following the guidelines in this document will ensure that contributed
drivers for NoSQLBench work in a familiar and reliable way for users from
one driver to another.
This is the document to read if you want to know if your NoSQLBench driver is complete.
Within this document, the phrase `conformant` will be taken to mean that a driver or feature
is implemented according to the design intent and standards of the NoSQLBench driver API.
While it may be possible to partially implement a driver for basic use, following the guidelines
in this document will ensure that contributed drivers for NoSQLBench work in a familiar and
reliable way for users from one driver to another.
Over time, the standards in this guide will be programmatically enforced by the NoSQLBench
driver API.
## Op Templates
The core building block of a NoSQLBench driver is the op template. This is the form of a
statement or operation that users add to a yaml or workload editor to represent a single operation.
For example, in the CQL driver, this is called a "statement template", but going forward, they will
all be called Op Templates and internal API names will reflect that.
It is the driver's responsibility to create a quick-draw version of an operation.
## Op Sequencing
A conformant driver should use the standard method of creating an operational sequence. This means
that a driver simply has to provide a function to map an OpTemplate to a more ready to use form that
is specific to the low level driver in question.
Over time, the standards in this guide will be programmatically enforced
by the NoSQLBench driver API.
## Terms
@ -37,6 +20,40 @@ is specific to the low level driver in question.
- Native driver - An underlying driver which is provided by a vendor or
project.
## Op Templates
The core building block of a NoSQLBench driver is the op template. This is
the form of a statement or operation that users add to a yaml or workload
editor to represent a single operation.
It is the driver's responsibility to create a quick-draw version of an
operation. This is done by using the OpTemplate API. Rules for how a
developer maps an op template to an op function are not set in stone, but
here are some guidelines:
1. Pre-compute as much as you can.
2. Store re-usable elements of an operation in thread-safe form and re-use
it wherever possible.
3. Allow as much to be deferred till cycle time as reasonable, assuming
you can cache it effectively.
A moderately advanced example of caching objects by name is included in
the pulsar driver.
In contrast to the rules about how you map your op templates to op
functions (and then ops), it is *crucial* tha tyou document the rules for
how the fields of an template are used. The content that users provide in
a YAML file are the substance of an op template. It is very important that
you document what this means for users, specifically in terms of how field
names and values map to a specific operation.
## Op Sequencing
A conformant driver should use the standard method of creating an
operational sequence. This means that a driver simply has to provide a
function to map an OpTemplate to a more ready to use form that is specific
to the low level driver in question.
## Metrics
At a minimum, a conformant driver should provide the following metrics:
@ -61,7 +78,8 @@ At a minimum, a conformant driver should provide the following metrics:
as those on the result timer, but they should be applied only in the
case of no exceptions during the operation's execution.
- errorcounts-... (counters)- Each uniquely named exception or error type
that is known to the native driver should be counted.
that is known to the native driver should be counted. This is provided
for you as a side effect of using the NBErrorHandler API.
- tries (histogram) - The number of tries for a given operation. This
number is incremented before each execution of a native operation, and
when the result timer is updated, this value should be updated as well
@ -77,72 +95,19 @@ without flexibility in error handling, users may not be able to do
reasonable testing for their requirements, thus configurable error
handling is essential.
Until the error handling subsystem is put in place, these types of error
handling levels are suggested:
1. stop
2. warn
3. retry
4. histogram
5. count
6. ignore
This serves as an error handling stack, where the user chooses the entry
point. From the user-selected entry point, all of the remaining actions
are taken until the end if possible.
### stop
If an exception occurs, and the user has selected the `stop` error
handler, then the activity should be stopped. This is triggered by
allowing the NB driver to propagate a runtime exception up the stack.
Since this error handler interrupts flow of an activity, no further error
handling is done.
### warn
If an exception occurs and the user has selected the `warn` error handler,
the the exception should be logged at WARN level.
The next error handler `retry` should also be called.
### retry
If an exception occurs and the user has selected the `retry` error
handler, **AND** the exception represents a type of error which could
reasonably be expected to be non-persistent, then the operation should be
re-submitted after incrementing the tries metric.
Whether or not the operation is retried, the next error handler
`histogram` should also be called.
### histogram
If an exception occurs and the user has selected the `histogram` error
handler,the error should be recorded with the help class
`ExceptionHistoMetrics`. This adds metrics under the `errorhistos` label
under the activity's name.
The next error handler `count` should also be called.
### count
If an exception occurs and the user has selected the `count` error
handler, then the error should be counted with the helper class
`ExceptionCountMetrics`. This adds metrics under the `errorcounts` label
under the activity's name.
The next exception handler `ignore` should also be called, but this is
simply a named 'no-op' which is generally the last fall-through case in a
switch statement.
TBD
A core library, NBErrorHandler is provided as a uniform way to handle
these errors. It is documented separately in this dev guide. If you add
this error handler to your action implementation, users will automatically
get a completely configurable and standard way to decide what happens for
specific errors in their workload.
## Result Validation
TBD
## Diagnostic Mode
TBD
## Naming Conventions
@ -173,18 +138,18 @@ with the help command `nb help <name>`. For example, if a driver module
contains `../src/main/resources/mydriver-specials.md`, then a user would
be able to find this help by running `nb help mydriver-specials`.
These sources of documentation can be wired into the main NoSQLBench documentation system with a set
of content descriptors.
These sources of documentation can be wired into the main NoSQLBench
documentation system with a set of content descriptors.
## Named Scenarios
Conformant driver implementations should come with one or more examples of a workload under the
activities directory path.
Useful driver implementations should come with one or more examples of a
workloads under the activities directory path. These examples should
employ the "named scenarios" format as described in the main docs. By
including named scenarios in the yaml format, these named scenarios then
become available to users when they look for scenarios to call with the
Conformant driver implementations should come with one or more examples of
a workload under the activities directory path. Useful driver
implementations should come with one or more examples of a workloads under
the activities directory path. These examples should employ the "named
scenarios" format as described in the main docs. By including named
scenarios in the yaml format, these named scenarios then become available
to users when they look for scenarios to call with the
`--list-scenarios` command.
To include such scenario, simply add a working yaml with a scenarios
@ -208,8 +173,8 @@ option in addition to the `--list-scenarios` command.
## Testing and Docs
Complete driver implementations should also come with a set of examples under the examples
directory path.
Complete driver implementations should also come with a set of examples
under the examples directory path.
Unit testing within the NB code base is necessary in many places, but not
in others. Use your judgement about when to *not* add unit testing, but
@ -232,69 +197,61 @@ become part of one whole. Particularly, docs should provide executable
examples which can also be used to explain how NB or drivers work. Until
this is done, use the guidelines above.
## Usage of the Op Template
The operation which will be executed in a driver should be derivable from
the YAML as provided by a user. Thankfully, NoSQLBench goes to great
lengths to make this easy for both to the user and to the driver
developer. In particular, NB presents the user with a set of formatting
options in the YAML which are highly flexible in terms of syntax and
structure. On the other side, it presents the driver developer with a
service interface which contains all the input from the user as a complete
data structure.
This means that the driver developer needs to make it clear how different
forms of content from the YAML will map into an operation. Fundamentally,
a driver is responsible for mapping the fully realized data structure of
an `op template` into an executable operation by NoSQLBench.
In some protocols or syntaxes, the phrase _the statement_ makes sense, as
it is the normative way to describe what an operation does. This is true,
for example, with CQL. In CQL You have a statement which provides a very
clear indication of what a user is expecting to happen. At a more abstract
level, and more correctly, the content that a user puts into the YAML is a
`statement template`, and more generally within NoSQLBench, and `operation
template`. This is simply called an `op template` going forward, but as a
driver developer, you should know that these are simply different levels
of detail around the same basic idea: an executable operation derived from
a templating format.
Since there are different ways to employ the op template, a few examples
are provided here. As a driver developer, you should make sure that your
primary docs include examples like these for users. Good NB driver docs
will make it clear to users how their op templates map to executable
operations.
### op template: statement form
In this form, the op template is provided as a map, which is also an
element of the statements array. The convention here is that the values
for and _the statement_name_ and _the statement_ are taken as the first
key and value. Otherwise, the special properties `name` and `stmt` are
explicitly recognized.
```text
statements:
- aname: the syntax of the statement with binding {b1}
tags:
tag1: tagvalue1
params:
param1: paramvalue1
freeparamfoo: freevaluebar
bindings:
b1: NumberNameToString()
```
### op template: map form
```text
statements:
- cmd_type:
```
Structural variations and conventions.
## Handling secrets
Reading passwords ...
## Parameter Use
Activity parameters *and* statement parameters must combine in intuitive
ways.
### ActivityType Parameters
The documentation for an activity type should have an explanation of all
the activity parameters that are unique to it. Examples of each of these
should be given. The default values for these parameters should be given.
Further, if there are some common settings that may be useful to users,
these should be included in the examples.
### Statement Parameters
The documentation for an activity type should have an explanation of all
the statement parameters that are unique to it. Examples of each of these
should be given. The default values for these parameters should be given.
### Additive Configuration
If there is a configuration element in the activity type which can be
modified in multiple ways that are not mutually exclusive, each time that
configuration element is modified, it should be done additively. This
means that users should not be surprised when they use multiple parameters
that modify the configuration element with only the last one being
applied. An example of this would be adding a load-balancing policy to a
cql driver and then, separately adding another. The second one should wrap
the first, as this is expected to be additive by nature of the native
driver's API.
### Parameter Conflicts
If it is possible for parameters to conflict with each other in a way that
would provide an invalid configuration when both are applied, or in a way
that the underlying API would not strictly allow, then these conditions
must be detected by the activity type, with an error thrown to the user
explaining the conflict.
### Parameter Diagnostics
Each and every activity parameter that is set on an activity *must* be
logged at DEBUG level with the
pattern `ACTIVITY PARAMETER: <activity alias>` included in the log line,
so that the user may verify applied parameter settings. Further, an
explanation for what this parameter does to the specific activity *should*
be included in a following log line.
Each and every statement parameter that is set on a statement *must* be
logged at DEBUG level with the
pattern `STATEMENT PARAMETER: <statement name>: ` included in the log
line, so that the user may verify applied statement settings. Further, an
explanation for what this parameter does to the specific statement *
should* be included in a following log line.

0
devdocs/contributing_drivers.md → devdocs/devguide/drivers/drivers_overview.md
View File

35
devdocs/devguide/drivers/nberrors_api.md Normal file
View File

@ -0,0 +1,35 @@
# NBErrorHandler API
NoSQLBench provides a standard way of configuring error handling across
driver implementations. This is provided by NBErrorHandler and associated
classes.
This error handler allows a chain of specific error handler behaviors to
be invoked based on which type of error occurs. Additionally, the error
handler is responsible for determining the cycle code to be used in
advanced testing as well as for indicating whether or not an operation is
eligible to be retried or not.
The entire implementation is of this error handler is in package:
io.nosqlbench.engine.api.activityapi.errorhandling.modular
The canonical example of using this API can be found in HttpAction.
## NBErrorHandler Sketch
As a feature that changes control-flow based on user input, it strongly
suggests a specific type of flow within an action. The following
pseudo-code explains this pattern.
1. When an activity is initialized, also initialize an instance of
NBErrorHandler
2. Each time an Action is called, loop while retries are not exhausted.
1. catch all exceptions and pass them to the error handler
2. Read the returned ErrorDetail.
3. If the error detail indicates that the error was retryable and
max_tries is not exhausted, retry the operation Otherwise, stop the
cycle.
4. In any case, when the cycle is complete, always return the cycle
code in the error detail, or 0 if no error was found

168
devdocs/devguide/drivers/optemplate_api.md Normal file
View File

@ -0,0 +1,168 @@
# OpTemplate API
The operation which will be executed in a driver should be derivable from
the YAML as provided by a user. Thankfully, NoSQLBench goes to great
lengths to make this easy for both to the user and to the driver
developer. In particular, NB presents the user with a set of formatting
options in the YAML which are highly flexible in terms of syntax and
structure. On the other side, it presents the driver developer with a
service interface which contains all the input from the user as a complete
data structure.
This means that the driver developer needs to make it clear how different
forms of content from the YAML will map into an operation. Fundamentally,
a driver is responsible for mapping the fully realized data structure of
an `op template` into an executable operation by NoSQLBench.
In some protocols or syntaxes, the phrase _the statement_ makes sense, as
it is the normative way to describe what an operation does. This is true,
for example, with CQL. In CQL You have a statement which provides a very
clear indication of what a user is expecting to happen. At a more abstract
level, and more correctly, the content that a user puts into the YAML is a
`statement template`, and more generally within NoSQLBench,
and `operation template`. This is simply called an `op template` going
forward, but as a driver developer, you should know that these are simply
different levels of detail around the same basic idea: an executable
operation derived from a templating format.
## Op Function Mapping
This is accomplished by using a technique called _Op Function Mapping_
-- the act of mapping an op template to an op function. There are two
functional steps in this process: First, at activity initialization, and
then later within each cycle.
This process is governed by a single type:
Function<OpTemplate, LongFunction<T>>
In this type, the operation you will use is generic type T, and is up to
you, the driver developer. At activity initialization time, a function of
the type above will be used to construct a sequence of LongFunction<T>.
Within each cycle, one of these LongFunction<T> will be selected from the
sequence and called with the cycle to produce a fully realized operation.
This is the most challenging part of building a NoSQLBench driver, but it
is arguably the most important. This section is worth some study if you
want to understand one of the key mechanisms of NoSQLBench.
## Preparing Operations
During execution, NB needs to be fast and lean in terms of the prep work
needed to execute an operation. This means that, at startup, an activity
is responsible for doing two critical steps:
1) (Driver Logic) Map the YAML structure to a set of driver-specific
operation templates.
2) (Core Logic, with Type specified by driver) Cache the operations in a
ready form which is efficient to use.
3) (Core Logic) Assemble these ready operations into a sequence based on
ratios.
Much of the boilerplate for doing this is handled already by the default
Activity implementation which is used by all drivers: the
_SimpleActivity_.
## Op Mapping
Step 1 above is all about mapping the YAML content for each statement to a
driver-specific type. For the sake of illustration, we'll call that simply
a ReadyOp. In order to provide this, you call
`super.createOpSequence(...)` with a function from _OpTemplate_ to your
ReadyOp type, and store the result in your activity.
As long as your function, say a _Function<OpTemplate,ReadyOp>_ knows what
the user means to do depending on what they put in the YAML structure,
steps #2 and #3 are already handled! You're done.
But wait, there's more. If you unpack the phrase above "what the user
means to do ...", you'll see that it is loaded with ambiguity. The rest of
this guide is meant to explain why and how you actually create that
all-important mapping function.
## Op Template Semantics
**NOTE:**
To fully understand this section, it is important that you understand what
the NB Standard YAML format looks like. A detailed explanation of this
format is found under the user guide, with detailed examples under the "
Designing Workloads", with each feature of the YAML format explained in
detail.
The actual type of mapping function provided by your driver will be up to
you, but it is not an actual operation to be executed. Each cycle will
synthesize the operation to be executed by combining your ready op type
above with a cycle number. Thus, from the yaml format to the operation,
there are two stages of resolution:
1. YAML to ready op, as controlled by your own Ready___Op implementation,
as a Function<OpTemplate,YourTypeHere>.
2. ready op to operation, as controlled by the
### Reserved parameters
Some param names used in the YAML format are reserved for current or
future use by the NoSQLBench core. These are:
- name (used to name all elements within the Standard YAML format)
- ratio (used to set ratios in the Standard YAML format)
- type,driver,context,instance (future use)
These parameter names are sometimes visible as part of the OpTemplate, but
driver implementors are not allowed to reassign what they mean or how they
are used.
## Op Function Mapping
Op Function Mapping, or simply _Op Mapping_ is the process of interpreting
the op template and rendering a partially resolved operation back to
NoSQLBench.
Generally, Op Function Mapping is controlled by a driver-provided function
of type Function<OpTemplate,LongFunction<T>>. The result of calling this
function is another function which is used later during each cycle to
render a fully realized and executable operation.
The OpTemplate is a parsed and assembled view of everything the user has
specified for a statement in the yaml, including layering in values from
blocks and root document levels. This includes bindings, params, tags, and
so on. For bindings, it also knows which fields are specified as static
and which ones are tied to bindings with the
`{binding}` syntax.
This allows you, as the developer, to know which parts of an operation's
definition are intended to be fixed and which ones are to be deferred
until a specific cycle. The methods on the OpTemplate allow you to
interrogate the data.
Ideally, you would create an operation that is as fully resolved as
possible. Sometimes this is quite possible and sometimes it is not. For
example, if a user specifies that the type of operation is controlled by a
binding like `optype: {myoptype}` it is impossible to know what that type
will be until the cycle is being executed. Generally, you want to reject
this type of configuration as invalid, since this defeats the ability to
streamline the execution of ops with pre-configuration.
NoSQLBench configures an activity by calling your Op Function Mapping
function for each active instance of an op template as defined in the
YAML.
## Op Binding
Later, the runtime of NoSQLBench will use the result of your provided
function to render an executable operation, fully qualified with payload
details and any other dynamic features you allowed for in the op template
documentation for your driver. This is known as _binding an op template to
a cycle_, or simply _op binding_. There is nothing else for you do to
enable this. The function produced by your Op Template Mapping phase will
be used to power this within each cycle.
## Mapping and Binding Infographic
The infographic below provides an overview of the flow of information and
types throughout this process. The driver developer must provide an
implementation of an Op mapping function which is assignable to
a `Function<OpTemplate,LongFunction<T>>` as shown. The 4 steps in the _Op
Mapping Logic_ box describe a generalized approach that can be used for
each driver.
![Mapping And Binding](optemplates.png)

8
devdocs/java_versions.md → devdocs/devguide/java_versions.md
View File

@ -5,14 +5,14 @@ the NoSQLBench project. If you change versions or need to do
version-specific troubleshooting, this list will help.
- In the Dockerfile, in the parent image "FROM" tag.
[here](../Dockerfile)
[here](../../Dockerfile)
- In the Maven pom files, under `<source>...</source>`, `<release>...
</release>`, `<target>...</target>`
[here](../mvn-defaults/pom.xml)
[here](../../mvn-defaults/pom.xml)
- In some cases, a Maven variable '<java.target.version>...</java.target.
version>` is used.
[here](../mvn-defaults/pom.xml)
[here](../../mvn-defaults/pom.xml)
- In the nb appimage build scripts under nb/build-bin.sh.
[here](../nb/build-bin.sh)
[here](../../nb/build-bin.sh)
- In the github actions workflows for the Java runtime version

37
devdocs/devguide/nb_docs.md Normal file
View File

@ -0,0 +1,37 @@
# Editing NoSQLBench Docs
The docs that are hosted at docs.nosqlbench.io are built from markdown
files found in this project. To make changes or to author these docs, you
need to know where to find them.
## NoSQLBench docs
The core docs are found under the [engine-docs](../../engine-docs)
module
under [/src/main/resources/docs-for-nb/](../../engine-docs/src/main/resources/docs-for-nb)
.
By browsing this directory structure and looking at the frontmatter on
each markdown file, you'll get a sense for how they are orgainzed.
## Driver Docs
Some of the other docs are found within each driver module. For example,
the cql docs are found in the resources root directory of the cql driver
module. This is the case for any basic docs provided for a driver. The
docs are bundled with modules to allow for them to be maintained by the
driver maintainers directly.
## Binding Function Docs
All of the binding function docs are generated automatically from source.
Javadoc source as well as annotation details are used to decorate the
binding functions so that the can be cataloged and shared to the doc site.
To improve the binding function docs, you must improve the markdown
rendering code which is responsible for this.
## Suggestions
The doc system in NoSQLBench is a core element, but it is not really great
yet nor is it done. Any help on improving it is appreciated, in any form.

68
devdocs/devguide/nbio_api.md Normal file
View File

@ -0,0 +1,68 @@
# NBIO API
The NBIO class provides a way to access file and URL resources from within
a running NoSQLBench process. This centralizes the logic for accessing
files in classpath, filesystem, or on the net.
Managing file, classpath, and URL resources when you need access to all of
them can be complicated business. The NBIO API was introduced to
consolidate this into one easy-to-use API with substantial testing. It
also provides a uniform interface for accessing all of these resource
types so that users have a consistent experience regardless of where their
source data lives.
## Design Intent
Unless there is a good reason to avoid it, developers should delegate to
the NBUI API _anywhere_ in NoSQLBench that a user needs to load a file.
This imbues such call sites with the ability to read local and remote
resources with no further effort required by the programmer.
## API Usage
To use this API, simply call one of the following methods and use method
chaining to delve into what you are looking for:
- `NBIO.all()...` - Look in the filesystem, the classpath, and on the
net (for any patterns which are a URL).
- `NBIO.fs()...` - Look only in the filesystem.
- `NBIO.classpath()...` - Look only in the classpath.
- `NBIO.local()...` - Look in filesystem or classpath.
- `NBIO.remote()...` - Look only for remote resources.
## Searching Semantics
The fluent-style API will layout the following search parameters for you.
The details on these are explained in the javadoc which will be provided
in most cases as you expand the API. Each of these search parameters
supports zero or more values, and all values are searched that are
provided.
- `prefix(...)` - Zero or more prefix paths to search.
- `name(...)` - Zero or more resources or filenames or urls to look for.
- `extension(...)` - Zero or more extensions to try to match with.
## Getting Results
At runtime, the local process has visibility to many things. Due to how
classpath resources are assembled in pre-JDPA days, it is possible to find
multiple resources under the same name. Thus, you have to be specific
about how many you expect to find, and thus what is considered an error
condition:
- `list()` - All found content in one list
- `one()` - A single source of content should be found -- more or less is
an error.
- `first()` - At least one source of content should be found, and you only
want to see the first one.
- `resolveEach()` - List of lists, allows full cardinality visibility when
using the bulk interface with multiple name terms.
## Consuming Content
To support efficient resolution and consumption of local and remote
content, the type of element returned by the NBIO API is a _Content_
object. However, it provides convenient accessors for getting content when
needed. Remote content will be read lazily only once per NBIO result.
Local content is also read lazily, but it is not cached until GC.

0
devdocs/nbui/README.md → devdocs/devguide/nbui/README.md
View File

0
devdocs/nbui/dev_mode.png → devdocs/devguide/nbui/dev_mode.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 128 KiB

After

Width:  |  Height:  |  Size: 128 KiB

0
devdocs/nbui/dev_mode.puml → devdocs/devguide/nbui/dev_mode.puml
View File

0
devdocs/nbui/embedded_mode.png → devdocs/devguide/nbui/embedded_mode.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 116 KiB

After

Width:  |  Height:  |  Size: 116 KiB

0
devdocs/nbui/embedded_mode.puml → devdocs/devguide/nbui/embedded_mode.puml
View File

0
devdocs/nbui/nbui_design_guide.md → devdocs/devguide/nbui/nbui_design_guide.md
View File

87
devdocs/devguide/project_structure.md Normal file
View File

@ -0,0 +1,87 @@
# Project Structure
nosqlbench is packaged as a
[Maven Reactor](https://maven.apache.org/guides/mini/guide-multiple-modules.html)
project.
## Defaults and Dependencies
Maven reactor projects often confuse developers. In this document, we'll
explain the basic structure of the nosqlbench project and the reasons for
it.
Firstly, there is a parent for each of the modules. In Maven parlance, you
can think of a parent project as a defaults template for projects that
reference it. One of the reasons you would do this is to supply common
build or dependency settings across many maven projects or modules. That
is exactly why we do that here. The 'parent' project for all nosqlbench
modules is aptly named 'mvn-defaults', as that is exactly what we use it
for.
As well, there is a "root" project, which is simply the project at the
project's base directory. It pulls in the modules of the project
explicitly as in:
~~~
<modules>
<module>mvn-defaults</module>
<module>nb</module>
<module>nb-api</module>
<module>nb-annotations</module>
...
~~~
This means that when you build the root project, it will build all the
modules included, but only after linearizing the build order around the
inter-module dependencies. This is an important detail, as it is often
overlooked that this is the purpose of a reactor-style project.
The dependencies between the modules is not implicit. Each module listed
in the root pom.xml has its own explicit dependencies to other modules in
the project. We could cause them to have a common set of dependencies by
adding those dependencies to the 'mvn-defaults' module, but this would
mostly prevent us from making the dependencies for each as lean and
specific as we like. That is why the dependencies in the mvn-defaults **
module** module are very limited. Only those modules which are to be taken
for granted as dependencies everywhere in the project should be added to
the mvn-defaults module.
The mvn-defaults module contains build, locale, and project identity
settings. You can consider these cross-cutting aspects of all of the other
modules in the project. If you want to put something in the mvn-defaults
module, and it is not strictly cross-cutting across the other modules,
then don't. That's how you keep maven reactor projects functioning and
maintainable.
To be clear, cross-cutting build behavior and per-module dependencies are
two separate axes of build management. Try to keep this in mind when
thinking about modular projects.
## Inter-module Dependencies
Modularity at runtime is enabled via the
[Java ServiceLoader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html)
.
The engine-core module uses the engine-api module to know the loadable
activity types. Driver implementations use the engine-api module to
implement the loadable activity types. In this way, they both depend on
the engine-api module to provide the common types needed for this to work.
In this way, multiple modules depending on a single API allows them to
speak together using the language of that API. This also means that the
API must be external to each of the participating modules which use it as
a communication layer.
The nb module allows the separate implementations of the core and the
activity type implementations to exist together in the same classpath.
This occurs specifically because the nb module is an _aggregator_
module which depends on multiple other modules. When the artifact for the
nb module is built, it has all these dependencies together. Since the nb
module is built in a way to include all dependencies, a jar is built that
contains them all in one single file.
## Module Naming
A consistent naming scheme is prescribed for all modules within
nosqlbench, as described in [MODULES](../../MODULES.md)

27
devdocs/devguide/scripting_extensions.md Normal file
View File

@ -0,0 +1,27 @@
# Scripting Extensions
When a scenario runs, it executes a scenario control script. This script *
is* the scenario in programmatic terms. Even when users don't see the
scenario script as such, it is still there as an intermediary between the
user commands and the core runtime of NoSQLBench. As an executive control
layer, the scenario script doesn't directly run operations with drivers,
although it is able to observe and modify activity metrics and parameters
in real time.
## Scripting Environment
The scripting environment is a _javascript_ environment powered by
GraalJS. Initially, a set of service objects is published into this
environment that allows for the scenario, activity parameters, and
activity metrics to all be accessed directly.
## Scripting Extensions
Additional services can be published into the scripting environment when
it is initialized. These services are simply named objects. They are found
using SPI Java mechanism as long as they are published as services with
a `@Service(ScriptingPluginInfo.class)` annotation and
implement `ScriptingPluginInfo<T>`. This API is pretty basic. You can look
at the `ExamplePluginData` class for a clear example for how to build a
scripting extension.

4
devdocs/dynamic_endpoints.md
View File

@ -1,4 +0,0 @@
# Dynamic Endpoints
[JAX-RS 2.0 spec](http://download.oracle.com/otn-pub/jcp/jaxrs-2_0-fr-eval-spec/jsr339-jaxrs-2.0-final-spec.pdf)

33
engine-docs/src/main/resources/docs-for-ebhistoric/dev-guide/scripting_extensions.md
View File

@ -1,33 +0,0 @@
---
title: Scripting Extensions
weight: 32
menu:
main:
parent: Dev Guide
identifier: Scripting Extensions
weight: 12
---
## Requirements
- Java 8
- Maven dependency:
## Scripting Extensions
When a new scripting environment is initialized in nosqlbench, a new instance of each scripting extension is published into it as a variable. This variable acts as a named service endpoint within the scripting environment. For example, an extension for saving a JSON map to disk could be published into the scripting environment as "savejson", and you might invoke it as "savejson.save('somefile.json',myjson);".
## Loading Scripting Extensions
In order to share these with the nosqlbench runtime, the ServerLoader API is used. The scripting environment will load every plugin implementing the SandboxPluginData interface, as long as it has the proper data in META-INF/services/ in the classpath. There are examples of how to do this via Maven in the source repo under the engine-extensions module.
## Maven Dependencies
~~~
<dependency>
<groupId>io.nosqlbench</groupId>
<artifactId>engine-api</artifactId>
<version>1.0.17</version>
<type>pom</type>
</dependency>
~~~