diff --git a/nb-engine/nb-engine-core/src/main/java/io/nosqlbench/engine/api/activityimpl/uniform/actions/scaffold.svg b/nb-engine/nb-engine-core/src/main/java/io/nosqlbench/engine/api/activityimpl/uniform/actions/scaffold.svg new file mode 100644 index 000000000..d544827f6 --- /dev/null +++ b/nb-engine/nb-engine-core/src/main/java/io/nosqlbench/engine/api/activityimpl/uniform/actions/scaffold.svg @@ -0,0 +1,395 @@ + + + + + + + + +activity commandloader(workload)rawworkloadtemplateEach workload template is a datastructure which can be provided as yaml, json, etc. The raw text of the workload template is first updated with template variables for any TEMPLATE(...) patterns before being deserialized into a native data structure in memory.Op TemplateNormalization(raw template)The raw op template structure is de-sugared and layered according to the rules of _Uniform Workload Specification_, combining document, block, and op-level details together to provide a normalized view at the op level.(normal)op templateA normalized op template is self-standing, with all document and block level default and settings applied. This allows activities to view op templates as a simple list of op recipes to be executedadapterselectionThe adapter selected to interpret the op template can be specified in the op template directly as 'driver', or it can come from the 'driver' parameter of the activity params.AdapterResolver(parent,name,config)->adapterThe adapter is loaded according to the specified adapter name (whether from the op template directly, or from the default provided in the activity parameters)ParsedOpResolver(parent, adapter, optemplate)The parsed op resolver creates a parsed op out of the op template and any subset of the adapter parameters which are relevant. The op fields take priority, then any block elements, then any document-level elements, and then if needed, specific op fields may be back-filled from the adapter parameters, but only those indicated by the related adapter's config model.OpMapper(adapter, parsedOp)The Op Mapper selects the type of dispenser to create based on the fit of the op template to the documented examples. The op mapper is not responsible for dispensing executable operations. It is merely a type routing mechanism. It is meant to determine what type of operation to stage, but then the rest of the work is handed off to the op dispenser layer. The logic which builds an operation may be constructed by either the op mapper or the op dispenser, so long as the work of building the operations is actually executed by the dispenser.OpDispenser(cycle)Op dispensers are the workhorse layer of op synthesis. They hold lambdas which map cycles to operations. These lambdas can be created by op mappers and then passed into op dispensers, or the op dispensers can build the lambdas on their own. In any case, the lambdas should be constructed before or in the dispenser constructors.execute(op)ParsedOpThe ParsedOp is a representation of an op template, which provides a toolkit of methods for up-converting the template into functions over cycle values. In other words, it helps developers build the exact lambdas which are needed to fill in the last-minute details of a specific executable operation. This can be considered a precedural generation construction kit, driven by the recipes provided by users in their op templates. This is handled in two stages, see below.spaceFThe space function is kept as a separate mapping layer because it allows op dispensers to dynamically select a state space for an adapter. Thus, it needs to be provided directly to the op mapper so that it may be used to create op dispensers. The space function is a no-op by default, meaning only one space is ever created for an adapter. However, if the user configures a 'space' op field, then the value of this field determines the distribution of spaces over cycles for this op template.opcycleResultactivity paramsAn activity is started with a set of named parameters and arguments. These parameters may apply to multiple matching parameters by name: 1. The template variables in any workload templates 2. The standard activity parameters, like threads, cycles, ... 3. Any specific overrides allowed for related op templatesrawoptemplateThe raw op template is a minimal "sugared" format which allows users to specify op fields and structure in a minimal form...[other templatesprocessed inthe same way]raw optemplateparentWithin the NB component tree, the parent for an adapter is the activity which loaded it. This parent is provided to the adapter resolver to be injected into the adapter when it is loaded.adapternameEach adapter is identified by a canonical name, like 'http' or 'cql'activityconfigThe adapter is initialized with a subset of matching activity parameters which are valid for it.adaptercacheadapter instances are cached by adapter name, and only one instance of any given adapter name is realized for a given activity.optemplateparentadapterSpaceFparsedOpcycleWhen the op dispenser is called with a given cycle value, it produces an executable operation. These operations are deterministic by cycle, meaning that they can be reproduced or replayed for a given set of cycles. Calling op dispensers often takes microseconds, given the efficiency of Java lambdas.opworkload[other params]driverselectedadapter[otheradapters]spacecacheA cache of stateful spaces is maintained per adapter. This is the designated area to keep per-instance state for a given native driver, such as the native driver instance itself and any related object wiring needed for it to function as a normal application would. Think of the space as a mini-application, but just the parts that deal with the native driver API.selectedspace[otherspaces] workloaddeserialization workloadsourceselection templatevariableinterpolation Each workload template is a datastructure which can be provided as yaml, json, etc. The raw text of the workload template is first updated with template variables for any TEMPLATE(...) patterns before being deserialized into a native data structure in memory. + + + + + + + + + + + + +The raw op template structure is de-sugared and layered according to the rules of _Uniform Workload Specification_, combining document, block, and op-level details together to provide a normalized view at the op level. + + + + + + + + + + + + +A normalized op template is self-standing, with all document and block level default and settings applied. This allows activities to view op templates as a simple list of op recipes to be executed + + + + + + + + + + + + +The adapter selected to interpret the op template can be specified in the op template directly as 'driver', or it can come from the 'driver' parameter of the activity params. + + + + + + + + + + + + +The adapter is loaded according to the specified adapter name (whether from the op template directly, or from the default provided in the activity parameters) + + + + + + + + + + + + +The parsed op resolver creates a parsed op out of the op template and any subset of the adapter parameters which are relevant. The op fields take priority, then any block elements, then any document-level elements, and then if needed, specific op fields may be back-filled from the adapter parameters, but only those indicated by the related adapter's config model. + + + + + + + + + + + + +The Op Mapper selects the type of dispenser to create based on the fit of the op template to the documented examples. The op mapper is not responsible for dispensing executable operations. It is merely a type routing mechanism. It is meant to determine what type of operation to stage, but then the rest of the work is handed off to the op dispenser layer. The logic which builds an operation may be constructed by either the op mapper or the op dispenser, so long as the work of building the operations is actually executed by the dispenser. + + + + + + + + + + + + +Op dispensers are the workhorse layer of op synthesis. They hold lambdas which map cycles to operations. These lambdas can be created by op mappers and then passed into op dispensers, or the op dispensers can build the lambdas on their own. In any case, the lambdas should be constructed before or in the dispenser constructors. + + + + + + + + + + + + +The ParsedOp is a representation of an op template, which provides a toolkit of methods for up-converting the template into functions over cycle values. In other words, it helps developers build the exact lambdas which are needed to fill in the last-minute details of a specific executable operation. This can be considered a precedural generation construction kit, driven by the recipes provided by users in their op templates. This is handled in two stages, see below. + + + + + + + + + + + + +The space function is kept as a separate mapping layer because it allows op dispensers to dynamically select a state space for an adapter. Thus, it needs to be provided directly to the op mapper so that it may be used to create op dispensers. The space function is a no-op by default, meaning only one space is ever created for an adapter. However, if the user configures a 'space' op field, then the value of this field determines the distribution of spaces over cycles for this op template. + + + + + + + + + + + + +An activity is started with a set of named parameters and arguments. These parameters may apply to multiple matching parameters by name: 1. The template variables in any workload templates 2. The standard activity parameters, like threads, cycles, ... 3. Any specific overrides allowed for related op templates + + + + + + + + + + + + +The raw op template is a minimal "sugared" format which allows users to specify op fields and structure in a minimal form + + + + + + + + + + + + +Within the NB component tree, the parent for an adapter is the activity which loaded it. This parent is provided to the adapter resolver to be injected into the adapter when it is loaded. + + + + + + + + + + + + +Each adapter is identified by a canonical name, like 'http' or 'cql' + + + + + + + + + + + + +The adapter is initialized with a subset of matching activity parameters which are valid for it. + + + + + + + + + + + + +adapter instances are cached by adapter name, and only one instance of any given adapter name is realized for a given activity. + + + + + + + + + + + + +When the op dispenser is called with a given cycle value, it produces an executable operation. These operations are deterministic by cycle, meaning that they can be reproduced or replayed for a given set of cycles. Calling op dispensers often takes microseconds, given the efficiency of Java lambdas. + + + + + + + + + + + + +A cache of stateful spaces is maintained per adapter. This is the designated area to keep per-instance state for a given native driver, such as the native driver instance itself and any related object wiring needed for it to function as a normal application would. Think of the space as a mini-application, but just the parts that deal with the native driver API. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +