IntenseFinance/nosqlbench
3
0
Fork 0
mirror of https://github.com/nosqlbench/nosqlbench.git synced 2025-02-25 18:55:28 -06:00
Code Issues Packages Projects Releases Wiki Activity

spectest fixes and additional documentation

Browse Source
This commit is contained in:
Jonathan Shook
2022-06-27 23:44:50 -05:00
parent 007d90218d
commit 929b4fe929
4 changed files with 440 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

384
adapters-api/src/main/resources/workload_definition/op_template_payloads.md Normal file
View File

@@ -0,0 +1,384 @@
# Op Template Payloads
Payloads in NoSQLBench op templates can be of any type that you can create from bindings, string
templates, data structure, or any combination thereof. Yet, the recipe format for these bindings is
simple and direct. If you know what you want the op to do, then one of the supported forms here
should get you there.
The payload of an operation is handled as any content which is assigned to the `op` field in the
uniform op structure.
## assigned ops as string
When you are using the command, it is possible to specify `op=...` instead of providing a workload
path. When you do that, a workload description is automatically derived for you on-the-fly.
`nb run driver=stdout op='cycle number {{NumberNameToString}'`
The equivalent structure has the same effect as if you had created a workload description like this:
*yaml:*
```yaml
ops: "cycle number '{{NumberNameToString}}'"
```
*json:*
```json5
{
"ops": "cycle number '{{NumberNameToString}}'"
}
```
*ops:*
```json5
[
{
"tags": {
"name": "block0--stmt1",
"block": "block0"
},
"op": {
"stmt": "cycle number '{{NumberNameToString}}'"
},
"name": "block0--stmt1"
}
]
```
## assigned ops as a list of strings
If the value type of the top-level ops field is in list form, then each value is processed
individually.
*yaml:*
```yaml
ops:
- "even cycle '{{NumberNameToString}}'"
- "odd cycle '{{NumberNameToString}}'"
```
*json:*
```json5
{
"ops": [
"even cycle '{{NumberNameToString}}'",
"odd cycle '{{NumberNameToString}}'"
]
}
```
*ops:*
```json5
[
{
"tags": {
"name": "block0--stmt1",
"block": "block0"
},
"op": {
"stmt": "even cycle '{{NumberNameToString}}'"
},
"name": "block0--stmt1"
},
{
"tags": {
"name": "block0--stmt2",
"block": "block0"
},
"op": {
"stmt": "odd cycle '{{NumberNameToString}}'"
},
"name": "block0--stmt2"
}
]
```
## assigned ops as a map of strings
A more preferable way to add ops to a structure is in map form. This is also covered elsewhere, but
it is important for examples further below so we'll refresh it here:
*yaml:*
```yaml
ops:
myop1: "even cycle '{{NumberNameToString}}'"
myop2: "odd cycle '{{NumberNameToString}}'"
```
*json:*
```json5
{
"ops": {
"myop1": "even cycle '{{NumberNameToString}}'",
"myop2": "odd cycle '{{NumberNameToString}}'"
}
}
```
*ops:*
```json5
[
{
"tags": {
"name": "block0--myop1",
"block": "block0"
},
"op": {
"stmt": "even cycle '{{NumberNameToString}}'"
},
"name": "block0--myop1"
},
{
"tags": {
"name": "block0--myop2",
"block": "block0"
},
"op": {
"stmt": "odd cycle '{{NumberNameToString}}'"
},
"name": "block0--myop2"
}
]
```
## assigned op payload as a simple map of op fields
When your operation takes on a non-statement form, you simply provide a map structure at the
top-level of the op:
*yaml:*
```yaml
ops:
op1:
opfield1: opvalue1
opfield2: opvalue2
```
*json:*
```json5
{
"ops": {
"op1": {
"opfield1": "opvalue1",
"opfield2": "opvalue2"
}
}
}
```
*ops:*
```json5
[
{
"tags": {
"name": "block0--op1",
"block": "block0"
},
"op": {
"opfield1": "opvalue1",
"opfield2": "opvalue2"
},
"name": "block0--op1"
}
]
```
## assigned op payload as an array of values
When your operation takes on a non-statement form, you simply provide a map structure at the
top-level of the op. Notice that the structurally normalized form shows the field values moved
underneath the canonical 'op' field by default. This is because the structurally normalized
op template form *always* has a map in the op field. The structural short-hand for creating an
op template that is list based simply moves any list entries at the op template level down in
the named 'op' field for you.
*yaml:*
```yaml
ops:
op1:
- opvalue1
- opvalue2
```
*json:*
```json5
{
"ops": {
"op1": [
"opvalue1",
"opvalue2"
]
}
}
```
*ops:*
```json5
[
{
"tags": {
"name": "block0--op1",
"block": "block0"
},
"op": {
"stmt": [
"opvalue1",
"opvalue2"
]
},
"name": "block0--op1"
}
]
```
## op payload with structured value types
From the examples above, it's clear that you can use string and maps in your op fields. Here, we
show how you can use arbitrary levels of structured types based on commodity collection types like
List and Map for Java and JSON objects and arrays.
*yaml:*
```yaml
ops:
op1:
index_map:
username: user_index
geocode: georef_v23
rollups:
- by_username/@4h
- by_session_len/@1h
```
*json:*
```json5
{
"ops": {
"op1": {
"index_map": {
"username": "user_index",
"geocode": "georef_v23"
},
"rollups": [
"by_username/@4h",
"by_session_len/@1h"
]
}
}
}
```
*ops:*
```json5
[
{
"tags": {
"name": "block0--op1",
"block": "block0"
},
"op": {
"index_map": {
"username": "user_index",
"geocode": "georef_v23"
},
"rollups": [
"by_username/@4h",
"by_session_len/@1h"
]
},
"name": "block0--op1"
}
]
```
## Binding Points at any position in op field data
You can use either named binding points references like `{userid}` or binding definitions such
as `{{Template('user-{}',ToString())}}`. The only exception to this rule is that you may not
(yet) use dynamic values for keys within your structure.
*yaml:*
```yaml
bindings:
user_index: Mod(1000L); ToString();
ops:
op1:
index_map:
username: "{user_index}"
geocode: "{{Template('georef_v{}',HashRange(0,23))}}"
rollups:
- by_username/@4h
- by_session_len/@1h
```
*json:*
```json5
{
"bindings": {
"user_index": "Mod(1000L); ToString();"
},
"ops": {
"op1": {
"index_map": {
"username": "{user_index}",
"geocode": "{{Template('georef_v{}',HashRange(0,23))}}"
},
"rollups": [
"by_username/@4h",
"by_session_len/@1h"
]
}
}
}
```
*ops:*
```json5
[
{
"bindings": {
"user_index": "Mod(1000L); ToString();"
},
"tags": {
"name": "block0--op1",
"block": "block0"
},
"op": {
"index_map": {
"username": "{user_index}",
"geocode": "{{Template('georef_v{}',HashRange(0,23))}}"
},
"rollups": [
"by_username/@4h",
"by_session_len/@1h"
]
},
"name": "block0--op1"
}
]
```

12
adapters-api/src/main/resources/workload_definition/spectest_formatting.md
View File

@@ -18,23 +18,23 @@ body of example
```
Further, to match the pattern above, these must occur in sequences like the following, with no other
intervening content:
intervening content. If the second fenced code section is a JSON array, then each object within
it is compared pair-wise with the yaml structure as in a multi-doc scenario. The following
example is actually tested along with the non-empty templates. It is valid because the second
block is in array form, and thus compares 0 pair-wise elements.
*yaml:*
```yaml
# some yaml here
```
*json:*
```
```json5
[]
```
*ops:*
```
```json5
[]
```

24
adapters-api/src/test/java/io/nosqlbench/engine/api/activityconfig/rawyaml/UniformWorkloadSpecificationTest.java
View File

@@ -22,20 +22,38 @@ import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.junit.jupiter.api.Test;
import java.util.regex.Pattern;
import static io.nosqlbench.nb.spectest.traversal.STPredicateVerbs.*;
public class UniformWorkloadSpecificationTest {
private final static Logger logger = LogManager.getLogger(UniformWorkloadSpecificationTest.class);
private final static Object[] mdPredicate = new Object[] {
depth(deepany("yaml:"), ".*"), depth(FencedCodeBlock.class, ".*"),
depth(deepany("json:"), ".*"), ref(1),
depth(deepany("ops:"), ".*"), ref(1)
};
@Test
public void testTemplatedWorkloads() {
SpecTest specTester = SpecTest.builder()
.path("target/classes/workload_definition/")
.matchNodes(
Pattern.compile("\\*(.*?)\\*\n?"), FencedCodeBlock.class, 0, 1, 0 ,1)
.matchNodes(mdPredicate)
.validators(new YamlSpecValidator())
.build();
specTester.run();
}
// @Test
// public void testWithStructuredPredicate() {
// SpecTest specTester = SpecTest.builder()
// .path("target/classes/workload_definition/op_template_payloads.md")
// .matchNodes(mdPredicate)
// .validators(new YamlSpecValidator())
//// .debug()
// .build();
// specTester.run();
// }
//
}

58
adapters-api/src/test/java/io/nosqlbench/engine/api/activityconfig/rawyaml/YamlSpecValidator.java
View File

@@ -16,19 +16,16 @@
package io.nosqlbench.engine.api.activityconfig.rawyaml;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonElement;
import com.google.gson.JsonParser;
import com.google.gson.*;
import com.google.gson.reflect.TypeToken;
import io.nosqlbench.engine.api.activityconfig.StatementsLoader;
import io.nosqlbench.engine.api.activityconfig.yaml.OpTemplate;
import io.nosqlbench.engine.api.activityconfig.yaml.StmtsDocList;
import io.nosqlbench.nb.spectest.api.STAssemblyValidator;
import io.nosqlbench.nb.spectest.core.STNodeAssembly;
import io.nosqlbench.nb.spectest.loaders.STDefaultLoader;
import io.nosqlbench.nb.spectest.testtypes.STNamedCodeTuples;
import io.nosqlbench.nb.spectest.testtypes.STNodeReference;
import io.nosqlbench.nb.spectest.types.STAssemblyValidator;
import io.nosqlbench.nb.spectest.testmodels.STNamedCodeTuples;
import io.nosqlbench.nb.spectest.testmodels.STNodeReference;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.assertj.core.api.Assertions;
@@ -73,14 +70,9 @@ public class YamlSpecValidator implements STAssemblyValidator {
if (testblock.size() == 6) {
STNamedCodeTuples tuples = testblock.getAsNameAndCodeTuples();
String name = tuples.getTypeSignature();
System.out.println(testblock.get(0).getDesc());
System.out.println(testblock.get(0).getLocationRef());
if (tuples.getTypeSignature().equals("yaml->json->ops")) {
testBracket_YamlJsonOps(tuples);
}
testBracket_YamlJsonOps(tuples);
} else {
throw new RuntimeException("Test block sized " + testblock.size() + " unrecognized by test loader.");
}
@@ -92,13 +84,13 @@ public class YamlSpecValidator implements STAssemblyValidator {
validateYamlWithJson(
tuples.get(0).getName(),
tuples.get(0).getData(),
tuples.get(1).getName(),
tuples.get(1)
);
tuples.get(1).getData(),
tuples.get(1),
false);
validateYamlWithOpsModel(
tuples.get(1).getName(),
tuples.get(1).getData(),
tuples.get(0).getName(),
tuples.get(0).getData(),
tuples.get(2).getData(),
tuples.get(2)
);
@@ -118,17 +110,18 @@ public class YamlSpecValidator implements STAssemblyValidator {
List<OpTemplate> stmts = stmtsDocs.getStmts();
List<Map<String, Object>> stmt_objs = stmts.stream().map(OpTemplate::asData).collect(Collectors.toList());
Assertions.assertThat(stmt_objs).isEqualTo(expectedList);
try {
Assertions.assertThat(stmt_objs).isEqualTo(expectedList);
} catch (Throwable t) {
System.out.println("JSON version:\n"+gson.toJson(stmt_objs)+"\n");
throw t;
}
} else {
throw new RuntimeException("The structurally normalized op template must be a list.");
}
System.out.println("OK");
} catch (Exception e) {
// System.out.println("Error while validating equivalence between the yaml and the rendered op context:");
// System.out.println("yaml:");
// System.out.println(yaml);
// System.out.println("ops:");
// System.out.println(json);
throw new RuntimeException(e);
}
@@ -140,19 +133,24 @@ public class YamlSpecValidator implements STAssemblyValidator {
* Compare one or more raw yaml docs to JSON5 representation of the same.
* For clarity in the docs, a single object is allowed in the json5, in which case
* an error is thrown if the yaml side contains more or less than 1 element.
*
* @param desc A moniker describing the test
* @param desc A moniker describing the test
* @param yaml YAML describing a templated workload
* @param json JSON describing a templated workload
* @param debug
*/
private void validateYamlWithJson(String desc, String yaml, String json, STNodeReference testset) {
private void validateYamlWithJson(String desc, String yaml, String json, STNodeReference testset, boolean debug) {
System.out.format("%-40s", "- checking yaml->json");
// StmtsDocList stmts = StatementsLoader.loadString(yaml);
try {
List<Map<String, Object>> docmaps = new RawYamlLoader().loadString(logger, yaml);
JsonElement elem = JsonParser.parseString(json);
JsonElement elem = null;
try {
elem = JsonParser.parseString(json);
} catch (JsonSyntaxException jse) {
throw new RuntimeException("Not recongized as JSON:\n" + json);
}
if (elem.isJsonArray()) {
Type type = new TypeToken<List<Map<String, Object>>>() {
}.getType();
@@ -168,6 +166,8 @@ public class YamlSpecValidator implements STAssemblyValidator {
throw new RuntimeException("comparator expected a single object, but found " + docmaps.size());
}
System.out.println("OK");
} else if (elem.isJsonPrimitive()) {
throw new RuntimeException("Invalid type: JsonPrimitive types are not compatible with this validator. Objects and Arrays are.");
} else {
System.out.println("ERROR");
throw new RuntimeException("unknown type in comparator: " + json);