IntenseFinance/nosqlbench
3
0
Fork 0
mirror of https://github.com/nosqlbench/nosqlbench.git synced 2024-12-29 10:11:08 -06:00
Code Issues Packages Projects Releases Wiki Activity

typo fixes

Browse Source
This commit is contained in:
Jonathan Shook 2021-02-04 17:47:03 -06:00
parent 4c726b0333
commit 4d87e1c9a7
4 changed files with 105 additions and 87 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

186
driver-http/src/main/resources/http.md
View File

@ -1,12 +1,14 @@
# HTTP driver # HTTP driver
This driver allows you to make http requests using the native HTTP client that is bundled with the This driver allows you to make http requests using the native HTTP client
JVM. It supports free-form construction of requests. that is bundled with the JVM. It supports free-form construction of
requests.
You specify what a request looks like by providing a set of request parameters. They can be in You specify what a request looks like by providing a set of request
either literal (static) form with no dynamic data binding, or they can each be in a string template parameters. They can be in either literal (static) form with no dynamic
form that draws from data bindings. Each cycle, a request is assembled from these parameters and data binding, or they can each be in a string template form that draws
executed. from data bindings. Each cycle, a request is assembled from these
parameters and executed.
## Example Statements ## Example Statements
@ -21,8 +23,8 @@ Or, you can have a list:
```yaml ```yaml
# A list of statements # A list of statements
statements: statements:
- http://google.com/ - http://google.com/
- http://amazon.com/ - http://amazon.com/
``` ```
Or you can template the values used in the URI, and even add ratios: Or you can template the values used in the URI, and even add ratios:
@ -30,92 +32,99 @@ Or you can template the values used in the URI, and even add ratios:
```yaml ```yaml
# A list of named statements with variable fields and specific ratios: # A list of named statements with variable fields and specific ratios:
statements: statements:
- s1: http://google.com/search?query={query} - s1: http://google.com/search?query={query}
ratio: 3 ratio: 3
- s2: https://www.amazon.com/s?k={query} - s2: https://www.amazon.com/s?k={query}
ratio: 2 ratio: 2
bindings: bindings:
query: > query: >
WeightedStrings('function generator;backup generator;static generator'); WeightedStrings('function generator;backup generator;static generator');
UrlEncode(); UrlEncode();
``` ```
You can even make a detailed request with custom headers and result verification conditions: You can even make a detailed request with custom headers and result
verification conditions:
```yaml ```yaml
# Require that the result be status code 200-299 match regex "OK, account id is .*" in the body # Require that the result be status code 200-299 match regex "OK, account id is .*" in the body
statements: statements:
- get-from-google: - get-from-google:
method: GET method: GET
uri: "https://google.com/" uri: "https://google.com/"
version: "HTTP/1.1" version: "HTTP/1.1"
Content-Type: "application/json" Content-Type: "application/json"
ok-status: "2[0-9][0-9]" ok-status: "2[0-9][0-9]"
ok-body: "^(OK, account id is .*)$" ok-body: "^(OK, account id is .*)$"
``` ```
For those familiar with what an HTTP request looks like on the wire, the format below may be For those familiar with what an HTTP request looks like on the wire, the
familiar. This isn't actually the content that is submitted, but it is recognized as a valid way to format below may be familiar. This isn't actually the content that is
express the request parameters in a familiar and condensed form. A custom config parser makes this submitted, but it is recognized as a valid way to express the request
form available fo rhose who want to emulate a well-known pattern: parameters in a familiar and condensed form. A custom config parser makes
this form available fo rhose who want to emulate a well-known pattern:
```yaml ```yaml
statements: statements:
- s1: | - s1: |
GET https://google.com/ HTTP/1.1 GET https://google.com/ HTTP/1.1
Content-Type: application/json Content-Type: application/json
ok-status: 2[0-9][0-9] ok-status: 2[0-9][0-9]
ok-body: ^(OK, account id is.*)$ ok-body: ^(OK, account id is.*)$
``` ```
Of course, in the above form, the response validators are still separate parameters. Of course, in the above form, the response validators are still separate
parameters.
## Bindings ## Bindings
All request fields can be made dynamic with binding functions. To make a request that has all All request fields can be made dynamic with binding functions. To make a
dynamic fields, you can do something like this: request that has all dynamic fields, you can do something like this:
```yaml ```yaml
statements: statements:
- s1: | - s1: |
{method} {scheme}://{host}:{port}/{path}?{query} {version} {method} {scheme}://{host}:{port}/{path}?{query} {version}
Content-Type: {content_type} Content-Type: {content_type}
Token: {mybearertoken} Token: {mybearertoken}
{body} {body}
``` ```
The above example is in the inline request form. It is parsed and interpreted internally as if you The above example is in the inline request form. It is parsed and
had configured your op template like this: interpreted internally as if you had configured your op template like
this:
```yaml ```yaml
statements: statements:
- method: {method} - method: { method }
uri: {scheme}://{host}:{port}/{path}?{query} uri: { scheme }://{host}:{port}/{path}?{query}
version: {version} version: { version }
"Content-Type": {content_type} "Content-Type": { content_type }
"Token": {mybearertoken} "Token": { mybearertoken }
body: {body} body: { body }
``` ```
The above two examples are semantically identical, only the format is different. Notice that the The above two examples are semantically identical, only the format is
expansion of the URI is still captured in a field called uri, with all of the dynamic pieces different. Notice that the expansion of the URI is still captured in a
stitched together in the value. You can't use arbitrary request fields. Every request field must field called uri, with all of the dynamic pieces stitched together in the
from (method, uri, version, body, ok-status, ok-body) or otherwise be capitalized to signify an HTTP value. You can't use arbitrary request fields. Every request field must
header. from (method, uri, version, body, ok-status, ok-body) or otherwise be
capitalized to signify an HTTP header.
The HTTP RFCs do not require headers to be capitalized, but they are capitalized ubiquitously in The HTTP RFCs do not require headers to be capitalized, but they are
practice, so we follow that convention here for clarity. Headers are in-fact case-insensitive, so capitalized ubiquitously in practice, so we follow that convention here
any issues created by this indicate a non-conformant server/application implementation. for clarity. Headers are in-fact case-insensitive, so any issues created
by this indicate a non-conformant server/application implementation.
For URIs which are fully static (There are no dynamic fields, request generation will be much
faster, since the request is fully built and cached at startup.
For URIs which are fully static (There are no dynamic fields, request
generation will be much faster, since the request is fully built and
cached at startup.
## Request Fields ## Request Fields
At a minimum, a **URI** must be provided. These are enough to build a request with. At a minimum, a **URI** must be provided. These are enough to build a
All other request fields are optional and have reasonable defaults: request with. All other request fields are optional and have reasonable
defaults:
- **uri** - This is the URI that you might put into the URL bar of your - **uri** - This is the URI that you might put into the URL bar of your
browser. There is no default. browser. There is no default.
@ -133,17 +142,22 @@ All other request fields are optional and have reasonable defaults:
ensure that the values that are inserted at binding points are produced ensure that the values that are inserted at binding points are produced
in a valid form for a URI. You can use the `URLEncode()` in a valid form for a URI. You can use the `URLEncode()`
binding function where needed to achieve this. binding function where needed to achieve this.
- **method** - An optional request method. If not provided, "GET" is assumed. Any method name will - **method** - An optional request method. If not provided, "GET" is
work here, even custom ones that are specific to a given target system. No validation is done for assumed. Any method name will work here, even custom ones that are
standard method names, as there is no way to know what method names may be valid. specific to a given target system. No validation is done for standard
- **version** - The HTTP version to use. If this value is not provided, the default version for the method names, as there is no way to know what method names may be valid.
Java HttpClient is used. If it is provided, it must be one of 'HTTP/1.1' or 'HTTP/2.0'. - **version** - The HTTP version to use. If this value is not provided,
- **body** - The content of the request body, for methods which support it. the default version for the Java HttpClient is used. If it is provided,
- **ok-status** - An optional set of rules to verify that a response is valid. This is a it must be one of 'HTTP/1.1' or 'HTTP/2.0'.
simple comma or space separated list of integer status codes or a pattern which is used as a regex - **body** - The content of the request body, for methods which support
against the string form of a status code. If any characters other than digits spaces and commas it.
are found in this value, then it is taken as a regex. If this is not provided, then any status - **ok-status** - An optional set of rules to verify that a response is
code which is >=200 and <300 is considered valid. valid. This is a simple comma or space separated list of integer status
codes or a pattern which is used as a regex against the string form of a
status code. If any characters other than digits spaces and commas are
found in this value, then it is taken as a regex. If this is not
provided, then any status code which is >=200 and <300 is considered
valid.
- **ok-body** - An optional regex pattern which will be applied to the - **ok-body** - An optional regex pattern which will be applied to the
body to verify that it is a valid response. If this is not provided, body to verify that it is a valid response. If this is not provided,
then content bodies are read, but any content is considered valid. then content bodies are read, but any content is considered valid.
@ -164,9 +178,8 @@ you have a specific exception type that indicates a retryable operation.
The HTTP driver is the first NB driver to include a completely The HTTP driver is the first NB driver to include a completely
configurable error handler chain. This is explained in the configurable error handler chain. This is explained in the
`error-handling` topic. By default, the HTTP activity's error handler is `error-handlers` topic. By default, the HTTP activity's error handler is
wired to stop the activity for any error encountered. For more details see wired to stop the activity for any error encountered.
the `error-handling` topic.
## SSL Support ## SSL Support
@ -177,24 +190,27 @@ configuration. If needed, more configurable SSL support will be added.
### TCP Sessions ### TCP Sessions
The HTTP clients are allocated one to each thread. The TCP connection caching is entirely left to The HTTP clients are allocated one to each thread. The TCP connection
the defaults for the current HttpClient library that is bundled within the JVM. caching is entirely left to the defaults for the current HttpClient
library that is bundled within the JVM.
### Chunked encoding and web sockets ### Chunked encoding and web sockets
Presently, this driver only does basic request-response style requests. Thus, adding headers which Presently, this driver only does basic request-response style requests.
take TCP socket control away from the HttpClient will likely yield inconsistent (or undefined) Thus, adding headers which take TCP socket control away from the
results. Support may be added for long-lived connections in a future release. HttpClient will likely yield inconsistent (or undefined)
results. Support may be added for long-lived connections in a future
release.
## HTTP Activity Parameters ## HTTP Activity Parameters
- **client_scope** - default: activity - One of activity, or thread. This controls how many - **client_scope** - default: activity - One of activity, or thread. This
clients instances you use with an HTTP activity. By default, all threads will use the same controls how many clients instances you use with an HTTP activity. By
client instance. default, all threads will use the same client instance.
- **follow_redirects** - default: normal - One of never, always, or - **follow_redirects** - default: normal - One of never, always, or
normal. Normal redirects normal. Normal redirects are those which do not redirect from HTTPS to
are those which do not redirect from HTTPS to HTTP. HTTP.
- **diagnostics** - default: none - synonym: **diag** - **diagnostics** - default: none - synonym: **diag**
example: `diag=brief,1000` - print diagnostics for every 1000th cycle, example: `diag=brief,1000` - print diagnostics for every 1000th cycle,

1
engine-clients/src/main/java/io/nosqlbench/engine/clients/grafana/analyzer/GrafanaRegionAnalyzer.java
View File

@ -39,6 +39,7 @@ public class GrafanaRegionAnalyzer implements Runnable {
ccfg.setBaseUri(baseUrl); ccfg.setBaseUri(baseUrl);
GrafanaClient newclient = new GrafanaClient(ccfg); GrafanaClient newclient = new GrafanaClient(ccfg);
Supplier<String> namer = () -> "nosqlbench-" + SystemId.getNodeId() + "-" + System.currentTimeMillis(); Supplier<String> namer = () -> "nosqlbench-" + SystemId.getNodeId() + "-" + System.currentTimeMillis();
newclient.cacheApiToken(namer, "Admin", Long.MAX_VALUE, Path.of("grafana_apikey"), "admin", "admin"); newclient.cacheApiToken(namer, "Admin", Long.MAX_VALUE, Path.of("grafana_apikey"), "admin", "admin");
this.gclient = newclient; this.gclient = newclient;
} }

2
engine-clients/src/test/java/io/nosqlbench/engine/clients/grafana/analyzer/GrafanaRegionAnalyzerTest.java
View File

@ -16,4 +16,4 @@ public class GrafanaRegionAnalyzerTest {
} }
} }

3
nb-api/src/main/java/io/nosqlbench/nb/api/content/NBPathsAPI.java
View File

@ -89,7 +89,8 @@ public interface NBPathsAPI {
List<List<Content<?>>> resolveEach(); List<List<Content<?>>> resolveEach();
/** /**
* Provide a list of all matching content that was matched by the search qualifers * Provide a list of all matching content that was matched by the search qualifiers
*
* @return a list of content * @return a list of content
*/ */
List<Content<?>> list(); List<Content<?>> list();