Transformation: added support for excluding/including rows based on their values. (#26884)

* Adding FilterByValue transformer skeleton

* Connecting options with Editor

* Improving UI and making deep copy of options on change.

* Improving Transformation Editor UI

* Implementing Regex filtering

* Adding valueFilters.ts and creating filter registry

* Connecting the test function

* Correcting TypeScript errors

* Using FilterInstance instead of simple Filter test function

* Adding field.type as filter options

* Improving UI. Adding custom placeholder depending on filter.

* Implementing a few more filter types

* Implementing more filters

* Return original data if no filter were processed

* Improving UI

* Correcting TS errors

* Making sure inequality transform are invalid until the filterExpression is not empty

* Cleanup in the UI file

* Improving UI (highlight invalid fields)

* Only show filterType that are supported for the selected field

* Adding tests + correction of a filter

* Adding transformer test

* Adding doc

* Cleanup

* Typing props for FilterSelectorRow component

Co-authored-by: Marcus Andersson <systemvetaren@gmail.com>

* Moving rendering in the JSX

Co-authored-by: Marcus Andersson <systemvetaren@gmail.com>

* Memoizing filterTypeOptions computation

Co-authored-by: Marcus Andersson <systemvetaren@gmail.com>

* Improve code compactness

Co-authored-by: Marcus Andersson <systemvetaren@gmail.com>

* Cleanup + solving TS errors

* Updating some labels

* Wrapping stuff around useMemo and useCallback

* Using cloneDeep from lodash

* Don't highlight field name input if null

* Removing time type fields in selectable options

* We want loose equality in this scenario.

* Adding `onChange` to useCallback dependencies

Co-authored-by: Marcus Andersson <systemvetaren@gmail.com>

* Include or exclude matching any or all conditions

* Correcting field name matching

* Cleanup

* Don't highlight the filterExpression input when its empty

* Adding Range filter

* Updating doc

* Correcting TS error

* Only showing the Match All/Match Any option if more than one condition

* Two inputs for the Range filter instead of one

* Improving invalid highlight for Range filter type

* Cleanup

* Improving labels in UI

* Using ButtonSelect to improve UI

* editor UI updates.

* Updating tests

* Adding component for Regex

* Improve TS typing

* Adding components for the other filter types.

* Cleanup

* Correct error

* Updating valueFilter.test.ts

* Updating filterByValue.test.ts

* Reverting and removing Range filter

* Update docs/sources/panels/transformations.md

* starting to implement poc.

* added a small poc.

* wip

* added tests.

* added structure for dynamic value matcher editors.

* added more support.

* added some more value matchers.

* removed unused value filters.

* added some more matchers.

* adding more matchers.

* added a range matcher.

* fixing some tests.

* fixing tests.

* remove unused dep.

* making the matching a bit more performant.

* UX improvements and alignment fixes

* fixed delete button.

* fixed some spacing in the UI.

* added docs for matchers.

* adding docs and exposing value matcher types.

* will store dateTime as string.

* updated docs according to feedback.

* moved filter by value in transformation list.

* Improved description.

* added regex value filter.

* added support for regex.

* fixing failing tests.

Co-authored-by: Marcus Andersson <systemvetaren@gmail.com>
Co-authored-by: Marcus Andersson <marcus.andersson@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>

docs/sources/panels/transformations/types-options.md

@@ -10,6 +10,7 @@ Grafana comes with the following transformations:
   - [Reduce](#reduce)
   - [Filter by name](#filter-by-name)
   - [Filter data by query](#filter-data-by-query)
+  - [Filter data by value](#filter-data-by-value)
   - [Organize fields](#organize-fields)
   - [Outer join](#join-by-field-outer-join)
   - [Series to rows](#series-to-rows)
@@ -276,3 +277,63 @@ Here is the result after applying the Series to rows transformation.
 | 2020-07-07 10:31:22 | Temperature | 22    |
 | 2020-07-07 09:30:57 | Humidity    | 33    |
 | 2020-07-07 09:30:05 | Temperature | 19    |
+
+## Filter by value
+
+
+This transformation allows you to filter your data directly in Grafana and remove some data points from your query result. You have the option to include or exclude data that match one or more conditions you define. The conditions are applied on a selected field.
+
+The available conditions are:
+
+- **Regex**: match a regex expression
+- **Is Null**: match if the value is null
+- **Is Not Null**: match if the value is not null
+- **Equal**: match if the value is equal to the specified value
+- **Different**: match if the value is different than the specified value
+- **Greater**\*: match if the value is greater than the specified value
+- **Lower**\*: match if the value is lower than the specified value
+- **Greater or equal**\*: match if the value is greater or equal
+- **Lower or equal**\*: match if the value is lower or equal
+- **Range**\*: match a range between a specified minimum and maximum, min and max included
+
+\* Those conditions are only available for number fields.
+
+Consider the following data set:
+
+| Time                | Temperature | Altitude
+|---------------------|-------------|----------
+| 2020-07-07 11:34:23 | 32          | 101
+| 2020-07-07 11:34:22 | 28          | 125
+| 2020-07-07 11:34:21 | 26          | 110
+| 2020-07-07 11:34:20 | 23          | 98
+| 2020-07-07 10:32:24 | 31          | 95
+| 2020-07-07 10:31:22 | 20          | 85
+| 2020-07-07 09:30:57 | 19          | 101
+
+If you **Include** the data points that have a temperature below 30°C, the configuration will look as follows:
+
+- Filter Type: `Include`
+- Condition: Rows where `Temperature` matches `Lower Than` `100`
+
+And you will get the following result, where only the temperatures below 30°C are included:
+
+
+| Time                | Temperature | Altitude
+|---------------------|-------------|----------
+| 2020-07-07 11:34:22 | 28          | 125
+| 2020-07-07 11:34:21 | 26          | 110
+| 2020-07-07 11:34:20 | 23          | 98
+| 2020-07-07 10:31:22 | 20          | 85
+| 2020-07-07 09:30:57 | 19          | 101
+
+You can add more than one condition to the filter. For example, you might want to include the data only if the altitude is greater than 100. To do so, add that condition to the following configuration:
+
+- Filter type: `Include` rows that `Match All` conditions
+- Condition 1: Rows where `Temperature` matches `Lower` than `30`
+- Condition 2: Rows where `Altitude` matches `Greater` than `100`
+
+When you have more than one condition, you can choose if you want the action (include / exclude) to be applied on rows that **Match all** conditions or **Match any** of the conditions you added.
+
+In the example above we chose **Match all** because we wanted to include the rows that have a temperature lower than 30 _AND_ an altitude higher than 100. If we wanted to include the rows that have a temperature lower than 30 _OR_ an altitude higher than 100 instead, then we would select **Match any**. This would include the first row in the original data, which has a temperature of 32°C (does not match the first condition) but an altitude of 101 (which matches the second condition), so it is included.
+
+Conditions that are invalid or incompletely configured are ignored.

e2e/suite1/specs/panelEdit_base.spec.ts

@@ -26,7 +26,7 @@ e2e.scenario({
           expect(li.text()).equals('Query1'); // there's already a query so therefore Query + 1
         });
         e2e.components.QueryTab.content().should('be.visible');
-        e2e.components.TransformTab.content().should('not.be.visible');
+        e2e.components.TransformTab.content().should('not.exist');
         e2e.components.AlertTab.content().should('not.be.visible');
 
         //  Bottom pane tabs
@@ -37,7 +37,9 @@ e2e.scenario({
         e2e.components.Tab.active().within((li: JQuery<HTMLLIElement>) => {
           expect(li.text()).equals('Transform0'); // there's no transform so therefore Transform + 0
         });
-        e2e.components.TransformTab.content().should('be.visible');
+        e2e.components.Transforms.card('Merge')
+          .scrollIntoView()
+          .should('be.visible');
         e2e.components.QueryTab.content().should('not.be.visible');
         e2e.components.AlertTab.content().should('not.be.visible');
 
@@ -50,7 +52,7 @@ e2e.scenario({
         });
         e2e.components.AlertTab.content().should('be.visible');
         e2e.components.QueryTab.content().should('not.be.visible');
-        e2e.components.TransformTab.content().should('not.be.visible');
+        e2e.components.TransformTab.content().should('not.exist');
 
         e2e.components.Tab.title('Query')
           .should('be.visible')

packages/grafana-data/src/index.ts

@@ -13,4 +13,9 @@ export * from './text';
 export * from './valueFormats';
 export * from './field';
 export * from './events';
+export {
+  ValueMatcherOptions,
+  BasicValueMatcherOptions,
+  RangeValueMatcherOptions,
+} from './transformations/matchers/valueMatchers/types';
 export { PanelPlugin } from './panel/PanelPlugin';

packages/grafana-data/src/transformations/matchers.ts

@@ -9,9 +9,16 @@ import {
   FrameMatcherInfo,
   FieldMatcher,
   FrameMatcher,
+  ValueMatcherInfo,
+  ValueMatcher,
 } from '../types/transformations';
 import { Registry } from '../utils/Registry';
+import { getNullValueMatchers } from './matchers/valueMatchers/nullMatchers';
+import { getNumericValueMatchers } from './matchers/valueMatchers/numericMatchers';
+import { getEqualValueMatchers } from './matchers/valueMatchers/equalMatchers';
+import { getRangeValueMatchers } from './matchers/valueMatchers/rangeMatchers';
 import { getSimpleFieldMatchers } from './matchers/simpleFieldMatcher';
+import { getRegexValueMatcher } from './matchers/valueMatchers/regexMatchers';
 
 /**
  * Registry that contains all of the built in field matchers.
@@ -38,6 +45,20 @@ export const frameMatchers = new Registry<FrameMatcherInfo>(() => {
   ];
 });
 
+/**
+ * Registry that contains all of the built in value matchers.
+ * @public
+ */
+export const valueMatchers = new Registry<ValueMatcherInfo>(() => {
+  return [
+    ...getNullValueMatchers(),
+    ...getNumericValueMatchers(),
+    ...getEqualValueMatchers(),
+    ...getRangeValueMatchers(),
+    ...getRegexValueMatcher(),
+  ];
+});
+
 /**
  * Resolves a field matcher from the registry for given config.
  * Will throw an error if matcher can not be resolved.
@@ -45,6 +66,9 @@ export const frameMatchers = new Registry<FrameMatcherInfo>(() => {
  */
 export function getFieldMatcher(config: MatcherConfig): FieldMatcher {
   const info = fieldMatchers.get(config.id);
+  if (!info) {
+    throw new Error('Unknown field matcher: ' + config.id);
+  }
   return info.get(config.options);
 }
 
@@ -55,5 +79,21 @@ export function getFieldMatcher(config: MatcherConfig): FieldMatcher {
  */
 export function getFrameMatchers(config: MatcherConfig): FrameMatcher {
   const info = frameMatchers.get(config.id);
+  if (!info) {
+    throw new Error('Unknown frame matcher: ' + config.id);
+  }
+  return info.get(config.options);
+}
+
+/**
+ * Resolves a value matcher from the registry for given config.
+ * Will throw an error if matcher can not be resolved.
+ * @public
+ */
+export function getValueMatcher(config: MatcherConfig): ValueMatcher {
+  const info = valueMatchers.get(config.id);
+  if (!info) {
+    throw new Error('Unknown value matcher: ' + config.id);
+  }
   return info.get(config.options);
 }

packages/grafana-data/src/transformations/matchers/ids.ts

@@ -37,3 +37,19 @@ export enum FrameMatcherID {
   byIndex = 'byIndex',
   byLabel = 'byLabel',
 }
+
+/**
+ * @public
+ */
+export enum ValueMatcherID {
+  regex = 'regex',
+  isNull = 'isNull',
+  isNotNull = 'isNotNull',
+  greater = 'greater',
+  greaterOrEqual = 'greaterOrEqual',
+  lower = 'lower',
+  lowerOrEqual = 'lowerOrEqual',
+  equal = 'equal',
+  notEqual = 'notEqual',
+  between = 'between',
+}

packages/grafana-data/src/transformations/matchers/valueMatchers/equalMatchers.test.ts

@@ -0,0 +1,108 @@
+import { toDataFrame } from '../../../dataframe';
+import { DataFrame } from '../../../types/dataFrame';
+import { getValueMatcher } from '../../matchers';
+import { ValueMatcherID } from '../ids';
+
+describe('value equals to matcher', () => {
+  const data: DataFrame[] = [
+    toDataFrame({
+      fields: [
+        {
+          name: 'temp',
+          values: [23, null, 10, 'asd', '23'],
+        },
+      ],
+    }),
+  ];
+
+  const matcher = getValueMatcher({
+    id: ValueMatcherID.equal,
+    options: {
+      value: 23,
+    },
+  });
+
+  it('should match when option value is same', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 0;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should not match when option value is different', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 2;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+
+  it('should not match when option value is different type', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 3;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+
+  it('should match when option value is different type but same', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 4;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+});
+
+describe('value not equals matcher', () => {
+  const data: DataFrame[] = [
+    toDataFrame({
+      fields: [
+        {
+          name: 'temp',
+          values: [23, null, 10, 'asd', '23'],
+        },
+      ],
+    }),
+  ];
+
+  const matcher = getValueMatcher({
+    id: ValueMatcherID.notEqual,
+    options: {
+      value: 23,
+    },
+  });
+
+  it('should not match when option value is same', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 0;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+
+  it('should match when option value is different', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 2;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should match when option value is different type', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 3;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should not match when option value is different type but same', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 4;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+});

packages/grafana-data/src/transformations/matchers/valueMatchers/equalMatchers.ts

@@ -0,0 +1,42 @@
+import { Field } from '../../../types/dataFrame';
+import { ValueMatcherInfo } from '../../../types/transformations';
+import { ValueMatcherID } from '../ids';
+import { BasicValueMatcherOptions } from './types';
+
+const isEqualValueMatcher: ValueMatcherInfo<BasicValueMatcherOptions> = {
+  id: ValueMatcherID.equal,
+  name: 'Is equal',
+  description: 'Match where value for given field is equal to options value.',
+  get: options => {
+    return (valueIndex: number, field: Field) => {
+      const value = field.values.get(valueIndex);
+      // eslint-disable-next-line eqeqeq
+      return value == options.value;
+    };
+  },
+  getOptionsDisplayText: () => {
+    return `Matches all rows where field is null.`;
+  },
+  isApplicable: () => true,
+  getDefaultOptions: () => ({ value: '' }),
+};
+
+const isNotEqualValueMatcher: ValueMatcherInfo<BasicValueMatcherOptions> = {
+  id: ValueMatcherID.notEqual,
+  name: 'Is not equal',
+  description: 'Match where value for given field is not equal to options value.',
+  get: options => {
+    return (valueIndex: number, field: Field) => {
+      const value = field.values.get(valueIndex);
+      // eslint-disable-next-line eqeqeq
+      return value != options.value;
+    };
+  },
+  getOptionsDisplayText: () => {
+    return `Matches all rows where field is not null.`;
+  },
+  isApplicable: () => true,
+  getDefaultOptions: () => ({ value: '' }),
+};
+
+export const getEqualValueMatchers = (): ValueMatcherInfo[] => [isEqualValueMatcher, isNotEqualValueMatcher];

packages/grafana-data/src/transformations/matchers/valueMatchers/nullMatchers.test.ts

@@ -0,0 +1,72 @@
+import { toDataFrame } from '../../../dataframe';
+import { DataFrame } from '../../../types/dataFrame';
+import { getValueMatcher } from '../../matchers';
+import { ValueMatcherID } from '../ids';
+
+describe('value null matcher', () => {
+  const data: DataFrame[] = [
+    toDataFrame({
+      fields: [
+        {
+          name: 'temp',
+          values: [23, null, 10],
+        },
+      ],
+    }),
+  ];
+
+  const matcher = getValueMatcher({
+    id: ValueMatcherID.isNull,
+    options: {},
+  });
+
+  it('should match null values', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 1;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should not match non-null values', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 0;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+});
+
+describe('value not null matcher', () => {
+  const data: DataFrame[] = [
+    toDataFrame({
+      fields: [
+        {
+          name: 'temp',
+          values: [23, null, 10],
+        },
+      ],
+    }),
+  ];
+
+  const matcher = getValueMatcher({
+    id: ValueMatcherID.isNotNull,
+    options: {},
+  });
+
+  it('should match not null values', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 0;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should match non-null values', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 1;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+});

packages/grafana-data/src/transformations/matchers/valueMatchers/nullMatchers.ts

@@ -0,0 +1,40 @@
+import { Field } from '../../../types/dataFrame';
+import { ValueMatcherInfo } from '../../../types/transformations';
+import { ValueMatcherID } from '../ids';
+import { ValueMatcherOptions } from './types';
+
+const isNullValueMatcher: ValueMatcherInfo<ValueMatcherOptions> = {
+  id: ValueMatcherID.isNull,
+  name: 'Is null',
+  description: 'Match where value for given field is null.',
+  get: () => {
+    return (valueIndex: number, field: Field) => {
+      const value = field.values.get(valueIndex);
+      return value === null;
+    };
+  },
+  getOptionsDisplayText: () => {
+    return `Matches all rows where field is null.`;
+  },
+  isApplicable: () => true,
+  getDefaultOptions: () => ({}),
+};
+
+const isNotNullValueMatcher: ValueMatcherInfo<ValueMatcherOptions> = {
+  id: ValueMatcherID.isNotNull,
+  name: 'Is not null',
+  description: 'Match where value for given field is not null.',
+  get: () => {
+    return (valueIndex: number, field: Field) => {
+      const value = field.values.get(valueIndex);
+      return value !== null;
+    };
+  },
+  getOptionsDisplayText: () => {
+    return `Matches all rows where field is not null.`;
+  },
+  isApplicable: () => true,
+  getDefaultOptions: () => ({}),
+};
+
+export const getNullValueMatchers = (): ValueMatcherInfo[] => [isNullValueMatcher, isNotNullValueMatcher];

packages/grafana-data/src/transformations/matchers/valueMatchers/numericMatchers.test.ts

@@ -0,0 +1,180 @@
+import { toDataFrame } from '../../../dataframe';
+import { DataFrame } from '../../../types/dataFrame';
+import { getValueMatcher } from '../../matchers';
+import { ValueMatcherID } from '../ids';
+
+describe('value greater than matcher', () => {
+  const data: DataFrame[] = [
+    toDataFrame({
+      fields: [
+        {
+          name: 'temp',
+          values: [23, 11, 10],
+        },
+      ],
+    }),
+  ];
+
+  const matcher = getValueMatcher({
+    id: ValueMatcherID.greater,
+    options: {
+      value: 11,
+    },
+  });
+
+  it('should match values greater than 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 0;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should not match values equlas to 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 1;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+
+  it('should not match values lower than 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 2;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+});
+
+describe('value greater than or equal matcher', () => {
+  const data: DataFrame[] = [
+    toDataFrame({
+      fields: [
+        {
+          name: 'temp',
+          values: [23, 11, 10],
+        },
+      ],
+    }),
+  ];
+
+  const matcher = getValueMatcher({
+    id: ValueMatcherID.greaterOrEqual,
+    options: {
+      value: 11,
+    },
+  });
+
+  it('should match values greater than 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 0;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should match values equlas to 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 1;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should not match values lower than 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 2;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+});
+
+describe('value lower than matcher', () => {
+  const data: DataFrame[] = [
+    toDataFrame({
+      fields: [
+        {
+          name: 'temp',
+          values: [23, 11, 10],
+        },
+      ],
+    }),
+  ];
+
+  const matcher = getValueMatcher({
+    id: ValueMatcherID.lower,
+    options: {
+      value: 11,
+    },
+  });
+
+  it('should match values lower than 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 2;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should not match values equal to 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 1;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+
+  it('should not match values greater than 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 0;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+});
+
+describe('value lower than or equal matcher', () => {
+  const data: DataFrame[] = [
+    toDataFrame({
+      fields: [
+        {
+          name: 'temp',
+          values: [23, 11, 10],
+        },
+      ],
+    }),
+  ];
+
+  const matcher = getValueMatcher({
+    id: ValueMatcherID.lowerOrEqual,
+    options: {
+      value: 11,
+    },
+  });
+
+  it('should match values lower than 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 2;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should match values equal to 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 1;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should not match values greater than 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 0;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+});

packages/grafana-data/src/transformations/matchers/valueMatchers/numericMatchers.ts

@@ -0,0 +1,91 @@
+import { Field, FieldType } from '../../../types/dataFrame';
+import { ValueMatcherInfo } from '../../../types/transformations';
+import { ValueMatcherID } from '../ids';
+import { BasicValueMatcherOptions } from './types';
+
+const isGreaterValueMatcher: ValueMatcherInfo<BasicValueMatcherOptions<number>> = {
+  id: ValueMatcherID.greater,
+  name: 'Is greater',
+  description: 'Match when field value is greater than option.',
+  get: options => {
+    return (valueIndex: number, field: Field) => {
+      const value = field.values.get(valueIndex);
+      if (isNaN(value)) {
+        return false;
+      }
+      return value > options.value;
+    };
+  },
+  getOptionsDisplayText: options => {
+    return `Matches all rows where field value is greater than: ${options.value}.`;
+  },
+  isApplicable: field => field.type === FieldType.number,
+  getDefaultOptions: () => ({ value: 0 }),
+};
+
+const isGreaterOrEqualValueMatcher: ValueMatcherInfo<BasicValueMatcherOptions<number>> = {
+  id: ValueMatcherID.greaterOrEqual,
+  name: 'Is greater or equal',
+  description: 'Match when field value is lower or greater than option.',
+  get: options => {
+    return (valueIndex: number, field: Field) => {
+      const value = field.values.get(valueIndex);
+      if (isNaN(value)) {
+        return false;
+      }
+      return value >= options.value;
+    };
+  },
+  getOptionsDisplayText: options => {
+    return `Matches all rows where field value is lower or greater than: ${options.value}.`;
+  },
+  isApplicable: field => field.type === FieldType.number,
+  getDefaultOptions: () => ({ value: 0 }),
+};
+
+const isLowerValueMatcher: ValueMatcherInfo<BasicValueMatcherOptions<number>> = {
+  id: ValueMatcherID.lower,
+  name: 'Is lower',
+  description: 'Match when field value is lower than option.',
+  get: options => {
+    return (valueIndex: number, field: Field) => {
+      const value = field.values.get(valueIndex);
+      if (isNaN(value)) {
+        return false;
+      }
+      return value < options.value;
+    };
+  },
+  getOptionsDisplayText: options => {
+    return `Matches all rows where field value is lower than: ${options.value}.`;
+  },
+  isApplicable: field => field.type === FieldType.number,
+  getDefaultOptions: () => ({ value: 0 }),
+};
+
+const isLowerOrEqualValueMatcher: ValueMatcherInfo<BasicValueMatcherOptions<number>> = {
+  id: ValueMatcherID.lowerOrEqual,
+  name: 'Is lower or equal',
+  description: 'Match when field value is lower or equal than option.',
+  get: options => {
+    return (valueIndex: number, field: Field) => {
+      const value = field.values.get(valueIndex);
+      if (isNaN(value)) {
+        return false;
+      }
+      return value <= options.value;
+    };
+  },
+  getOptionsDisplayText: options => {
+    return `Matches all rows where field value is lower or equal than: ${options.value}.`;
+  },
+  isApplicable: field => field.type === FieldType.number,
+  getDefaultOptions: () => ({ value: 0 }),
+};
+
+export const getNumericValueMatchers = (): ValueMatcherInfo[] => [
+  isGreaterValueMatcher,
+  isGreaterOrEqualValueMatcher,
+  isLowerValueMatcher,
+  isLowerOrEqualValueMatcher,
+];

packages/grafana-data/src/transformations/matchers/valueMatchers/rangeMatchers.test.ts

@@ -0,0 +1,49 @@
+import { toDataFrame } from '../../../dataframe';
+import { DataFrame } from '../../../types/dataFrame';
+import { getValueMatcher } from '../../matchers';
+import { ValueMatcherID } from '../ids';
+
+describe('value between matcher', () => {
+  const data: DataFrame[] = [
+    toDataFrame({
+      fields: [
+        {
+          name: 'temp',
+          values: [23, 11, 10, 25],
+        },
+      ],
+    }),
+  ];
+
+  const matcher = getValueMatcher({
+    id: ValueMatcherID.between,
+    options: {
+      from: 10,
+      to: 25,
+    },
+  });
+
+  it('should match values greater than 10 but lower than 25', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 0;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+  });
+
+  it('should not match values greater than 25', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 4;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+
+  it('should not match values lower than 11', () => {
+    const frame = data[0];
+    const field = frame.fields[0];
+    const valueIndex = 2;
+
+    expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+  });
+});

packages/grafana-data/src/transformations/matchers/valueMatchers/rangeMatchers.ts

@@ -0,0 +1,26 @@
+import { Field, FieldType } from '../../../types/dataFrame';
+import { ValueMatcherInfo } from '../../../types/transformations';
+import { ValueMatcherID } from '../ids';
+import { RangeValueMatcherOptions } from './types';
+
+const isBetweenValueMatcher: ValueMatcherInfo<RangeValueMatcherOptions<number>> = {
+  id: ValueMatcherID.between,
+  name: 'Is between',
+  description: 'Match when field value is between given option values.',
+  get: options => {
+    return (valueIndex: number, field: Field) => {
+      const value = field.values.get(valueIndex);
+      if (isNaN(value)) {
+        return false;
+      }
+      return value > options.from && value < options.to;
+    };
+  },
+  getOptionsDisplayText: options => {
+    return `Matches all rows where field value is between ${options.from} and ${options.to}.`;
+  },
+  isApplicable: field => field.type === FieldType.number,
+  getDefaultOptions: () => ({ from: 0, to: 100 }),
+};
+
+export const getRangeValueMatchers = (): ValueMatcherInfo[] => [isBetweenValueMatcher];

packages/grafana-data/src/transformations/matchers/valueMatchers/regexMatchers.test.ts

@@ -0,0 +1,85 @@
+import { toDataFrame } from '../../../dataframe';
+import { DataFrame } from '../../../types/dataFrame';
+import { getValueMatcher } from '../../matchers';
+import { ValueMatcherID } from '../ids';
+
+describe('regex value matcher', () => {
+  const data: DataFrame[] = [
+    toDataFrame({
+      fields: [
+        {
+          name: 'temp',
+          values: ['.', 'asdf', 100, '25.5'],
+        },
+      ],
+    }),
+  ];
+
+  describe('option with value .*', () => {
+    const matcher = getValueMatcher({
+      id: ValueMatcherID.regex,
+      options: {
+        value: '.*',
+      },
+    });
+
+    it('should match all values', () => {
+      const frame = data[0];
+      const field = frame.fields[0];
+
+      for (let i = 0; i < field.values.length; i++) {
+        expect(matcher(i, field, frame, data)).toBeTruthy();
+      }
+    });
+  });
+
+  describe('option with value \\w+', () => {
+    const matcher = getValueMatcher({
+      id: ValueMatcherID.regex,
+      options: {
+        value: '\\w+',
+      },
+    });
+
+    it('should match wordy values', () => {
+      const frame = data[0];
+      const field = frame.fields[0];
+      const valueIndex = 1;
+
+      expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+    });
+
+    it('should not match non-wordy values', () => {
+      const frame = data[0];
+      const field = frame.fields[0];
+      const valueIndex = 0;
+
+      expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+    });
+  });
+
+  describe('option with value \\d+', () => {
+    const matcher = getValueMatcher({
+      id: ValueMatcherID.regex,
+      options: {
+        value: '\\d+',
+      },
+    });
+
+    it('should match numeric values', () => {
+      const frame = data[0];
+      const field = frame.fields[0];
+      const valueIndex = 2;
+
+      expect(matcher(valueIndex, field, frame, data)).toBeTruthy();
+    });
+
+    it('should not match non-numeric values', () => {
+      const frame = data[0];
+      const field = frame.fields[0];
+      const valueIndex = 1;
+
+      expect(matcher(valueIndex, field, frame, data)).toBeFalsy();
+    });
+  });
+});

packages/grafana-data/src/transformations/matchers/valueMatchers/regexMatchers.ts

@@ -0,0 +1,25 @@
+import { Field } from '../../../types/dataFrame';
+import { ValueMatcherInfo } from '../../../types/transformations';
+import { ValueMatcherID } from '../ids';
+import { BasicValueMatcherOptions } from './types';
+
+const regexValueMatcher: ValueMatcherInfo<BasicValueMatcherOptions<string>> = {
+  id: ValueMatcherID.regex,
+  name: 'Regex',
+  description: 'Match when field value is matching regex.',
+  get: options => {
+    const regex = new RegExp(options.value);
+
+    return (valueIndex: number, field: Field) => {
+      const value = field.values.get(valueIndex);
+      return regex.test(value);
+    };
+  },
+  getOptionsDisplayText: options => {
+    return `Matches all rows where field value is matching regex: ${options.value}`;
+  },
+  isApplicable: () => true,
+  getDefaultOptions: () => ({ value: '.*' }),
+};
+
+export const getRegexValueMatcher = (): ValueMatcherInfo[] => [regexValueMatcher];

packages/grafana-data/src/transformations/matchers/valueMatchers/types.ts

@@ -0,0 +1,23 @@
+/**
+ * Describes a empty value matcher option.
+ * @public
+ */
+export interface ValueMatcherOptions {}
+
+/**
+ * Describes a basic value matcher option that has a single value.
+ * @public
+ */
+export interface BasicValueMatcherOptions<T = any> extends ValueMatcherOptions {
+  value: T;
+}
+
+/**
+ * Describes a range value matcher option that has a to and a from value to
+ * be able to match a range.
+ * @public
+ */
+export interface RangeValueMatcherOptions<T = any> extends ValueMatcherOptions {
+  from: T;
+  to: T;
+}

packages/grafana-data/src/transformations/transformers.ts

@@ -14,6 +14,7 @@ import { labelsToFieldsTransformer } from './transformers/labelsToFields';
 import { ensureColumnsTransformer } from './transformers/ensureColumns';
 import { groupByTransformer } from './transformers/groupBy';
 import { mergeTransformer } from './transformers/merge';
+import { filterByValueTransformer } from './transformers/filterByValue';
 
 export const standardTransformers = {
   noopTransformer,
@@ -21,6 +22,7 @@ export const standardTransformers = {
   filterFieldsByNameTransformer,
   filterFramesTransformer,
   filterFramesByRefIdTransformer,
+  filterByValueTransformer,
   orderFieldsTransformer,
   organizeFieldsTransformer,
   reduceTransformer,

packages/grafana-data/src/transformations/transformers/filterByValue.test.ts

@@ -0,0 +1,219 @@
+import { mockTransformationsRegistry } from '../../utils/tests/mockTransformationsRegistry';
+import { DataTransformerConfig, FieldType, MatcherConfig } from '../../types';
+import { ArrayVector } from '../../vector';
+import { transformDataFrame } from '../transformDataFrame';
+import { toDataFrame } from '../../dataframe/processDataFrame';
+import {
+  FilterByValueMatch,
+  filterByValueTransformer,
+  FilterByValueTransformerOptions,
+  FilterByValueType,
+} from './filterByValue';
+import { DataTransformerID } from './ids';
+import { ValueMatcherID } from '../matchers/ids';
+import { BasicValueMatcherOptions } from '../matchers/valueMatchers/types';
+
+const seriesAWithSingleField = toDataFrame({
+  name: 'A',
+  length: 7,
+  fields: [
+    { name: 'time', type: FieldType.time, values: new ArrayVector([1000, 2000, 3000, 4000, 5000, 6000, 7000]) },
+    { name: 'numbers', type: FieldType.number, values: new ArrayVector([1, 2, 3, 4, 5, 6, 7]) },
+  ],
+});
+
+describe('FilterByValue transformer', () => {
+  beforeAll(() => {
+    mockTransformationsRegistry([filterByValueTransformer]);
+  });
+
+  it('should exclude values', async () => {
+    const lower: MatcherConfig<BasicValueMatcherOptions<number>> = {
+      id: ValueMatcherID.lower,
+      options: { value: 6 },
+    };
+
+    const cfg: DataTransformerConfig<FilterByValueTransformerOptions> = {
+      id: DataTransformerID.filterByValue,
+      options: {
+        type: FilterByValueType.exclude,
+        match: FilterByValueMatch.all,
+        filters: [
+          {
+            fieldName: 'numbers',
+            config: lower,
+          },
+        ],
+      },
+    };
+
+    await expect(transformDataFrame([cfg], [seriesAWithSingleField])).toEmitValuesWith(received => {
+      const processed = received[0];
+
+      expect(processed.length).toEqual(1);
+      expect(processed[0].fields).toEqual([
+        {
+          name: 'time',
+          type: FieldType.time,
+          values: new ArrayVector([6000, 7000]),
+          state: { displayName: 'time' },
+          config: {},
+        },
+        {
+          name: 'numbers',
+          type: FieldType.number,
+          values: new ArrayVector([6, 7]),
+          state: { displayName: 'numbers' },
+          config: {},
+        },
+      ]);
+    });
+  });
+
+  it('should include values', async () => {
+    const lowerOrEqual: MatcherConfig<BasicValueMatcherOptions<number>> = {
+      id: ValueMatcherID.lowerOrEqual,
+      options: { value: 5 },
+    };
+
+    const cfg: DataTransformerConfig<FilterByValueTransformerOptions> = {
+      id: DataTransformerID.filterByValue,
+      options: {
+        type: FilterByValueType.include,
+        match: FilterByValueMatch.all,
+        filters: [
+          {
+            fieldName: 'numbers',
+            config: lowerOrEqual,
+          },
+        ],
+      },
+    };
+
+    await expect(transformDataFrame([cfg], [seriesAWithSingleField])).toEmitValuesWith(received => {
+      const processed = received[0];
+
+      expect(processed.length).toEqual(1);
+      expect(processed[0].fields).toEqual([
+        {
+          name: 'time',
+          type: FieldType.time,
+          values: new ArrayVector([1000, 2000, 3000, 4000, 5000]),
+          state: { displayName: 'time' },
+          config: {},
+        },
+        {
+          name: 'numbers',
+          type: FieldType.number,
+          values: new ArrayVector([1, 2, 3, 4, 5]),
+          state: { displayName: 'numbers' },
+          config: {},
+        },
+      ]);
+    });
+  });
+
+  it('should match any condition', async () => {
+    const lowerOrEqual: MatcherConfig<BasicValueMatcherOptions<number>> = {
+      id: ValueMatcherID.lowerOrEqual,
+      options: { value: 4 },
+    };
+
+    const equal: MatcherConfig<BasicValueMatcherOptions<number>> = {
+      id: ValueMatcherID.equal,
+      options: { value: 7 },
+    };
+
+    const cfg: DataTransformerConfig<FilterByValueTransformerOptions> = {
+      id: DataTransformerID.filterByValue,
+      options: {
+        type: FilterByValueType.include,
+        match: FilterByValueMatch.any,
+        filters: [
+          {
+            fieldName: 'numbers',
+            config: lowerOrEqual,
+          },
+          {
+            fieldName: 'numbers',
+            config: equal,
+          },
+        ],
+      },
+    };
+
+    await expect(transformDataFrame([cfg], [seriesAWithSingleField])).toEmitValuesWith(received => {
+      const processed = received[0];
+
+      expect(processed.length).toEqual(1);
+      expect(processed[0].fields).toEqual([
+        {
+          name: 'time',
+          type: FieldType.time,
+          values: new ArrayVector([1000, 2000, 3000, 4000, 7000]),
+          state: { displayName: 'time' },
+          config: {},
+        },
+        {
+          name: 'numbers',
+          type: FieldType.number,
+          values: new ArrayVector([1, 2, 3, 4, 7]),
+          state: { displayName: 'numbers' },
+          config: {},
+        },
+      ]);
+    });
+  });
+
+  it('should match all condition', async () => {
+    const greaterOrEqual: MatcherConfig<BasicValueMatcherOptions<number>> = {
+      id: ValueMatcherID.greaterOrEqual,
+      options: { value: 4 },
+    };
+
+    const lowerOrEqual: MatcherConfig<BasicValueMatcherOptions<number>> = {
+      id: ValueMatcherID.lowerOrEqual,
+      options: { value: 5 },
+    };
+
+    const cfg: DataTransformerConfig<FilterByValueTransformerOptions> = {
+      id: DataTransformerID.filterByValue,
+      options: {
+        type: FilterByValueType.include,
+        match: FilterByValueMatch.all,
+        filters: [
+          {
+            fieldName: 'numbers',
+            config: lowerOrEqual,
+          },
+          {
+            fieldName: 'numbers',
+            config: greaterOrEqual,
+          },
+        ],
+      },
+    };
+
+    await expect(transformDataFrame([cfg], [seriesAWithSingleField])).toEmitValuesWith(received => {
+      const processed = received[0];
+
+      expect(processed.length).toEqual(1);
+      expect(processed[0].fields).toEqual([
+        {
+          name: 'time',
+          type: FieldType.time,
+          values: new ArrayVector([4000, 5000]),
+          state: { displayName: 'time' },
+          config: {},
+        },
+        {
+          name: 'numbers',
+          type: FieldType.number,
+          values: new ArrayVector([4, 5]),
+          state: { displayName: 'numbers' },
+          config: {},
+        },
+      ]);
+    });
+  });
+});

packages/grafana-data/src/transformations/transformers/filterByValue.ts

@@ -0,0 +1,159 @@
+import { map } from 'rxjs/operators';
+
+import { noopTransformer } from './noop';
+import { DataTransformerID } from './ids';
+import { DataTransformerInfo, MatcherConfig } from '../../types/transformations';
+import { DataFrame, Field } from '../../types/dataFrame';
+import { getFieldDisplayName } from '../../field/fieldState';
+import { getValueMatcher } from '../matchers';
+import { ArrayVector } from '../../vector/ArrayVector';
+
+export enum FilterByValueType {
+  exclude = 'exclude',
+  include = 'include',
+}
+
+export enum FilterByValueMatch {
+  all = 'all',
+  any = 'any',
+}
+
+export interface FilterByValueFilter {
+  fieldName: string;
+  config: MatcherConfig;
+}
+
+export interface FilterByValueTransformerOptions {
+  filters: FilterByValueFilter[];
+  type: FilterByValueType;
+  match: FilterByValueMatch;
+}
+
+export const filterByValueTransformer: DataTransformerInfo<FilterByValueTransformerOptions> = {
+  id: DataTransformerID.filterByValue,
+  name: 'Filter data by values',
+  description: 'select a subset of results based on values',
+  defaultOptions: {
+    filters: [],
+    type: FilterByValueType.include,
+    match: FilterByValueMatch.any,
+  },
+
+  operator: options => source => {
+    const filters = options.filters;
+    const matchAll = options.match === FilterByValueMatch.all;
+    const include = options.type === FilterByValueType.include;
+
+    if (!Array.isArray(filters) || filters.length === 0) {
+      return source.pipe(noopTransformer.operator({}));
+    }
+
+    return source.pipe(
+      map(data => {
+        if (!Array.isArray(data) || data.length === 0) {
+          return data;
+        }
+
+        const rows = new Set<number>();
+
+        for (const frame of data) {
+          const fieldIndexByName = groupFieldIndexByName(frame, data);
+          const matchers = createFilterValueMatchers(filters, fieldIndexByName);
+
+          for (let index = 0; index < frame.length; index++) {
+            if (rows.has(index)) {
+              continue;
+            }
+
+            let matching = true;
+
+            for (const matcher of matchers) {
+              const match = matcher(index, frame, data);
+
+              if (!matchAll && match) {
+                matching = true;
+                break;
+              }
+
+              if (matchAll && !match) {
+                matching = false;
+                break;
+              }
+
+              matching = match;
+            }
+
+            if (matching) {
+              rows.add(index);
+            }
+          }
+        }
+
+        const processed: DataFrame[] = [];
+        const frameLength = include ? rows.size : data[0].length - rows.size;
+
+        for (const frame of data) {
+          const fields: Field[] = [];
+
+          for (const field of frame.fields) {
+            const buffer = [];
+
+            for (let index = 0; index < frame.length; index++) {
+              if (include && rows.has(index)) {
+                buffer.push(field.values.get(index));
+                continue;
+              }
+
+              if (!include && !rows.has(index)) {
+                buffer.push(field.values.get(index));
+                continue;
+              }
+            }
+
+            // TODO: what parts needs to be excluded from field.
+            fields.push({
+              ...field,
+              values: new ArrayVector(buffer),
+              config: {},
+            });
+          }
+
+          processed.push({
+            ...frame,
+            fields: fields,
+            length: frameLength,
+          });
+        }
+
+        return processed;
+      })
+    );
+  },
+};
+
+const createFilterValueMatchers = (
+  filters: FilterByValueFilter[],
+  fieldIndexByName: Record<string, number>
+): Array<(index: number, frame: DataFrame, data: DataFrame[]) => boolean> => {
+  const noop = () => false;
+
+  return filters.map(filter => {
+    const fieldIndex = fieldIndexByName[filter.fieldName] ?? -1;
+
+    if (fieldIndex < 0) {
+      console.warn(`[FilterByValue] Could not find index for field name: ${filter.fieldName}`);
+      return noop;
+    }
+
+    const matcher = getValueMatcher(filter.config);
+    return (index, frame, data) => matcher(index, frame.fields[fieldIndex], frame, data);
+  });
+};
+
+const groupFieldIndexByName = (frame: DataFrame, data: DataFrame[]): Record<string, number> => {
+  return frame.fields.reduce((all: Record<string, number>, field, fieldIndex) => {
+    const fieldName = getFieldDisplayName(field, frame, data);
+    all[fieldName] = fieldIndex;
+    return all;
+  }, {});
+};

packages/grafana-data/src/transformations/transformers/ids.ts

@@ -16,6 +16,7 @@ export enum DataTransformerID {
   filterFieldsByName = 'filterFieldsByName',
   filterFrames = 'filterFrames',
   filterByRefId = 'filterByRefId',
+  filterByValue = 'filterByValue',
   noop = 'noop',
   ensureColumns = 'ensureColumns',
   groupBy = 'groupBy',

packages/grafana-data/src/types/transformations.ts

@@ -25,8 +25,14 @@ export interface DataTransformerConfig<TOptions = any> {
   options: TOptions;
 }
 
-export type FieldMatcher = (field: Field, frame: DataFrame, allFrames: DataFrame[]) => boolean;
 export type FrameMatcher = (frame: DataFrame) => boolean;
+export type FieldMatcher = (field: Field, frame: DataFrame, allFrames: DataFrame[]) => boolean;
+
+/**
+ * Value matcher type to describe the matcher function
+ * @public
+ */
+export type ValueMatcher = (valueIndex: number, field: Field, frame: DataFrame, allFrames: DataFrame[]) => boolean;
 
 export interface FieldMatcherInfo<TOptions = any> extends RegistryItemWithOptions<TOptions> {
   get: (options: TOptions) => FieldMatcher;
@@ -36,6 +42,16 @@ export interface FrameMatcherInfo<TOptions = any> extends RegistryItemWithOption
   get: (options: TOptions) => FrameMatcher;
 }
 
+/**
+ * Registry item to represent all the different valu matchers supported
+ * in the Grafana platform.
+ * @public
+ */
+export interface ValueMatcherInfo<TOptions = any> extends RegistryItemWithOptions<TOptions> {
+  get: (options: TOptions) => ValueMatcher;
+  isApplicable: (field: Field) => boolean;
+  getDefaultOptions: (field: Field) => TOptions;
+}
 export interface MatcherConfig<TOptions = any> {
   id: string;
   options?: TOptions;

packages/grafana-e2e-selectors/src/selectors/components.ts

@@ -109,6 +109,7 @@ export const Components = {
     transformationEditorDebugger: (name: string) => `Transformation editor debugger ${name}`,
   },
   Transforms: {
+    card: (name: string) => `New transform ${name}`,
     Reduce: {
       modeLabel: 'Transform mode label',
       calculationsLabel: 'Transform calculations label',

packages/grafana-ui/src/components/Forms/RadioButtonGroup/RadioButton.tsx

@@ -80,9 +80,8 @@ const getRadioButtonStyles = stylesFactory((theme: GrafanaTheme, size: RadioButt
       background: ${bg};
       cursor: pointer;
       z-index: 1;
-      flex-grow: ${fullWidth ? 1 : 0};
+      flex: ${fullWidth ? `1 0 0` : 0};
       text-align: center;
-
       user-select: none;
 
       &:hover {

packages/grafana-ui/src/components/Forms/RadioButtonGroup/RadioButtonGroup.tsx

@@ -1,5 +1,5 @@
 import React, { useCallback, useRef } from 'react';
-import { css } from 'emotion';
+import { css, cx } from 'emotion';
 import uniqueId from 'lodash/uniqueId';
 import { SelectableValue } from '@grafana/data';
 import { RadioButtonSize, RadioButton } from './RadioButton';
@@ -44,6 +44,7 @@ interface RadioButtonGroupProps<T> {
   onChange?: (value?: T) => void;
   size?: RadioButtonSize;
   fullWidth?: boolean;
+  className?: string;
 }
 
 export function RadioButtonGroup<T>({
@@ -53,6 +54,7 @@ export function RadioButtonGroup<T>({
   disabled,
   disabledOptions,
   size = 'md',
+  className,
   fullWidth = false,
 }: RadioButtonGroupProps<T>) {
   const handleOnChange = useCallback(
@@ -70,7 +72,7 @@ export function RadioButtonGroup<T>({
   const styles = getRadioButtonGroupStyles();
 
   return (
-    <div className={styles.radioGroup}>
+    <div className={cx(styles.radioGroup, className)}>
       {options.map((o, i) => {
         const isItemDisabled = disabledOptions && o.value && disabledOptions.includes(o.value);
         return (

public/app/core/components/TransformersUI/FilterByValueTransformer/FilterByValueFilterEditor.tsx

@@ -0,0 +1,171 @@
+import React, { useCallback } from 'react';
+import { Button, Select } from '@grafana/ui';
+import { Field, SelectableValue, valueMatchers } from '@grafana/data';
+import { FilterByValueFilter } from '@grafana/data/src/transformations/transformers/filterByValue';
+import { valueMatchersUI } from './ValueMatchers/valueMatchersUI';
+
+interface Props {
+  onDelete: () => void;
+  onChange: (filter: FilterByValueFilter) => void;
+  filter: FilterByValueFilter;
+  fieldsInfo: DataFrameFieldsInfo;
+}
+
+export interface DataFrameFieldsInfo {
+  fieldsAsOptions: Array<SelectableValue<string>>;
+  fieldByDisplayName: Record<string, Field>;
+}
+
+export const FilterByValueFilterEditor: React.FC<Props> = props => {
+  const { onDelete, onChange, filter, fieldsInfo } = props;
+  const { fieldsAsOptions, fieldByDisplayName } = fieldsInfo;
+  const fieldName = getFieldName(filter, fieldsAsOptions) ?? '';
+  const field = fieldByDisplayName[fieldName];
+
+  if (!field) {
+    return null;
+  }
+
+  const matcherOptions = getMatcherOptions(field);
+  const matcherId = getSelectedMatcherId(filter, matcherOptions);
+  const editor = valueMatchersUI.getIfExists(matcherId);
+
+  if (!editor || !editor.component) {
+    return null;
+  }
+
+  const onChangeField = useCallback(
+    (selectable?: SelectableValue<string>) => {
+      if (!selectable?.value) {
+        return;
+      }
+      onChange({
+        ...filter,
+        fieldName: selectable.value,
+      });
+    },
+    [onChange, filter]
+  );
+
+  const onChangeMatcher = useCallback(
+    (selectable?: SelectableValue<string>) => {
+      if (!selectable?.value) {
+        return;
+      }
+
+      const id = selectable.value;
+      const options = valueMatchers.get(id).getDefaultOptions(field);
+
+      onChange({
+        ...filter,
+        config: { id, options },
+      });
+    },
+    [onChange, filter, field]
+  );
+
+  const onChangeMatcherOptions = useCallback(
+    options => {
+      onChange({
+        ...filter,
+        config: {
+          ...filter.config,
+          options,
+        },
+      });
+    },
+    [onChange, filter]
+  );
+
+  return (
+    <div className="gf-form-inline">
+      <div className="gf-form gf-form-spacing">
+        <div className="gf-form-label width-7">Field</div>
+        <Select
+          className="min-width-15 max-width-24"
+          placeholder="Field Name"
+          options={fieldsAsOptions}
+          value={filter.fieldName}
+          onChange={onChangeField}
+          menuPlacement="bottom"
+        />
+      </div>
+      <div className="gf-form gf-form-spacing">
+        <div className="gf-form-label">Match</div>
+        <Select
+          className="width-12"
+          placeholder="Select test"
+          options={matcherOptions}
+          value={matcherId}
+          onChange={onChangeMatcher}
+          menuPlacement="bottom"
+        />
+      </div>
+      <div className="gf-form gf-form--grow gf-form-spacing">
+        <div className="gf-form-label">Value</div>
+        <editor.component field={field} options={filter.config.options ?? {}} onChange={onChangeMatcherOptions} />
+      </div>
+      <div className="gf-form">
+        <Button icon="times" onClick={onDelete} variant="secondary" />
+      </div>
+    </div>
+  );
+};
+
+const getMatcherOptions = (field: Field): Array<SelectableValue<string>> => {
+  const options = [];
+
+  for (const matcher of valueMatchers.list()) {
+    if (!matcher.isApplicable(field)) {
+      continue;
+    }
+
+    const editor = valueMatchersUI.getIfExists(matcher.id);
+
+    if (!editor) {
+      continue;
+    }
+
+    options.push({
+      value: matcher.id,
+      label: matcher.name,
+      description: matcher.description,
+    });
+  }
+
+  return options;
+};
+
+const getSelectedMatcherId = (
+  filter: FilterByValueFilter,
+  matcherOptions: Array<SelectableValue<string>>
+): string | undefined => {
+  const matcher = matcherOptions.find(m => m.value === filter.config.id);
+
+  if (matcher && matcher.value) {
+    return matcher.value;
+  }
+
+  if (matcherOptions[0]?.value) {
+    return matcherOptions[0]?.value;
+  }
+
+  return;
+};
+
+const getFieldName = (
+  filter: FilterByValueFilter,
+  fieldOptions: Array<SelectableValue<string>>
+): string | undefined => {
+  const fieldName = fieldOptions.find(m => m.value === filter.fieldName);
+
+  if (fieldName && fieldName.value) {
+    return fieldName.value;
+  }
+
+  if (fieldOptions[0]?.value) {
+    return fieldOptions[0]?.value;
+  }
+
+  return;
+};

public/app/core/components/TransformersUI/FilterByValueTransformer/FilterByValueTransformerEditor.tsx

@@ -0,0 +1,180 @@
+import React, { useMemo, useCallback } from 'react';
+import { css } from 'emotion';
+import {
+  DataTransformerID,
+  standardTransformers,
+  TransformerRegistyItem,
+  TransformerUIProps,
+  getFieldDisplayName,
+  DataFrame,
+  SelectableValue,
+  FieldType,
+  ValueMatcherID,
+  valueMatchers,
+} from '@grafana/data';
+import { Button, RadioButtonGroup, stylesFactory } from '@grafana/ui';
+import cloneDeep from 'lodash/cloneDeep';
+import {
+  FilterByValueFilter,
+  FilterByValueMatch,
+  FilterByValueTransformerOptions,
+  FilterByValueType,
+} from '@grafana/data/src/transformations/transformers/filterByValue';
+
+import { DataFrameFieldsInfo, FilterByValueFilterEditor } from './FilterByValueFilterEditor';
+
+const filterTypes: Array<SelectableValue<FilterByValueType>> = [
+  { label: 'Include', value: FilterByValueType.include },
+  { label: 'Exclude', value: FilterByValueType.exclude },
+];
+
+const filterMatch: Array<SelectableValue<FilterByValueMatch>> = [
+  { label: 'Match all', value: FilterByValueMatch.all },
+  { label: 'Match any', value: FilterByValueMatch.any },
+];
+
+export const FilterByValueTransformerEditor: React.FC<TransformerUIProps<FilterByValueTransformerOptions>> = props => {
+  const { input, options, onChange } = props;
+  const styles = getEditorStyles();
+  const fieldsInfo = useFieldsInfo(input);
+
+  const onAddFilter = useCallback(() => {
+    const frame = input[0];
+    const field = frame.fields.find(f => f.type !== FieldType.time);
+
+    if (!field) {
+      return;
+    }
+
+    const filters = cloneDeep(options.filters);
+    const matcher = valueMatchers.get(ValueMatcherID.greater);
+
+    filters.push({
+      fieldName: getFieldDisplayName(field, frame, input),
+      config: {
+        id: matcher.id,
+        options: matcher.getDefaultOptions(field),
+      },
+    });
+    onChange({ ...options, filters });
+  }, [onChange, options, valueMatchers, input]);
+
+  const onDeleteFilter = useCallback(
+    (index: number) => {
+      let filters = cloneDeep(options.filters);
+      filters.splice(index, 1);
+      onChange({ ...options, filters });
+    },
+    [options, onChange]
+  );
+
+  const onChangeFilter = useCallback(
+    (filter: FilterByValueFilter, index: number) => {
+      let filters = cloneDeep(options.filters);
+      filters[index] = filter;
+      onChange({ ...options, filters });
+    },
+    [options, onChange]
+  );
+
+  const onChangeType = useCallback(
+    (type?: FilterByValueType) => {
+      onChange({
+        ...options,
+        type: type ?? FilterByValueType.include,
+      });
+    },
+    [options, onChange]
+  );
+
+  const onChangeMatch = useCallback(
+    (match?: FilterByValueMatch) => {
+      onChange({
+        ...options,
+        match: match ?? FilterByValueMatch.all,
+      });
+    },
+    [options, onChange]
+  );
+
+  return (
+    <div>
+      <div className="gf-form gf-form-inline">
+        <div className="gf-form-label width-8">Filter type</div>
+        <div className="width-15">
+          <RadioButtonGroup options={filterTypes} value={options.type} onChange={onChangeType} fullWidth />
+        </div>
+      </div>
+      <div className="gf-form gf-form-inline">
+        <div className="gf-form-label width-8">Conditions</div>
+        <div className="width-15">
+          <RadioButtonGroup options={filterMatch} value={options.match} onChange={onChangeMatch} fullWidth />
+        </div>
+      </div>
+      <div className={styles.conditions}>
+        {options.filters.map((filter, idx) => (
+          <FilterByValueFilterEditor
+            key={idx}
+            filter={filter}
+            fieldsInfo={fieldsInfo}
+            onChange={filter => onChangeFilter(filter, idx)}
+            onDelete={() => onDeleteFilter(idx)}
+          />
+        ))}
+        <div className="gf-form">
+          <Button icon="plus" size="sm" onClick={onAddFilter} variant="secondary">
+            Add condition
+          </Button>
+        </div>
+      </div>
+    </div>
+  );
+};
+
+export const filterByValueTransformRegistryItem: TransformerRegistyItem<FilterByValueTransformerOptions> = {
+  id: DataTransformerID.filterByValue,
+  editor: FilterByValueTransformerEditor,
+  transformation: standardTransformers.filterByValueTransformer,
+  name: standardTransformers.filterByValueTransformer.name,
+  description:
+    'Removes rows of the query results using user definied filters. This is useful if you can not filter your data in the data source.',
+};
+
+const getEditorStyles = stylesFactory(() => ({
+  conditions: css`
+    padding-left: 16px;
+  `,
+}));
+
+const useFieldsInfo = (data: DataFrame[]): DataFrameFieldsInfo => {
+  return useMemo(() => {
+    const meta = {
+      fieldsAsOptions: [],
+      fieldByDisplayName: {},
+    };
+
+    if (!Array.isArray(data)) {
+      return meta;
+    }
+
+    return data.reduce((meta: DataFrameFieldsInfo, frame) => {
+      return frame.fields.reduce((meta, field) => {
+        const fieldName = getFieldDisplayName(field, frame, data);
+
+        if (meta.fieldByDisplayName[fieldName]) {
+          return meta;
+        }
+
+        meta.fieldsAsOptions.push({
+          label: fieldName,
+          value: fieldName,
+          type: field.type,
+        });
+
+        meta.fieldByDisplayName[fieldName] = field;
+
+        return meta;
+      }, meta);
+    }, meta);
+  }, [data]);
+};

public/app/core/components/TransformersUI/FilterByValueTransformer/ValueMatchers/BasicMatcherEditor.tsx

@@ -0,0 +1,104 @@
+import React, { useCallback, useState } from 'react';
+import { Input } from '@grafana/ui';
+import { ValueMatcherID, BasicValueMatcherOptions } from '@grafana/data';
+import { ValueMatcherEditorConfig, ValueMatcherUIProps, ValueMatcherUIRegistryItem } from './types';
+import { convertToType } from './utils';
+
+export function basicMatcherEditor<T = any>(
+  config: ValueMatcherEditorConfig
+): React.FC<ValueMatcherUIProps<BasicValueMatcherOptions<T>>> {
+  return ({ options, onChange, field }) => {
+    const { validator, converter = convertToType } = config;
+    const { value } = options;
+    const [isInvalid, setInvalid] = useState(!validator(value));
+
+    const onChangeValue = useCallback(
+      (event: React.FormEvent<HTMLInputElement>) => {
+        setInvalid(!validator(event.currentTarget.value));
+      },
+      [setInvalid, validator]
+    );
+
+    const onChangeOptions = useCallback(
+      (event: React.FocusEvent<HTMLInputElement>) => {
+        if (isInvalid) {
+          return;
+        }
+
+        const { value } = event.currentTarget;
+
+        onChange({
+          ...options,
+          value: converter(value, field),
+        });
+      },
+      [options, onChange, isInvalid, field, converter]
+    );
+
+    return (
+      <Input
+        className="flex-grow-1"
+        invalid={isInvalid}
+        defaultValue={String(options.value)}
+        placeholder="Value"
+        onChange={onChangeValue}
+        onBlur={onChangeOptions}
+      />
+    );
+  };
+}
+
+export const getBasicValueMatchersUI = (): Array<ValueMatcherUIRegistryItem<BasicValueMatcherOptions>> => {
+  return [
+    {
+      name: 'Is greater',
+      id: ValueMatcherID.greater,
+      component: basicMatcherEditor<number>({
+        validator: value => !isNaN(value),
+      }),
+    },
+    {
+      name: 'Is greater or equal',
+      id: ValueMatcherID.greaterOrEqual,
+      component: basicMatcherEditor<number>({
+        validator: value => !isNaN(value),
+      }),
+    },
+    {
+      name: 'Is lower',
+      id: ValueMatcherID.lower,
+      component: basicMatcherEditor<number>({
+        validator: value => !isNaN(value),
+      }),
+    },
+    {
+      name: 'Is lower or equal',
+      id: ValueMatcherID.lowerOrEqual,
+      component: basicMatcherEditor<number>({
+        validator: value => !isNaN(value),
+      }),
+    },
+    {
+      name: 'Is equal',
+      id: ValueMatcherID.equal,
+      component: basicMatcherEditor<any>({
+        validator: () => true,
+      }),
+    },
+    {
+      name: 'Is not equal',
+      id: ValueMatcherID.notEqual,
+      component: basicMatcherEditor<any>({
+        validator: () => true,
+      }),
+    },
+    {
+      name: 'Regex',
+      id: ValueMatcherID.regex,
+      component: basicMatcherEditor<string>({
+        validator: () => true,
+        converter: (value: any) => String(value),
+      }),
+    },
+  ];
+};

public/app/core/components/TransformersUI/FilterByValueTransformer/ValueMatchers/NoopMatcherEditor.tsx

@@ -0,0 +1,22 @@
+import { ValueMatcherID } from '@grafana/data';
+import React from 'react';
+import { ValueMatcherUIProps, ValueMatcherUIRegistryItem } from './types';
+
+export const NoopMatcherEditor: React.FC<ValueMatcherUIProps<any>> = () => {
+  return null;
+};
+
+export const getNoopValueMatchersUI = (): Array<ValueMatcherUIRegistryItem<any>> => {
+  return [
+    {
+      name: 'Is null',
+      id: ValueMatcherID.isNull,
+      component: NoopMatcherEditor,
+    },
+    {
+      name: 'Is not null',
+      id: ValueMatcherID.isNotNull,
+      component: NoopMatcherEditor,
+    },
+  ];
+};

public/app/core/components/TransformersUI/FilterByValueTransformer/ValueMatchers/RangeMatcherEditor.tsx

@@ -0,0 +1,81 @@
+import React, { useCallback, useState } from 'react';
+import { Input } from '@grafana/ui';
+import { ValueMatcherID, RangeValueMatcherOptions } from '@grafana/data';
+import { ValueMatcherEditorConfig, ValueMatcherUIProps, ValueMatcherUIRegistryItem } from './types';
+import { convertToType } from './utils';
+
+type PropNames = 'from' | 'to';
+
+export function rangeMatcherEditor<T = any>(
+  config: ValueMatcherEditorConfig
+): React.FC<ValueMatcherUIProps<RangeValueMatcherOptions<T>>> {
+  return ({ options, onChange, field }) => {
+    const { validator } = config;
+    const [isInvalid, setInvalid] = useState({
+      from: !validator(options.from),
+      to: !validator(options.to),
+    });
+
+    const onChangeValue = useCallback(
+      (event: React.FormEvent<HTMLInputElement>, prop: PropNames) => {
+        setInvalid({
+          ...isInvalid,
+          [prop]: !validator(event.currentTarget.value),
+        });
+      },
+      [setInvalid, validator, isInvalid]
+    );
+
+    const onChangeOptions = useCallback(
+      (event: React.FocusEvent<HTMLInputElement>, prop: PropNames) => {
+        if (isInvalid[prop]) {
+          return;
+        }
+
+        const { value } = event.currentTarget;
+
+        onChange({
+          ...options,
+          [prop]: convertToType(value, field),
+        });
+      },
+      [options, onChange, isInvalid, field]
+    );
+
+    return (
+      <>
+        <Input
+          className="flex-grow-1 gf-form-spacing"
+          invalid={isInvalid['from']}
+          defaultValue={String(options.from)}
+          placeholder="From"
+          onChange={event => onChangeValue(event, 'from')}
+          onBlur={event => onChangeOptions(event, 'from')}
+        />
+        <div className="gf-form-label">and</div>
+        <Input
+          className="flex-grow-1"
+          invalid={isInvalid['to']}
+          defaultValue={String(options.to)}
+          placeholder="To"
+          onChange={event => onChangeValue(event, 'to')}
+          onBlur={event => onChangeOptions(event, 'to')}
+        />
+      </>
+    );
+  };
+}
+
+export const getRangeValueMatchersUI = (): Array<ValueMatcherUIRegistryItem<RangeValueMatcherOptions>> => {
+  return [
+    {
+      name: 'Is between',
+      id: ValueMatcherID.between,
+      component: rangeMatcherEditor<number>({
+        validator: value => {
+          return !isNaN(value);
+        },
+      }),
+    },
+  ];
+};

public/app/core/components/TransformersUI/FilterByValueTransformer/ValueMatchers/types.ts

@@ -0,0 +1,14 @@
+import { Field, RegistryItem } from '@grafana/data';
+export interface ValueMatcherUIRegistryItem<TOptions> extends RegistryItem {
+  component: React.ComponentType<ValueMatcherUIProps<TOptions>>;
+}
+
+export interface ValueMatcherUIProps<TOptions> {
+  options: TOptions;
+  onChange: (options: TOptions) => void;
+  field: Field;
+}
+export interface ValueMatcherEditorConfig {
+  validator: (value: any) => boolean;
+  converter?: (value: any, field: Field) => any;
+}

public/app/core/components/TransformersUI/FilterByValueTransformer/ValueMatchers/utils.ts

@@ -0,0 +1,34 @@
+import { Field, FieldType } from '@grafana/data';
+import { isString, isUndefined } from 'lodash';
+
+export function convertToType(value: any, field: Field): any {
+  switch (field.type) {
+    case FieldType.boolean:
+      if (isUndefined(value)) {
+        return false;
+      }
+      return convertToBool(value);
+
+    case FieldType.number:
+      if (isNaN(value)) {
+        return 0;
+      }
+      return parseFloat(value);
+
+    case FieldType.string:
+      if (!value) {
+        return '';
+      }
+      return String(value);
+
+    default:
+      return value;
+  }
+}
+
+const convertToBool = (value: any): boolean => {
+  if (isString(value)) {
+    return !(value[0] === 'F' || value[0] === 'f' || value[0] === '0');
+  }
+  return !!value;
+};

public/app/core/components/TransformersUI/FilterByValueTransformer/ValueMatchers/valueMatchersUI.ts

@@ -0,0 +1,9 @@
+import { Registry } from '@grafana/data';
+import { getBasicValueMatchersUI } from './BasicMatcherEditor';
+import { getNoopValueMatchersUI } from './NoopMatcherEditor';
+import { getRangeValueMatchersUI } from './RangeMatcherEditor';
+import { ValueMatcherUIRegistryItem } from './types';
+
+export const valueMatchersUI = new Registry<ValueMatcherUIRegistryItem<any>>(() => {
+  return [...getBasicValueMatchersUI(), ...getNoopValueMatchersUI(), ...getRangeValueMatchersUI()];
+});

public/app/core/utils/standardTransformers.ts

@@ -2,6 +2,7 @@ import { TransformerRegistyItem } from '@grafana/data';
 import { reduceTransformRegistryItem } from '../components/TransformersUI/ReduceTransformerEditor';
 import { filterFieldsByNameTransformRegistryItem } from '../components/TransformersUI/FilterByNameTransformerEditor';
 import { filterFramesByRefIdTransformRegistryItem } from '../components/TransformersUI/FilterByRefIdTransformerEditor';
+import { filterByValueTransformRegistryItem } from '../components/TransformersUI/FilterByValueTransformer/FilterByValueTransformerEditor';
 import { organizeFieldsTransformRegistryItem } from '../components/TransformersUI/OrganizeFieldsTransformerEditor';
 import { seriesToFieldsTransformerRegistryItem } from '../components/TransformersUI/SeriesToFieldsTransformerEditor';
 import { calculateFieldTransformRegistryItem } from '../components/TransformersUI/CalculateFieldTransformerEditor';
@@ -16,6 +17,7 @@ export const getStandardTransformers = (): Array<TransformerRegistyItem<any>> =>
     reduceTransformRegistryItem,
     filterFieldsByNameTransformRegistryItem,
     filterFramesByRefIdTransformRegistryItem,
+    filterByValueTransformRegistryItem,
     organizeFieldsTransformRegistryItem,
     seriesToFieldsTransformerRegistryItem,
     seriesToRowsTransformerRegistryItem,