Alerting: Respect "For" Duration for NoData alerts (#65574)

* Alerting: Respect "For" Duration for NoData alerts

This change modifies `resultNoData` to be more inline with the logic of the other state handlers.

The main effects of this are:

1) NoData states with NoDataState config set to Alerting will respect "For" duration.
2) Prevents zero value in StartsAt and EndsAt for alerts that have only even been in normal state. This includes state transitions from NoDataState=OK and ExecErrState=OK.
3) Better state transition logging.
This commit is contained in:
Matthew Jacobson
2023-03-31 12:05:15 -04:00
committed by GitHub
parent ed8628e39d
commit b9dc04139a
4 changed files with 152 additions and 104 deletions

View File

@@ -126,8 +126,8 @@ func (rs *ruleStates) getOrCreate(ctx context.Context, log log.Logger, alertRule
return state
}
// If the first result we get is alerting, set StartsAt to EvaluatedAt because we
// do not have data for determining StartsAt otherwise
// For new states, we set StartsAt & EndsAt to EvaluatedAt as this is the
// expected value for a Normal state during state transition.
newState := &State{
AlertRuleUID: alertRule.UID,
OrgID: alertRule.OrgID,
@@ -136,9 +136,8 @@ func (rs *ruleStates) getOrCreate(ctx context.Context, log log.Logger, alertRule
Annotations: annotations,
EvaluationDuration: result.EvaluationDuration,
Values: values,
}
if result.State == eval.Alerting {
newState.StartsAt = result.EvaluatedAt
StartsAt: result.EvaluatedAt,
EndsAt: result.EvaluatedAt,
}
rs.states[id] = newState
return newState