Commit Graph
61 Commits
Author SHA1 Message Date
Ganesh Vernekar 10f8a138ae Alerting: Deduplicate receivers during migration (#36812)
* Alerting: Deduplicate receivers during migration

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Better naming of contact points

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Handle some edge cases

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Don't add routes which point to default contact point

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-29 15:56:19 +02:00
Ganesh Vernekar e8ac802e4f Alerting: Remove unused fields in Pagerduty struct (#37198)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-27 10:41:48 +05:30
Ganesh Vernekar a65975cca0 Alerting: Remove the fixed wait for notification delivery (#37203)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-26 15:15:09 +02:00
Ganesh Vernekar 5806c66825 Alerting: Fix the table markdown docs (#36901)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-20 15:37:19 +05:30
Ganesh Vernekar 34699713ea Alerting: Update docs to mention the requirement of persistent disks (#36808)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-19 17:51:33 +02:00
Ganesh Vernekar 8efe1856e2 Alerting: A better and cleaner way to know if Alertmanager is initialised (#36659)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-12 18:53:01 +05:30
Ganesh Vernekar e19c690426 Alerting: Fix potential panic in Alertmanager when starting up (#36562)
* Alerting: Fix potential panic in Alertmanager when starting up

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix reviews

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-12 11:15:16 +05:30
Ganesh Vernekar 94d2520a84 Alerting: Allow space in label and annotation names (#36549)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-08 18:26:09 +05:30
Ganesh Vernekar 8fe58fc2e3 Alerting: Add additional newlines to Microsoft Teams notification message where necessary (#36126)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-24 20:52:00 +05:30
Ganesh Vernekar f2bb3faea8 Alerting: Add integration test case for email channel (#36029)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-24 20:31:29 +05:30
Ganesh Vernekar 9a5c1f06df Alerting: Template all possible variables in notification channels (#35749)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-22 15:12:54 +05:30
Ganesh Vernekar 33d6e11175 Alerting: Decouple default template from channel tests (#35239)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-21 07:59:09 +05:30
Ganesh Vernekar dcd4bf1615 Alerting: Fill the empty GeneratorURL (#35740)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-16 15:34:12 +05:30
Ganesh Vernekar 673b03671d Docs: Fix template example for unified alerting (#35620)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-14 17:38:24 +05:30
Ganesh Vernekar de3bcb6071 Alerting: Update docs for templating in rule labels (#35238)
* Alerting: Update docs for templating in rule labels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix comment

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-04 17:46:05 +05:30
Ganesh Vernekar 8417088969 Alerting: Expand {{$labels.xyz}} template in labels and annotations (#35159)
* Alerting: Expand `{{$labels.xyz}}` template in labels and annotations

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix annotation not updating for same alert

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-03 19:24:36 +02:00
Ganesh Vernekar a30e60a0b8 Alerting: Do not hard fail on templating errors in channels (#35165)
* Alerting: Do not hard fail on templating errors in channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-03 19:39:32 +05:30
Ganesh Vernekar aa4c5bbfe4 Alerting: Don't save Alertmanager config on migration when 0 channels (#35119)
* Alerting: Don't save Alertmanager config on migration when 0 channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-03 15:07:34 +02:00
Ganesh Vernekar a23674ef99 Alerting: Migrate tags as labels and not annotations (#34990)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-31 19:47:17 +05:30
Ganesh Vernekar 7c25465b3a Alerting: Migrate paused alerts to silences (#34898)
* Alerting: Migrate paused alerts to silences

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix lint

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-31 17:30:58 +05:30
Ganesh Vernekar ad6648b649 Alerting: Do no hard fail on finding discontinued channels on migration (#34894)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-28 16:06:30 +02:00
Ganesh Vernekar 8278c7bf77 Alerting: Fix flaky TestNotificationChannels take 2 (#34880)
* Alerting: Fix flaky TestNotificationChannels take 2

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix nits

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix lint

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-28 15:24:18 +05:30
Ganesh Vernekar 89262ac64d Fix flaky TestNotificationChannels (#34831)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-28 11:03:49 +05:30
Ganesh Vernekar d69c21acb6 NGAlert: Update the default template to include more URLs (#34715)
* NGAlert: Update the default template to include more URLs

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-26 16:49:39 +02:00
Ganesh Vernekar b168223029 NGAlert: Add integration tests for remaining notification channels (#34662)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-26 16:33:55 +05:30
Ganesh Vernekar 95ee5f01b5 NGAlert: Do not pick a random default channel if no default exists (#34465)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-20 10:30:09 +02:00
Ganesh Vernekar e52f718a87 NGAlert: Migrate old notification channel config (#33846)
* NGAlert: Migrate old notification channel config

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Copy the required structs and functions

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix lint

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments and error on discontinued channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Use both ID and UID to map channels, attach UID to channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix lint

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Bug fixes

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Use alert table instead of dashboard for channel IDs

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix offline reviews: don't add empty receivers and routes, add receiver for unmigrated channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Handle multiple default channels, pick default from used channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Include default channels in all receivers

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 21:10:12 +02:00
Ganesh Vernekar 533be16787 NGAlert: Add Threema notification channel (#34159)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 20:20:52 +02:00
Ganesh Vernekar b2e84277a3 NGAlert: Add Kafka notification channel (#34156)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 20:02:09 +02:00
Ganesh Vernekar ad1d0ae0bf NGAlert: Add VictorOps notification channel (#34161)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 19:52:14 +02:00
Ganesh Vernekar fb9223ab42 NGAlert: Add Line notification channel (#34157)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 19:04:48 +02:00
Ganesh Vernekar 01e0faf800 NGAlert: Add GoogleChat notification channel (#34153)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-19 18:24:04 +02:00
Ganesh Vernekar 89c2b5e863 NGAlert: Remove unwanted fields from notification channel config (#34036)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-18 10:04:47 +02:00
Ganesh Vernekar d5ae55c5dd NGAlert: Add message field to email notification channel (#34044)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-17 16:05:09 +05:30
Ganesh Vernekar ec3214bac2 NGAlert: Add integration tests for notification channels (#33431)
* NGAlert: Add integration tests for notification channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix the failing tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Override creation of rule UID, remove only namespace UID

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-13 22:58:19 +05:30
Ganesh Vernekar 5f44ccff0c NGAlert: Fix unit test to write files in temporary directory (#34032)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-13 16:08:12 +05:30
Ganesh Vernekar 8d442c9b44 NGAlert: Fix templating and remove unwanted default templates (#33918)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-12 15:13:43 +05:30
Ganesh Vernekar 1b8c0ce88b NGAlert: Fix some TODOs in notification channels (#33739)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-05 17:48:40 +05:30
Ganesh Vernekar 918552d34b NGAlert: Send list of available ngalert notification channels via API (#33489)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-04 13:58:39 +02:00
Ganesh Vernekar 467ab124dd NGAlert: Fix GET for Alertmanager config (#33379)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-27 20:48:19 +05:30
Ganesh Vernekar be1affe0a4 NGAlert: Fix flaky test (#33415)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-27 17:03:22 +05:30
Ganesh Vernekar 659ea20c3c NGAlert: Run the maintenance cycle for the silences (#33301)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-23 16:19:03 +02:00
Ganesh Vernekar d66a5e65a4 AlertingNG: Add webhook notification channel (#33229)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-23 18:59:28 +05:30
Ganesh Vernekar a0e567f80f AlertingNG: Add Dingding notification channel (#32995)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-22 19:30:49 +02:00
Ganesh Vernekar 4ec1edfca3 AlertingNG: Add Teams notification channel (#32979)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-22 18:16:26 +02:00
Ganesh Vernekar c9cd7ea701 AlertingNG: Add Telegram notification channel (#32795)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-22 17:24:59 +02:00
Ganesh Vernekar 0a03d5c29e AlertingNG: Correctly set StartsAt, EndsAt, UpdatedAt after alert reception (#33109)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-22 20:42:18 +05:30
Ganesh VernekarandJosue Abreu 3056f86f76 AlertingNG: Fix TODOs in email notification channel (#33169)
* AlertingNG: Fix TODOs in email notification channel

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Test fixup

* Remove the receiver field it is not needed for the email notification

Co-authored-by: Josue Abreu <josue@grafana.com>
2021-04-22 10:01:55 -04:00
Ganesh Vernekar 6cc7a20789 NGAlert: Remove all pending references to the old API package (#33269)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-22 18:36:32 +05:30
Ganesh Vernekar 6271777ec6 AlertingNG: Remove the receivers field from postable alerts (#33068)
* AlertingNG: Remove the receivers field from postable alerts and update tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-19 12:28:44 +05:30
Ganesh Vernekar 04a8d5407e AlertingNG: Slack notification channel (#32675)
* AlertingNG: Slack notification channel

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Add tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments and small refactoring

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-15 16:01:41 +05:30
Ganesh Vernekar e3a1d3d158 AlertingNG: PagerDuty notification channel (#32604)
* AlertingNG: PagerDuty notification channel

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Add tests

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix lint

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix reviews

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-08 19:21:09 +02:00
Ganesh Vernekar b1c84c795f AlertingNG: Add a global registry for notification channels (#32781)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-08 22:01:23 +05:30
Ganesh Vernekar 0f7d8ae6d2 Update email template for AlertingNG (#32691)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-04-07 14:52:48 +05:30
Ganesh Vernekar 740c5813d4 AlertingNG: Fix dispatcher metrics in notifier (#32434)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-03-29 20:35:15 +05:30
Ganesh Vernekar a0db4dce32 Render new email template and fix the title (#32314)
* Render new email template and fix the title

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix nit

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-03-26 12:53:14 +05:30
Ganesh Vernekar 093e5947f4 Upgrade Prometheus Alertmanager and small fixes (#32280)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-03-25 12:51:44 +01:00
Ganesh Vernekar 8854001b67 AlertingNG: Refactor notifier to support config reloads (#32099)
* AlertingNG: Refactor notifier to support config reloads

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments and make reloading of config a sync operation

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review comments

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-03-19 09:26:00 +01:00
Ganesh Vernekar 0b788b5ce8 AlertingNG: Notification channel for emails (#31768)
* Email notification channel in ngalert

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Use existing templating system

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Update template and add unit tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2021-03-18 14:55:11 +00:00
Ganesh Vernekar 974ccf8091 AlertingNG: Fix the alerting stage for legacy alerts (#32025)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
2021-03-17 16:38:33 +00:00
Ganesh VernekarandJosue Abreu ecbc98ba5d AlertingNG: Add alert provider and basic structure with dispatcher, silences and delivery stages (#31833)
* AlertingNG: Add alert provider

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Add unit tests

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Alertmanager WIP

* Merge alertmanager into notifier

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fixes for PR 31833 (#31990)

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Use alertmanager from upgrad-uuid temporarily to unblock

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

* Fix lint

Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>

Co-authored-by: Josue Abreu <josue@grafana.com>
2021-03-16 10:44:52 +01:00