Chore: Add CI Pipeline to generate Grafana's OpenAPI specification (#75393)

* chore: move over initial changes from sofia's pr #58029

* chore: remove go_image

* chore: begin removing edition, remove unused imports

* chore: remove edition from swagger_gen.star and generate .drone.yml

* chore: regen drone.yml

* fix: fix order of load statements

* fix: try #2 fix order of load statements

* linter fixes

* chore: add doc comment explaining purpose of new clone_pr_branch step

* fix: add placeholder documentation for ver_mode arg

* attempt #1 to import and use clone_enterprise_step_pr

* Update scripts/drone/pipelines/swagger_gen.star

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* attempt #2 to import and use clone_enterprise_step_pr

* fix: fix drone lint err invalid or unknown step dependency

* fix: regen hmac to make drone run

* fix: fix drone lint err

* fix: update swagger-clean cmd in step

* attempt to return non zero exit code

* test to see if pipeline fails

* fix: add git to clone pr branch step

* fix: add git and make to swagger-gen step

* try to rerun drone

* debug: figure out why cannot find make swagger-clean

* debug: see if cd grafana/grafana fixes things

* debug: undo cd grafana/grafana

* debug: add more logging statements

* debug: try and remove cd grafana in swagger-gen

* debug: removed grafana after clone statement; add debug before cloning

* fix: remove disable clone

* regen specs to see if swagger-gen step passes now

* add descriptive error message to swagger-gen step

* remove api-spec.json from .gitignore

* revert backend change, regen spec

* add back backend change, regen specs

* Update scripts/drone/pipelines/swagger_gen.star

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Update scripts/drone/pipelines/swagger_gen.star

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* revert gitignore change, set enterprise source to drone source branch

* chore: remove unused variable, change committish var name to source

* testing functionality: make a new BE change without gen spec, pipeline should fail

* test functionality: does removing err msg cause step to fail properly

* test functionality: add back err msg and && after

* chore: remove debug statements from swagger gen step

* chore: remove debug lines from clone_pr_branch step

* test functionality: regen specs, swagger_gen step should pass

* test funcionality: regen specs again ????

* chore: update swagger-gen step err msg

* test functionality: make BE change dont regen spec, swagger gen should fail

* test functionality: regen the specs, swagger-gen should pass

* chore: revert test BE change, regen spec

* chore: remove unused clone step

* chore: regen drone.yml

---------

Co-authored-by: Timur Olzhabayev <timur.olzhabayev@grafana.com>
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

scripts/drone/events/pr.star

@@ -32,6 +32,10 @@ load(
     "scripts/drone/pipelines/shellcheck.star",
     "shellcheck_pipeline",
 )
+load(
+    "scripts/drone/pipelines/swagger_gen.star",
+    "swagger_gen",
+)
 load(
     "scripts/drone/pipelines/test_backend.star",
     "test_backend",
@@ -137,6 +141,10 @@ def pr_pipelines():
         ),
         docs_pipelines(ver_mode, trigger_docs_pr()),
         shellcheck_pipeline(),
+        swagger_gen(
+            get_pr_trigger(include_paths = ["pkg/**"]),
+            ver_mode,
+        ),
         integration_benchmarks(
             prefix = ver_mode,
         ),

scripts/drone/pipelines/swagger_gen.star

@@ -0,0 +1,56 @@
+"""
+This module returns all pipelines used in OpenAPI specification generation of Grafana HTTP APIs
+"""
+
+load(
+    "scripts/drone/steps/lib.star",
+    "clone_enterprise_step_pr",
+)
+load(
+    "scripts/drone/utils/images.star",
+    "images",
+)
+load(
+    "scripts/drone/utils/utils.star",
+    "pipeline",
+)
+load(
+    "scripts/drone/vault.star",
+    "from_secret",
+)
+
+def swagger_gen_step(ver_mode):
+    if ver_mode != "pr":
+        return None
+
+    return {
+        "name": "swagger-gen",
+        "image": images["go"],
+        "environment": {
+            "GITHUB_TOKEN": from_secret("github_token"),
+        },
+        "commands": [
+            "apk add --update git make",
+            "make swagger-clean && make openapi3-gen",
+            "for f in public/api-merged.json public/openapi3.json; do git add $f; done",
+            'if [ -z "$(git diff --name-only --cached)" ]; then echo "Everything seems up to date!"; else echo "Please ensure the branch is up-to-date, then regenerate the specification by running make swagger-clean && make openapi3-gen" && return 1; fi',
+        ],
+        "depends_on": [
+            "clone-enterprise",
+        ],
+    }
+
+def swagger_gen(trigger, ver_mode, source = "${DRONE_SOURCE_BRANCH}"):
+    test_steps = [
+        clone_enterprise_step_pr(source = source),
+        swagger_gen_step(ver_mode = ver_mode),
+    ]
+
+    p = pipeline(
+        name = "{}-swagger-gen".format(ver_mode),
+        trigger = trigger,
+        services = [],
+        steps = test_steps,
+    )
+
+    return p