Docs: Settings updates at runtime (#34169)

* Docs: Settings updates on runtime docs skeleton

* Docs: Settings HTTP API

* Docs: Minor changes on settings updates at runtime docs

* Docs: Settings updates at runtime description

* Docs: Minor fix

* Docs: Move PUT /api/admin/settings docs into Admin API page

* Docs: Added longer explanation into 'Settings updates at runtime' page

* Apply suggestions from code review

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: Leonard Gram <leo@xlson.com>

* Docs: Include order of precedence description at settings updates page

* Update docs/sources/http_api/admin.md

* Docs: Add link to main configuration page from Settings updates at runtime

* Apply suggestions from code review

Co-authored-by: Mitch Seaman <mjseaman@users.noreply.github.com>

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
Co-authored-by: Mitch Seaman <mjseaman@users.noreply.github.com>

docs/sources/enterprise/_index.md

@@ -52,6 +52,7 @@ With Grafana Enterprise, you get access to new features, including:
 - [Vault integration]({{< relref "vault.md" >}}) to manage your configuration or provisioning secrets with Vault.
 - [Auditing]({{< relref "auditing.md" >}}) tracks important changes to your Grafana instance to help you manage and mitigate suspicious activity and meet compliance requirements.
 - [Request security]({{< relref "request-security.md" >}}) makes it possible to restrict outgoing requests from the Grafana server.
+- [Settings updates at runtime]({{< relref "settings-updates.md" >}}) allows you to update Grafana settings at runtime without requiring a restart.
 
 ## Enterprise data sources
 
@@ -75,4 +76,3 @@ With a Grafana Enterprise license, you get access to premium data sources, inclu
 ## Try Grafana Enterprise
 
 To purchase or obtain a trial license contact the Grafana Labs [Sales Team](https://grafana.com/contact?about=support&topic=Grafana%20Enterprise).
-

docs/sources/enterprise/license/license-expiration.md

@@ -77,3 +77,7 @@ Auditing is not affected by an expired license.
 The concurrent session limit remains active for seven days after the expiration date, after which it will be turned off.
 
 The active users limit is turned off immediately.
+
+### Settings updates at runtime
+
+Settings updates at runtime are not affected by an expired license.

docs/sources/enterprise/settings-updates.md

@@ -0,0 +1,80 @@
++++
+title = "Settings updates at runtime"
+description = "Settings updates at runtime"
+keywords = ["grafana", "runtime", "settings"]
+weight = 500
++++
+
+# Settings updates at runtime
+
+> **Note:** Available in Grafana Enterprise v8.0+.
+
+Settings updates at runtime allows you to update Grafana settings with no need to restart the Grafana server.
+
+Updates that happen at runtime are stored in the database and override
+[settings from the other sources](https://grafana.com/docs/grafana/latest/administration/configuration/)
+(arguments, environment variables, settings file, etc). Therefore, every time a specific setting key is removed at runtime,
+the value used for that key is the inherited one from the other sources in the reverse order of precedence 
+(`arguments > environment variables > settings file`), being the application default the value used when no one provided
+through one of these, at least.
+
+Currently, **it only supports updates on the `auth.saml` section.** 
+
+## Update settings via the API
+
+You can update settings through the [Admin API]({{< relref "../http_api/admin.md#update-settings" >}}).
+
+When you submit a settings update via API, Grafana verifies if the given settings updates are allowed and valid. If they are, then Grafana stores the settings in the database and reloads
+Grafana services with no need to restart the instance.
+
+So, the payload of a `PUT` request to the update settings endpoint (`/api/admin/settings`) 
+should contain (either one or both):
+- An `updates` map with a key, and a value per section you want to set.
+- A `removals` list with keys per section you want to unset.
+
+For example, if you provide the following `updates`:
+
+```json
+{
+  "auth.saml": {
+    "enabled": "true",
+    "single_logout": "false"
+}
+```
+it would enable SAML and disable single logouts. And, if you provide the following `removals`:
+```json
+{
+  "auth.saml": ["allow_idp_initiated"]
+}
+```
+
+it would remove the key/value setting identified by `allow_idp_initiated` within the `auth.saml`.
+So, the SAML service would be reloaded and that value would be inherited for either (settings `.ini` file,
+environment variable, command line arguments or any other accepted mechanism to provide configuration).
+
+Therefore, the complete HTTP payload would looks like:
+
+```json
+{
+  "updates": {
+    "auth.saml": {
+      "enabled": "true",
+      "single_logout": "false"
+    }
+  },
+  "removals": {
+    "auth.saml": ["allow_idp_initiated"]
+  }
+}
+```
+
+In case any of these settings cannot be overridden nor valid, it would return an error and these settings
+won't be persisted into the database.
+
+## Background job (high availability set-ups)
+
+Grafana Enterprise has a built-in scheduled background job that looks into the database every minute for
+settings updates. If there are updates, it reloads the Grafana services affected by the detected changes. 
+
+The background job synchronizes settings between instances in high availability set-ups. So, after you perform some changes through the
+HTTP API, then the other instances are synchronized through the database and the background job.

docs/sources/http_api/admin.md

@@ -14,7 +14,7 @@ must have the Grafana Admin permission. (The default admin user is called `admin
 > If you are running Grafana Enterprise and have [Fine-grained access control]({{< relref "../enterprise/access-control/_index.md" >}}) enabled, for some endpoints you would need to have relevant permissions.
 Refer to specific resources to understand what permissions are required.
 
-## Settings
+## Fetch settings
 
 `GET /api/admin/settings`
 
@@ -172,6 +172,57 @@ Content-Type: application/json
   }
 }
 ```
+
+## Update settings
+
+`PUT /api/admin/settings`
+
+> **Note:** Available in Grafana Enterprise v8.0+.
+
+Updates / removes and reloads database settings. You must provide either `updates`, `removals` or both.
+
+This endpoint only supports changes to `auth.saml` configuration.
+
+**Example request:**
+
+```http
+PUT /api/admin/settings
+Accept: application/json
+Content-Type: application/json
+Authorization: Bearer eyJrIjoiT0tTcG1pUlY2RnVKZTFVaDFsNFZXdE9ZWmNrMkZYbk
+
+{
+  "updates": {
+    "auth.saml": {
+      "enabled": "true"
+    }
+  },
+  "removals": {
+    "auth.saml": ["single_logout"]
+  },
+}
+```
+
+**Example response:**
+
+```http
+HTTP/1.1 200 OK
+Content-Type: application/json
+Content-Length: 32
+
+{
+  "message":"Settings updated"
+}
+```
+
+Status codes:
+
+- **200** - OK
+- **400** - Bad Request
+- **401** - Unauthorized
+- **403** - Forbidden
+- **500** - Internal Server Error
+
 ## Grafana Stats
 
 `GET /api/admin/stats`

public/app/features/admin/UpgradePage.tsx

@@ -130,6 +130,7 @@ const FeatureListing: React.FC = () => {
       <Item title="Team Sync">LDAP, GitHub OAuth, Auth Proxy, Okta</Item>
       <Item title="White labeling" />
       <Item title="Auditing" />
+      <Item title="Settings updates at runtime" />
       <Item title="Grafana usage insights">
         <List nested={true}>
           <Item title="Sort dashboards by popularity in search" />