Merge branch 'main' into rt-backport-changelog

CHANGELOG.md

@@ -6,11 +6,17 @@ UPGRADE NOTES:
 * When making outgoing HTTPS or other TLS connections as a client, Terraform now requires the server to support TLS v1.2. TLS v1.0 and v1.1 are no longer supported. Any safely up-to-date server should support TLS 1.2, and mainstream web browsers have required it since 2020.
 * When making outgoing HTTPS or other TLS connections as a client, Terraform will no longer accept CA certificates signed using the SHA-1 hash function. Publicly trusted Certificate Authorities have not issued SHA-1 certificates since 2015.
 
-(Note: the changes to Terraform's requirements when interacting with TLS servers apply only to requests made by Terraform CLI itself, such as provider/module installation and state storage requests. Terraform provider plugins include their own TLS clients which may have different requirements, and may add new requirements in their own releases, independently of Terraform CLI changes.)
+    (Note: the changes to Terraform's requirements when interacting with TLS servers apply only to requests made by Terraform CLI itself, such as provider/module installation and state storage requests. Terraform provider plugins include their own TLS clients which may have different requirements, and may add new requirements in their own releases, independently of Terraform CLI changes.)
+* If you use the [third-party credentials helper plugin terraform-credentials-env](https://github.com/apparentlymart/terraform-credentials-env), you should disable it as part of upgrading to Terraform v1.2 because similar functionality is now built in to Terraform itself.
+
+    The new behavior supports the same environment variable naming scheme but has a difference in priority order from the credentials helper: `TF_TOKEN_...` environment variables will now take priority over credentials blocks in CLI configuration and credentials stored automatically by terraform login, which is not true for credentials provided by any credentials helper plugin. If you see Terraform using different credentials after upgrading, check to make sure you do not specify credentials for the same host in multiple locations.
+
+    If you use the credentials helper in conjunction with the [hashicorp/tfe](https://registry.terraform.io/providers/hashicorp/tfe) Terraform provider to manage Terraform Cloud or Terraform Enterprise objects with Terraform, you should also upgrade to version 0.31 of that provider, which added the corresponding built-in support for these environment variables.
 
 NEW FEATURES:
 
 * `precondition` and `postcondition` check blocks for resources, data sources, and module output values: module authors can now document assumptions and assertions about configuration and state values. If these conditions are not met, Terraform will report a custom error message to the user and halt further evaluation.
+* You may specify remote network service credentials using an environment variable named after the host name with a `TF_TOKEN_` prefix. For example, the value of a variable named `TF_TOKEN_app_terraform_io` will be used as a bearer authorization token when the CLI makes service requests to the host name "app.terraform.io".
 
 ENHANCEMENTS:
 

internal/command/cliconfig/credentials.go

@@ -114,45 +114,77 @@ func (c *Config) credentialsSource(helperType string, helper svcauth.Credentials
 	}
 }
 
+func collectCredentialsFromEnv() map[svchost.Hostname]string {
+	const prefix = "TF_TOKEN_"
+
+	ret := make(map[svchost.Hostname]string)
+	for _, ev := range os.Environ() {
+		eqIdx := strings.Index(ev, "=")
+		if eqIdx < 0 {
+			continue
+		}
+		name := ev[:eqIdx]
+		value := ev[eqIdx+1:]
+		if !strings.HasPrefix(name, prefix) {
+			continue
+		}
+		rawHost := name[len(prefix):]
+
+		// We accept double underscores in place of hyphens because hyphens are not valid
+		// identifiers in most shells and are therefore hard to set.
+		// This is unambiguous with replacing single underscores below because
+		// hyphens are not allowed at the beginning or end of a label and therefore
+		// odd numbers of underscores will not appear together in a valid variable name.
+		rawHost = strings.ReplaceAll(rawHost, "__", "-")
+
+		// We accept underscores in place of dots because dots are not valid
+		// identifiers in most shells and are therefore hard to set.
+		// Underscores are not valid in hostnames, so this is unambiguous for
+		// valid hostnames.
+		rawHost = strings.ReplaceAll(rawHost, "_", ".")
+
+		// Because environment variables are often set indirectly by OS
+		// libraries that might interfere with how they are encoded, we'll
+		// be tolerant of them being given either directly as UTF-8 IDNs
+		// or in Punycode form, normalizing to Punycode form here because
+		// that is what the Terraform credentials helper protocol will
+		// use in its requests.
+		//
+		// Using ForDisplay first here makes this more liberal than Terraform
+		// itself would usually be in that it will tolerate pre-punycoded
+		// hostnames that Terraform normally rejects in other contexts in order
+		// to ensure stored hostnames are human-readable.
+		dispHost := svchost.ForDisplay(rawHost)
+		hostname, err := svchost.ForComparison(dispHost)
+		if err != nil {
+			// Ignore invalid hostnames
+			continue
+		}
+
+		ret[hostname] = value
+	}
+
+	return ret
+}
+
 // hostCredentialsFromEnv returns a token credential by searching for a hostname-specific
 // environment variable. The host parameter is expected to be in the "comparison" form,
 // for example, hostnames containing non-ASCII characters like "café.fr"
 // should be expressed as "xn--caf-dma.fr". If the variable based on the hostname is not
-// defined, nil is returned. Variable names must have dot characters translated to
-// underscores, which are not allowed in DNS names. For example, token credentials
-// for app.terraform.io should be set in the variable named TF_TOKEN_app_terraform_io.
+// defined, nil is returned.
 //
-// Hyphen characters are allowed in environment variable names, but are not valid POSIX
-// variable names. Usually, it's still possible to set variable names with hyphens using
-// utilities like env or docker. But, as a fallback, host names may encode their
-// hyphens as double underscores in the variable name. For the example "café.fr",
-// the variable name "TF_TOKEN_xn____caf__dma_fr" or "TF_TOKEN_xn--caf-dma_fr"
-// may be used.
+// Hyphen and period characters are allowed in environment variable names, but are not valid POSIX
+// variable names. However, it's still possible to set variable names with these characters using
+// utilities like env or docker. Variable names may have periods translated to underscores and
+// hyphens translated to double underscores in the variable name.
+// For the example "café.fr", you may use the variable names "TF_TOKEN_xn____caf__dma_fr",
+// "TF_TOKEN_xn--caf-dma_fr", or "TF_TOKEN_xn--caf-dma.fr"
 func hostCredentialsFromEnv(host svchost.Hostname) svcauth.HostCredentials {
-	if len(host) == 0 {
+	token, ok := collectCredentialsFromEnv()[host]
+	if !ok {
 		return nil
 	}
-
-	// Convert dots to underscores when looking for environment configuration for a specific host.
-	// DNS names do not allow underscore characters so this is unambiguous.
-	translated := strings.ReplaceAll(host.String(), ".", "_")
-
-	if token, ok := os.LookupEnv(fmt.Sprintf("TF_TOKEN_%s", translated)); ok {
-		return svcauth.HostCredentialsToken(token)
-	}
-
-	if strings.ContainsRune(translated, '-') {
-		// This host name contains a hyphen. Replace hyphens with double underscores as a fallback
-		// (see godoc above for details)
-		translated = strings.ReplaceAll(host.String(), "-", "__")
-		translated = strings.ReplaceAll(translated, ".", "_")
-
-		if token, ok := os.LookupEnv(fmt.Sprintf("TF_TOKEN_%s", translated)); ok {
-			return svcauth.HostCredentialsToken(token)
-		}
-	}
-
-	return nil
+	return svcauth.HostCredentialsToken(token)
 }
 
 // CredentialsSource is an implementation of svcauth.CredentialsSource

internal/command/cliconfig/credentials_test.go

@@ -156,6 +156,56 @@ func TestCredentialsForHost(t *testing.T) {
 			t.Errorf("wrong result\ngot: %s\nwant: %s", got, expectedToken)
 		}
 	})
+
+	t.Run("periods are ok", func(t *testing.T) {
+		envName := "TF_TOKEN_configured.example.com"
+		expectedToken := "configured-by-env"
+		t.Cleanup(func() {
+			os.Unsetenv(envName)
+		})
+
+		os.Setenv(envName, expectedToken)
+
+		hostname, _ := svchost.ForComparison("configured.example.com")
+		creds, err := credSrc.ForHost(hostname)
+
+		if err != nil {
+			t.Fatalf("unexpected error: %s", err)
+		}
+
+		if creds == nil {
+			t.Fatal("no credentials found")
+		}
+
+		if got := creds.Token(); got != expectedToken {
+			t.Errorf("wrong result\ngot: %s\nwant: %s", got, expectedToken)
+		}
+	})
+
+	t.Run("casing is insensitive", func(t *testing.T) {
+		envName := "TF_TOKEN_CONFIGUREDUPPERCASE_EXAMPLE_COM"
+		expectedToken := "configured-by-env"
+
+		os.Setenv(envName, expectedToken)
+		t.Cleanup(func() {
+			os.Unsetenv(envName)
+		})
+
+		hostname, _ := svchost.ForComparison("configureduppercase.example.com")
+		creds, err := credSrc.ForHost(hostname)
+
+		if err != nil {
+			t.Fatalf("unexpected error: %s", err)
+		}
+
+		if creds == nil {
+			t.Fatal("no credentials found")
+		}
+
+		if got := creds.Token(); got != expectedToken {
+			t.Errorf("wrong result\ngot: %s\nwant: %s", got, expectedToken)
+		}
+	})
 }
 
 func TestCredentialsStoreForget(t *testing.T) {

website/docs/cli/config/config-file.mdx

@@ -117,17 +117,21 @@ Terraform Cloud responds to API calls at both its current hostname
 
 ### Environment Variable Credentials
 
-If you would prefer not to store your API tokens directly in the CLI
-configuration, you may use a host-specific environment variable. Environment variable names should
-have the prefix `TF_TOKEN_` added to the domain name, with periods encoded as underscores.
-For example, the value of a variable named `TF_TOKEN_app_terraform_io` will be used as a
-bearer authorization token when the CLI makes service requests to the hostname "app.terraform.io".
-If multiple variables evaluate to the same hostname, Terraform will use the one defined later in the
-operating system's variable table.
+If you would prefer not to store your API tokens directly in the CLI configuration, you may use
+a host-specific environment variable. Environment variable names should have the prefix
+`TF_TOKEN_` added to the domain name, with periods encoded as underscores. For example, the
+value of a variable named `TF_TOKEN_app_terraform_io` will be used as a bearer authorization
+token when the CLI makes service requests to the hostname `app.terraform.io`.
 
-When using domain names as a variable name, you must convert domain names containing non-ASCII characters to their [punycode equivalent](https://www.charset.org/punycode) with an ACE prefix. For example, token credentials for 例えば.com must be set in a variable called `TF_TOKEN_xn--r8j3dr99h_com`.
+You must convert domain names containing non-ASCII characters to their [punycode equivalent](https://www.charset.org/punycode)
+with an ACE prefix. For example, token credentials for 例えば.com must be set in a variable
+called `TF_TOKEN_xn--r8j3dr99h_com`.
 
-Some tools like the `env` utility allow hyphens in variable names, but hyphens create invalid POSIX variable names. Therefore, you can encode hyphens as double underscores when you set variables with interactive tools like Bash or Zsh. For example, you can set a token for the domain name "café.fr" as either `TF_TOKEN_xn--caf-dma_fr` or `TF_TOKEN_xn____caf__dma_fr`. If both are defined, Terraform will use the version containing hyphens.
+Hyphens are also valid within host names but usually invalid as variable names and
+may be encoded as double underscores. For example, you can set a token for the domain name
+`café.fr` as `TF_TOKEN_xn--caf-dma.fr`, `TF_TOKEN_xn--caf-dma_fr`,  or `TF_TOKEN_xn____caf__dma_fr`.
+If multiple variables evaluate to the same hostname, Terraform will choose the one defined last
+in the operating system's variable table.
 
 ### Credentials Helpers