From 2449fadf0639f3be8fd1f959e69ff60890621dd2 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 30 Mar 2022 17:00:13 -0400 Subject: [PATCH 01/54] Revising preconditions and postconditions --- .../preconditions-postconditions.mdx | 307 ++++++++---------- 1 file changed, 144 insertions(+), 163 deletions(-) diff --git a/website/docs/language/expressions/preconditions-postconditions.mdx b/website/docs/language/expressions/preconditions-postconditions.mdx index 635cb1b63c..f9953bf193 100644 --- a/website/docs/language/expressions/preconditions-postconditions.mdx +++ b/website/docs/language/expressions/preconditions-postconditions.mdx @@ -1,31 +1,143 @@ --- -page_title: Preconditions and Postconditions - Configuration Language +page_title: Custom Data Validation - Configuration Language +description: >- + Validate requirements so Terraform can produce more detailed error messages. --- -# Preconditions and Postconditions +# Custom Data Validation -Terraform providers can automatically detect and report problems related to -the remote system they are interacting with, but they typically do so using -language that describes implementation details of the target system, which -can sometimes make it hard to find the root cause of the problem in your -Terraform configuration. +You can create validation checks for configuration requirements that return detailed error messages to consumers. Custom validations are optional, but they can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. -Preconditions and postconditions allow you to optionally describe the -assumptions you are making as a module author, so that Terraform can detect -situations where those assumptions don't hold and potentially return an -error earlier or an error with better context about where the problem -originated. +You can create custom validations with the following types of expressions. +- Add [`validation` blocks](input-variable-validation) inside input `variable` blocks. +- Add []`precondition` and `postcondition`](#preconditions-and-postconditions) blocks inside `lifecycle` blocks and `precondition` blocks inside `output blocks`. -Preconditions and postconditions both follow a similar structure, and differ -only in when Terraform evaluates them: Terraform checks a precondition prior -to evaluating the object it is associated with, and a postcondition _after_ -evaluating the object. That means that preconditions are useful for stating -assumptions about data from elsewhere that the resource configuration relies -on, while postconditions are more useful for stating assumptions about the -result of the resource itself. +The `validation`, `precondition`, and `postcondition` blocks all require a [`condition` attribute](#condition-expressions) that describes the validation requirements and an [`error_message` attribute](#error-messages) that contains explanatory text that Terraform will display to the user. -The following example shows some different possible uses of preconditions and -postconditions. +-> **Note:** Input variable validation is available in Terraform CLI v0.13.0 and later. Preconditions and postconditions are available in Terraform CLI v1.2.0 and later. + +## When Terraform Evaluates Custom Validations + +Terraform will evaluate the conditions specified in `validation`,`precondition`, and `postcondition` blocks as early as possible. + +If the condition expression depends on a resource attribute that won't be known +until the apply phase then Terraform will delay checking the condition until +the apply phase, but Terraform can check all other expressions during the +planning phase, and therefore block applying a plan that would violate the +conditions. + +For example, Terraform would typically be able to detect invalid AMI tags during the planning phase, as long as `var.aws_ami_id` is not derived from another resource. However, Terraform will not detect a non-encrypted root volume until the EC2 instance is created during the apply step because that condition depends on the root volume's assigned ID, which AWS decides only when the EC2 instance is actually started. + +For conditions which Terraform must defer to the apply phase, a _precondition_ +will prevent taking whatever action was planned for a related resource, whereas +a _postcondition_ will merely halt processing after that action was already +taken, preventing any downstream actions that rely on it but not undoing the +action. + +Terraform typically has less information during the initial creation of a +full configuration than when applying subsequent changes to that configuration. +Conditions checked only during apply during initial creation may therefore +be checked during planning on subsequent updates, detecting problems sooner +in that case. + + +## Input Variable Validation + +To specify custom validation rules for a variable, add a `validation` block within the corresponding `variable` block. + +The following example checks whether the AMI ID has valid syntax. + +```hcl +variable "image_id" { + type = string + description = "The id of the machine image (AMI) to use for the server." + + validation { + condition = length(var.image_id) > 4 && substr(var.image_id, 0, 4) == "ami-" + error_message = "The image_id value must be a valid AMI id, starting with \"ami-\"." + } +} +``` + +The [`condition` argument](#condition-expressions) is an expression that must use the value of the variable to return `true` if the value is valid, or `false` if it is invalid. + +If the failure of an expression is the basis of the validation decision, use [the `can` function](/language/functions/can) to detect such errors, as demonstrated in the following example. + +```hcl +variable "image_id" { + type = string + description = "The id of the machine image (AMI) to use for the server." + + validation { + # regex(...) fails if it cannot find a match + condition = can(regex("^ami-", var.image_id)) + error_message = "The image_id value must be a valid AMI id, starting with \"ami-\"." + } +} +``` + +If `condition` evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple `validation` blocks, Terraform returns error messages for _all_ failed conditions. + + +## Preconditions and Postconditions + +Terraform checks a precondition _before_ evaluating the object it is associated with, and checks a postcondition _after_ evaluating the object. + +Terraform supports preconditions and postconditions in the following locations. + +* The `lifecycle` block inside a `resource` or `data` block can include both + `precondition` and `postcondition` blocks associated with the containing + resource. + + Terraform evaluates resource preconditions before evaluating the resource's + configuration arguments. Resource preconditions can take precedence over + argument evaluation errors. + + Terraform evaluates resource postconditions after planning and after + applying changes to a managed resource, or after reading from a data + resource. Resource postcondition failures will therefore prevent applying + changes to other resources that depend on the failing resource. + +* An `output` block declaring an output value can include a `precondition` + block. + + Terraform evaluates output value preconditions before evaluating the + `value` expression to finalize the result. Output value preconditions + can take precedence over potential errors in the `value` expression. + + Output value preconditions can be particularly useful in a root module, + to prevent saving an invalid new output value in the state and to preserve + the value from the previous apply, if any. + + Output value preconditions can serve a symmetrical purpose to input + variable `validation` blocks: whereas input variable validation checks + assumptions the module makes about its inputs, output value preconditions + check guarantees that the module makes about its outputs. + +### When to Use Preconditions and PostConditions + +You can often implement a validation check as either a postcondition of the resource producing the data or as a precondition of a resource or output value using the data. To decide which is most appropriate, consider whether the check is representing either an assumption or a guarantee. + +- **Assumption:** A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. + + Assumptions should typically be written as preconditions, so that future maintainers can find them close to the other expressions that rely on that condition, and know more about what different variations that resource is intended to allow. + +- **Guarantee:** A characteristic or behavior of an object that the rest of + the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. + + Guarantees should typically be written as postconditions, so that + future maintainers can find them close to the resource configuration that is responsible for implementing those guarantees and more easily see which behaviors are important to preserve when changing the configuration. + +We recommend also considering the following factors. + +- Which resource or output value would be most helpful to report in the error message. Terraform will always report errors in the location where the condition was declared. +- Which approach is more convenient. If a particular resource has many dependencies that all make an assumption about that resource, it can be pragmatic to declare that once as a post-condition of the resource, rather than declaring it many times as preconditions on each of the dependencies. +- Whether it is helpful to declare the same or similar conditions as both preconditions and postconditions. This can be useful if the postcondition is in a different module than the precondition because it lets the modules verify one another as they evolve independently. + + +## Usage Examples + +The following example shows possible uses for variable validation, preconditions and postconditions. ```hcl variable "aws_ami_id" { @@ -100,15 +212,7 @@ output "api_base_url" { } ``` -The input variable validation rule, preconditions, and postconditions in the -above example declare explicitly some assumptions and guarantees that the -module developer is making in the design of this module: - -* The caller of the module must provide a syntactically-valid AMI ID in the - `aws_ami_id` input variable. - - This would detect if the caller accidentally assigned an AMI name to the - argument, instead of an AMI ID. +The preconditions, and postconditions declare the following assumptions and guarantees. * The AMI ID must refer to an AMI that exists and that has been tagged as being intended for the component "nomad-server". @@ -141,65 +245,19 @@ module developer is making in the design of this module: problem immediately, before any other components rely on the insecurely-configured component. -Writing explicit preconditions and postconditions is always optional, but it -can be helpful to users and future maintainers of a Terraform module by -capturing assumptions that might otherwise be only implied, and by allowing -Terraform to check those assumptions and halt more quickly if they don't -hold in practice for a particular set of input variables. +## Condition Expressions -## Precondition and Postcondition Locations +The variable `validation` block and the `precondition` and `postcondition` blocks all require an argument named `condition`, whose value is a boolean expression which should return `true` if the intended assumption holds or `false` if it does not. -Terraform supports preconditions and postconditions in a number of different -locations in a module: +Condition expressions have the following requirements. -* The `lifecycle` block inside a `resource` or `data` block can include both - `precondition` and `postcondition` blocks associated with the containing - resource. +- For variables, the expression can refer only to the variable that the condition applies to, and must not produce errors. +- For preconditions and postconditions, the expression can refer to any other objects in the same module, as long as the references don't create any cyclic dependencies. Resource postconditions can additionally refer to attributes of each instance of the resource where they are configured, using the special symbol `self`. For example, `self.private_dns` refers to the `private_dns` attribute of each instance of the containing resource. - Terraform evaluates resource preconditions before evaluating the resource's - configuration arguments. Resource preconditions can take precedence over - argument evaluation errors. +You can use any of Terraform's built-in functions or language operators +in a condition as long as the expression is valid and returns a boolean result. - Terraform evaluates resource postconditions after planning and after - applying changes to a managed resource, or after reading from a data - resource. Resource postcondition failures will therefore prevent applying - changes to other resources that depend on the failing resource. - -* An `output` block declaring an output value can include a `precondition` - block. - - Terraform evaluates output value preconditions before evaluating the - `value` expression to finalize the result. Output value preconditions - can take precedence over potential errors in the `value` expression. - - Output value preconditions can be particularly useful in a root module, - to prevent saving an invalid new output value in the state and to preserve - the value from the previous apply, if any. - - Output value preconditions can serve a symmetrical purpose to input - variable `validation` blocks: whereas input variable validation checks - assumptions the module makes about its inputs, output value preconditions - check guarantees that the module makes about its outputs. - -## Condition Expressions - -`precondition` and `postcondition` blocks both require an argument named -`condition`, whose value is a boolean expression which should return `true` -if the intended assumption holds or `false` if it does not. - -Preconditions and postconditions can both refer to any other objects in the -same module, as long as the references don't create any cyclic dependencies. - -Resource postconditions can additionally refer to attributes of each instance -of the resource where they are configured, using the special symbol `self`. -For example, `self.private_dns` refers to the `private_dns` attribute of -each instance of the containing resource. - -Condition expressions are otherwise just normal Terraform expressions, and -so you can use any of Terraform's built-in functions or language operators -as long as the expression is valid and returns a boolean result. - -### Common Condition Expression Features +## Common Condition Expression Features Because condition expressions must produce boolean results, they can often use built-in functions and language features that are less common elsewhere @@ -274,7 +332,7 @@ useful when writing condition expressions: You can also use `can` with attribute access or index operators to concisely test whether a collection or structural value has a particular element or index: - + ```hcl # var.example must have an attribute named "foo" condition = can(var.example.foo) @@ -288,38 +346,9 @@ useful when writing condition expressions: # intent of the condition.) ``` -## Early Evaluation - -Terraform will evaluate conditions as early as possible. - -If the condition expression depends on a resource attribute that won't be known -until the apply phase then Terraform will delay checking the condition until -the apply phase, but Terraform can check all other expressions during the -planning phase, and therefore block applying a plan that would violate the -conditions. - -In the earlier example on this page, Terraform would typically be able to -detect invalid AMI tags during the planning phase, as long as `var.aws_ami_id` -is not itself derived from another resource. However, Terraform will not -detect a non-encrypted root volume until the EC2 instance was already created -during the apply step, because that condition depends on the root volume's -assigned ID, which AWS decides only when the EC2 instance is actually started. - -For conditions which Terraform must defer to the apply phase, a _precondition_ -will prevent taking whatever action was planned for a related resource, whereas -a _postcondition_ will merely halt processing after that action was already -taken, preventing any downstream actions that rely on it but not undoing the -action. - -Terraform typically has less information during the initial creation of a -full configuration than when applying subsequent changes to that configuration. -Conditions checked only during apply during initial creation may therefore -be checked during planning on subsequent updates, detecting problems sooner -in that case. - ## Error Messages -Each `precondition` or `postcondition` block must include an argument +Each `validation`, `precondition` or `postcondition` block must include an argument `error_message`, which provides some custom error sentences that Terraform will include as part of error messages when it detects an unmet condition. @@ -345,51 +374,3 @@ style similar to Terraform's own error messages. Terraform will show the given message alongside the name of the resource that detected the problem and any outside values used as part of the condition expression. -## Preconditions or Postconditions? - -Because preconditions can refer to the result attributes of other resources -in the same module, it's typically true that a particular check could be -implemented either as a postcondition of the resource producing the data -or as a precondition of a resource or output value using the data. - -To decide which is most appropriate for a particular situation, consider -whether the check is representing either an assumption or a guarantee: - -* An _assumption_ is a condition that must be true in order for the - configuration of a particular resource to be usable. In the earlier - example on this page, the `aws_instance` configuration had the _assumption_ - that the given AMI will always be for the `x86_64` CPU architecture. - - Assumptions should typically be written as preconditions, so that future - maintainers can find them close to the other expressions that rely on - that condition, and thus know more about what different variations that - resource is intended to allow. - -* A _guarantee_ is a characteristic or behavior of an object that the rest of - the configuration ought to be able to rely on. In the earlier example on - this page, the `aws_instance` configuration had the _guarantee_ that the - EC2 instance will be running in a network that assigns it a private DNS - record. - - Guarantees should typically be written as postconditions, so that - future maintainers can find them close to the resource configuration that - is responsible for implementing those guarantees and more easily see - which behaviors are important to preserve when changing the configuration. - -In practice though, the distinction between these two is subjective: is the -AMI being tagged as Component `"nomad-server"` a guarantee about the AMI or -an assumption made by the EC2 instance? To decide, it might help to consider -which resource or output value would be most helpful to report in a resulting -error message, because Terraform will always report errors in the location -where the condition was declared. - -The decision between the two may also be a matter of convenience. If a -particular resource has many dependencies that _all_ make an assumption about -that resource then it can be pragmatic to declare that just once as a -post-condition of the resource, rather than many times as preconditions on -each of the dependencies. - -It may sometimes be helpful to declare the same or similar conditions as both -preconditions _and_ postconditions, particularly if the postcondition is -in a different module than the precondition, so that they can verify one -another as the two modules evolve independently. From 42d73fe3242495349a054101fe3303bf833e22d5 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 30 Mar 2022 17:11:23 -0400 Subject: [PATCH 02/54] Rename page in sidebar and change filename --- website/data/language-nav-data.json | 4 ++-- ...postconditions.mdx => custom-data-validation.mdx} | 12 +++++------- 2 files changed, 7 insertions(+), 9 deletions(-) rename website/docs/language/expressions/{preconditions-postconditions.mdx => custom-data-validation.mdx} (94%) diff --git a/website/data/language-nav-data.json b/website/data/language-nav-data.json index 74310b84b5..837a46614e 100644 --- a/website/data/language-nav-data.json +++ b/website/data/language-nav-data.json @@ -280,8 +280,8 @@ "path": "expressions/version-constraints" }, { - "title": "Pre and Postconditions", - "path": "expressions/preconditions-postconditions" + "title": "Custom Data Validation", + "path": "expressions/custom-data-validation" } ] }, diff --git a/website/docs/language/expressions/preconditions-postconditions.mdx b/website/docs/language/expressions/custom-data-validation.mdx similarity index 94% rename from website/docs/language/expressions/preconditions-postconditions.mdx rename to website/docs/language/expressions/custom-data-validation.mdx index f9953bf193..a2ffb6999f 100644 --- a/website/docs/language/expressions/preconditions-postconditions.mdx +++ b/website/docs/language/expressions/custom-data-validation.mdx @@ -6,13 +6,13 @@ description: >- # Custom Data Validation -You can create validation checks for configuration requirements that return detailed error messages to consumers. Custom validations are optional, but they can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. +You can create validation checks with custom error messages for several types of objects in a configuration. Custom validations are optional, but they can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. You can create custom validations with the following types of expressions. - Add [`validation` blocks](input-variable-validation) inside input `variable` blocks. -- Add []`precondition` and `postcondition`](#preconditions-and-postconditions) blocks inside `lifecycle` blocks and `precondition` blocks inside `output blocks`. +- Add [`precondition` and `postcondition`](#preconditions-and-postconditions) blocks inside `lifecycle` blocks and `precondition` blocks inside `output blocks`. -The `validation`, `precondition`, and `postcondition` blocks all require a [`condition` attribute](#condition-expressions) that describes the validation requirements and an [`error_message` attribute](#error-messages) that contains explanatory text that Terraform will display to the user. +All of these methods require a [`condition` attribute](#condition-expressions) that describes the validation requirements and an [`error_message` attribute](#error-messages) that contains explanatory text that Terraform will display to the user. -> **Note:** Input variable validation is available in Terraform CLI v0.13.0 and later. Preconditions and postconditions are available in Terraform CLI v1.2.0 and later. @@ -135,7 +135,7 @@ We recommend also considering the following factors. - Whether it is helpful to declare the same or similar conditions as both preconditions and postconditions. This can be useful if the postcondition is in a different module than the precondition because it lets the modules verify one another as they evolve independently. -## Usage Examples +### Usage Examples The following example shows possible uses for variable validation, preconditions and postconditions. @@ -245,7 +245,7 @@ The preconditions, and postconditions declare the following assumptions and guar problem immediately, before any other components rely on the insecurely-configured component. -## Condition Expressions +## Condition Expressions The variable `validation` block and the `precondition` and `postcondition` blocks all require an argument named `condition`, whose value is a boolean expression which should return `true` if the intended assumption holds or `false` if it does not. @@ -257,8 +257,6 @@ Condition expressions have the following requirements. You can use any of Terraform's built-in functions or language operators in a condition as long as the expression is valid and returns a boolean result. -## Common Condition Expression Features - Because condition expressions must produce boolean results, they can often use built-in functions and language features that are less common elsewhere in the Terraform language. The following language features are particularly From 5effda41704cf33334ca5918b8ccac1651df7ab0 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 30 Mar 2022 17:18:16 -0400 Subject: [PATCH 03/54] Rename page again --- website/data/language-nav-data.json | 4 +- ...dation.mdx => custom-validation-rules.mdx} | 0 website/docs/language/values/variables.mdx | 39 ++----------------- 3 files changed, 5 insertions(+), 38 deletions(-) rename website/docs/language/expressions/{custom-data-validation.mdx => custom-validation-rules.mdx} (100%) diff --git a/website/data/language-nav-data.json b/website/data/language-nav-data.json index 837a46614e..ecb6900c7b 100644 --- a/website/data/language-nav-data.json +++ b/website/data/language-nav-data.json @@ -280,8 +280,8 @@ "path": "expressions/version-constraints" }, { - "title": "Custom Data Validation", - "path": "expressions/custom-data-validation" + "title": "Custom Validation Rules", + "path": "expressions/custom-validation-rules" } ] }, diff --git a/website/docs/language/expressions/custom-data-validation.mdx b/website/docs/language/expressions/custom-validation-rules.mdx similarity index 100% rename from website/docs/language/expressions/custom-data-validation.mdx rename to website/docs/language/expressions/custom-validation-rules.mdx diff --git a/website/docs/language/values/variables.mdx b/website/docs/language/values/variables.mdx index 434c88a24f..6c2f42e06b 100644 --- a/website/docs/language/values/variables.mdx +++ b/website/docs/language/values/variables.mdx @@ -160,9 +160,7 @@ commentary for module maintainers, use comments. -> This feature was introduced in Terraform CLI v0.13.0. -In addition to Type Constraints as described above, a module author can specify -arbitrary custom validation rules for a particular variable using a `validation` -block nested within the corresponding `variable` block: +In addition to Type Constraints, you can specify custom validation rules for a particular variable using a `validation` block nested within the corresponding `variable` block. The example below checks whether the AMI ID has the correct syntax. ```hcl variable "image_id" { @@ -175,38 +173,7 @@ variable "image_id" { } } ``` - -The `condition` argument is an expression that must use the value of the -variable to return `true` if the value is valid, or `false` if it is invalid. -The expression can refer only to the variable that the condition applies to, -and _must not_ produce errors. - -If the failure of an expression is the basis of the validation decision, use -[the `can` function](/language/functions/can) to detect such errors. For example: - -```hcl -variable "image_id" { - type = string - description = "The id of the machine image (AMI) to use for the server." - - validation { - # regex(...) fails if it cannot find a match - condition = can(regex("^ami-", var.image_id)) - error_message = "The image_id value must be a valid AMI id, starting with \"ami-\"." - } -} -``` - -If `condition` evaluates to `false`, Terraform will produce an error message -that includes the result of the `error_message` expression. The error message -should be at least one full sentence explaining the constraint that failed, -using a sentence structure similar to the above examples. - -Error messages can be literal strings, heredocs, or template expressions. The -only valid reference in an error message is the variable under validation. - -Multiple `validation` blocks can be declared in which case error messages -will be returned for _all_ failed conditions. +Refer to [Custom Data Validation](/language/expressions/custom-validation-rules) for more details. ### Suppressing Values in CLI Output @@ -319,7 +286,7 @@ may assign the value `null` to the variable. ``` variable "example" { type = string - nullable = false + nullable = false } ``` From d12b170ef23e00bf1dbe9449488f946d4b281564 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 30 Mar 2022 17:48:43 -0400 Subject: [PATCH 04/54] more page edits for flow and style --- .../expressions/custom-validation-rules.mdx | 216 +++++++----------- 1 file changed, 79 insertions(+), 137 deletions(-) diff --git a/website/docs/language/expressions/custom-validation-rules.mdx b/website/docs/language/expressions/custom-validation-rules.mdx index a2ffb6999f..a8b47de200 100644 --- a/website/docs/language/expressions/custom-validation-rules.mdx +++ b/website/docs/language/expressions/custom-validation-rules.mdx @@ -1,10 +1,10 @@ --- -page_title: Custom Data Validation - Configuration Language +page_title: Custom Validation Rules - Configuration Language description: >- - Validate requirements so Terraform can produce more detailed error messages. + Validate requirements for variables, outputs, and within lifecycle blocks so Terraform can produce better error messages in context. --- -# Custom Data Validation +# Custom Validation Rules You can create validation checks with custom error messages for several types of objects in a configuration. Custom validations are optional, but they can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. @@ -43,9 +43,7 @@ in that case. ## Input Variable Validation -To specify custom validation rules for a variable, add a `validation` block within the corresponding `variable` block. - -The following example checks whether the AMI ID has valid syntax. +To specify custom validation rules for a variable, add a `validation` block within the corresponding `variable` block. The following example checks whether the AMI ID has valid syntax. ```hcl variable "image_id" { @@ -81,38 +79,19 @@ If `condition` evaluates to `false`, Terraform will produce an [error message](# ## Preconditions and Postconditions -Terraform checks a precondition _before_ evaluating the object it is associated with, and checks a postcondition _after_ evaluating the object. +Terraform checks a precondition _before_ evaluating the object it is associated with, and checks a postcondition _after_ evaluating the object. You can add preconditions and postconditions within the following configuration blocks. -Terraform supports preconditions and postconditions in the following locations. +The `lifecycle` block inside a `resource` or `data` block can include both `precondition` and `postcondition` blocks associated with the containing resource. -* The `lifecycle` block inside a `resource` or `data` block can include both - `precondition` and `postcondition` blocks associated with the containing - resource. +- Terraform evaluates resource preconditions before evaluating the resource's configuration arguments. Resource preconditions can take precedence over argument evaluation errors. +- Terraform evaluates resource postconditions after planning and after applying changes to a managed resource, or after reading from a data resource. Resource postcondition failures will therefore prevent applying changes to other resources that depend on the failing resource. - Terraform evaluates resource preconditions before evaluating the resource's - configuration arguments. Resource preconditions can take precedence over - argument evaluation errors. +An `output` block can include a `precondition` block. - Terraform evaluates resource postconditions after planning and after - applying changes to a managed resource, or after reading from a data - resource. Resource postcondition failures will therefore prevent applying - changes to other resources that depend on the failing resource. - -* An `output` block declaring an output value can include a `precondition` - block. - - Terraform evaluates output value preconditions before evaluating the - `value` expression to finalize the result. Output value preconditions - can take precedence over potential errors in the `value` expression. - - Output value preconditions can be particularly useful in a root module, - to prevent saving an invalid new output value in the state and to preserve - the value from the previous apply, if any. - - Output value preconditions can serve a symmetrical purpose to input - variable `validation` blocks: whereas input variable validation checks - assumptions the module makes about its inputs, output value preconditions - check guarantees that the module makes about its outputs. +- Terraform evaluates output value preconditions before evaluating the +`value` expression to finalize the result. Output value preconditions can take precedence over potential errors in the `value` expression. +- Preconditions can be particularly useful in a root module to prevent saving an invalid new output value in the state and to preserve the value from the previous apply, if any. +- Preconditions can serve a symmetrical purpose to input variable `validation` blocks. Whereas input variable validation checks assumptions the module makes about its inputs, output value preconditions check guarantees that the module makes about its outputs. ### When to Use Preconditions and PostConditions @@ -137,7 +116,7 @@ We recommend also considering the following factors. ### Usage Examples -The following example shows possible uses for variable validation, preconditions and postconditions. +The following example shows several possible uses for preconditions and postconditions. ```hcl variable "aws_ami_id" { @@ -212,38 +191,17 @@ output "api_base_url" { } ``` -The preconditions, and postconditions declare the following assumptions and guarantees. +The preconditions and postconditions declare the following assumptions and guarantees. -* The AMI ID must refer to an AMI that exists and that has been tagged as - being intended for the component "nomad-server". +- **The AMI ID must refer to an AMI that exists and that has been tagged with "nomad-server".** This would detect if the caller accidentally provided an AMI intended for some other system component, which might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. - This would detect if the caller accidentally provided an AMI intended for - some other system component, which might otherwise be detected only after - booting the EC2 instance and noticing that the expected network service - isn't running. Terraform can therefore detect that problem earlier and - return a more actionable error message for it. +- **The AMI ID must refer to an AMI that contains an operating system for the + `x86_64` architecture.** This would detect if the caller accidentally built an AMI for a different + architecture, which may not be able to run the software this virtual machine is intended to host. -* The AMI ID must refer to an AMI which contains an operating system for the - `x86_64` architecture. +- **The EC2 instance must be allocated a private DNS hostname.** In Amazon Web Services, EC2 instances are assigned private DNS hostnames only if they belong to a virtual network configured in a certain way. This would detect if the selected virtual network is not configured correctly, giving explicit feedback to prompt the user to debug the network settings. - This would detect if the caller accidentally built an AMI for a different - architecture, which might therefore not be able to run the software this - virtual machine is intended to host. - -* The EC2 instance must be allocated a private DNS hostname. - - In AWS, EC2 instances are assigned private DNS hostnames only if they - belong to a virtual network configured in a certain way. This would - detect if the selected virtual network is not configured correctly, - giving explicit feedback to prompt the user to debug the network settings. - -* The EC2 instance will have an encrypted root volume. - - This ensures that the root volume is encrypted even though the software - running in this EC2 instance would probably still operate as expected - on an unencrypted volume. Therefore Terraform can draw attention to the - problem immediately, before any other components rely on the - insecurely-configured component. +- **The EC2 instance will have an encrypted root volume.** This ensures that the root volume is encrypted even though the software running in this EC2 instance would probably still operate as expected on an unencrypted volume. This lets Terraform produce an error immediately, before any other components rely on the insecurely-configured component. ## Condition Expressions @@ -255,100 +213,84 @@ Condition expressions have the following requirements. - For preconditions and postconditions, the expression can refer to any other objects in the same module, as long as the references don't create any cyclic dependencies. Resource postconditions can additionally refer to attributes of each instance of the resource where they are configured, using the special symbol `self`. For example, `self.private_dns` refers to the `private_dns` attribute of each instance of the containing resource. You can use any of Terraform's built-in functions or language operators -in a condition as long as the expression is valid and returns a boolean result. +in a condition as long as the expression is valid and returns a boolean result. The following language features are particularly +useful when writing condition expressions. -Because condition expressions must produce boolean results, they can often -use built-in functions and language features that are less common elsewhere -in the Terraform language. The following language features are particularly -useful when writing condition expressions: +### `contains` Function +You can use the built-in function `contains` to test whether a given value is one of a set of predefined valid values. -* You can use the built-in function `contains` to test whether a given - value is one of a set of predefined valid values: +```hcl + condition = contains(["STAGE", "PROD"], var.environment) +``` - ```hcl - condition = contains(["STAGE", "PROD"], var.environment) - ``` +### Boolean Operators +You can use the boolean operators `&&` (AND), `||` (OR), and `!` (NOT) to combine multiple simpler conditions together. -* You can use the boolean operators `&&` (AND), `||` (OR), and `!` (NOT) to - combine multiple simpler conditions together: +```hcl + condition = var.name != "" && lower(var.name) == var.name + ``` - ```hcl - condition = var.name != "" && lower(var.name) == var.name - ``` +### `length` -* You can require a non-empty list or map by testing the collection's length: +You can require a non-empty list or map by testing the collection's length. - ```hcl - condition = length(var.items) != 0 - ``` +```hcl + condition = length(var.items) != 0 +``` +This is a better approach than directly comparing with another collection using `==` or `!=`, because the comparison operators can only return `true` if both operands have exactly the same type, which is often ambiguous for empty collections. - This is a better approach than directly comparing with another collection - using `==` or `!=`, because the comparison operators can only return `true` - if both operands have exactly the same type, which is often ambiguous - for empty collections. +### `for` Expressions +You can use `for` expressions which produce lists of boolean results +themselves in conjunction with the functions `alltrue` and `anytrue` to test whether a condition holds for all or for any elements of a collection. -* You can use `for` expressions which produce lists of boolean results - themselves in conjunction with the functions `alltrue` and `anytrue` to - test whether a condition holds for all or for any elements of a collection: +```hcl + condition = alltrue([ + for v in var.instances : contains(["t2.micro", "m3.medium"], v.type) + ]) +``` - ```hcl - condition = alltrue([ - for v in var.instances : contains(["t2.micro", "m3.medium"], v.type) - ]) - ``` +### `can` Function -* You can use the `can` function to concisely use the validity of an expression - as a condition. It returns `true` if its given expression evaluates - successfully and `false` if it returns any error, so you can use various - other functions that typically return errors as a part of your condition - expressions. +You can use the `can` function to concisely use the validity of an expression as a condition. It returns `true` if its given expression evaluates successfully and `false` if it returns any error, so you can use various other functions that typically return errors as a part of your condition expressions. - For example, you can use `can` with `regex` to test if a string matches - a particular pattern, because `regex` returns an error when given a - non-matching string: +For example, you can use `can` with `regex` to test if a string matches a particular pattern, because `regex` returns an error when given a non-matching string. - ```hcl - condition = can(regex("^[a-z]+$", var.name) - ``` +```hcl + condition = can(regex("^[a-z]+$", var.name) +``` - You can also use `can` with the type conversion functions to test whether - a value is convertible to a type or type constraint: +You can also use `can` with the type conversion functions to test whether a value is convertible to a type or type constraint. - ```hcl - # This remote output value must have a value that can - # be used as a string, which includes strings themselves - # but also allows numbers and boolean values. - condition = can(tostring(data.terraform_remote_state.example.outputs["name"])) - ``` +```hcl + # This remote output value must have a value that can + # be used as a string, which includes strings themselves + # but also allows numbers and boolean values. + condition = can(tostring(data.terraform_remote_state.example.outputs["name"])) +``` - ```hcl - # This remote output value must be convertible to a list - # type of with element type. - condition = can(tolist(data.terraform_remote_state.example.outputs["items"])) - ``` +```hcl + # This remote output value must be convertible to a list + # type of with element type. + condition = can(tolist(data.terraform_remote_state.example.outputs["items"])) +``` - You can also use `can` with attribute access or index operators to - concisely test whether a collection or structural value has a particular - element or index: +You can also use `can` with attribute access or index operators to concisely test whether a collection or structural value has a particular element or index. - ```hcl - # var.example must have an attribute named "foo" - condition = can(var.example.foo) - ``` +```hcl + # var.example must have an attribute named "foo" + condition = can(var.example.foo) ``` - ```hcl - # var.example must be a sequence with at least one element - condition = can(var.example[0]) - # (although it would typically be clearer to write this as a - # test like length(var.example) > 0 to better represent the - # intent of the condition.) - ``` +```hcl + # var.example must be a sequence with at least one element + condition = can(var.example[0]) + # (although it would typically be clearer to write this as a + # test like length(var.example) > 0 to better represent the + # intent of the condition.) +``` ## Error Messages -Each `validation`, `precondition` or `postcondition` block must include an argument -`error_message`, which provides some custom error sentences that Terraform -will include as part of error messages when it detects an unmet condition. +Each `validation`, `precondition` or `postcondition` block must include an argument `error_message`, which contains the text that Terraform will include as part of error messages when it detects an unmet condition. ``` Error: Resource postcondition failed @@ -362,12 +304,12 @@ Error: Resource postcondition failed The selected AMI must be tagged with the Component value "nomad-server". ``` -The `error_message` argument can be any expression which evaluates to a string. +The `error_message` argument can be any expression that evaluates to a string. This includes literal strings, heredocs, and template expressions. Multi-line error messages are supported, and lines with leading whitespace will not be word wrapped. -Error message should typically be written as one or more full sentences in a +We recommend writing error messages as one or more full sentences in a style similar to Terraform's own error messages. Terraform will show the given message alongside the name of the resource that detected the problem and any outside values used as part of the condition expression. From b0f491f2d1360b2d6d5ba2e01cb86e2478dee18f Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 30 Mar 2022 17:57:03 -0400 Subject: [PATCH 05/54] update section title --- website/docs/language/expressions/custom-validation-rules.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/language/expressions/custom-validation-rules.mdx b/website/docs/language/expressions/custom-validation-rules.mdx index a8b47de200..b2cc2dec87 100644 --- a/website/docs/language/expressions/custom-validation-rules.mdx +++ b/website/docs/language/expressions/custom-validation-rules.mdx @@ -93,7 +93,7 @@ An `output` block can include a `precondition` block. - Preconditions can be particularly useful in a root module to prevent saving an invalid new output value in the state and to preserve the value from the previous apply, if any. - Preconditions can serve a symmetrical purpose to input variable `validation` blocks. Whereas input variable validation checks assumptions the module makes about its inputs, output value preconditions check guarantees that the module makes about its outputs. -### When to Use Preconditions and PostConditions +### Choosing Between Preconditions and PostConditions You can often implement a validation check as either a postcondition of the resource producing the data or as a precondition of a resource or output value using the data. To decide which is most appropriate, consider whether the check is representing either an assumption or a guarantee. From 040985f6e10b245a1f870f9c8c0bff7de8de3984 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 30 Mar 2022 17:59:03 -0400 Subject: [PATCH 06/54] more edits --- .../docs/language/expressions/custom-validation-rules.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/language/expressions/custom-validation-rules.mdx b/website/docs/language/expressions/custom-validation-rules.mdx index b2cc2dec87..3c2caae4c6 100644 --- a/website/docs/language/expressions/custom-validation-rules.mdx +++ b/website/docs/language/expressions/custom-validation-rules.mdx @@ -209,8 +209,8 @@ The variable `validation` block and the `precondition` and `postcondition` block Condition expressions have the following requirements. -- For variables, the expression can refer only to the variable that the condition applies to, and must not produce errors. -- For preconditions and postconditions, the expression can refer to any other objects in the same module, as long as the references don't create any cyclic dependencies. Resource postconditions can additionally refer to attributes of each instance of the resource where they are configured, using the special symbol `self`. For example, `self.private_dns` refers to the `private_dns` attribute of each instance of the containing resource. +- For variable `validation` blocks, the expression can refer only to the variable that the condition applies to and must not produce errors. +- For `precondition` and `postcondition` blocks, the expression can refer to any other objects in the same module, as long as the references don't create any cyclic dependencies. Resource postconditions can additionally refer to attributes of each instance of the resource where they are configured, using the special symbol `self`. For example, `self.private_dns` refers to the `private_dns` attribute of each instance of the containing resource. You can use any of Terraform's built-in functions or language operators in a condition as long as the expression is valid and returns a boolean result. The following language features are particularly @@ -230,7 +230,7 @@ You can use the boolean operators `&&` (AND), `||` (OR), and `!` (NOT) to combin condition = var.name != "" && lower(var.name) == var.name ``` -### `length` +### `length` Function You can require a non-empty list or map by testing the collection's length. From b2576a3df366b54c31aaaf520819b38e2643134b Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Tue, 5 Apr 2022 15:50:24 -0400 Subject: [PATCH 07/54] Update page name to Custom Conditions per feedback --- website/data/language-nav-data.json | 4 ++-- ...stom-validation-rules.mdx => custom-conditions.mdx} | 10 ++++++---- 2 files changed, 8 insertions(+), 6 deletions(-) rename website/docs/language/expressions/{custom-validation-rules.mdx => custom-conditions.mdx} (96%) diff --git a/website/data/language-nav-data.json b/website/data/language-nav-data.json index ecb6900c7b..656cbe1019 100644 --- a/website/data/language-nav-data.json +++ b/website/data/language-nav-data.json @@ -280,8 +280,8 @@ "path": "expressions/version-constraints" }, { - "title": "Custom Validation Rules", - "path": "expressions/custom-validation-rules" + "title": "Custom Conditions", + "path": "expressions/custom-conditions" } ] }, diff --git a/website/docs/language/expressions/custom-validation-rules.mdx b/website/docs/language/expressions/custom-conditions.mdx similarity index 96% rename from website/docs/language/expressions/custom-validation-rules.mdx rename to website/docs/language/expressions/custom-conditions.mdx index 3c2caae4c6..6b57f4f4a6 100644 --- a/website/docs/language/expressions/custom-validation-rules.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -1,14 +1,16 @@ --- -page_title: Custom Validation Rules - Configuration Language +page_title: Custom Conditions - Configuration Language description: >- Validate requirements for variables, outputs, and within lifecycle blocks so Terraform can produce better error messages in context. --- -# Custom Validation Rules +# Custom Conditions -You can create validation checks with custom error messages for several types of objects in a configuration. Custom validations are optional, but they can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. +You can create custom conditions that produce custom error messages for several types of objects in a configuration. For example, you may want to check to whether an incoming image ID is formatted properly. -You can create custom validations with the following types of expressions. +Custom condition checks are optional, but they can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. + +You can create custom conditions with the following types of expressions. - Add [`validation` blocks](input-variable-validation) inside input `variable` blocks. - Add [`precondition` and `postcondition`](#preconditions-and-postconditions) blocks inside `lifecycle` blocks and `precondition` blocks inside `output blocks`. From af7e6888599c4b53e31f29de85f9d9baee52260a Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Tue, 5 Apr 2022 17:01:21 -0400 Subject: [PATCH 08/54] more language cleanup for clarity --- .../expressions/custom-conditions.mdx | 127 +++++++++--------- 1 file changed, 61 insertions(+), 66 deletions(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 6b57f4f4a6..0a6f8d706d 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -6,46 +6,24 @@ description: >- # Custom Conditions -You can create custom conditions that produce custom error messages for several types of objects in a configuration. For example, you may want to check to whether an incoming image ID is formatted properly. +You can create conditions that produce custom error messages for several types of objects in a configuration. For example, you can add a condition to an input variable that checks whether incoming image IDs are formatted properly. -Custom condition checks are optional, but they can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. +Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. You can create custom conditions with the following types of expressions. -- Add [`validation` blocks](input-variable-validation) inside input `variable` blocks. -- Add [`precondition` and `postcondition`](#preconditions-and-postconditions) blocks inside `lifecycle` blocks and `precondition` blocks inside `output blocks`. - -All of these methods require a [`condition` attribute](#condition-expressions) that describes the validation requirements and an [`error_message` attribute](#error-messages) that contains explanatory text that Terraform will display to the user. +- [Input variable validation](input-variable-validation) +- [Preconditions and postconditions](#preconditions-and-postconditions) for resources, data sources, and outputs -> **Note:** Input variable validation is available in Terraform CLI v0.13.0 and later. Preconditions and postconditions are available in Terraform CLI v1.2.0 and later. -## When Terraform Evaluates Custom Validations - -Terraform will evaluate the conditions specified in `validation`,`precondition`, and `postcondition` blocks as early as possible. - -If the condition expression depends on a resource attribute that won't be known -until the apply phase then Terraform will delay checking the condition until -the apply phase, but Terraform can check all other expressions during the -planning phase, and therefore block applying a plan that would violate the -conditions. - -For example, Terraform would typically be able to detect invalid AMI tags during the planning phase, as long as `var.aws_ami_id` is not derived from another resource. However, Terraform will not detect a non-encrypted root volume until the EC2 instance is created during the apply step because that condition depends on the root volume's assigned ID, which AWS decides only when the EC2 instance is actually started. - -For conditions which Terraform must defer to the apply phase, a _precondition_ -will prevent taking whatever action was planned for a related resource, whereas -a _postcondition_ will merely halt processing after that action was already -taken, preventing any downstream actions that rely on it but not undoing the -action. - -Terraform typically has less information during the initial creation of a -full configuration than when applying subsequent changes to that configuration. -Conditions checked only during apply during initial creation may therefore -be checked during planning on subsequent updates, detecting problems sooner -in that case. - ## Input Variable Validation -To specify custom validation rules for a variable, add a `validation` block within the corresponding `variable` block. The following example checks whether the AMI ID has valid syntax. +To specify custom validation rules for a variable, add one or more `validation` blocks within the corresponding `variable` block. + +The [`condition` argument](#condition-expressions) is an expression that must use the value of the variable to return `true` if the value is valid, or `false` if it is invalid. If `condition` evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple `validation` blocks, Terraform returns error messages for _all_ failed conditions. + +The following example checks whether the AMI ID has valid syntax. ```hcl variable "image_id" { @@ -59,8 +37,6 @@ variable "image_id" { } ``` -The [`condition` argument](#condition-expressions) is an expression that must use the value of the variable to return `true` if the value is valid, or `false` if it is invalid. - If the failure of an expression is the basis of the validation decision, use [the `can` function](/language/functions/can) to detect such errors, as demonstrated in the following example. ```hcl @@ -76,44 +52,26 @@ variable "image_id" { } ``` -If `condition` evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple `validation` blocks, Terraform returns error messages for _all_ failed conditions. - ## Preconditions and Postconditions Terraform checks a precondition _before_ evaluating the object it is associated with, and checks a postcondition _after_ evaluating the object. You can add preconditions and postconditions within the following configuration blocks. +### Resources and Data Sources + The `lifecycle` block inside a `resource` or `data` block can include both `precondition` and `postcondition` blocks associated with the containing resource. -- Terraform evaluates resource preconditions before evaluating the resource's configuration arguments. Resource preconditions can take precedence over argument evaluation errors. -- Terraform evaluates resource postconditions after planning and after applying changes to a managed resource, or after reading from a data resource. Resource postcondition failures will therefore prevent applying changes to other resources that depend on the failing resource. +- Terraform evaluates preconditions before evaluating the resource's configuration arguments. Preconditions can take precedence over argument evaluation errors. +- Terraform evaluates postconditions after planning and applying changes to a managed resource, or after reading from a data source. Postcondition failures prevent changes to other resources that depend on the failing resource. + +### Outputs An `output` block can include a `precondition` block. - Terraform evaluates output value preconditions before evaluating the -`value` expression to finalize the result. Output value preconditions can take precedence over potential errors in the `value` expression. -- Preconditions can be particularly useful in a root module to prevent saving an invalid new output value in the state and to preserve the value from the previous apply, if any. -- Preconditions can serve a symmetrical purpose to input variable `validation` blocks. Whereas input variable validation checks assumptions the module makes about its inputs, output value preconditions check guarantees that the module makes about its outputs. - -### Choosing Between Preconditions and PostConditions - -You can often implement a validation check as either a postcondition of the resource producing the data or as a precondition of a resource or output value using the data. To decide which is most appropriate, consider whether the check is representing either an assumption or a guarantee. - -- **Assumption:** A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. - - Assumptions should typically be written as preconditions, so that future maintainers can find them close to the other expressions that rely on that condition, and know more about what different variations that resource is intended to allow. - -- **Guarantee:** A characteristic or behavior of an object that the rest of - the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. - - Guarantees should typically be written as postconditions, so that - future maintainers can find them close to the resource configuration that is responsible for implementing those guarantees and more easily see which behaviors are important to preserve when changing the configuration. - -We recommend also considering the following factors. - -- Which resource or output value would be most helpful to report in the error message. Terraform will always report errors in the location where the condition was declared. -- Which approach is more convenient. If a particular resource has many dependencies that all make an assumption about that resource, it can be pragmatic to declare that once as a post-condition of the resource, rather than declaring it many times as preconditions on each of the dependencies. -- Whether it is helpful to declare the same or similar conditions as both preconditions and postconditions. This can be useful if the postcondition is in a different module than the precondition because it lets the modules verify one another as they evolve independently. +`value` expression to finalize the result. Preconditions can take precedence over potential errors in the `value` expression. +- Preconditions can be particularly useful in a root module to prevent saving an invalid new output value in the state. You can also use them to preserve the value from the previous apply. +- Preconditions can serve a symmetrical purpose to input variable `validation` blocks. Whereas input variable validation checks assumptions the module makes about its inputs, preconditions check guarantees that the module makes about its outputs. ### Usage Examples @@ -140,7 +98,8 @@ data "aws_ami" "example" { # AMI attributes. postcondition { condition = self.tags["Component"] == "nomad-server" - error_message = "The selected AMI must be tagged with the Component value \"nomad-server\"." + error_message = "The selected AMI must be tagged with the + Component value \"nomad-server\"." } } } @@ -168,11 +127,11 @@ resource "aws_instance" "example" { } data "aws_ebs_volume" "example" { - # We can use data resources that refer to other resources in order to + # Use data resources that refer to other resources in order to # load extra data that isn't directly exported by a resource. # - # This example reads the details about the root storage volume for - # the EC2 instance declared by aws_instance.example, using the exported ID. + # Read the details about the root storage volume for the EC2 instance + # declared by aws_instance.example, using the exported ID. filter { name = "volume-id" @@ -195,7 +154,7 @@ output "api_base_url" { The preconditions and postconditions declare the following assumptions and guarantees. -- **The AMI ID must refer to an AMI that exists and that has been tagged with "nomad-server".** This would detect if the caller accidentally provided an AMI intended for some other system component, which might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. +- **The AMI ID must refer to an AMI that exists and that has been tagged with "nomad-server".** This would detect if the caller accidentally provided an AMI intended for some other system component. This might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. - **The AMI ID must refer to an AMI that contains an operating system for the `x86_64` architecture.** This would detect if the caller accidentally built an AMI for a different @@ -203,7 +162,28 @@ The preconditions and postconditions declare the following assumptions and guara - **The EC2 instance must be allocated a private DNS hostname.** In Amazon Web Services, EC2 instances are assigned private DNS hostnames only if they belong to a virtual network configured in a certain way. This would detect if the selected virtual network is not configured correctly, giving explicit feedback to prompt the user to debug the network settings. -- **The EC2 instance will have an encrypted root volume.** This ensures that the root volume is encrypted even though the software running in this EC2 instance would probably still operate as expected on an unencrypted volume. This lets Terraform produce an error immediately, before any other components rely on the insecurely-configured component. +- **The EC2 instance will have an encrypted root volume.** This ensures that the root volume is encrypted even though the software running in this EC2 instance would probably still operate as expected on an unencrypted volume. This lets Terraform produce an error immediately, before any other components rely on the component. + +### Choosing Between Preconditions and PostConditions + +You can often implement a validation check as either a postcondition of the resource producing the data or as a precondition of a resource or output value using the data. To decide which is most appropriate, consider whether the check is representing either an assumption or a guarantee. + +- **Assumption:** A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. + + Assumptions should typically be written as preconditions, so that future maintainers can find them close to the other expressions that rely on that condition, and know more about what different variations that resource is intended to allow. + +- **Guarantee:** A characteristic or behavior of an object that the rest of + the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. + + Guarantees should typically be written as postconditions, so that + future maintainers can find them close to the resource configuration that is responsible for implementing those guarantees and more easily see which behaviors are important to preserve when changing the configuration. + +We recommend also considering the following factors. + +- Which resource or output value would be most helpful to report in the error message. Terraform will always report errors in the location where the condition was declared. +- Which approach is more convenient. If a particular resource has many dependencies that all make an assumption about that resource, it can be pragmatic to declare that once as a post-condition of the resource, rather than declaring it many times as preconditions on each of the dependencies. +- Whether it is helpful to declare the same or similar conditions as both preconditions and postconditions. This can be useful if the postcondition is in a different module than the precondition because it lets the modules verify one another as they evolve independently. + ## Condition Expressions @@ -316,3 +296,18 @@ style similar to Terraform's own error messages. Terraform will show the given message alongside the name of the resource that detected the problem and any outside values used as part of the condition expression. +## When Terraform Evaluates Custom Conditions + +Terraform evaluates custom conditions as early as possible. + +Input variable validations can only refer to the variable value, so Terraform always evaluates them immediately. When Terraform evaluates preconditions and postconditions depends on whether the value(s) associated with the condition are known before or after applying the configuration. + +- **Known before apply:** Terraform checks the condition during the planning phase. For example, Terraform can know the value of an image ID during planning as long as it is not generated from another resource. +- **Known after apply:** Terraform delays checking that condition until the apply phase. For example, AWS only assigns the root volume ID when it starts an EC2 instance, so Terraform cannot know this value until apply. + +During the apply phase, a failed _precondition_ +will prevent Terraform from implementing planned actions for the associated resource. However, a failed _postcondition_ will halt processing after Terraform has already implemented these actions. The failed postcondition prevents any further downstream actions that rely on the resource, but does not undo the actions Terraform has already taken. + +Terraform typically has less information during the initial creation of a +full configuration than when applying subsequent changes. Therefore, conditions checked during apply for initial creation may be checked earlier during planning for subsequent updates. + From 3ae238a448c4150dc8b4b68e1d79db3486c2b575 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 6 Apr 2022 14:57:20 -0400 Subject: [PATCH 09/54] More language edits --- .../expressions/custom-conditions.mdx | 85 ++++++++----------- 1 file changed, 36 insertions(+), 49 deletions(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 0a6f8d706d..b8e9c89310 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -10,18 +10,18 @@ You can create conditions that produce custom error messages for several types o Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. -You can create custom conditions with the following types of expressions. -- [Input variable validation](input-variable-validation) +Use the following types of expressions to create custom conditions. +- [Validation](#input-variable-validation) for input variables - [Preconditions and postconditions](#preconditions-and-postconditions) for resources, data sources, and outputs --> **Note:** Input variable validation is available in Terraform CLI v0.13.0 and later. Preconditions and postconditions are available in Terraform CLI v1.2.0 and later. - ## Input Variable Validation +-> **Note:** Input variable validation is available in Terraform CLI v0.13.0 and later. + To specify custom validation rules for a variable, add one or more `validation` blocks within the corresponding `variable` block. -The [`condition` argument](#condition-expressions) is an expression that must use the value of the variable to return `true` if the value is valid, or `false` if it is invalid. If `condition` evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple `validation` blocks, Terraform returns error messages for _all_ failed conditions. +The [`condition` argument](#condition-expressions) is an expression that must use the value of the variable to return `true` if the value is valid, or `false` if it is invalid. If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple `validation` blocks, Terraform returns error messages for all failed conditions. The following example checks whether the AMI ID has valid syntax. @@ -37,7 +37,7 @@ variable "image_id" { } ``` -If the failure of an expression is the basis of the validation decision, use [the `can` function](/language/functions/can) to detect such errors, as demonstrated in the following example. +If the failure of an expression determines the validation decision, use the [`can` function](/language/functions/can) as demonstrated in the following example. ```hcl variable "image_id" { @@ -55,47 +55,49 @@ variable "image_id" { ## Preconditions and Postconditions -Terraform checks a precondition _before_ evaluating the object it is associated with, and checks a postcondition _after_ evaluating the object. You can add preconditions and postconditions within the following configuration blocks. +-> **Note:** Preconditions and postconditions are available in Terraform CLI v1.2.0 and later. + +Use `precondition` and `postcondition` blocks to add custom conditions to resources, data sources, and output values. Terraform checks a precondition _before_ evaluating the object it is associated with, and checks a postcondition _after_ evaluating the object. + +Terraform evaluates these custom conditions as early as possibly in the plan and apply cycle, but must wait to evaluate conditions that depend on unknown values until the apply phase. Refer to [Delaying Evaluation Until Apply](#delaying-evaluation-until-apply) for more details. + +You can add preconditions and postconditions within the following configuration blocks. ### Resources and Data Sources -The `lifecycle` block inside a `resource` or `data` block can include both `precondition` and `postcondition` blocks associated with the containing resource. +The `lifecycle` block inside a `resource` or `data` block can include both `precondition` and `postcondition` blocks associated with the containing resource. Terraform evaluates these blocks as follows: -- Terraform evaluates preconditions before evaluating the resource's configuration arguments. Preconditions can take precedence over argument evaluation errors. -- Terraform evaluates postconditions after planning and applying changes to a managed resource, or after reading from a data source. Postcondition failures prevent changes to other resources that depend on the failing resource. +- Preconditions after evaluating existing `count` and `for_each` arguments. This lets Terraform evaluate the precondition separately for each instance and then make `each.key`, `count.index`, etc. available to those conditions. +- Preconditions before evaluating the resource's configuration arguments. Preconditions can take precedence over argument evaluation errors. +- Postconditions after planning and applying changes to a managed resource, or after reading from a data source. Postcondition failures prevent changes to other resources that depend on the failing resource. ### Outputs An `output` block can include a `precondition` block. -- Terraform evaluates output value preconditions before evaluating the -`value` expression to finalize the result. Preconditions can take precedence over potential errors in the `value` expression. -- Preconditions can be particularly useful in a root module to prevent saving an invalid new output value in the state. You can also use them to preserve the value from the previous apply. -- Preconditions can serve a symmetrical purpose to input variable `validation` blocks. Whereas input variable validation checks assumptions the module makes about its inputs, preconditions check guarantees that the module makes about its outputs. +Preconditions can serve a symmetrical purpose to input variable `validation` blocks. Whereas input variable validation checks assumptions the module makes about its inputs, preconditions check guarantees that the module makes about its outputs. You can use preconditions to prevent Terraform from saving an invalid new output value in the state. You can also use them to preserve the output value from the previous apply, if applicable. +Terraform evaluates output value preconditions before evaluating the `value` expression to finalize the result. Preconditions can take precedence over potential errors in the `value` expression. ### Usage Examples -The following example shows several possible uses for preconditions and postconditions. +The following example shows several use cases for preconditions and postconditions. The preconditions and postconditions declare the following assumptions and guarantees. + +- **The AMI ID must refer to an existing AMI that has the tag "nomad-server".** This would detect if the caller accidentally provided an AMI intended for some other system component. This might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. + +- **The AMI ID must refer to an AMI that contains an operating system for the +`x86_64` architecture.** This would detect if the caller accidentally built an AMI for a different architecture, which may not be able to run the software this virtual machine is intended to host. + +- **The EC2 instance must be allocated a private DNS hostname.** In Amazon Web Services, EC2 instances are assigned private DNS hostnames only if they belong to a virtual network configured in a certain way. This would detect if the selected virtual network is not configured correctly, giving explicit feedback to prompt the user to debug the network settings. + +- **The EC2 instance will have an encrypted root volume.** This ensures that the root volume is encrypted even though the software running in this EC2 instance would probably still operate as expected on an unencrypted volume. This lets Terraform produce an error immediately, before any other components rely on the component. ```hcl -variable "aws_ami_id" { - type = string - - # Input variable validation can check that the AMI ID is syntactically valid. - validation { - condition = can(regex("^ami-", var.aws_ami_id)) - error_message = "The AMI ID must have the prefix \"ami-\"." - } -} - data "aws_ami" "example" { id = var.aws_ami_id lifecycle { - # A data resource with a postcondition can ensure that the selected AMI - # meets this module's expectations, by reacting to the dynamically-loaded - # AMI attributes. + # The AMI ID must refer to an existing AMI that has the tag "nomad-server". postcondition { condition = self.tags["Component"] == "nomad-server" error_message = "The selected AMI must be tagged with the @@ -109,16 +111,14 @@ resource "aws_instance" "example" { ami = "ami-abc123" lifecycle { - # A resource with a precondition can ensure that the selected AMI - # is set up correctly to work with the instance configuration. + # The AMI ID must refer to an AMI that contains an operating system + # for the `x86_64` architecture. precondition { condition = data.aws_ami.example.architecture == "x86_64" error_message = "The selected AMI must be for the x86_64 architecture." } - # A resource with a postcondition can react to server-decided values - # during the apply step and halt work immediately if the result doesn't - # meet expectations. + # The EC2 instance must be allocated a private DNS hostname. postcondition { condition = self.private_dns != "" error_message = "EC2 instance must be in a VPC that has private DNS hostnames enabled." @@ -127,7 +127,7 @@ resource "aws_instance" "example" { } data "aws_ebs_volume" "example" { - # Use data resources that refer to other resources in order to + # Use data resources that refer to other resources to # load extra data that isn't directly exported by a resource. # # Read the details about the root storage volume for the EC2 instance @@ -142,9 +142,7 @@ data "aws_ebs_volume" "example" { output "api_base_url" { value = "https://${aws_instance.example.private_dns}:8433/" - # An output value with a precondition can check the object that the - # output value is describing to make sure it meets expectations before - # any caller of this module can use it. + # The EC2 instance will have an encrypted root volume. precondition { condition = data.aws_ebs_volume.example.encrypted error_message = "The server's root volume is not encrypted." @@ -152,17 +150,6 @@ output "api_base_url" { } ``` -The preconditions and postconditions declare the following assumptions and guarantees. - -- **The AMI ID must refer to an AMI that exists and that has been tagged with "nomad-server".** This would detect if the caller accidentally provided an AMI intended for some other system component. This might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. - -- **The AMI ID must refer to an AMI that contains an operating system for the - `x86_64` architecture.** This would detect if the caller accidentally built an AMI for a different - architecture, which may not be able to run the software this virtual machine is intended to host. - -- **The EC2 instance must be allocated a private DNS hostname.** In Amazon Web Services, EC2 instances are assigned private DNS hostnames only if they belong to a virtual network configured in a certain way. This would detect if the selected virtual network is not configured correctly, giving explicit feedback to prompt the user to debug the network settings. - -- **The EC2 instance will have an encrypted root volume.** This ensures that the root volume is encrypted even though the software running in this EC2 instance would probably still operate as expected on an unencrypted volume. This lets Terraform produce an error immediately, before any other components rely on the component. ### Choosing Between Preconditions and PostConditions @@ -296,7 +283,7 @@ style similar to Terraform's own error messages. Terraform will show the given message alongside the name of the resource that detected the problem and any outside values used as part of the condition expression. -## When Terraform Evaluates Custom Conditions +## Delaying Evaluation Until Apply Terraform evaluates custom conditions as early as possible. From 62a7b51ba786e5a17a7ea8041dab459befa38785 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 6 Apr 2022 15:41:02 -0400 Subject: [PATCH 10/54] more edits --- .../expressions/custom-conditions.mdx | 71 ++++++++++--------- 1 file changed, 36 insertions(+), 35 deletions(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index b8e9c89310..ca2acb4af5 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -10,16 +10,18 @@ You can create conditions that produce custom error messages for several types o Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. -Use the following types of expressions to create custom conditions. -- [Validation](#input-variable-validation) for input variables -- [Preconditions and postconditions](#preconditions-and-postconditions) for resources, data sources, and outputs +This page explains the following: + - Creating [validation conditions](#input-variable-validation) for input variables + - Creating [preconditions and postconditions](#preconditions-and-postconditions) for resources, data sources, and outputs + - Writing effective [condition expressions](#condition-expressions) and [error messages](#error-messages) + - When Terraform [evaluates custom conditions](#delaying-evaluation-until-apply) during the plan and apply cycle ## Input Variable Validation -> **Note:** Input variable validation is available in Terraform CLI v0.13.0 and later. -To specify custom validation rules for a variable, add one or more `validation` blocks within the corresponding `variable` block. +To specify custom validation conditions for a variable, add one or more `validation` blocks within the corresponding `variable` block. The [`condition` argument](#condition-expressions) is an expression that must use the value of the variable to return `true` if the value is valid, or `false` if it is invalid. If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple `validation` blocks, Terraform returns error messages for all failed conditions. @@ -57,11 +59,30 @@ variable "image_id" { -> **Note:** Preconditions and postconditions are available in Terraform CLI v1.2.0 and later. -Use `precondition` and `postcondition` blocks to add custom conditions to resources, data sources, and output values. Terraform checks a precondition _before_ evaluating the object it is associated with, and checks a postcondition _after_ evaluating the object. +Use `precondition` and `postcondition` blocks to create custom rules for resources, data sources, and outputs. -Terraform evaluates these custom conditions as early as possibly in the plan and apply cycle, but must wait to evaluate conditions that depend on unknown values until the apply phase. Refer to [Delaying Evaluation Until Apply](#delaying-evaluation-until-apply) for more details. +Terraform checks a precondition _before_ evaluating the object it is associated with and checks a postcondition _after_ evaluating the object. Terraform evaluates custom conditions as early as possible, but must defer conditions that depend on unknown values until the apply phase. Refer to [Delaying Evaluation Until Apply](#delaying-evaluation-until-apply) for more details. -You can add preconditions and postconditions within the following configuration blocks. +Each `precondition` and `postcondition` block requires a [`condition` argument](#condition-expressions). This is an expression that must return `true` if the conditition is fufilled or `false` if it is invalid. If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple precondition or postcondition blocks, Terraform returns error messages for all failed conditions. + +The following example uses a `postcondition` to detect if the caller accidentally provided an AMI intended for the wrong system component. This might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. + +``` hcl +data "aws_ami" "example" { + id = var.aws_ami_id + + lifecycle { + # The AMI ID must refer to an existing AMI that has the tag "nomad-server". + postcondition { + condition = self.tags["Component"] == "nomad-server" + error_message = "The selected AMI must be tagged with the + Component value \"nomad-server\"." + } + } +} +``` + +You can add `precondition` and `postcondition` blocks in the following locations within your configuration. ### Resources and Data Sources @@ -81,9 +102,7 @@ Terraform evaluates output value preconditions before evaluating the `value` exp ### Usage Examples -The following example shows several use cases for preconditions and postconditions. The preconditions and postconditions declare the following assumptions and guarantees. - -- **The AMI ID must refer to an existing AMI that has the tag "nomad-server".** This would detect if the caller accidentally provided an AMI intended for some other system component. This might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. +The following example shows use cases for preconditions and postconditions. The preconditions and postconditions declare the following assumptions and guarantees. - **The AMI ID must refer to an AMI that contains an operating system for the `x86_64` architecture.** This would detect if the caller accidentally built an AMI for a different architecture, which may not be able to run the software this virtual machine is intended to host. @@ -93,18 +112,6 @@ The following example shows several use cases for preconditions and postconditio - **The EC2 instance will have an encrypted root volume.** This ensures that the root volume is encrypted even though the software running in this EC2 instance would probably still operate as expected on an unencrypted volume. This lets Terraform produce an error immediately, before any other components rely on the component. ```hcl -data "aws_ami" "example" { - id = var.aws_ami_id - - lifecycle { - # The AMI ID must refer to an existing AMI that has the tag "nomad-server". - postcondition { - condition = self.tags["Component"] == "nomad-server" - error_message = "The selected AMI must be tagged with the - Component value \"nomad-server\"." - } - } -} resource "aws_instance" "example" { instance_type = "t2.micro" @@ -150,20 +157,14 @@ output "api_base_url" { } ``` - ### Choosing Between Preconditions and PostConditions You can often implement a validation check as either a postcondition of the resource producing the data or as a precondition of a resource or output value using the data. To decide which is most appropriate, consider whether the check is representing either an assumption or a guarantee. -- **Assumption:** A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. - - Assumptions should typically be written as preconditions, so that future maintainers can find them close to the other expressions that rely on that condition, and know more about what different variations that resource is intended to allow. - -- **Guarantee:** A characteristic or behavior of an object that the rest of - the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. - - Guarantees should typically be written as postconditions, so that - future maintainers can find them close to the resource configuration that is responsible for implementing those guarantees and more easily see which behaviors are important to preserve when changing the configuration. +| Type | Definition | Expression to Use | +| -----|------------|--------------| +| Assumption | A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. | Preconditions, so that future maintainers can find them close to the other expressions that rely on that condition. This lets them understand more about what that resource is intended to allow.| +| Guarantee | A characteristic or behavior of an object that the rest of the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. | Postconditions, so that future maintainers can find them close to the resource configuration that is responsible for implementing those guarantees. This lets them more easily determine which behaviors they should preserve when changing the configuration. | We recommend also considering the following factors. @@ -174,12 +175,12 @@ We recommend also considering the following factors. ## Condition Expressions -The variable `validation` block and the `precondition` and `postcondition` blocks all require an argument named `condition`, whose value is a boolean expression which should return `true` if the intended assumption holds or `false` if it does not. +Input variable validation, preconditions, and postconditions all require a `condition` argument. This is a boolean expression that should return `true` if the intended assumption or guarantee is fulfilled or `false` if it does not. Condition expressions have the following requirements. -- For variable `validation` blocks, the expression can refer only to the variable that the condition applies to and must not produce errors. -- For `precondition` and `postcondition` blocks, the expression can refer to any other objects in the same module, as long as the references don't create any cyclic dependencies. Resource postconditions can additionally refer to attributes of each instance of the resource where they are configured, using the special symbol `self`. For example, `self.private_dns` refers to the `private_dns` attribute of each instance of the containing resource. +- **Input variable validation:** The expression can refer only to the variable that the condition applies to and must not produce errors. +- **Preconditions and postconditions:** The expression can refer to any other objects in the same module, as long as the references do not create cyclic dependencies. Resource postconditions can also use the symbol `self` to refer to attributes of each instance of the resource where they are configured. For example, `self.private_dns` refers to the `private_dns` attribute of each instance of the containing resource. You can use any of Terraform's built-in functions or language operators in a condition as long as the expression is valid and returns a boolean result. The following language features are particularly From 5803963be8dfb0ec93229013a1e8bddfd389675c Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 6 Apr 2022 15:45:27 -0400 Subject: [PATCH 11/54] more nitpicks --- website/docs/language/expressions/custom-conditions.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index ca2acb4af5..9eaf3f1f71 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -63,9 +63,9 @@ Use `precondition` and `postcondition` blocks to create custom rules for resourc Terraform checks a precondition _before_ evaluating the object it is associated with and checks a postcondition _after_ evaluating the object. Terraform evaluates custom conditions as early as possible, but must defer conditions that depend on unknown values until the apply phase. Refer to [Delaying Evaluation Until Apply](#delaying-evaluation-until-apply) for more details. -Each `precondition` and `postcondition` block requires a [`condition` argument](#condition-expressions). This is an expression that must return `true` if the conditition is fufilled or `false` if it is invalid. If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple precondition or postcondition blocks, Terraform returns error messages for all failed conditions. +Each precondition and postcondition requires a [`condition` argument](#condition-expressions). This is an expression that must return `true` if the conditition is fufilled or `false` if it is invalid. If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple preconditions or postconditions, Terraform returns error messages for all failed conditions. -The following example uses a `postcondition` to detect if the caller accidentally provided an AMI intended for the wrong system component. This might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. +The following example uses a postcondition to detect if the caller accidentally provided an AMI intended for the wrong system component. This might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. ``` hcl data "aws_ami" "example" { @@ -260,7 +260,7 @@ You can also use `can` with attribute access or index operators to concisely tes ## Error Messages -Each `validation`, `precondition` or `postcondition` block must include an argument `error_message`, which contains the text that Terraform will include as part of error messages when it detects an unmet condition. +Input variable validations, preconditions, and postconditions all must include the `error_message` argument. This contains the text that Terraform will include as part of error messages when it detects an unmet condition. ``` Error: Resource postcondition failed @@ -282,7 +282,7 @@ word wrapped. We recommend writing error messages as one or more full sentences in a style similar to Terraform's own error messages. Terraform will show the given message alongside the name of the resource that detected the problem and any -outside values used as part of the condition expression. +outside values included in the condition expression. ## Delaying Evaluation Until Apply From 04d329a9e1341cbaa30e94feb9d970f4e852091a Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 6 Apr 2022 16:11:36 -0400 Subject: [PATCH 12/54] Add references to custom conditions on related pages --- website/docs/language/data-sources/index.mdx | 24 +++++++++++++++++++ .../language/meta-arguments/lifecycle.mdx | 24 +++++++++++++++++++ .../language/modules/develop/composition.mdx | 23 ++++++++++++++++++ website/docs/language/resources/syntax.mdx | 24 +++++++++++++++++++ website/docs/language/values/outputs.mdx | 21 ++++++++++++++++ website/docs/language/values/variables.mdx | 2 +- 6 files changed, 117 insertions(+), 1 deletion(-) diff --git a/website/docs/language/data-sources/index.mdx b/website/docs/language/data-sources/index.mdx index 82e99e442b..6f5dc27c21 100644 --- a/website/docs/language/data-sources/index.mdx +++ b/website/docs/language/data-sources/index.mdx @@ -122,6 +122,30 @@ referencing the managed resource values through a `local` value. ~> **NOTE:** **In Terraform 0.12 and earlier**, due to the data resource behavior of deferring the read until the apply phase when depending on values that are not yet known, using `depends_on` with `data` resources will force the read to always be deferred to the apply phase, and therefore a configuration that uses `depends_on` with a `data` resource can never converge. Due to this behavior, we do not recommend using `depends_on` with data resources. +## Custom Conditions + +You can use `precondition` and `postcondition` blocks to specify assumptions and guarantees about how the data source operates. The following examples creates a postcondition that checks whether the AMI has the correct tags. + +``` hcl +data "aws_ami" "example" { + id = var.aws_ami_id + + lifecycle { + # The AMI ID must refer to an existing AMI that has the tag "nomad-server". + postcondition { + condition = self.tags["Component"] == "nomad-server" + error_message = "The selected AMI must be tagged with the + Component value \"nomad-server\"." + } + } +} +``` + +Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. + +Refer to [Custom Conditions](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. + + ## Multiple Resource Instances Data resources support [`count`](/language/meta-arguments/count) diff --git a/website/docs/language/meta-arguments/lifecycle.mdx b/website/docs/language/meta-arguments/lifecycle.mdx index 1554bcb397..3dd7ad0135 100644 --- a/website/docs/language/meta-arguments/lifecycle.mdx +++ b/website/docs/language/meta-arguments/lifecycle.mdx @@ -110,6 +110,30 @@ The following arguments can be used within a `lifecycle` block: Only attributes defined by the resource type can be ignored. `ignore_changes` cannot be applied to itself or to any other meta-arguments. +## Custom Conditions + +You can add `precondition` and `postcondition` blocks with a lifecycle block to specify assumptions and guarantees about how resources and data sources operate. The following examples creates a postcondition that checks whether the AMI is properly configured. + +```hcl +resource "aws_instance" "example" { + instance_type = "t2.micro" + ami = "ami-abc123" + + lifecycle { + # The AMI ID must refer to an AMI that contains an operating system + # for the `x86_64` architecture. + precondition { + condition = data.aws_ami.example.architecture == "x86_64" + error_message = "The selected AMI must be for the x86_64 architecture." + } + } +} +``` + +Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. + +Refer to [Custom Conditions](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. + ## Literal Values Only The `lifecycle` settings all affect how Terraform constructs and traverses diff --git a/website/docs/language/modules/develop/composition.mdx b/website/docs/language/modules/develop/composition.mdx index a083597e58..574262108d 100644 --- a/website/docs/language/modules/develop/composition.mdx +++ b/website/docs/language/modules/develop/composition.mdx @@ -186,6 +186,29 @@ be given inline as a single resource, but we can also compose together multiple modules as described elsewhere on this page in situations where the dependencies themselves are complicated enough to benefit from abstractions. +## Assumptions and Guarantees + +Every module has implicit assumptions and guarantees that define what data it expects and what data it produces for consumers. + +- **Assumption:** A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. +- **Guarantee:** A characteristic or behavior of an object that the rest of the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. + +We recommend using [custom conditions](/language/expressions/custom-conditions) help capture and test for assumptions and guarantees. This helps future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. + +The following examples creates a precondition that checks whether the EC2 instance has an encrypted root volume. + +```hcl +output "api_base_url" { + value = "https://${aws_instance.example.private_dns}:8433/" + + # The EC2 instance must have an encrypted root volume. + precondition { + condition = data.aws_ebs_volume.example.encrypted + error_message = "The server's root volume is not encrypted." + } +} +``` + ## Multi-cloud Abstractions Terraform itself intentionally does not attempt to abstract over similar diff --git a/website/docs/language/resources/syntax.mdx b/website/docs/language/resources/syntax.mdx index 485609ca8f..ef0ddf0622 100644 --- a/website/docs/language/resources/syntax.mdx +++ b/website/docs/language/resources/syntax.mdx @@ -131,6 +131,30 @@ The following meta-arguments are documented on separate pages: - [`lifecycle`, for lifecycle customizations](/language/meta-arguments/lifecycle) - [`provisioner`, for taking extra actions after resource creation](/language/resources/provisioners/syntax) +## Custom Conditions + +You can use `precondition` and `postcondition` blocks to specify assumptions and guarantees about how the resource operates. The following examples creates a postcondition that checks whether the AMI is properly configured. + +```hcl +resource "aws_instance" "example" { + instance_type = "t2.micro" + ami = "ami-abc123" + + lifecycle { + # The AMI ID must refer to an AMI that contains an operating system + # for the `x86_64` architecture. + precondition { + condition = data.aws_ami.example.architecture == "x86_64" + error_message = "The selected AMI must be for the x86_64 architecture." + } + } +} +``` + +Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. + +Refer to [Custom Conditions](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. + ## Operation Timeouts Some resource types provide a special `timeouts` nested block argument that diff --git a/website/docs/language/values/outputs.mdx b/website/docs/language/values/outputs.mdx index 777400e19d..e7fe6cb634 100644 --- a/website/docs/language/values/outputs.mdx +++ b/website/docs/language/values/outputs.mdx @@ -62,6 +62,27 @@ In a parent module, outputs of child modules are available in expressions as `web_server` declared an output named `instance_ip_addr`, you could access that value as `module.web_server.instance_ip_addr`. + +## Custom Conditions + +You can use `precondition` blocks to specify guarantees about output data. The following examples creates a precondition that checks whether the EC2 instance has an encrypted root volume. + +```hcl +output "api_base_url" { + value = "https://${aws_instance.example.private_dns}:8433/" + + # The EC2 instance must have an encrypted root volume. + precondition { + condition = data.aws_ebs_volume.example.encrypted + error_message = "The server's root volume is not encrypted." + } +} +``` + +Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. + +Refer to [Custom Conditions](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. + ## Optional Arguments `output` blocks can optionally include `description`, `sensitive`, and `depends_on` arguments, which are described in the following sections. diff --git a/website/docs/language/values/variables.mdx b/website/docs/language/values/variables.mdx index 6c2f42e06b..dd46103207 100644 --- a/website/docs/language/values/variables.mdx +++ b/website/docs/language/values/variables.mdx @@ -173,7 +173,7 @@ variable "image_id" { } } ``` -Refer to [Custom Data Validation](/language/expressions/custom-validation-rules) for more details. +Refer to [Custom Conditions](/language/expressions/custom-conditions#input-variable-validation) for more details. ### Suppressing Values in CLI Output From eea860e0cfe711b5aa832c4231f97bdd002eddb8 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 6 Apr 2022 16:55:43 -0400 Subject: [PATCH 13/54] Fixing typos and doing a read through --- .../expressions/custom-conditions.mdx | 57 ++++++++++--------- .../language/meta-arguments/lifecycle.mdx | 2 +- .../language/modules/develop/composition.mdx | 2 +- website/docs/language/resources/syntax.mdx | 2 +- website/docs/language/values/variables.mdx | 2 +- 5 files changed, 34 insertions(+), 31 deletions(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 9eaf3f1f71..d952ebcb4a 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -21,9 +21,11 @@ This page explains the following: -> **Note:** Input variable validation is available in Terraform CLI v0.13.0 and later. -To specify custom validation conditions for a variable, add one or more `validation` blocks within the corresponding `variable` block. +Add one or more `validation` blocks within the `variable` block to specify custom conditions. -The [`condition` argument](#condition-expressions) is an expression that must use the value of the variable to return `true` if the value is valid, or `false` if it is invalid. If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple `validation` blocks, Terraform returns error messages for all failed conditions. +Each validation requires a [`condition` argument](#condition-expressions), an expression that must use the value of the variable to return `true` if the value is valid, or `false` if it is invalid. The expression can refer only to the containing variable and must not produce errors. + +If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple `validation` blocks, Terraform returns error messages for all failed conditions. The following example checks whether the AMI ID has valid syntax. @@ -63,7 +65,11 @@ Use `precondition` and `postcondition` blocks to create custom rules for resourc Terraform checks a precondition _before_ evaluating the object it is associated with and checks a postcondition _after_ evaluating the object. Terraform evaluates custom conditions as early as possible, but must defer conditions that depend on unknown values until the apply phase. Refer to [Delaying Evaluation Until Apply](#delaying-evaluation-until-apply) for more details. -Each precondition and postcondition requires a [`condition` argument](#condition-expressions). This is an expression that must return `true` if the conditition is fufilled or `false` if it is invalid. If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple preconditions or postconditions, Terraform returns error messages for all failed conditions. +### Usage + +Each precondition and postcondition requires a [`condition` argument](#condition-expressions). This is an expression that must return `true` if the conditition is fufilled or `false` if it is invalid. The expression can refer to any other objects in the same module, as long as the references do not create cyclic dependencies. Resource postconditions can also use the symbol `self` to refer to attributes of each instance of the resource where they are configured. For example, `self.private_dns` refers to the `private_dns` attribute of each instance of the containing resource. + +If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple preconditions or postconditions, Terraform returns error messages for all failed conditions. The following example uses a postcondition to detect if the caller accidentally provided an AMI intended for the wrong system component. This might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. @@ -84,15 +90,15 @@ data "aws_ami" "example" { You can add `precondition` and `postcondition` blocks in the following locations within your configuration. -### Resources and Data Sources +#### Resources and Data Sources -The `lifecycle` block inside a `resource` or `data` block can include both `precondition` and `postcondition` blocks associated with the containing resource. Terraform evaluates these blocks as follows: +The `lifecycle` block inside a `resource` or `data` block can include both `precondition` and `postcondition` blocks. Terraform evaluates these blocks as follows: - Preconditions after evaluating existing `count` and `for_each` arguments. This lets Terraform evaluate the precondition separately for each instance and then make `each.key`, `count.index`, etc. available to those conditions. - Preconditions before evaluating the resource's configuration arguments. Preconditions can take precedence over argument evaluation errors. - Postconditions after planning and applying changes to a managed resource, or after reading from a data source. Postcondition failures prevent changes to other resources that depend on the failing resource. -### Outputs +#### Outputs An `output` block can include a `precondition` block. @@ -100,16 +106,16 @@ Preconditions can serve a symmetrical purpose to input variable `validation` blo Terraform evaluates output value preconditions before evaluating the `value` expression to finalize the result. Preconditions can take precedence over potential errors in the `value` expression. -### Usage Examples +### Examples The following example shows use cases for preconditions and postconditions. The preconditions and postconditions declare the following assumptions and guarantees. - **The AMI ID must refer to an AMI that contains an operating system for the -`x86_64` architecture.** This would detect if the caller accidentally built an AMI for a different architecture, which may not be able to run the software this virtual machine is intended to host. +`x86_64` architecture.** The precondition would detect if the caller accidentally built an AMI for a different architecture, which may not be able to run the software this virtual machine is intended to host. -- **The EC2 instance must be allocated a private DNS hostname.** In Amazon Web Services, EC2 instances are assigned private DNS hostnames only if they belong to a virtual network configured in a certain way. This would detect if the selected virtual network is not configured correctly, giving explicit feedback to prompt the user to debug the network settings. +- **The EC2 instance must be allocated a private DNS hostname.** In Amazon Web Services, EC2 instances are assigned private DNS hostnames only if they belong to a virtual network configured in a certain way. The postcondition would detect if the selected virtual network is not configured correctly, prompting the user to debug the network settings. -- **The EC2 instance will have an encrypted root volume.** This ensures that the root volume is encrypted even though the software running in this EC2 instance would probably still operate as expected on an unencrypted volume. This lets Terraform produce an error immediately, before any other components rely on the component. +- **The EC2 instance will have an encrypted root volume.** The precondition ensures that the root volume is encrypted, even though the software running in this EC2 instance would probably still operate as expected on an unencrypted volume. This lets Terraform produce an error immediately, before any other components rely on the component. ```hcl @@ -177,24 +183,21 @@ We recommend also considering the following factors. Input variable validation, preconditions, and postconditions all require a `condition` argument. This is a boolean expression that should return `true` if the intended assumption or guarantee is fulfilled or `false` if it does not. -Condition expressions have the following requirements. - -- **Input variable validation:** The expression can refer only to the variable that the condition applies to and must not produce errors. -- **Preconditions and postconditions:** The expression can refer to any other objects in the same module, as long as the references do not create cyclic dependencies. Resource postconditions can also use the symbol `self` to refer to attributes of each instance of the resource where they are configured. For example, `self.private_dns` refers to the `private_dns` attribute of each instance of the containing resource. - You can use any of Terraform's built-in functions or language operators in a condition as long as the expression is valid and returns a boolean result. The following language features are particularly useful when writing condition expressions. ### `contains` Function -You can use the built-in function `contains` to test whether a given value is one of a set of predefined valid values. + +Use the built-in function `contains` to test whether a given value is one of a set of predefined valid values. ```hcl condition = contains(["STAGE", "PROD"], var.environment) ``` ### Boolean Operators -You can use the boolean operators `&&` (AND), `||` (OR), and `!` (NOT) to combine multiple simpler conditions together. + +Use the boolean operators `&&` (AND), `||` (OR), and `!` (NOT) to combine multiple conditions together. ```hcl condition = var.name != "" && lower(var.name) == var.name @@ -202,16 +205,16 @@ You can use the boolean operators `&&` (AND), `||` (OR), and `!` (NOT) to combin ### `length` Function -You can require a non-empty list or map by testing the collection's length. +Require a non-empty list or map by testing the collection's length. ```hcl condition = length(var.items) != 0 ``` -This is a better approach than directly comparing with another collection using `==` or `!=`, because the comparison operators can only return `true` if both operands have exactly the same type, which is often ambiguous for empty collections. +This is a better approach than directly comparing with another collection using `==` or `!=`. This is because the comparison operators can only return `true` if both operands have exactly the same type, which is often ambiguous for empty collections. ### `for` Expressions -You can use `for` expressions which produce lists of boolean results -themselves in conjunction with the functions `alltrue` and `anytrue` to test whether a condition holds for all or for any elements of a collection. + +Use `for` expressions in conjunction with the functions `alltrue` and `anytrue` to test whether a condition holds for all or for any elements of a collection. ```hcl condition = alltrue([ @@ -221,9 +224,9 @@ themselves in conjunction with the functions `alltrue` and `anytrue` to test whe ### `can` Function -You can use the `can` function to concisely use the validity of an expression as a condition. It returns `true` if its given expression evaluates successfully and `false` if it returns any error, so you can use various other functions that typically return errors as a part of your condition expressions. +Use the `can` function to concisely use the validity of an expression as a condition. It returns `true` if its given expression evaluates successfully and `false` if it returns any error, so you can use various other functions that typically return errors as a part of your condition expressions. -For example, you can use `can` with `regex` to test if a string matches a particular pattern, because `regex` returns an error when given a non-matching string. +For example, you can use `can` with `regex` to test if a string matches a particular pattern because `regex` returns an error when given a non-matching string. ```hcl condition = can(regex("^[a-z]+$", var.name) @@ -244,7 +247,7 @@ You can also use `can` with the type conversion functions to test whether a valu condition = can(tolist(data.terraform_remote_state.example.outputs["items"])) ``` -You can also use `can` with attribute access or index operators to concisely test whether a collection or structural value has a particular element or index. +You can also use `can` with attribute access or index operators to test whether a collection or structural value has a particular element or index. ```hcl # var.example must have an attribute named "foo" @@ -280,9 +283,9 @@ error messages are supported, and lines with leading whitespace will not be word wrapped. We recommend writing error messages as one or more full sentences in a -style similar to Terraform's own error messages. Terraform will show the given +style similar to Terraform's own error messages. Terraform will show the message alongside the name of the resource that detected the problem and any -outside values included in the condition expression. +external values included in the condition expression. ## Delaying Evaluation Until Apply @@ -297,5 +300,5 @@ During the apply phase, a failed _precondition_ will prevent Terraform from implementing planned actions for the associated resource. However, a failed _postcondition_ will halt processing after Terraform has already implemented these actions. The failed postcondition prevents any further downstream actions that rely on the resource, but does not undo the actions Terraform has already taken. Terraform typically has less information during the initial creation of a -full configuration than when applying subsequent changes. Therefore, conditions checked during apply for initial creation may be checked earlier during planning for subsequent updates. +full configuration than when applying subsequent changes. Therefore, conditions checked during apply for initial creation may be checked during planning for subsequent updates. diff --git a/website/docs/language/meta-arguments/lifecycle.mdx b/website/docs/language/meta-arguments/lifecycle.mdx index 3dd7ad0135..88e49d96f0 100644 --- a/website/docs/language/meta-arguments/lifecycle.mdx +++ b/website/docs/language/meta-arguments/lifecycle.mdx @@ -112,7 +112,7 @@ The following arguments can be used within a `lifecycle` block: ## Custom Conditions -You can add `precondition` and `postcondition` blocks with a lifecycle block to specify assumptions and guarantees about how resources and data sources operate. The following examples creates a postcondition that checks whether the AMI is properly configured. +You can add `precondition` and `postcondition` blocks with a `lifecycle` block to specify assumptions and guarantees about how resources and data sources operate. The following examples creates a precondition that checks whether the AMI is properly configured. ```hcl resource "aws_instance" "example" { diff --git a/website/docs/language/modules/develop/composition.mdx b/website/docs/language/modules/develop/composition.mdx index 574262108d..e9758b58ac 100644 --- a/website/docs/language/modules/develop/composition.mdx +++ b/website/docs/language/modules/develop/composition.mdx @@ -193,7 +193,7 @@ Every module has implicit assumptions and guarantees that define what data it ex - **Assumption:** A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. - **Guarantee:** A characteristic or behavior of an object that the rest of the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. -We recommend using [custom conditions](/language/expressions/custom-conditions) help capture and test for assumptions and guarantees. This helps future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. +We recommend using [custom conditions](/language/expressions/custom-conditions) help capture and test for assumptions and guarantees. This helps future maintainers understand the configuration design and intent. Custom conditions also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. The following examples creates a precondition that checks whether the EC2 instance has an encrypted root volume. diff --git a/website/docs/language/resources/syntax.mdx b/website/docs/language/resources/syntax.mdx index ef0ddf0622..8f140351ab 100644 --- a/website/docs/language/resources/syntax.mdx +++ b/website/docs/language/resources/syntax.mdx @@ -133,7 +133,7 @@ The following meta-arguments are documented on separate pages: ## Custom Conditions -You can use `precondition` and `postcondition` blocks to specify assumptions and guarantees about how the resource operates. The following examples creates a postcondition that checks whether the AMI is properly configured. +You can use `precondition` and `postcondition` blocks to specify assumptions and guarantees about how the resource operates. The following examples creates a precondition that checks whether the AMI is properly configured. ```hcl resource "aws_instance" "example" { diff --git a/website/docs/language/values/variables.mdx b/website/docs/language/values/variables.mdx index dd46103207..db9318d8e1 100644 --- a/website/docs/language/values/variables.mdx +++ b/website/docs/language/values/variables.mdx @@ -160,7 +160,7 @@ commentary for module maintainers, use comments. -> This feature was introduced in Terraform CLI v0.13.0. -In addition to Type Constraints, you can specify custom validation rules for a particular variable using a `validation` block nested within the corresponding `variable` block. The example below checks whether the AMI ID has the correct syntax. +You can specify custom validation rules for a particular variable by adding a `validation` block within the corresponding `variable` block. The example below checks whether the AMI ID has the correct syntax. ```hcl variable "image_id" { From d69cb58ff5be227ac9aa8395d7a83b3c184e798e Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 6 Apr 2022 17:01:04 -0400 Subject: [PATCH 14/54] Add additional sections to condition expressions section --- .../expressions/custom-conditions.mdx | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index d952ebcb4a..3b8a8f922c 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -261,6 +261,55 @@ You can also use `can` with attribute access or index operators to test whether # intent of the condition.) ``` +### `self` Object + +Use the `self` object in postcondition blocks to refer to attributes of the instance under evaluation. + +```hcl +resource "aws_instance" "example" { + instance_type = "t2.micro" + ami = "ami-abc123" + + lifecycle { + postcondition { + condition = self.instance_state == "running" + error_message = "EC2 instance must be running." + } + } +} +``` + +### `each` and `count` Objects + +In blocks where `for_each` or `count` are set, use `each` and `count` objects to refer to other resources that are expanded in a chain. + +```hcl +variable "vpc_cidrs" { + type = set(string) +} + +data "aws_vpc" "example" { + for_each = var.vpc_cidrs + + filter { + name = "cidr" + values = [each.key] + } +} + +resource "aws_internet_gateway" "example" { + for_each = aws_vpc.example + vpc_id = each.value.id + + lifecycle { + precondition { + condition = aws_vpc.example[each.key].state == "available" + error_message = "VPC ${each.key} must be available." + } + } +} +``` + ## Error Messages Input variable validations, preconditions, and postconditions all must include the `error_message` argument. This contains the text that Terraform will include as part of error messages when it detects an unmet condition. From 6b986c94bc4389336b354357fa28def7245ee7c9 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Thu, 7 Apr 2022 10:12:49 -0400 Subject: [PATCH 15/54] Update website/docs/language/expressions/custom-conditions.mdx Co-authored-by: Martin Atkins --- website/docs/language/expressions/custom-conditions.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 3b8a8f922c..b215006b5b 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -167,7 +167,7 @@ output "api_base_url" { You can often implement a validation check as either a postcondition of the resource producing the data or as a precondition of a resource or output value using the data. To decide which is most appropriate, consider whether the check is representing either an assumption or a guarantee. -| Type | Definition | Expression to Use | +| Type | Definition | Condition Type to Use | | -----|------------|--------------| | Assumption | A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. | Preconditions, so that future maintainers can find them close to the other expressions that rely on that condition. This lets them understand more about what that resource is intended to allow.| | Guarantee | A characteristic or behavior of an object that the rest of the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. | Postconditions, so that future maintainers can find them close to the resource configuration that is responsible for implementing those guarantees. This lets them more easily determine which behaviors they should preserve when changing the configuration. | From 4ab955eeab799e05636eef502424babb2c016f6c Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Thu, 7 Apr 2022 10:13:41 -0400 Subject: [PATCH 16/54] Update website/docs/language/expressions/custom-conditions.mdx Co-authored-by: Martin Atkins --- website/docs/language/expressions/custom-conditions.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index b215006b5b..0faec5232a 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -163,7 +163,7 @@ output "api_base_url" { } ``` -### Choosing Between Preconditions and PostConditions +### Choosing Between Preconditions and Postconditions You can often implement a validation check as either a postcondition of the resource producing the data or as a precondition of a resource or output value using the data. To decide which is most appropriate, consider whether the check is representing either an assumption or a guarantee. From b997c983a53932787360d30c45ad14915e20cfda Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Thu, 7 Apr 2022 11:39:41 -0400 Subject: [PATCH 17/54] Update website/docs/language/expressions/custom-conditions.mdx Co-authored-by: Martin Atkins --- website/docs/language/expressions/custom-conditions.mdx | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 0faec5232a..2dcd5a158e 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -81,8 +81,7 @@ data "aws_ami" "example" { # The AMI ID must refer to an existing AMI that has the tag "nomad-server". postcondition { condition = self.tags["Component"] == "nomad-server" - error_message = "The selected AMI must be tagged with the - Component value \"nomad-server\"." + error_message = "The selected AMI must be tagged with the Component value \"nomad-server\"." } } } From 3bc3bc52f855dec39ffa1876f6df41ff5f250c9d Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Thu, 7 Apr 2022 11:40:01 -0400 Subject: [PATCH 18/54] Update website/docs/language/expressions/custom-conditions.mdx Co-authored-by: Martin Atkins --- website/docs/language/expressions/custom-conditions.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 2dcd5a158e..44dbef7e8b 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -114,7 +114,7 @@ The following example shows use cases for preconditions and postconditions. The - **The EC2 instance must be allocated a private DNS hostname.** In Amazon Web Services, EC2 instances are assigned private DNS hostnames only if they belong to a virtual network configured in a certain way. The postcondition would detect if the selected virtual network is not configured correctly, prompting the user to debug the network settings. -- **The EC2 instance will have an encrypted root volume.** The precondition ensures that the root volume is encrypted, even though the software running in this EC2 instance would probably still operate as expected on an unencrypted volume. This lets Terraform produce an error immediately, before any other components rely on the component. +- **The EC2 instance will have an encrypted root volume.** The precondition ensures that the root volume is encrypted, even though the software running in this EC2 instance would probably still operate as expected on an unencrypted volume. This lets Terraform produce an error immediately, before any other components rely on the new EC2 instance. ```hcl From 375b3583fdc23076c603e7768bbe75e7de82e9a6 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Thu, 7 Apr 2022 12:12:13 -0400 Subject: [PATCH 19/54] APply suggestions from PR review --- .../expressions/custom-conditions.mdx | 24 ++++++++++--------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 44dbef7e8b..9bc8e1b982 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -63,7 +63,7 @@ variable "image_id" { Use `precondition` and `postcondition` blocks to create custom rules for resources, data sources, and outputs. -Terraform checks a precondition _before_ evaluating the object it is associated with and checks a postcondition _after_ evaluating the object. Terraform evaluates custom conditions as early as possible, but must defer conditions that depend on unknown values until the apply phase. Refer to [Delaying Evaluation Until Apply](#delaying-evaluation-until-apply) for more details. +Terraform checks a precondition _before_ evaluating the object it is associated with and checks a postcondition _after_ evaluating the object. Terraform evaluates custom conditions as early as possible, but must defer conditions that depend on unknown values until the apply phase. Refer to [Conditions Checked Only During Apply](#conditions-checked-only-during-apply) for more details. ### Usage @@ -91,11 +91,10 @@ You can add `precondition` and `postcondition` blocks in the following locations #### Resources and Data Sources -The `lifecycle` block inside a `resource` or `data` block can include both `precondition` and `postcondition` blocks. Terraform evaluates these blocks as follows: +The `lifecycle` block inside a `resource` or `data` block can include both `precondition` and `postcondition` blocks. -- Preconditions after evaluating existing `count` and `for_each` arguments. This lets Terraform evaluate the precondition separately for each instance and then make `each.key`, `count.index`, etc. available to those conditions. -- Preconditions before evaluating the resource's configuration arguments. Preconditions can take precedence over argument evaluation errors. -- Postconditions after planning and applying changes to a managed resource, or after reading from a data source. Postcondition failures prevent changes to other resources that depend on the failing resource. +- Terraform evaluates `precondition` blocks after evaluating existing `count` and `for_each` arguments. This lets Terraform evaluate the precondition separately for each instance and then make `each.key`, `count.index`, etc. available to those conditions. Terraform also evaluates preconditions before evaluating the resource's configuration arguments. Preconditions can take precedence over argument evaluation errors. +- Terraform evaluates `postcondition` blocks after planning and applying changes to a managed resource, or after reading from a data source. Postcondition failures prevent changes to other resources that depend on the failing resource. #### Outputs @@ -166,12 +165,15 @@ output "api_base_url" { You can often implement a validation check as either a postcondition of the resource producing the data or as a precondition of a resource or output value using the data. To decide which is most appropriate, consider whether the check is representing either an assumption or a guarantee. -| Type | Definition | Condition Type to Use | -| -----|------------|--------------| -| Assumption | A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. | Preconditions, so that future maintainers can find them close to the other expressions that rely on that condition. This lets them understand more about what that resource is intended to allow.| -| Guarantee | A characteristic or behavior of an object that the rest of the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. | Postconditions, so that future maintainers can find them close to the resource configuration that is responsible for implementing those guarantees. This lets them more easily determine which behaviors they should preserve when changing the configuration. | +**Assumption:** A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. -We recommend also considering the following factors. +We recommend using preconditions for assumptions, so that future maintainers can find them close to the other expressions that rely on that condition. This lets them understand more about what that resource is intended to allow. + +**Guarantee:** A characteristic or behavior of an object that the rest of the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. + +We recommend using postconditions for guarantees, so that future maintainers can find them close to the resource configuration that is responsible for implementing those guarantees. This lets them more easily determine which behaviors they should preserve when changing the configuration. + +You should also consider the following factors. - Which resource or output value would be most helpful to report in the error message. Terraform will always report errors in the location where the condition was declared. - Which approach is more convenient. If a particular resource has many dependencies that all make an assumption about that resource, it can be pragmatic to declare that once as a post-condition of the resource, rather than declaring it many times as preconditions on each of the dependencies. @@ -335,7 +337,7 @@ style similar to Terraform's own error messages. Terraform will show the message alongside the name of the resource that detected the problem and any external values included in the condition expression. -## Delaying Evaluation Until Apply +## Conditions Checked Only During Apply Terraform evaluates custom conditions as early as possible. From 4c097842df71d42a3c4e6c19ceec22c5e54c1014 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Thu, 7 Apr 2022 13:48:44 -0400 Subject: [PATCH 20/54] Move conditionals information to conditional expressions page --- .../language/expressions/conditionals.mdx | 125 +++++++++++++++++ .../expressions/custom-conditions.mdx | 126 +----------------- 2 files changed, 126 insertions(+), 125 deletions(-) diff --git a/website/docs/language/expressions/conditionals.mdx b/website/docs/language/expressions/conditionals.mdx index 8f5fca61f2..f542483f85 100644 --- a/website/docs/language/expressions/conditionals.mdx +++ b/website/docs/language/expressions/conditionals.mdx @@ -39,6 +39,131 @@ The condition can be any expression that resolves to a boolean value. This will usually be an expression that uses the equality, comparison, or logical operators. +The following language features are particularly useful when writing condition expressions. + +### `contains` Function + +Use the built-in function `contains` to test whether a given value is one of a set of predefined valid values. + +```hcl + condition = contains(["STAGE", "PROD"], var.environment) +``` + +### Boolean Operators + +Use the boolean operators `&&` (AND), `||` (OR), and `!` (NOT) to combine multiple conditions together. + +```hcl + condition = var.name != "" && lower(var.name) == var.name + ``` + +### `length` Function + +Require a non-empty list or map by testing the collection's length. + +```hcl + condition = length(var.items) != 0 +``` +This is a better approach than directly comparing with another collection using `==` or `!=`. This is because the comparison operators can only return `true` if both operands have exactly the same type, which is often ambiguous for empty collections. + +### `for` Expressions + +Use `for` expressions in conjunction with the functions `alltrue` and `anytrue` to test whether a condition holds for all or for any elements of a collection. + +```hcl + condition = alltrue([ + for v in var.instances : contains(["t2.micro", "m3.medium"], v.type) + ]) +``` + +### `can` Function + +Use the `can` function to concisely use the validity of an expression as a condition. It returns `true` if its given expression evaluates successfully and `false` if it returns any error, so you can use various other functions that typically return errors as a part of your condition expressions. + +For example, you can use `can` with `regex` to test if a string matches a particular pattern because `regex` returns an error when given a non-matching string. + +```hcl + condition = can(regex("^[a-z]+$", var.name) +``` + +You can also use `can` with the type conversion functions to test whether a value is convertible to a type or type constraint. + +```hcl + # This remote output value must have a value that can + # be used as a string, which includes strings themselves + # but also allows numbers and boolean values. + condition = can(tostring(data.terraform_remote_state.example.outputs["name"])) +``` + +```hcl + # This remote output value must be convertible to a list + # type of with element type. + condition = can(tolist(data.terraform_remote_state.example.outputs["items"])) +``` + +You can also use `can` with attribute access or index operators to test whether a collection or structural value has a particular element or index. + +```hcl + # var.example must have an attribute named "foo" + condition = can(var.example.foo) ``` + +```hcl + # var.example must be a sequence with at least one element + condition = can(var.example[0]) + # (although it would typically be clearer to write this as a + # test like length(var.example) > 0 to better represent the + # intent of the condition.) +``` + +### `self` Object + +Use the `self` object in postcondition blocks to refer to attributes of the instance under evaluation. + +```hcl +resource "aws_instance" "example" { + instance_type = "t2.micro" + ami = "ami-abc123" + + lifecycle { + postcondition { + condition = self.instance_state == "running" + error_message = "EC2 instance must be running." + } + } +} +``` + +### `each` and `count` Objects + +In blocks where `for_each` or `count` are set, use `each` and `count` objects to refer to other resources that are expanded in a chain. + +```hcl +variable "vpc_cidrs" { + type = set(string) +} + +data "aws_vpc" "example" { + for_each = var.vpc_cidrs + + filter { + name = "cidr" + values = [each.key] + } +} + +resource "aws_internet_gateway" "example" { + for_each = aws_vpc.example + vpc_id = each.value.id + + lifecycle { + precondition { + condition = aws_vpc.example[each.key].state == "available" + error_message = "VPC ${each.key} must be available." + } + } +} +``` + ## Result Types The two result values may be of any type, but they must both diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 9bc8e1b982..abf984bb38 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -185,131 +185,7 @@ You should also consider the following factors. Input variable validation, preconditions, and postconditions all require a `condition` argument. This is a boolean expression that should return `true` if the intended assumption or guarantee is fulfilled or `false` if it does not. You can use any of Terraform's built-in functions or language operators -in a condition as long as the expression is valid and returns a boolean result. The following language features are particularly -useful when writing condition expressions. - -### `contains` Function - -Use the built-in function `contains` to test whether a given value is one of a set of predefined valid values. - -```hcl - condition = contains(["STAGE", "PROD"], var.environment) -``` - -### Boolean Operators - -Use the boolean operators `&&` (AND), `||` (OR), and `!` (NOT) to combine multiple conditions together. - -```hcl - condition = var.name != "" && lower(var.name) == var.name - ``` - -### `length` Function - -Require a non-empty list or map by testing the collection's length. - -```hcl - condition = length(var.items) != 0 -``` -This is a better approach than directly comparing with another collection using `==` or `!=`. This is because the comparison operators can only return `true` if both operands have exactly the same type, which is often ambiguous for empty collections. - -### `for` Expressions - -Use `for` expressions in conjunction with the functions `alltrue` and `anytrue` to test whether a condition holds for all or for any elements of a collection. - -```hcl - condition = alltrue([ - for v in var.instances : contains(["t2.micro", "m3.medium"], v.type) - ]) -``` - -### `can` Function - -Use the `can` function to concisely use the validity of an expression as a condition. It returns `true` if its given expression evaluates successfully and `false` if it returns any error, so you can use various other functions that typically return errors as a part of your condition expressions. - -For example, you can use `can` with `regex` to test if a string matches a particular pattern because `regex` returns an error when given a non-matching string. - -```hcl - condition = can(regex("^[a-z]+$", var.name) -``` - -You can also use `can` with the type conversion functions to test whether a value is convertible to a type or type constraint. - -```hcl - # This remote output value must have a value that can - # be used as a string, which includes strings themselves - # but also allows numbers and boolean values. - condition = can(tostring(data.terraform_remote_state.example.outputs["name"])) -``` - -```hcl - # This remote output value must be convertible to a list - # type of with element type. - condition = can(tolist(data.terraform_remote_state.example.outputs["items"])) -``` - -You can also use `can` with attribute access or index operators to test whether a collection or structural value has a particular element or index. - -```hcl - # var.example must have an attribute named "foo" - condition = can(var.example.foo) ``` - -```hcl - # var.example must be a sequence with at least one element - condition = can(var.example[0]) - # (although it would typically be clearer to write this as a - # test like length(var.example) > 0 to better represent the - # intent of the condition.) -``` - -### `self` Object - -Use the `self` object in postcondition blocks to refer to attributes of the instance under evaluation. - -```hcl -resource "aws_instance" "example" { - instance_type = "t2.micro" - ami = "ami-abc123" - - lifecycle { - postcondition { - condition = self.instance_state == "running" - error_message = "EC2 instance must be running." - } - } -} -``` - -### `each` and `count` Objects - -In blocks where `for_each` or `count` are set, use `each` and `count` objects to refer to other resources that are expanded in a chain. - -```hcl -variable "vpc_cidrs" { - type = set(string) -} - -data "aws_vpc" "example" { - for_each = var.vpc_cidrs - - filter { - name = "cidr" - values = [each.key] - } -} - -resource "aws_internet_gateway" "example" { - for_each = aws_vpc.example - vpc_id = each.value.id - - lifecycle { - precondition { - condition = aws_vpc.example[each.key].state == "available" - error_message = "VPC ${each.key} must be available." - } - } -} -``` +in a condition as long as the expression is valid and returns a boolean result. Refer to [Conditional Expressions](/language/expressions/conditionals) for more details and examples. ## Error Messages From 3ed8c5c4679fb4b5d8f8efc11dd52812dc9fd453 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Thu, 7 Apr 2022 13:51:56 -0400 Subject: [PATCH 21/54] Fix in-text link --- website/docs/language/expressions/custom-conditions.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index abf984bb38..3bb8ec442a 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -14,7 +14,7 @@ This page explains the following: - Creating [validation conditions](#input-variable-validation) for input variables - Creating [preconditions and postconditions](#preconditions-and-postconditions) for resources, data sources, and outputs - Writing effective [condition expressions](#condition-expressions) and [error messages](#error-messages) - - When Terraform [evaluates custom conditions](#delaying-evaluation-until-apply) during the plan and apply cycle + - When Terraform [evaluates custom conditions](#conditions-checked-only-during-apply) during the plan and apply cycle ## Input Variable Validation @@ -185,7 +185,7 @@ You should also consider the following factors. Input variable validation, preconditions, and postconditions all require a `condition` argument. This is a boolean expression that should return `true` if the intended assumption or guarantee is fulfilled or `false` if it does not. You can use any of Terraform's built-in functions or language operators -in a condition as long as the expression is valid and returns a boolean result. Refer to [Conditional Expressions](/language/expressions/conditionals) for more details and examples. +in a condition as long as the expression is valid and returns a boolean result. Refer to [Conditional Expressions](/language/expressions/conditionals#conditions) for more details and examples. ## Error Messages From f9462d5d518a5946b932fc7d188308740f863b73 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Thu, 7 Apr 2022 16:25:09 -0400 Subject: [PATCH 22/54] Update links to say "Custom Condition Checks" --- website/data/language-nav-data.json | 8 ++++---- website/docs/language/data-sources/index.mdx | 4 ++-- website/docs/language/expressions/custom-conditions.mdx | 6 +++--- website/docs/language/meta-arguments/lifecycle.mdx | 2 +- website/docs/language/resources/syntax.mdx | 4 ++-- website/docs/language/values/outputs.mdx | 4 ++-- 6 files changed, 14 insertions(+), 14 deletions(-) diff --git a/website/data/language-nav-data.json b/website/data/language-nav-data.json index 656cbe1019..8bd51e9834 100644 --- a/website/data/language-nav-data.json +++ b/website/data/language-nav-data.json @@ -271,6 +271,10 @@ "title": "Dynamic Blocks", "path": "expressions/dynamic-blocks" }, + { + "title": "Custom Condition Checks", + "path": "expressions/custom-conditions" + }, { "title": "Type Constraints", "path": "expressions/type-constraints" @@ -278,10 +282,6 @@ { "title": "Version Constraints", "path": "expressions/version-constraints" - }, - { - "title": "Custom Conditions", - "path": "expressions/custom-conditions" } ] }, diff --git a/website/docs/language/data-sources/index.mdx b/website/docs/language/data-sources/index.mdx index 6f5dc27c21..2bc9e56648 100644 --- a/website/docs/language/data-sources/index.mdx +++ b/website/docs/language/data-sources/index.mdx @@ -122,7 +122,7 @@ referencing the managed resource values through a `local` value. ~> **NOTE:** **In Terraform 0.12 and earlier**, due to the data resource behavior of deferring the read until the apply phase when depending on values that are not yet known, using `depends_on` with `data` resources will force the read to always be deferred to the apply phase, and therefore a configuration that uses `depends_on` with a `data` resource can never converge. Due to this behavior, we do not recommend using `depends_on` with data resources. -## Custom Conditions +## Custom Condition Checks You can use `precondition` and `postcondition` blocks to specify assumptions and guarantees about how the data source operates. The following examples creates a postcondition that checks whether the AMI has the correct tags. @@ -143,7 +143,7 @@ data "aws_ami" "example" { Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. -Refer to [Custom Conditions](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. +Refer to [Custom Condition Checks](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. ## Multiple Resource Instances diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 3bb8ec442a..6ef293c6d7 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -1,10 +1,10 @@ --- -page_title: Custom Conditions - Configuration Language +page_title: Custom Condition Checks - Configuration Language description: >- - Validate requirements for variables, outputs, and within lifecycle blocks so Terraform can produce better error messages in context. + Check custom requirements for variables, outputs, data sources, and resources and provide better error messages in context. --- -# Custom Conditions +# Custom Condition Checks You can create conditions that produce custom error messages for several types of objects in a configuration. For example, you can add a condition to an input variable that checks whether incoming image IDs are formatted properly. diff --git a/website/docs/language/meta-arguments/lifecycle.mdx b/website/docs/language/meta-arguments/lifecycle.mdx index 88e49d96f0..40ee9cc58c 100644 --- a/website/docs/language/meta-arguments/lifecycle.mdx +++ b/website/docs/language/meta-arguments/lifecycle.mdx @@ -110,7 +110,7 @@ The following arguments can be used within a `lifecycle` block: Only attributes defined by the resource type can be ignored. `ignore_changes` cannot be applied to itself or to any other meta-arguments. -## Custom Conditions +## Custom Condition Checks You can add `precondition` and `postcondition` blocks with a `lifecycle` block to specify assumptions and guarantees about how resources and data sources operate. The following examples creates a precondition that checks whether the AMI is properly configured. diff --git a/website/docs/language/resources/syntax.mdx b/website/docs/language/resources/syntax.mdx index 8f140351ab..fee42fa633 100644 --- a/website/docs/language/resources/syntax.mdx +++ b/website/docs/language/resources/syntax.mdx @@ -131,7 +131,7 @@ The following meta-arguments are documented on separate pages: - [`lifecycle`, for lifecycle customizations](/language/meta-arguments/lifecycle) - [`provisioner`, for taking extra actions after resource creation](/language/resources/provisioners/syntax) -## Custom Conditions +## Custom Condition Checks You can use `precondition` and `postcondition` blocks to specify assumptions and guarantees about how the resource operates. The following examples creates a precondition that checks whether the AMI is properly configured. @@ -153,7 +153,7 @@ resource "aws_instance" "example" { Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. -Refer to [Custom Conditions](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. +Refer to [Custom Condition Checks](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. ## Operation Timeouts diff --git a/website/docs/language/values/outputs.mdx b/website/docs/language/values/outputs.mdx index e7fe6cb634..cb599deff9 100644 --- a/website/docs/language/values/outputs.mdx +++ b/website/docs/language/values/outputs.mdx @@ -63,7 +63,7 @@ In a parent module, outputs of child modules are available in expressions as value as `module.web_server.instance_ip_addr`. -## Custom Conditions +## Custom Condition Checks You can use `precondition` blocks to specify guarantees about output data. The following examples creates a precondition that checks whether the EC2 instance has an encrypted root volume. @@ -81,7 +81,7 @@ output "api_base_url" { Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. -Refer to [Custom Conditions](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. +Refer to [Custom Condition Checks](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. ## Optional Arguments From 01628f0d50361cc957f7d6b9e2ea48d098aa959f Mon Sep 17 00:00:00 2001 From: James Bardin Date: Mon, 11 Apr 2022 09:41:22 -0400 Subject: [PATCH 23/54] data schema changes may prevent state decoding Data sources do not have state migrations, so there may be no way to decode the prior state when faced with incompatible type changes. Because prior state is only informational to the plan, and its existence should not effect the planning process, we can skip decoding when faced with errors. --- internal/terraform/context_plan2_test.go | 63 +++++++++++++++++++ internal/terraform/node_resource_abstract.go | 12 ++-- .../terraform/node_resource_plan_instance.go | 3 +- internal/terraform/upgrade_resource_state.go | 2 +- 4 files changed, 74 insertions(+), 6 deletions(-) diff --git a/internal/terraform/context_plan2_test.go b/internal/terraform/context_plan2_test.go index b5c65d8ee8..63fa95cdbe 100644 --- a/internal/terraform/context_plan2_test.go +++ b/internal/terraform/context_plan2_test.go @@ -2963,3 +2963,66 @@ output "a" { } } } + +func TestContext2Plan_dataSchemaChange(t *testing.T) { + // We can't decode the prior state when a data source upgrades the schema + // in an incompatible way. Since prior state for data sources is purely + // informational, decoding should be skipped altogether. + m := testModuleInline(t, map[string]string{ + "main.tf": ` +data "test_object" "a" { + obj { + # args changes from a list to a map + args = { + val = "string" + } + } +} +`, + }) + + p := new(MockProvider) + p.GetProviderSchemaResponse = getProviderSchemaResponseFromProviderSchema(&ProviderSchema{ + DataSources: map[string]*configschema.Block{ + "test_object": { + Attributes: map[string]*configschema.Attribute{ + "id": { + Type: cty.String, + Computed: true, + }, + }, + BlockTypes: map[string]*configschema.NestedBlock{ + "obj": { + Block: configschema.Block{ + Attributes: map[string]*configschema.Attribute{ + "args": {Type: cty.Map(cty.String), Optional: true}, + }, + }, + Nesting: configschema.NestingSet, + }, + }, + }, + }, + }) + + p.ReadDataSourceFn = func(req providers.ReadDataSourceRequest) (resp providers.ReadDataSourceResponse) { + resp.State = req.Config + return resp + } + + state := states.BuildState(func(s *states.SyncState) { + s.SetResourceInstanceCurrent(mustResourceInstanceAddr(`data.test_object.a`), &states.ResourceInstanceObjectSrc{ + AttrsJSON: []byte(`{"id":"old","obj":[{"args":["string"]}]}`), + Status: states.ObjectReady, + }, mustProviderConfig(`provider["registry.terraform.io/hashicorp/test"]`)) + }) + + ctx := testContext2(t, &ContextOpts{ + Providers: map[addrs.Provider]providers.Factory{ + addrs.NewDefaultProvider("test"): testProviderFuncFixed(p), + }, + }) + + _, diags := ctx.Plan(m, state, DefaultPlanOpts) + assertNoErrors(t, diags) +} diff --git a/internal/terraform/node_resource_abstract.go b/internal/terraform/node_resource_abstract.go index a15214b790..73e8c4859b 100644 --- a/internal/terraform/node_resource_abstract.go +++ b/internal/terraform/node_resource_abstract.go @@ -389,15 +389,19 @@ func (n *NodeAbstractResource) readResourceInstanceState(ctx EvalContext, addr a } diags = diags.Append(upgradeDiags) if diags.HasErrors() { - // Note that we don't have any channel to return warnings here. We'll - // accept that for now since warnings during a schema upgrade would - // be pretty weird anyway, since this operation is supposed to seem - // invisible to the user. return nil, diags } obj, err := src.Decode(schema.ImpliedType()) if err != nil { + // In the case of a data source which contains incompatible state + // migrations, we can just ignore decoding errors and skip comparing + // the prior state. + if addr.Resource.Resource.Mode == addrs.DataResourceMode { + log.Printf("[DEBUG] readResourceInstanceState: data source schema change for %s prevents decoding: %s", addr, err) + return nil, diags + } + diags = diags.Append(err) } diff --git a/internal/terraform/node_resource_plan_instance.go b/internal/terraform/node_resource_plan_instance.go index bafbb88f49..8d5e449b01 100644 --- a/internal/terraform/node_resource_plan_instance.go +++ b/internal/terraform/node_resource_plan_instance.go @@ -84,7 +84,8 @@ func (n *NodePlannableResourceInstance) dataResourceExecute(ctx EvalContext) (di // However, note that we don't have any explicit mechanism for upgrading // data resource results as we do for managed resources, and so the // prevRunState might not conform to the current schema if the - // previous run was with a different provider version. + // previous run was with a different provider version. In that case the + // snapshot will be null if we could not decode it at all. diags = diags.Append(n.writeResourceInstanceState(ctx, state, prevRunState)) if diags.HasErrors() { return diags diff --git a/internal/terraform/upgrade_resource_state.go b/internal/terraform/upgrade_resource_state.go index 2b748c5c69..647db361cc 100644 --- a/internal/terraform/upgrade_resource_state.go +++ b/internal/terraform/upgrade_resource_state.go @@ -127,7 +127,7 @@ func stripRemovedStateAttributes(state []byte, ty cty.Type) []byte { if err != nil { // we just log any errors here, and let the normal decode process catch // invalid JSON. - log.Printf("[ERROR] UpgradeResourceState: %s", err) + log.Printf("[ERROR] UpgradeResourceState: stripRemovedStateAttributes: %s", err) return state } From a7987dec9fc75798283cae647ab1a92f26461c72 Mon Sep 17 00:00:00 2001 From: James Bardin Date: Mon, 11 Apr 2022 10:12:35 -0400 Subject: [PATCH 24/54] remove redundant readResourceInstanceState --- internal/terraform/node_resource_apply_instance.go | 6 ------ 1 file changed, 6 deletions(-) diff --git a/internal/terraform/node_resource_apply_instance.go b/internal/terraform/node_resource_apply_instance.go index 2fa29ea5ce..8940810d3f 100644 --- a/internal/terraform/node_resource_apply_instance.go +++ b/internal/terraform/node_resource_apply_instance.go @@ -264,12 +264,6 @@ func (n *NodeApplyableResourceInstance) managedResourceExecute(ctx EvalContext) return diags } - state, readDiags = n.readResourceInstanceState(ctx, n.ResourceInstanceAddr()) - diags = diags.Append(readDiags) - if diags.HasErrors() { - return diags - } - diffApply = reducePlan(addr, diffApply, false) // reducePlan may have simplified our planned change // into a NoOp if it only requires destroying, since destroying From 29ecac08084970842c2dd0016400aa7753c9f4d5 Mon Sep 17 00:00:00 2001 From: James Bardin Date: Mon, 11 Apr 2022 10:19:45 -0400 Subject: [PATCH 25/54] remove the use of data source prior state entirely After data source handling was moved from a separate refresh phase into the planning phase, reading the existing state was only used for informational purposes. This had been reduced to reporting warnings when the provider returned an unexpected value to try and help locate legacy provider bugs, but any actual issues located from those warnings were very few and far between. Because the prior state cannot be reliably decoded when faced with incompatible provider schema upgrades, and there is no longer any significant reason to try and get the prior state at all, we can skip the process entirely. --- .../node_resource_abstract_instance.go | 33 +------------------ .../terraform/node_resource_plan_instance.go | 22 +------------ 2 files changed, 2 insertions(+), 53 deletions(-) diff --git a/internal/terraform/node_resource_abstract_instance.go b/internal/terraform/node_resource_abstract_instance.go index 7b8d1a1dbc..6a75274bb1 100644 --- a/internal/terraform/node_resource_abstract_instance.go +++ b/internal/terraform/node_resource_abstract_instance.go @@ -1477,7 +1477,7 @@ func (n *NodeAbstractResourceInstance) providerMetas(ctx EvalContext) (cty.Value // value, but it still matches the previous state, then we can record a NoNop // change. If the states don't match then we record a Read change so that the // new value is applied to the state. -func (n *NodeAbstractResourceInstance) planDataSource(ctx EvalContext, currentState *states.ResourceInstanceObject, checkRuleSeverity tfdiags.Severity) (*plans.ResourceInstanceChange, *states.ResourceInstanceObject, instances.RepetitionData, tfdiags.Diagnostics) { +func (n *NodeAbstractResourceInstance) planDataSource(ctx EvalContext, checkRuleSeverity tfdiags.Severity) (*plans.ResourceInstanceChange, *states.ResourceInstanceObject, instances.RepetitionData, tfdiags.Diagnostics) { var diags tfdiags.Diagnostics var keyData instances.RepetitionData var configVal cty.Value @@ -1500,9 +1500,6 @@ func (n *NodeAbstractResourceInstance) planDataSource(ctx EvalContext, currentSt objTy := schema.ImpliedType() priorVal := cty.NullVal(objTy) - if currentState != nil { - priorVal = currentState.Value - } forEach, _ := evaluateForEachExpression(config.ForEach, ctx) keyData = EvalDataForInstanceKey(n.ResourceInstanceAddr().Resource.Key, forEach) @@ -1515,9 +1512,6 @@ func (n *NodeAbstractResourceInstance) planDataSource(ctx EvalContext, currentSt ) diags = diags.Append(checkDiags) if diags.HasErrors() { - diags = diags.Append(ctx.Hook(func(h Hook) (HookAction, error) { - return h.PostApply(n.Addr, states.CurrentGen, priorVal, diags.Err()) - })) return nil, nil, keyData, diags // failed preconditions prevent further evaluation } @@ -1529,9 +1523,6 @@ func (n *NodeAbstractResourceInstance) planDataSource(ctx EvalContext, currentSt } unmarkedConfigVal, configMarkPaths := configVal.UnmarkDeepWithPaths() - // We drop marks on the values used here as the result is only - // temporarily used for validation. - unmarkedPriorVal, _ := priorVal.UnmarkDeep() configKnown := configVal.IsWhollyKnown() // If our configuration contains any unknown values, or we depend on any @@ -1581,28 +1572,6 @@ func (n *NodeAbstractResourceInstance) planDataSource(ctx EvalContext, currentSt return nil, nil, keyData, diags } - // if we have a prior value, we can check for any irregularities in the response - if !priorVal.IsNull() { - // While we don't propose planned changes for data sources, we can - // generate a proposed value for comparison to ensure the data source - // is returning a result following the rules of the provider contract. - proposedVal := objchange.ProposedNew(schema, unmarkedPriorVal, unmarkedConfigVal) - if errs := objchange.AssertObjectCompatible(schema, proposedVal, newVal); len(errs) > 0 { - // Resources have the LegacyTypeSystem field to signal when they are - // using an SDK which may not produce precise values. While data - // sources are read-only, they can still return a value which is not - // compatible with the config+schema. Since we can't detect the legacy - // type system, we can only warn about this for now. - var buf strings.Builder - fmt.Fprintf(&buf, "[WARN] Provider %q produced an unexpected new value for %s.", - n.ResolvedProvider, n.Addr) - for _, err := range errs { - fmt.Fprintf(&buf, "\n - %s", tfdiags.FormatError(err)) - } - log.Print(buf.String()) - } - } - plannedNewState := &states.ResourceInstanceObject{ Value: newVal, Status: states.ObjectReady, diff --git a/internal/terraform/node_resource_plan_instance.go b/internal/terraform/node_resource_plan_instance.go index 8d5e449b01..4b1da59438 100644 --- a/internal/terraform/node_resource_plan_instance.go +++ b/internal/terraform/node_resource_plan_instance.go @@ -71,26 +71,6 @@ func (n *NodePlannableResourceInstance) dataResourceExecute(ctx EvalContext) (di return diags } - state, readDiags := n.readResourceInstanceState(ctx, addr) - diags = diags.Append(readDiags) - if diags.HasErrors() { - return diags - } - - // We'll save a snapshot of what we just read from the state into the - // prevRunState which will capture the result read in the previous - // run, possibly tweaked by any upgrade steps that - // readResourceInstanceState might've made. - // However, note that we don't have any explicit mechanism for upgrading - // data resource results as we do for managed resources, and so the - // prevRunState might not conform to the current schema if the - // previous run was with a different provider version. In that case the - // snapshot will be null if we could not decode it at all. - diags = diags.Append(n.writeResourceInstanceState(ctx, state, prevRunState)) - if diags.HasErrors() { - return diags - } - diags = diags.Append(validateSelfRef(addr.Resource, config.Config, providerSchema)) if diags.HasErrors() { return diags @@ -101,7 +81,7 @@ func (n *NodePlannableResourceInstance) dataResourceExecute(ctx EvalContext) (di checkRuleSeverity = tfdiags.Warning } - change, state, repeatData, planDiags := n.planDataSource(ctx, state, checkRuleSeverity) + change, state, repeatData, planDiags := n.planDataSource(ctx, checkRuleSeverity) diags = diags.Append(planDiags) if diags.HasErrors() { return diags From 74885b1108090461879b0e0aeadb13dabafe0f46 Mon Sep 17 00:00:00 2001 From: James Bardin Date: Mon, 11 Apr 2022 11:55:53 -0400 Subject: [PATCH 26/54] remove data sources from state read and upgrade Data sources should not require reading the previous versions. While we previously skipped the decoding if it were to fail, this removes the need for any prior state at all. The only place where the prior state was functionally used was in the destroy path. Because a data source destroy is only for cleanup purposes to clean out the state using the same code paths as a managed resource, we can substitute the prior state in the change change with a null value to maintain the same behavior. --- internal/terraform/node_resource_abstract.go | 8 ----- .../terraform/node_resource_plan_destroy.go | 35 +++++++++++++++++++ internal/terraform/upgrade_resource_state.go | 13 ++++--- 3 files changed, 43 insertions(+), 13 deletions(-) diff --git a/internal/terraform/node_resource_abstract.go b/internal/terraform/node_resource_abstract.go index 73e8c4859b..4903cdbe37 100644 --- a/internal/terraform/node_resource_abstract.go +++ b/internal/terraform/node_resource_abstract.go @@ -394,14 +394,6 @@ func (n *NodeAbstractResource) readResourceInstanceState(ctx EvalContext, addr a obj, err := src.Decode(schema.ImpliedType()) if err != nil { - // In the case of a data source which contains incompatible state - // migrations, we can just ignore decoding errors and skip comparing - // the prior state. - if addr.Resource.Resource.Mode == addrs.DataResourceMode { - log.Printf("[DEBUG] readResourceInstanceState: data source schema change for %s prevents decoding: %s", addr, err) - return nil, diags - } - diags = diags.Append(err) } diff --git a/internal/terraform/node_resource_plan_destroy.go b/internal/terraform/node_resource_plan_destroy.go index 907f20519c..dd8216445d 100644 --- a/internal/terraform/node_resource_plan_destroy.go +++ b/internal/terraform/node_resource_plan_destroy.go @@ -1,10 +1,13 @@ package terraform import ( + "fmt" + "github.com/hashicorp/terraform/internal/addrs" "github.com/hashicorp/terraform/internal/plans" "github.com/hashicorp/terraform/internal/states" "github.com/hashicorp/terraform/internal/tfdiags" + "github.com/zclconf/go-cty/cty" ) // NodePlanDestroyableResourceInstance represents a resource that is ready @@ -39,6 +42,19 @@ func (n *NodePlanDestroyableResourceInstance) DestroyAddr() *addrs.AbsResourceIn func (n *NodePlanDestroyableResourceInstance) Execute(ctx EvalContext, op walkOperation) (diags tfdiags.Diagnostics) { addr := n.ResourceInstanceAddr() + switch addr.Resource.Resource.Mode { + case addrs.ManagedResourceMode: + return n.managedResourceExecute(ctx, op) + case addrs.DataResourceMode: + return n.dataResourceExecute(ctx, op) + default: + panic(fmt.Errorf("unsupported resource mode %s", n.Config.Mode)) + } +} + +func (n *NodePlanDestroyableResourceInstance) managedResourceExecute(ctx EvalContext, op walkOperation) (diags tfdiags.Diagnostics) { + addr := n.ResourceInstanceAddr() + // Declare a bunch of variables that are used for state during // evaluation. These are written to by address in the EvalNodes we // declare below. @@ -85,3 +101,22 @@ func (n *NodePlanDestroyableResourceInstance) Execute(ctx EvalContext, op walkOp diags = diags.Append(n.writeChange(ctx, change, "")) return diags } + +func (n *NodePlanDestroyableResourceInstance) dataResourceExecute(ctx EvalContext, op walkOperation) (diags tfdiags.Diagnostics) { + + // We may not be able to read a prior data source from the state if the + // schema was upgraded and we are destroying before ever refreshing that + // data source. Regardless, a data source "destroy" is simply writing a + // null state, which we can do with a null prior state too. + change := &plans.ResourceInstanceChange{ + Addr: n.ResourceInstanceAddr(), + PrevRunAddr: n.prevRunAddr(ctx), + Change: plans.Change{ + Action: plans.Delete, + Before: cty.NullVal(cty.DynamicPseudoType), + After: cty.NullVal(cty.DynamicPseudoType), + }, + ProviderAddr: n.ResolvedProvider, + } + return diags.Append(n.writeChange(ctx, change, "")) +} diff --git a/internal/terraform/upgrade_resource_state.go b/internal/terraform/upgrade_resource_state.go index 647db361cc..906898e281 100644 --- a/internal/terraform/upgrade_resource_state.go +++ b/internal/terraform/upgrade_resource_state.go @@ -21,6 +21,14 @@ import ( // If any errors occur during upgrade, error diagnostics are returned. In that // case it is not safe to proceed with using the original state object. func upgradeResourceState(addr addrs.AbsResourceInstance, provider providers.Interface, src *states.ResourceInstanceObjectSrc, currentSchema *configschema.Block, currentVersion uint64) (*states.ResourceInstanceObjectSrc, tfdiags.Diagnostics) { + if addr.Resource.Resource.Mode != addrs.ManagedResourceMode { + // We only do state upgrading for managed resources. + // This was a part of the normal workflow in older versions and + // returned early, so we are only going to log the error for now. + log.Printf("[ERROR] data resource %s should not require state upgrade", addr) + return src, nil + } + // Remove any attributes from state that are not present in the schema. // This was previously taken care of by the provider, but data sources do // not go through the UpgradeResourceState process. @@ -32,11 +40,6 @@ func upgradeResourceState(addr addrs.AbsResourceInstance, provider providers.Int src.AttrsJSON = stripRemovedStateAttributes(src.AttrsJSON, currentSchema.ImpliedType()) } - if addr.Resource.Resource.Mode != addrs.ManagedResourceMode { - // We only do state upgrading for managed resources. - return src, nil - } - stateIsFlatmap := len(src.AttrsJSON) == 0 // TODO: This should eventually use a proper FQN. From 34114286ffed6ab5d312f05ea178907e72757e34 Mon Sep 17 00:00:00 2001 From: Sebastian Rivera Date: Mon, 4 Apr 2022 15:37:44 -0400 Subject: [PATCH 27/54] Cloud e2e tests for configuring with env vars --- internal/cloud/e2e/env_variables_test.go | 260 +++++++++++++++++++++++ internal/cloud/e2e/helper_test.go | 45 ++++ internal/cloud/e2e/main_test.go | 8 +- 3 files changed, 311 insertions(+), 2 deletions(-) create mode 100644 internal/cloud/e2e/env_variables_test.go diff --git a/internal/cloud/e2e/env_variables_test.go b/internal/cloud/e2e/env_variables_test.go new file mode 100644 index 0000000000..2ee4dd18cd --- /dev/null +++ b/internal/cloud/e2e/env_variables_test.go @@ -0,0 +1,260 @@ +package main + +import ( + "context" + "fmt" + "testing" + + "github.com/hashicorp/go-tfe" +) + +func Test_cloud_organization_env_var(t *testing.T) { + t.Parallel() + + ctx := context.Background() + org, cleanup := createOrganization(t) + t.Cleanup(cleanup) + + cases := testCases{ + "with TF_ORGANIZATION set": { + operations: []operationSets{ + { + prep: func(t *testing.T, orgName, dir string) { + remoteWorkspace := "cloud-workspace" + tfBlock := terraformConfigCloudBackendOmitOrg(remoteWorkspace) + writeMainTF(t, tfBlock, dir) + }, + commands: []tfCommand{ + { + command: []string{"init"}, + expectedCmdOutput: `Terraform Cloud has been successfully initialized!`, + }, + { + command: []string{"apply", "-auto-approve"}, + postInputOutput: []string{`Apply complete!`}, + }, + }, + }, + }, + validations: func(t *testing.T, orgName string) { + expectedName := "cloud-workspace" + ws, err := tfeClient.Workspaces.Read(ctx, org.Name, expectedName) + if err != nil { + t.Fatal(err) + } + if ws == nil { + t.Fatalf("Expected workspace %s to be present, but is not.", expectedName) + } + }, + }, + } + + testRunner(t, cases, 0, fmt.Sprintf("TF_ORGANIZATION=%s", org.Name)) +} + +func Test_cloud_workspace_name_env_var(t *testing.T) { + t.Parallel() + + org, orgCleanup := createOrganization(t) + t.Cleanup(orgCleanup) + + wk := createWorkspace(t, org.Name, tfe.WorkspaceCreateOptions{ + Name: tfe.String("cloud-workspace"), + }) + + validCases := testCases{ + "a workspace that exists": { + operations: []operationSets{ + { + prep: func(t *testing.T, orgName, dir string) { + tfBlock := terraformConfigCloudBackendOmitWorkspaces(org.Name) + writeMainTF(t, tfBlock, dir) + }, + commands: []tfCommand{ + { + command: []string{"init"}, + expectedCmdOutput: `Terraform Cloud has been successfully initialized!`, + }, + { + command: []string{"apply", "-auto-approve"}, + postInputOutput: []string{`Apply complete!`}, + }, + }, + }, + { + prep: func(t *testing.T, orgName, dir string) { + tfBlock := terraformConfigCloudBackendOmitWorkspaces(org.Name) + writeMainTF(t, tfBlock, dir) + }, + commands: []tfCommand{ + { + command: []string{"init"}, + expectedCmdOutput: `Terraform Cloud has been successfully initialized!`, + }, + { + command: []string{"workspace", "show"}, + expectedCmdOutput: wk.Name, + }, + }, + }, + }, + }, + } + + errCases := testCases{ + "a workspace that doesn't exist": { + operations: []operationSets{ + { + prep: func(t *testing.T, orgName, dir string) { + tfBlock := terraformConfigCloudBackendOmitWorkspaces(org.Name) + writeMainTF(t, tfBlock, dir) + }, + commands: []tfCommand{ + { + command: []string{"init"}, + expectError: true, + }, + }, + }, + }, + }, + } + + testRunner(t, validCases, 0, fmt.Sprintf(`TF_WORKSPACE=%s`, wk.Name)) + testRunner(t, errCases, 0, fmt.Sprintf(`TF_WORKSPACE=%s`, "the-fires-of-mt-doom")) +} + +func Test_cloud_workspace_tags_env_var(t *testing.T) { + t.Parallel() + + org, orgCleanup := createOrganization(t) + t.Cleanup(orgCleanup) + + wkValid := createWorkspace(t, org.Name, tfe.WorkspaceCreateOptions{ + Name: tfe.String("cloud-workspace"), + Tags: []*tfe.Tag{ + {Name: "cloud"}, + }, + }) + + // this will be a workspace that won't have a tag listed in our test configuration + wkInvalid := createWorkspace(t, org.Name, tfe.WorkspaceCreateOptions{ + Name: tfe.String("cloud-workspace-2"), + }) + + validCases := testCases{ + "a workspace with valid tag": { + operations: []operationSets{ + { + prep: func(t *testing.T, orgName, dir string) { + tfBlock := terraformConfigCloudBackendTags(org.Name, wkValid.TagNames[0]) + writeMainTF(t, tfBlock, dir) + }, + commands: []tfCommand{ + { + command: []string{"init"}, + expectedCmdOutput: `Terraform Cloud has been successfully initialized!`, + }, + { + command: []string{"apply", "-auto-approve"}, + postInputOutput: []string{`Apply complete!`}, + }, + }, + }, + { + prep: func(t *testing.T, orgName, dir string) { + tfBlock := terraformConfigCloudBackendTags(org.Name, wkValid.TagNames[0]) + writeMainTF(t, tfBlock, dir) + }, + commands: []tfCommand{ + { + command: []string{"init"}, + expectedCmdOutput: `Terraform Cloud has been successfully initialized!`, + }, + { + command: []string{"workspace", "show"}, + expectedCmdOutput: wkValid.Name, + }, + }, + }, + }, + }, + } + + errCases := testCases{ + "a workspace not specified by tags": { + operations: []operationSets{ + { + prep: func(t *testing.T, orgName, dir string) { + tfBlock := terraformConfigCloudBackendTags(org.Name, wkValid.TagNames[0]) + writeMainTF(t, tfBlock, dir) + }, + commands: []tfCommand{ + { + command: []string{"init"}, + expectError: true, + }, + }, + }, + }, + }, + } + + testRunner(t, validCases, 0, fmt.Sprintf(`TF_WORKSPACE=%s`, wkValid.Name)) + testRunner(t, errCases, 0, fmt.Sprintf(`TF_WORKSPACE=%s`, wkInvalid.Name)) +} + +func Test_cloud_null_config(t *testing.T) { + t.Parallel() + + org, cleanup := createOrganization(t) + t.Cleanup(cleanup) + + wk := createWorkspace(t, org.Name, tfe.WorkspaceCreateOptions{ + Name: tfe.String("cloud-workspace"), + }) + + cases := testCases{ + "with all env vars set": { + operations: []operationSets{ + { + prep: func(t *testing.T, orgName, dir string) { + tfBlock := terraformConfigCloudBackendOmitConfig() + writeMainTF(t, tfBlock, dir) + }, + commands: []tfCommand{ + { + command: []string{"init"}, + expectedCmdOutput: `Terraform Cloud has been successfully initialized!`, + }, + { + command: []string{"apply", "-auto-approve"}, + postInputOutput: []string{`Apply complete!`}, + }, + }, + }, + { + prep: func(t *testing.T, orgName, dir string) { + tfBlock := terraformConfigCloudBackendOmitConfig() + writeMainTF(t, tfBlock, dir) + }, + commands: []tfCommand{ + { + command: []string{"init"}, + expectedCmdOutput: `Terraform Cloud has been successfully initialized!`, + }, + { + command: []string{"workspace", "show"}, + expectedCmdOutput: wk.Name, + }, + }, + }, + }, + }, + } + + testRunner(t, cases, 1, + fmt.Sprintf(`TF_ORGANIZATION=%s`, org.Name), + fmt.Sprintf(`TF_HOSTNAME=%s`, tfeHostname), + fmt.Sprintf(`TF_WORKSPACE=%s`, wk.Name)) +} diff --git a/internal/cloud/e2e/helper_test.go b/internal/cloud/e2e/helper_test.go index 084edf8f2d..4a9f2ee2e7 100644 --- a/internal/cloud/e2e/helper_test.go +++ b/internal/cloud/e2e/helper_test.go @@ -191,6 +191,51 @@ output "val" { `, tfeHostname, org, name) } +func terraformConfigCloudBackendOmitOrg(workspaceName string) string { + return fmt.Sprintf(` +terraform { + cloud { + hostname = "%s" + + workspaces { + name = "%s" + } + } +} + +output "val" { + value = "${terraform.workspace}" +} +`, tfeHostname, workspaceName) +} + +func terraformConfigCloudBackendOmitWorkspaces(orgName string) string { + return fmt.Sprintf(` +terraform { + cloud { + hostname = "%s" + organization = "%s" + } +} + +output "val" { + value = "${terraform.workspace}" +} +`, tfeHostname, orgName) +} + +func terraformConfigCloudBackendOmitConfig() string { + return ` +terraform { + cloud {} +} + +output "val" { + value = "${terraform.workspace}" +} +` +} + func writeMainTF(t *testing.T, block string, dir string) { f, err := os.Create(fmt.Sprintf("%s/main.tf", dir)) if err != nil { diff --git a/internal/cloud/e2e/main_test.go b/internal/cloud/e2e/main_test.go index ae78519b3a..44fc606395 100644 --- a/internal/cloud/e2e/main_test.go +++ b/internal/cloud/e2e/main_test.go @@ -98,11 +98,15 @@ func testRunner(t *testing.T, cases testCases, orgCount int, tfEnvFlags ...strin var orgName string for index, op := range tc.operations { - if orgCount == 1 { + switch orgCount { + case 0: + orgName = "" + case 1: orgName = orgNames[0] - } else { + default: orgName = orgNames[index] } + op.prep(t, orgName, tf.WorkDir()) for _, tfCmd := range op.commands { cmd := tf.Cmd(tfCmd.command...) From b191faf8a4b363336950a1ba1a0b04d5689296e9 Mon Sep 17 00:00:00 2001 From: Sebastian Rivera Date: Tue, 12 Apr 2022 17:32:39 -0400 Subject: [PATCH 28/54] Add skip test if missing vars helper --- internal/cloud/e2e/env_variables_test.go | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/internal/cloud/e2e/env_variables_test.go b/internal/cloud/e2e/env_variables_test.go index 2ee4dd18cd..3cf0096440 100644 --- a/internal/cloud/e2e/env_variables_test.go +++ b/internal/cloud/e2e/env_variables_test.go @@ -10,6 +10,7 @@ import ( func Test_cloud_organization_env_var(t *testing.T) { t.Parallel() + skipIfMissingEnvVar(t) ctx := context.Background() org, cleanup := createOrganization(t) @@ -54,6 +55,7 @@ func Test_cloud_organization_env_var(t *testing.T) { func Test_cloud_workspace_name_env_var(t *testing.T) { t.Parallel() + skipIfMissingEnvVar(t) org, orgCleanup := createOrganization(t) t.Cleanup(orgCleanup) @@ -126,6 +128,7 @@ func Test_cloud_workspace_name_env_var(t *testing.T) { func Test_cloud_workspace_tags_env_var(t *testing.T) { t.Parallel() + skipIfMissingEnvVar(t) org, orgCleanup := createOrganization(t) t.Cleanup(orgCleanup) @@ -206,6 +209,7 @@ func Test_cloud_workspace_tags_env_var(t *testing.T) { func Test_cloud_null_config(t *testing.T) { t.Parallel() + skipIfMissingEnvVar(t) org, cleanup := createOrganization(t) t.Cleanup(cleanup) From a38a0ee8a8ca486037f326fa8e3e3c4c8b87532b Mon Sep 17 00:00:00 2001 From: Brandon Croft Date: Tue, 12 Apr 2022 13:14:01 -0600 Subject: [PATCH 29/54] test(cloud): ensure mocks are used for backend configure tests Also adds a few new tests for cloud configuration using environment variables --- internal/cloud/backend.go | 48 ++++++++------- internal/cloud/backend_test.go | 106 ++++++++++++++++++++++++++++----- internal/cloud/testing.go | 38 ++++++++++++ 3 files changed, 155 insertions(+), 37 deletions(-) diff --git a/internal/cloud/backend.go b/internal/cloud/backend.go index ca895df46f..cdb59cf2d8 100644 --- a/internal/cloud/backend.go +++ b/internal/cloud/backend.go @@ -253,30 +253,32 @@ func (b *Cloud) Configure(obj cty.Value) tfdiags.Diagnostics { return diags } - cfg := &tfe.Config{ - Address: service.String(), - BasePath: service.Path, - Token: token, - Headers: make(http.Header), - RetryLogHook: b.retryLogHook, - } + if b.client == nil { + cfg := &tfe.Config{ + Address: service.String(), + BasePath: service.Path, + Token: token, + Headers: make(http.Header), + RetryLogHook: b.retryLogHook, + } - // Set the version header to the current version. - cfg.Headers.Set(tfversion.Header, tfversion.Version) - cfg.Headers.Set(headerSourceKey, headerSourceValue) + // Set the version header to the current version. + cfg.Headers.Set(tfversion.Header, tfversion.Version) + cfg.Headers.Set(headerSourceKey, headerSourceValue) - // Create the TFC/E API client. - b.client, err = tfe.NewClient(cfg) - if err != nil { - diags = diags.Append(tfdiags.Sourceless( - tfdiags.Error, - "Failed to create the Terraform Cloud/Enterprise client", - fmt.Sprintf( - `Encountered an unexpected error while creating the `+ - `Terraform Cloud/Enterprise client: %s.`, err, - ), - )) - return diags + // Create the TFC/E API client. + b.client, err = tfe.NewClient(cfg) + if err != nil { + diags = diags.Append(tfdiags.Sourceless( + tfdiags.Error, + "Failed to create the Terraform Cloud/Enterprise client", + fmt.Sprintf( + `Encountered an unexpected error while creating the `+ + `Terraform Cloud/Enterprise client: %s.`, err, + ), + )) + return diags + } } // Check if the organization exists by reading its entitlements. @@ -383,7 +385,7 @@ func (b *Cloud) setConfigurationFields(obj cty.Value) tfdiags.Diagnostics { var tags []string err := gocty.FromCtyValue(val, &tags) if err != nil { - log.Panicf("An unxpected error occurred: %s", err) + log.Panicf("An unexpected error occurred: %s", err) } b.WorkspaceMapping.Tags = tags diff --git a/internal/cloud/backend_test.go b/internal/cloud/backend_test.go index 9b0c0fbad8..b6ed803781 100644 --- a/internal/cloud/backend_test.go +++ b/internal/cloud/backend_test.go @@ -223,6 +223,7 @@ func TestCloud_PrepareConfigWithEnvVars(t *testing.T) { func TestCloud_configWithEnvVars(t *testing.T) { cases := map[string]struct { + setup func(b *Cloud) config cty.Value vars map[string]string expectedOrganization string @@ -271,13 +272,13 @@ func TestCloud_configWithEnvVars(t *testing.T) { }), }), vars: map[string]string{ - "TF_HOSTNAME": "app.terraform.io", + "TF_HOSTNAME": "private.hashicorp.engineering", }, - expectedHostname: "app.terraform.io", + expectedHostname: "private.hashicorp.engineering", }, "with hostname and env var specified": { config: cty.ObjectVal(map[string]cty.Value{ - "hostname": cty.StringVal("app.terraform.io"), + "hostname": cty.StringVal("private.hashicorp.engineering"), "token": cty.NullVal(cty.String), "organization": cty.StringVal("hashicorp"), "workspaces": cty.ObjectVal(map[string]cty.Value{ @@ -288,11 +289,11 @@ func TestCloud_configWithEnvVars(t *testing.T) { vars: map[string]string{ "TF_HOSTNAME": "mycool.tfe-host.io", }, - expectedHostname: "app.terraform.io", + expectedHostname: "private.hashicorp.engineering", }, "an invalid workspace env var": { config: cty.ObjectVal(map[string]cty.Value{ - "hostname": cty.StringVal("app.terraform.io"), + "hostname": cty.NullVal(cty.String), "token": cty.NullVal(cty.String), "organization": cty.StringVal("hashicorp"), "workspaces": cty.NullVal(cty.Object(map[string]cty.Type{ @@ -307,25 +308,98 @@ func TestCloud_configWithEnvVars(t *testing.T) { }, "workspaces and env var specified": { config: cty.ObjectVal(map[string]cty.Value{ - "hostname": cty.StringVal("app.terraform.io"), + "hostname": cty.NullVal(cty.String), "token": cty.NullVal(cty.String), - "organization": cty.StringVal("hashicorp"), + "organization": cty.StringVal("mordor"), "workspaces": cty.ObjectVal(map[string]cty.Value{ - "name": cty.StringVal("prod"), + "name": cty.StringVal("mt-doom"), "tags": cty.NullVal(cty.Set(cty.String)), }), }), vars: map[string]string{ - "TF_WORKSPACE": "mt-doom", + "TF_WORKSPACE": "shire", }, - expectedWorkspaceName: "prod", + expectedWorkspaceName: "mt-doom", + }, + "env var workspace does not have specified tag": { + setup: func(b *Cloud) { + b.client.Organizations.Create(context.Background(), tfe.OrganizationCreateOptions{ + Name: tfe.String("mordor"), + }) + + b.client.Workspaces.Create(context.Background(), "mordor", tfe.WorkspaceCreateOptions{ + Name: tfe.String("shire"), + }) + }, + config: cty.ObjectVal(map[string]cty.Value{ + "hostname": cty.NullVal(cty.String), + "token": cty.NullVal(cty.String), + "organization": cty.StringVal("mordor"), + "workspaces": cty.ObjectVal(map[string]cty.Value{ + "name": cty.NullVal(cty.String), + "tags": cty.SetVal([]cty.Value{ + cty.StringVal("cloud"), + }), + }), + }), + vars: map[string]string{ + "TF_WORKSPACE": "shire", + }, + expectedErr: "Terraform failed to find workspace \"shire\" with the tags specified in your configuration:\n[cloud]", + }, + "env var workspace has specified tag": { + setup: func(b *Cloud) { + b.client.Organizations.Create(context.Background(), tfe.OrganizationCreateOptions{ + Name: tfe.String("mordor"), + }) + + b.client.Workspaces.Create(context.Background(), "mordor", tfe.WorkspaceCreateOptions{ + Name: tfe.String("shire"), + Tags: []*tfe.Tag{ + { + Name: "hobbity", + }, + }, + }) + }, + config: cty.ObjectVal(map[string]cty.Value{ + "hostname": cty.NullVal(cty.String), + "token": cty.NullVal(cty.String), + "organization": cty.StringVal("mordor"), + "workspaces": cty.ObjectVal(map[string]cty.Value{ + "name": cty.NullVal(cty.String), + "tags": cty.SetVal([]cty.Value{ + cty.StringVal("hobbity"), + }), + }), + }), + vars: map[string]string{ + "TF_WORKSPACE": "shire", + }, + expectedWorkspaceName: "", // No error is raised, but workspace is not set + }, + "with everything set as env vars": { + config: cty.ObjectVal(map[string]cty.Value{ + "hostname": cty.NullVal(cty.String), + "token": cty.NullVal(cty.String), + "organization": cty.NullVal(cty.String), + "workspaces": cty.NullVal(cty.String), + }), + vars: map[string]string{ + "TF_ORGANIZATION": "mordor", + "TF_WORKSPACE": "mt-doom", + "TF_HOSTNAME": "mycool.tfe-host.io", + }, + expectedOrganization: "mordor", + expectedWorkspaceName: "mt-doom", + expectedHostname: "mycool.tfe-host.io", }, } for name, tc := range cases { t.Run(name, func(t *testing.T) { - s := testServer(t) - b := New(testDisco(s)) + b, cleanup := testUnconfiguredBackend(t) + t.Cleanup(cleanup) for k, v := range tc.vars { os.Setenv(k, v) @@ -342,6 +416,10 @@ func TestCloud_configWithEnvVars(t *testing.T) { t.Fatalf("%s: unexpected validation result: %v", name, valDiags.Err()) } + if tc.setup != nil { + tc.setup(b) + } + diags := b.Configure(tc.config) if (diags.Err() != nil || tc.expectedErr != "") && (diags.Err() == nil || !strings.Contains(diags.Err().Error(), tc.expectedErr)) { @@ -466,8 +544,8 @@ func TestCloud_config(t *testing.T) { } for name, tc := range cases { - s := testServer(t) - b := New(testDisco(s)) + b, cleanup := testUnconfiguredBackend(t) + t.Cleanup(cleanup) // Validate _, valDiags := b.PrepareConfig(tc.config) diff --git a/internal/cloud/testing.go b/internal/cloud/testing.go index e7be4748ce..e5dd538066 100644 --- a/internal/cloud/testing.go +++ b/internal/cloud/testing.go @@ -178,6 +178,44 @@ func testBackend(t *testing.T, obj cty.Value) (*Cloud, func()) { return b, s.Close } +// testUnconfiguredBackend is used for testing the configuration of the backend +// with the mock client +func testUnconfiguredBackend(t *testing.T) (*Cloud, func()) { + s := testServer(t) + b := New(testDisco(s)) + + // Normally, the client is created during configuration, but the configuration uses the + // client to read entitlements. + var err error + b.client, err = tfe.NewClient(&tfe.Config{ + Token: "fake-token", + }) + if err != nil { + t.Fatal(err) + } + + // Get a new mock client. + mc := NewMockClient() + + // Replace the services we use with our mock services. + b.CLI = cli.NewMockUi() + b.client.Applies = mc.Applies + b.client.ConfigurationVersions = mc.ConfigurationVersions + b.client.CostEstimates = mc.CostEstimates + b.client.Organizations = mc.Organizations + b.client.Plans = mc.Plans + b.client.PolicyChecks = mc.PolicyChecks + b.client.Runs = mc.Runs + b.client.StateVersions = mc.StateVersions + b.client.Variables = mc.Variables + b.client.Workspaces = mc.Workspaces + + // Set local to a local test backend. + b.local = testLocalBackend(t, b) + + return b, s.Close +} + func testLocalBackend(t *testing.T, cloud *Cloud) backend.Enhanced { b := backendLocal.NewWithBackend(cloud) From 0dc26a9585556dfbf86bc4769facb6cacf0f87f9 Mon Sep 17 00:00:00 2001 From: Brandon Croft Date: Wed, 13 Apr 2022 09:27:13 -0600 Subject: [PATCH 30/54] test(cloud): nonexisting org not a valid test when using mocks --- internal/cloud/backend_test.go | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/internal/cloud/backend_test.go b/internal/cloud/backend_test.go index b6ed803781..f857c5cef5 100644 --- a/internal/cloud/backend_test.go +++ b/internal/cloud/backend_test.go @@ -447,18 +447,6 @@ func TestCloud_config(t *testing.T) { confErr string valErr string }{ - "with_a_nonexisting_organization": { - config: cty.ObjectVal(map[string]cty.Value{ - "hostname": cty.NullVal(cty.String), - "organization": cty.StringVal("nonexisting"), - "token": cty.NullVal(cty.String), - "workspaces": cty.ObjectVal(map[string]cty.Value{ - "name": cty.StringVal("prod"), - "tags": cty.NullVal(cty.Set(cty.String)), - }), - }), - confErr: "organization \"nonexisting\" at host app.terraform.io not found", - }, "with_an_unknown_host": { config: cty.ObjectVal(map[string]cty.Value{ "hostname": cty.StringVal("nonexisting.local"), From ed59bd7299d07f26973098b06a349a62c235eba5 Mon Sep 17 00:00:00 2001 From: Shanee D Date: Wed, 13 Apr 2022 16:13:20 -0700 Subject: [PATCH 31/54] fix: Fix typo in v1.0 upgrade guide --- website/docs/language/upgrade-guides/1-0.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/language/upgrade-guides/1-0.mdx b/website/docs/language/upgrade-guides/1-0.mdx index 47892dae45..b099980202 100644 --- a/website/docs/language/upgrade-guides/1-0.mdx +++ b/website/docs/language/upgrade-guides/1-0.mdx @@ -29,7 +29,7 @@ recommend upgrading one major version at a time until you reach Terraform v0.14, following the upgrade guides of each of those versions, because those earlier versions include mechanisms to automatically detect necessary changes to your configuration, and in some cases also automatically edit your configuration -to include those changes. One you reach Terraform v0.14 you can then skip +to include those changes. Once you reach Terraform v0.14 you can then skip directly from there to Terraform v1.0. The following table summarizes the above recommendations. In each case, we From 746af015ea34b44daa188ab82a139b508383928d Mon Sep 17 00:00:00 2001 From: Radek Simko Date: Thu, 14 Apr 2022 16:14:50 +0100 Subject: [PATCH 32/54] internal/getproviders: Add URL to error message for clarity (#30810) * internal/getproviders: Add URL to error message for clarity Occasionally `terraform init` on some providers may return the following error message: Error while installing citrix/citrixadc v1.13.0: could not query provider registry for registry.terraform.io/citrix/citrixadc: failed to retrieve authentication checksums for provider: 403 Forbidden The 403 is most often returned from GitHub (rather than Registry API) and this change makes it more obvious. * Use Host instead of full URL --- internal/getproviders/registry_client.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/internal/getproviders/registry_client.go b/internal/getproviders/registry_client.go index a31405ec01..428d362e2c 100644 --- a/internal/getproviders/registry_client.go +++ b/internal/getproviders/registry_client.go @@ -437,7 +437,7 @@ func (c *registryClient) getFile(url *url.URL) ([]byte, error) { defer resp.Body.Close() if resp.StatusCode != http.StatusOK { - return nil, fmt.Errorf("%s", resp.Status) + return nil, fmt.Errorf("%s returned from %s", resp.Status, resp.Request.Host) } data, err := ioutil.ReadAll(resp.Body) @@ -478,7 +478,7 @@ func maxRetryErrorHandler(resp *http.Response, err error, numTries int) (*http.R // both response and error. var errMsg string if resp != nil { - errMsg = fmt.Sprintf(": %s returned from %s", resp.Status, resp.Request.URL) + errMsg = fmt.Sprintf(": %s returned from %s", resp.Status, resp.Request.Host) } else if err != nil { errMsg = fmt.Sprintf(": %s", err) } From ad86e5a06f45425421145ec974d7106c0a143ed9 Mon Sep 17 00:00:00 2001 From: Radek Simko Date: Thu, 14 Apr 2022 16:39:36 +0100 Subject: [PATCH 33/54] Update CHANGELOG.md --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8ed4cb82cd..3acb93b88c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,7 @@ ENHANCEMENTS: * Add `TF_ORGANIZATION` environment variable fallback for `organization` in the cloud configuration * Add `TF_HOSTNAME` environment variable fallback for `hostname` in the cloud configuration * When running on macOS, Terraform will now use platform APIs to validate certificates presented by TLS (HTTPS) servers. This may change exactly which root certificates Terraform will accept as valid. ([#30768](https://github.com/hashicorp/terraform/issues/30768)) +* Show remote host in error message for clarity when installation of provider fails ([#30810](https://github.com/hashicorp/terraform/issues/30810)) BUG FIXES: From 545346b331baf84336498fc021b3cae50c3d36b7 Mon Sep 17 00:00:00 2001 From: Kevin Wang Date: Thu, 14 Apr 2022 12:04:35 -0400 Subject: [PATCH 34/54] chore: move content into `docs` (#30837) --- website/{ => docs}/guides/index.mdx | 0 .../{ => docs}/guides/terraform-provider-development-program.mdx | 0 website/{ => docs}/intro/core-workflow.mdx | 0 website/{ => docs}/intro/index.mdx | 0 website/{ => docs}/intro/terraform-editions.mdx | 0 website/{ => docs}/intro/use-cases.mdx | 0 website/{ => docs}/intro/vs/boto.mdx | 0 website/{ => docs}/intro/vs/chef-puppet.mdx | 0 website/{ => docs}/intro/vs/cloudformation.mdx | 0 website/{ => docs}/intro/vs/custom.mdx | 0 website/{ => docs}/intro/vs/index.mdx | 0 11 files changed, 0 insertions(+), 0 deletions(-) rename website/{ => docs}/guides/index.mdx (100%) rename website/{ => docs}/guides/terraform-provider-development-program.mdx (100%) rename website/{ => docs}/intro/core-workflow.mdx (100%) rename website/{ => docs}/intro/index.mdx (100%) rename website/{ => docs}/intro/terraform-editions.mdx (100%) rename website/{ => docs}/intro/use-cases.mdx (100%) rename website/{ => docs}/intro/vs/boto.mdx (100%) rename website/{ => docs}/intro/vs/chef-puppet.mdx (100%) rename website/{ => docs}/intro/vs/cloudformation.mdx (100%) rename website/{ => docs}/intro/vs/custom.mdx (100%) rename website/{ => docs}/intro/vs/index.mdx (100%) diff --git a/website/guides/index.mdx b/website/docs/guides/index.mdx similarity index 100% rename from website/guides/index.mdx rename to website/docs/guides/index.mdx diff --git a/website/guides/terraform-provider-development-program.mdx b/website/docs/guides/terraform-provider-development-program.mdx similarity index 100% rename from website/guides/terraform-provider-development-program.mdx rename to website/docs/guides/terraform-provider-development-program.mdx diff --git a/website/intro/core-workflow.mdx b/website/docs/intro/core-workflow.mdx similarity index 100% rename from website/intro/core-workflow.mdx rename to website/docs/intro/core-workflow.mdx diff --git a/website/intro/index.mdx b/website/docs/intro/index.mdx similarity index 100% rename from website/intro/index.mdx rename to website/docs/intro/index.mdx diff --git a/website/intro/terraform-editions.mdx b/website/docs/intro/terraform-editions.mdx similarity index 100% rename from website/intro/terraform-editions.mdx rename to website/docs/intro/terraform-editions.mdx diff --git a/website/intro/use-cases.mdx b/website/docs/intro/use-cases.mdx similarity index 100% rename from website/intro/use-cases.mdx rename to website/docs/intro/use-cases.mdx diff --git a/website/intro/vs/boto.mdx b/website/docs/intro/vs/boto.mdx similarity index 100% rename from website/intro/vs/boto.mdx rename to website/docs/intro/vs/boto.mdx diff --git a/website/intro/vs/chef-puppet.mdx b/website/docs/intro/vs/chef-puppet.mdx similarity index 100% rename from website/intro/vs/chef-puppet.mdx rename to website/docs/intro/vs/chef-puppet.mdx diff --git a/website/intro/vs/cloudformation.mdx b/website/docs/intro/vs/cloudformation.mdx similarity index 100% rename from website/intro/vs/cloudformation.mdx rename to website/docs/intro/vs/cloudformation.mdx diff --git a/website/intro/vs/custom.mdx b/website/docs/intro/vs/custom.mdx similarity index 100% rename from website/intro/vs/custom.mdx rename to website/docs/intro/vs/custom.mdx diff --git a/website/intro/vs/index.mdx b/website/docs/intro/vs/index.mdx similarity index 100% rename from website/intro/vs/index.mdx rename to website/docs/intro/vs/index.mdx From ad15263cf74fa6481ffb9a8befe47ba69a4263c5 Mon Sep 17 00:00:00 2001 From: Sebastian Rivera Date: Thu, 14 Apr 2022 13:33:41 -0400 Subject: [PATCH 35/54] Minor changelog modifications Removed the run task entry since the feature will be backported to v1.1. I've also added the missing enhancement entry for using `TF_WORKSPACE` to configure your cloud block. --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3acb93b88c..a5b21b3e59 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,7 +11,6 @@ UPGRADE NOTES: NEW FEATURES: * `precondition` and `postcondition` check blocks for resources, data sources, and module output values: module authors can now document assumptions and assertions about configuration and state values. If these conditions are not met, Terraform will report a custom error message to the user and halt further evaluation. -* Terraform now supports [run tasks](https://www.terraform.io/cloud-docs/workspaces/settings/run-tasks), a Terraform Cloud integration for executing remote operations, for the post plan stage of a run. ENHANCEMENTS: @@ -22,6 +21,7 @@ ENHANCEMENTS: * When running `terraform plan`, only show external changes which may have contributed to the current plan ([#30486](https://github.com/hashicorp/terraform/issues/30486)) * Add `TF_ORGANIZATION` environment variable fallback for `organization` in the cloud configuration * Add `TF_HOSTNAME` environment variable fallback for `hostname` in the cloud configuration +* `TF_WORKSPACE` can now be used to configure the `workspaces` attribute in your cloud configuration * When running on macOS, Terraform will now use platform APIs to validate certificates presented by TLS (HTTPS) servers. This may change exactly which root certificates Terraform will accept as valid. ([#30768](https://github.com/hashicorp/terraform/issues/30768)) * Show remote host in error message for clarity when installation of provider fails ([#30810](https://github.com/hashicorp/terraform/issues/30810)) From dc9940332537d56b4d620cab536f574983618333 Mon Sep 17 00:00:00 2001 From: Brandon Croft Date: Wed, 13 Apr 2022 14:52:08 -0600 Subject: [PATCH 36/54] Update CHANGELOG.md --- CHANGELOG.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3acb93b88c..c9bbf361d2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,14 +4,16 @@ UPGRADE NOTES: * The official Linux packages for the v1.2 series now require Linux kernel version 2.6.32 or later. * When making outgoing HTTPS or other TLS connections as a client, Terraform now requires the server to support TLS v1.2. TLS v1.0 and v1.1 are no longer supported. Any safely up-to-date server should support TLS 1.2, and mainstream web browsers have required it since 2020. -* When making outgoing HTTPS or other TLS connections as a client, Terraform will no longer accept CA certificates signed using the SHA-1 hash function. Publicly trusted Certificate Authorities have not issued SHA-1 certificates since 2015. - -(Note: the changes to Terraform's requirements when interacting with TLS servers apply only to requests made by Terraform CLI itself, such as provider/module installation and state storage requests. Terraform provider plugins include their own TLS clients which may have different requirements, and may add new requirements in their own releases, independently of Terraform CLI changes.) +* When making outgoing HTTPS or other TLS connections as a client, Terraform will no longer accept CA certificates signed using the SHA-1 hash function. Publicly trusted Certificate Authorities have not issued SHA-1 certificates since 2015.
(Note: the changes to Terraform's requirements when interacting with TLS servers apply only to requests made by Terraform CLI itself, such as provider/module installation and state storage requests. Terraform provider plugins include their own TLS clients which may have different requirements, and may add new requirements in their own releases, independently of Terraform CLI changes.) +* If you use the [third-party credentials helper plugin terraform-credentials-env](https://github.com/apparentlymart/terraform-credentials-env), you should disable it as part of upgrading to Terraform v1.2 because similar functionality is now built in to Terraform itself. +
The new behavior supports the same environment variable naming scheme but has a difference in priority order from the credentials helper: `TF_TOKEN_...` environment variables will now take priority over credentials blocks in CLI configuration and credentials stored automatically by terraform login, which is not true for credentials provided by any credentials helper plugin. If you see Terraform using different credentials after upgrading, check to make sure you do not specify credentials for the same host in multiple locations. +
If you use the credentials helper in conjunction with the [hashicorp/tfe](https://registry.terraform.io/providers/hashicorp/tfe) Terraform provider to manage Terraform Cloud or Terraform Enterprise objects with Terraform, you should also upgrade to version 0.31 of that provider, which added the corresponding built-in support for these environment variables. NEW FEATURES: * `precondition` and `postcondition` check blocks for resources, data sources, and module output values: module authors can now document assumptions and assertions about configuration and state values. If these conditions are not met, Terraform will report a custom error message to the user and halt further evaluation. * Terraform now supports [run tasks](https://www.terraform.io/cloud-docs/workspaces/settings/run-tasks), a Terraform Cloud integration for executing remote operations, for the post plan stage of a run. +* You may specify remote network service credentials using an environment variable named after the host name with a `TF_TOKEN_` prefix. For example, the value of a variable named `TF_TOKEN_app_terraform_io` will be used as a bearer authorization token when the CLI makes service requests to the host name "app.terraform.io". ENHANCEMENTS: From 3a7263990d283e5166e847389ea9d1929d9c8880 Mon Sep 17 00:00:00 2001 From: Martin Atkins Date: Fri, 15 Apr 2022 08:41:49 -0700 Subject: [PATCH 37/54] Update CHANGELOG.md --- CHANGELOG.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c9bbf361d2..87053bb357 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,10 +4,14 @@ UPGRADE NOTES: * The official Linux packages for the v1.2 series now require Linux kernel version 2.6.32 or later. * When making outgoing HTTPS or other TLS connections as a client, Terraform now requires the server to support TLS v1.2. TLS v1.0 and v1.1 are no longer supported. Any safely up-to-date server should support TLS 1.2, and mainstream web browsers have required it since 2020. -* When making outgoing HTTPS or other TLS connections as a client, Terraform will no longer accept CA certificates signed using the SHA-1 hash function. Publicly trusted Certificate Authorities have not issued SHA-1 certificates since 2015.
(Note: the changes to Terraform's requirements when interacting with TLS servers apply only to requests made by Terraform CLI itself, such as provider/module installation and state storage requests. Terraform provider plugins include their own TLS clients which may have different requirements, and may add new requirements in their own releases, independently of Terraform CLI changes.) +* When making outgoing HTTPS or other TLS connections as a client, Terraform will no longer accept CA certificates signed using the SHA-1 hash function. Publicly trusted Certificate Authorities have not issued SHA-1 certificates since 2015. + + (Note: the changes to Terraform's requirements when interacting with TLS servers apply only to requests made by Terraform CLI itself, such as provider/module installation and state storage requests. Terraform provider plugins include their own TLS clients which may have different requirements, and may add new requirements in their own releases, independently of Terraform CLI changes.) * If you use the [third-party credentials helper plugin terraform-credentials-env](https://github.com/apparentlymart/terraform-credentials-env), you should disable it as part of upgrading to Terraform v1.2 because similar functionality is now built in to Terraform itself. -
The new behavior supports the same environment variable naming scheme but has a difference in priority order from the credentials helper: `TF_TOKEN_...` environment variables will now take priority over credentials blocks in CLI configuration and credentials stored automatically by terraform login, which is not true for credentials provided by any credentials helper plugin. If you see Terraform using different credentials after upgrading, check to make sure you do not specify credentials for the same host in multiple locations. -
If you use the credentials helper in conjunction with the [hashicorp/tfe](https://registry.terraform.io/providers/hashicorp/tfe) Terraform provider to manage Terraform Cloud or Terraform Enterprise objects with Terraform, you should also upgrade to version 0.31 of that provider, which added the corresponding built-in support for these environment variables. + + The new behavior supports the same environment variable naming scheme but has a difference in priority order from the credentials helper: `TF_TOKEN_...` environment variables will now take priority over credentials blocks in CLI configuration and credentials stored automatically by terraform login, which is not true for credentials provided by any credentials helper plugin. If you see Terraform using different credentials after upgrading, check to make sure you do not specify credentials for the same host in multiple locations. + + If you use the credentials helper in conjunction with the [hashicorp/tfe](https://registry.terraform.io/providers/hashicorp/tfe) Terraform provider to manage Terraform Cloud or Terraform Enterprise objects with Terraform, you should also upgrade to version 0.31 of that provider, which added the corresponding built-in support for these environment variables. NEW FEATURES: From 1943af51a22c1453f032df5b5653e9382296ab06 Mon Sep 17 00:00:00 2001 From: Brandon Croft Date: Fri, 15 Apr 2022 11:19:46 -0600 Subject: [PATCH 38/54] fix(creds): allow periods in TF_TOKEN_... credentials vars --- internal/command/cliconfig/credentials.go | 94 +++++++++++++------ .../command/cliconfig/credentials_test.go | 50 ++++++++++ website/docs/cli/config/config-file.mdx | 22 +++-- 3 files changed, 126 insertions(+), 40 deletions(-) diff --git a/internal/command/cliconfig/credentials.go b/internal/command/cliconfig/credentials.go index e39734eb2f..a85a1b7cd2 100644 --- a/internal/command/cliconfig/credentials.go +++ b/internal/command/cliconfig/credentials.go @@ -114,45 +114,77 @@ func (c *Config) credentialsSource(helperType string, helper svcauth.Credentials } } +func collectCredentialsFromEnv() map[svchost.Hostname]string { + const prefix = "TF_TOKEN_" + + ret := make(map[svchost.Hostname]string) + for _, ev := range os.Environ() { + eqIdx := strings.Index(ev, "=") + if eqIdx < 0 { + continue + } + name := ev[:eqIdx] + value := ev[eqIdx+1:] + if !strings.HasPrefix(name, prefix) { + continue + } + rawHost := name[len(prefix):] + + // We accept double underscores in place of hyphens because hyphens are not valid + // identifiers in most shells and are therefore hard to set. + // This is unambiguous with replacing single underscores below because + // hyphens are not allowed at the beginning or end of a label and therefore + // odd numbers of underscores will not appear together in a valid variable name. + rawHost = strings.ReplaceAll(rawHost, "__", "-") + + // We accept underscores in place of dots because dots are not valid + // identifiers in most shells and are therefore hard to set. + // Underscores are not valid in hostnames, so this is unambiguous for + // valid hostnames. + rawHost = strings.ReplaceAll(rawHost, "_", ".") + + // Because environment variables are often set indirectly by OS + // libraries that might interfere with how they are encoded, we'll + // be tolerant of them being given either directly as UTF-8 IDNs + // or in Punycode form, normalizing to Punycode form here because + // that is what the Terraform credentials helper protocol will + // use in its requests. + // + // Using ForDisplay first here makes this more liberal than Terraform + // itself would usually be in that it will tolerate pre-punycoded + // hostnames that Terraform normally rejects in other contexts in order + // to ensure stored hostnames are human-readable. + dispHost := svchost.ForDisplay(rawHost) + hostname, err := svchost.ForComparison(dispHost) + if err != nil { + // Ignore invalid hostnames + continue + } + + ret[hostname] = value + } + + return ret +} + // hostCredentialsFromEnv returns a token credential by searching for a hostname-specific // environment variable. The host parameter is expected to be in the "comparison" form, // for example, hostnames containing non-ASCII characters like "café.fr" // should be expressed as "xn--caf-dma.fr". If the variable based on the hostname is not -// defined, nil is returned. Variable names must have dot characters translated to -// underscores, which are not allowed in DNS names. For example, token credentials -// for app.terraform.io should be set in the variable named TF_TOKEN_app_terraform_io. +// defined, nil is returned. // -// Hyphen characters are allowed in environment variable names, but are not valid POSIX -// variable names. Usually, it's still possible to set variable names with hyphens using -// utilities like env or docker. But, as a fallback, host names may encode their -// hyphens as double underscores in the variable name. For the example "café.fr", -// the variable name "TF_TOKEN_xn____caf__dma_fr" or "TF_TOKEN_xn--caf-dma_fr" -// may be used. +// Hyphen and period characters are allowed in environment variable names, but are not valid POSIX +// variable names. However, it's still possible to set variable names with these characters using +// utilities like env or docker. Variable names may have periods translated to underscores and +// hyphens translated to double underscores in the variable name. +// For the example "café.fr", you may use the variable names "TF_TOKEN_xn____caf__dma_fr", +// "TF_TOKEN_xn--caf-dma_fr", or "TF_TOKEN_xn--caf-dma.fr" func hostCredentialsFromEnv(host svchost.Hostname) svcauth.HostCredentials { - if len(host) == 0 { + token, ok := collectCredentialsFromEnv()[host] + if !ok { return nil } - - // Convert dots to underscores when looking for environment configuration for a specific host. - // DNS names do not allow underscore characters so this is unambiguous. - translated := strings.ReplaceAll(host.String(), ".", "_") - - if token, ok := os.LookupEnv(fmt.Sprintf("TF_TOKEN_%s", translated)); ok { - return svcauth.HostCredentialsToken(token) - } - - if strings.ContainsRune(translated, '-') { - // This host name contains a hyphen. Replace hyphens with double underscores as a fallback - // (see godoc above for details) - translated = strings.ReplaceAll(host.String(), "-", "__") - translated = strings.ReplaceAll(translated, ".", "_") - - if token, ok := os.LookupEnv(fmt.Sprintf("TF_TOKEN_%s", translated)); ok { - return svcauth.HostCredentialsToken(token) - } - } - - return nil + return svcauth.HostCredentialsToken(token) } // CredentialsSource is an implementation of svcauth.CredentialsSource diff --git a/internal/command/cliconfig/credentials_test.go b/internal/command/cliconfig/credentials_test.go index 3b200c6401..50ed443b7d 100644 --- a/internal/command/cliconfig/credentials_test.go +++ b/internal/command/cliconfig/credentials_test.go @@ -156,6 +156,56 @@ func TestCredentialsForHost(t *testing.T) { t.Errorf("wrong result\ngot: %s\nwant: %s", got, expectedToken) } }) + + t.Run("periods are ok", func(t *testing.T) { + envName := "TF_TOKEN_configured.example.com" + expectedToken := "configured-by-env" + t.Cleanup(func() { + os.Unsetenv(envName) + }) + + os.Setenv(envName, expectedToken) + + hostname, _ := svchost.ForComparison("configured.example.com") + creds, err := credSrc.ForHost(hostname) + + if err != nil { + t.Fatalf("unexpected error: %s", err) + } + + if creds == nil { + t.Fatal("no credentials found") + } + + if got := creds.Token(); got != expectedToken { + t.Errorf("wrong result\ngot: %s\nwant: %s", got, expectedToken) + } + }) + + t.Run("casing is insensitive", func(t *testing.T) { + envName := "TF_TOKEN_CONFIGUREDUPPERCASE_EXAMPLE_COM" + expectedToken := "configured-by-env" + + os.Setenv(envName, expectedToken) + t.Cleanup(func() { + os.Unsetenv(envName) + }) + + hostname, _ := svchost.ForComparison("configureduppercase.example.com") + creds, err := credSrc.ForHost(hostname) + + if err != nil { + t.Fatalf("unexpected error: %s", err) + } + + if creds == nil { + t.Fatal("no credentials found") + } + + if got := creds.Token(); got != expectedToken { + t.Errorf("wrong result\ngot: %s\nwant: %s", got, expectedToken) + } + }) } func TestCredentialsStoreForget(t *testing.T) { diff --git a/website/docs/cli/config/config-file.mdx b/website/docs/cli/config/config-file.mdx index 0e74a63a9b..a10fca1040 100644 --- a/website/docs/cli/config/config-file.mdx +++ b/website/docs/cli/config/config-file.mdx @@ -117,17 +117,21 @@ Terraform Cloud responds to API calls at both its current hostname ### Environment Variable Credentials -If you would prefer not to store your API tokens directly in the CLI -configuration, you may use a host-specific environment variable. Environment variable names should -have the prefix `TF_TOKEN_` added to the domain name, with periods encoded as underscores. -For example, the value of a variable named `TF_TOKEN_app_terraform_io` will be used as a -bearer authorization token when the CLI makes service requests to the hostname "app.terraform.io". -If multiple variables evaluate to the same hostname, Terraform will use the one defined later in the -operating system's variable table. +If you would prefer not to store your API tokens directly in the CLI configuration, you may use +a host-specific environment variable. Environment variable names should have the prefix +`TF_TOKEN_` added to the domain name, with periods encoded as underscores. For example, the +value of a variable named `TF_TOKEN_app_terraform_io` will be used as a bearer authorization +token when the CLI makes service requests to the hostname `app.terraform.io`. -When using domain names as a variable name, you must convert domain names containing non-ASCII characters to their [punycode equivalent](https://www.charset.org/punycode) with an ACE prefix. For example, token credentials for 例えば.com must be set in a variable called `TF_TOKEN_xn--r8j3dr99h_com`. +You must convert domain names containing non-ASCII characters to their [punycode equivalent](https://www.charset.org/punycode) +with an ACE prefix. For example, token credentials for 例えば.com must be set in a variable +called `TF_TOKEN_xn--r8j3dr99h_com`. -Some tools like the `env` utility allow hyphens in variable names, but hyphens create invalid POSIX variable names. Therefore, you can encode hyphens as double underscores when you set variables with interactive tools like Bash or Zsh. For example, you can set a token for the domain name "café.fr" as either `TF_TOKEN_xn--caf-dma_fr` or `TF_TOKEN_xn____caf__dma_fr`. If both are defined, Terraform will use the version containing hyphens. +Hyphens are also valid within host names but usually invalid as variable names and +may be encoded as double underscores. For example, you can set a token for the domain name +`café.fr` as `TF_TOKEN_xn--caf-dma.fr`, `TF_TOKEN_xn--caf-dma_fr`, or `TF_TOKEN_xn____caf__dma_fr`. +If multiple variables evaluate to the same hostname, Terraform will choose the one defined last +in the operating system's variable table. ### Credentials Helpers From 87b09b1ee173e3b92c1d08ffae6c44242600c62b Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 20 Apr 2022 11:17:16 -0400 Subject: [PATCH 39/54] Fix broken HCL in example --- website/docs/language/data-sources/index.mdx | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/website/docs/language/data-sources/index.mdx b/website/docs/language/data-sources/index.mdx index 2bc9e56648..c9c993a892 100644 --- a/website/docs/language/data-sources/index.mdx +++ b/website/docs/language/data-sources/index.mdx @@ -134,8 +134,7 @@ data "aws_ami" "example" { # The AMI ID must refer to an existing AMI that has the tag "nomad-server". postcondition { condition = self.tags["Component"] == "nomad-server" - error_message = "The selected AMI must be tagged with the - Component value \"nomad-server\"." + error_message = error_message = "tags[\"Component\"] must be \"nomad-server\"." } } } From e8743143e33a2a8c55ebf0fc04438aeff135272b Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 20 Apr 2022 11:17:45 -0400 Subject: [PATCH 40/54] Update website/docs/language/data-sources/index.mdx Co-authored-by: Alisdair McDiarmid --- website/docs/language/data-sources/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/language/data-sources/index.mdx b/website/docs/language/data-sources/index.mdx index c9c993a892..7eb889b3a6 100644 --- a/website/docs/language/data-sources/index.mdx +++ b/website/docs/language/data-sources/index.mdx @@ -140,7 +140,7 @@ data "aws_ami" "example" { } ``` -Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. +Custom conditions can help capture assumptions that might otherwise be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. Refer to [Custom Condition Checks](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. From f660f54b87dc5b8239517560ef3626f16c907f8b Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 20 Apr 2022 11:21:11 -0400 Subject: [PATCH 41/54] Final nits from review --- website/docs/language/data-sources/index.mdx | 2 +- website/docs/language/expressions/custom-conditions.mdx | 2 +- website/docs/language/values/variables.mdx | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/language/data-sources/index.mdx b/website/docs/language/data-sources/index.mdx index 7eb889b3a6..651f800b90 100644 --- a/website/docs/language/data-sources/index.mdx +++ b/website/docs/language/data-sources/index.mdx @@ -140,7 +140,7 @@ data "aws_ami" "example" { } ``` -Custom conditions can help capture assumptions that might otherwise be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. +Custom conditions can help capture assumptions, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. Refer to [Custom Condition Checks](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 6ef293c6d7..95fb3a4c34 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -8,7 +8,7 @@ description: >- You can create conditions that produce custom error messages for several types of objects in a configuration. For example, you can add a condition to an input variable that checks whether incoming image IDs are formatted properly. -Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. +Custom conditions can help capture assumptions, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. This page explains the following: - Creating [validation conditions](#input-variable-validation) for input variables diff --git a/website/docs/language/values/variables.mdx b/website/docs/language/values/variables.mdx index db9318d8e1..7669c3087f 100644 --- a/website/docs/language/values/variables.mdx +++ b/website/docs/language/values/variables.mdx @@ -173,7 +173,7 @@ variable "image_id" { } } ``` -Refer to [Custom Conditions](/language/expressions/custom-conditions#input-variable-validation) for more details. +Refer to [Custom Condition Checks](/language/expressions/custom-conditions#input-variable-validation) for more details. ### Suppressing Values in CLI Output From ce757244f284a8270210b6dd85bea554a9fb5b7f Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 20 Apr 2022 11:22:11 -0400 Subject: [PATCH 42/54] fix confusing sentence in outputs --- website/docs/language/values/outputs.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/language/values/outputs.mdx b/website/docs/language/values/outputs.mdx index cb599deff9..9a9ab935b3 100644 --- a/website/docs/language/values/outputs.mdx +++ b/website/docs/language/values/outputs.mdx @@ -79,7 +79,7 @@ output "api_base_url" { } ``` -Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. +Custom conditions can help capture assumptions, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. Refer to [Custom Condition Checks](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. From 4d4d774aef4a7abbc77b0c951e2f124af1a2327c Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 20 Apr 2022 11:23:15 -0400 Subject: [PATCH 43/54] fix confusing sentence on resources page --- website/docs/language/resources/syntax.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/language/resources/syntax.mdx b/website/docs/language/resources/syntax.mdx index fee42fa633..7ed899f971 100644 --- a/website/docs/language/resources/syntax.mdx +++ b/website/docs/language/resources/syntax.mdx @@ -151,7 +151,7 @@ resource "aws_instance" "example" { } ``` -Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. +Custom conditions can help capture assumptions, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. Refer to [Custom Condition Checks](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. From 686dbcdb8dfcf45b2063bb28960310b243847614 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 20 Apr 2022 11:24:23 -0400 Subject: [PATCH 44/54] fix confusing sentence on lifecycle page --- website/docs/language/meta-arguments/lifecycle.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/language/meta-arguments/lifecycle.mdx b/website/docs/language/meta-arguments/lifecycle.mdx index 40ee9cc58c..7d1e83fe5d 100644 --- a/website/docs/language/meta-arguments/lifecycle.mdx +++ b/website/docs/language/meta-arguments/lifecycle.mdx @@ -130,7 +130,7 @@ resource "aws_instance" "example" { } ``` -Custom conditions can help capture assumptions that might be only implied, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. +Custom conditions can help capture assumptions, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. Refer to [Custom Conditions](/language/expressions/custom-conditions#preconditions-and-postconditions) for more details. From ba3bb5ad5dcfc63c0c72f9ed569f3bf603e191da Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 20 Apr 2022 11:30:20 -0400 Subject: [PATCH 45/54] Apply suggestions from PR review --- .../language/expressions/conditionals.mdx | 157 +---------------- .../expressions/custom-conditions.mdx | 159 +++++++++++++++++- 2 files changed, 162 insertions(+), 154 deletions(-) diff --git a/website/docs/language/expressions/conditionals.mdx b/website/docs/language/expressions/conditionals.mdx index f542483f85..fc241f9e67 100644 --- a/website/docs/language/expressions/conditionals.mdx +++ b/website/docs/language/expressions/conditionals.mdx @@ -39,159 +39,10 @@ The condition can be any expression that resolves to a boolean value. This will usually be an expression that uses the equality, comparison, or logical operators. -The following language features are particularly useful when writing condition expressions. +### Custom Condition Checks -### `contains` Function +You can create conditions that produce custom error messages for several types of objects in a configuration. For example, you can add a condition to an input variable that checks whether incoming image IDs are formatted properly. -Use the built-in function `contains` to test whether a given value is one of a set of predefined valid values. +Custom conditions can help capture assumptions, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. -```hcl - condition = contains(["STAGE", "PROD"], var.environment) -``` - -### Boolean Operators - -Use the boolean operators `&&` (AND), `||` (OR), and `!` (NOT) to combine multiple conditions together. - -```hcl - condition = var.name != "" && lower(var.name) == var.name - ``` - -### `length` Function - -Require a non-empty list or map by testing the collection's length. - -```hcl - condition = length(var.items) != 0 -``` -This is a better approach than directly comparing with another collection using `==` or `!=`. This is because the comparison operators can only return `true` if both operands have exactly the same type, which is often ambiguous for empty collections. - -### `for` Expressions - -Use `for` expressions in conjunction with the functions `alltrue` and `anytrue` to test whether a condition holds for all or for any elements of a collection. - -```hcl - condition = alltrue([ - for v in var.instances : contains(["t2.micro", "m3.medium"], v.type) - ]) -``` - -### `can` Function - -Use the `can` function to concisely use the validity of an expression as a condition. It returns `true` if its given expression evaluates successfully and `false` if it returns any error, so you can use various other functions that typically return errors as a part of your condition expressions. - -For example, you can use `can` with `regex` to test if a string matches a particular pattern because `regex` returns an error when given a non-matching string. - -```hcl - condition = can(regex("^[a-z]+$", var.name) -``` - -You can also use `can` with the type conversion functions to test whether a value is convertible to a type or type constraint. - -```hcl - # This remote output value must have a value that can - # be used as a string, which includes strings themselves - # but also allows numbers and boolean values. - condition = can(tostring(data.terraform_remote_state.example.outputs["name"])) -``` - -```hcl - # This remote output value must be convertible to a list - # type of with element type. - condition = can(tolist(data.terraform_remote_state.example.outputs["items"])) -``` - -You can also use `can` with attribute access or index operators to test whether a collection or structural value has a particular element or index. - -```hcl - # var.example must have an attribute named "foo" - condition = can(var.example.foo) ``` - -```hcl - # var.example must be a sequence with at least one element - condition = can(var.example[0]) - # (although it would typically be clearer to write this as a - # test like length(var.example) > 0 to better represent the - # intent of the condition.) -``` - -### `self` Object - -Use the `self` object in postcondition blocks to refer to attributes of the instance under evaluation. - -```hcl -resource "aws_instance" "example" { - instance_type = "t2.micro" - ami = "ami-abc123" - - lifecycle { - postcondition { - condition = self.instance_state == "running" - error_message = "EC2 instance must be running." - } - } -} -``` - -### `each` and `count` Objects - -In blocks where `for_each` or `count` are set, use `each` and `count` objects to refer to other resources that are expanded in a chain. - -```hcl -variable "vpc_cidrs" { - type = set(string) -} - -data "aws_vpc" "example" { - for_each = var.vpc_cidrs - - filter { - name = "cidr" - values = [each.key] - } -} - -resource "aws_internet_gateway" "example" { - for_each = aws_vpc.example - vpc_id = each.value.id - - lifecycle { - precondition { - condition = aws_vpc.example[each.key].state == "available" - error_message = "VPC ${each.key} must be available." - } - } -} -``` - -## Result Types - -The two result values may be of any type, but they must both -be of the _same_ type so that Terraform can determine what type the whole -conditional expression will return without knowing the condition value. - -If the two result expressions don't produce the same type then Terraform will -attempt to find a type that they can both convert to, and make those -conversions automatically if so. - -For example, the following expression is valid and will always return a string, -because in Terraform all numbers can convert automatically to a string using -decimal digits: - -```hcl -var.example ? 12 : "hello" -``` - -Relying on this automatic conversion behavior can be confusing for those who -are not familiar with Terraform's conversion rules though, so we recommend -being explicit using type conversion functions in any situation where there may -be some uncertainty about the expected result type. - -The following example is contrived because it would be easier to write the -constant `"12"` instead of the type conversion in this case, but shows how to -use [`tostring`](/language/functions/tostring) to explicitly convert a number to -a string. - -```hcl -var.example ? tostring(12) : "hello" -``` +Refer to [Custom Condition Checks](/language/expressions/custom-conditions#input-variable-validation) for details. diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 95fb3a4c34..657849051b 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -185,7 +185,164 @@ You should also consider the following factors. Input variable validation, preconditions, and postconditions all require a `condition` argument. This is a boolean expression that should return `true` if the intended assumption or guarantee is fulfilled or `false` if it does not. You can use any of Terraform's built-in functions or language operators -in a condition as long as the expression is valid and returns a boolean result. Refer to [Conditional Expressions](/language/expressions/conditionals#conditions) for more details and examples. +in a condition as long as the expression is valid and returns a boolean result. + +The following language features are particularly useful when writing condition expressions. + +### `contains` Function + +Use the built-in function `contains` to test whether a given value is one of a set of predefined valid values. + +```hcl + condition = contains(["STAGE", "PROD"], var.environment) +``` + +### Boolean Operators + +Use the boolean operators `&&` (AND), `||` (OR), and `!` (NOT) to combine multiple conditions together. + +```hcl + condition = var.name != "" && lower(var.name) == var.name +``` + +### `length` Function + +Require a non-empty list or map by testing the collection's length. + +```hcl + condition = length(var.items) != 0 +``` +This is a better approach than directly comparing with another collection using `==` or `!=`. This is because the comparison operators can only return `true` if both operands have exactly the same type, which is often ambiguous for empty collections. + +### `for` Expressions + +Use `for` expressions in conjunction with the functions `alltrue` and `anytrue` to test whether a condition holds for all or for any elements of a collection. + +```hcl + condition = alltrue([ + for v in var.instances : contains(["t2.micro", "m3.medium"], v.type) + ]) +``` + +### `can` Function + +Use the `can` function to concisely use the validity of an expression as a condition. It returns `true` if its given expression evaluates successfully and `false` if it returns any error, so you can use various other functions that typically return errors as a part of your condition expressions. + +For example, you can use `can` with `regex` to test if a string matches a particular pattern because `regex` returns an error when given a non-matching string. + +```hcl + condition = can(regex("^[a-z]+$", var.name) +``` + +You can also use `can` with the type conversion functions to test whether a value is convertible to a type or type constraint. + +```hcl + # This remote output value must have a value that can + # be used as a string, which includes strings themselves + # but also allows numbers and boolean values. + condition = can(tostring(data.terraform_remote_state.example.outputs["name"])) +``` + +```hcl + # This remote output value must be convertible to a list + # type of with element type. + condition = can(tolist(data.terraform_remote_state.example.outputs["items"])) +``` + +You can also use `can` with attribute access or index operators to test whether a collection or structural value has a particular element or index. + +```hcl + # var.example must have an attribute named "foo" + condition = can(var.example.foo) ``` + +```hcl + # var.example must be a sequence with at least one element + condition = can(var.example[0]) + # (although it would typically be clearer to write this as a + # test like length(var.example) > 0 to better represent the + # intent of the condition.) +``` + +### `self` Object + +Use the `self` object in postcondition blocks to refer to attributes of the instance under evaluation. + +```hcl +resource "aws_instance" "example" { + instance_type = "t2.micro" + ami = "ami-abc123" + + lifecycle { + postcondition { + condition = self.instance_state == "running" + error_message = "EC2 instance must be running." + } + } +} +``` + +### `each` and `count` Objects + +In blocks where `for_each` or `count` are set, use `each` and `count` objects to refer to other resources that are expanded in a chain. + +```hcl +variable "vpc_cidrs" { + type = set(string) +} + +data "aws_vpc" "example" { + for_each = var.vpc_cidrs + + filter { + name = "cidr" + values = [each.key] + } +} + +resource "aws_internet_gateway" "example" { + for_each = aws_vpc.example + vpc_id = each.value.id + + lifecycle { + precondition { + condition = aws_vpc.example[each.key].state == "available" + error_message = "VPC ${each.key} must be available." + } + } +} +``` + +## Result Types + +The two result values may be of any type, but they must both +be of the _same_ type so that Terraform can determine what type the whole +conditional expression will return without knowing the condition value. + +If the two result expressions don't produce the same type then Terraform will +attempt to find a type that they can both convert to, and make those +conversions automatically if so. + +For example, the following expression is valid and will always return a string, +because in Terraform all numbers can convert automatically to a string using +decimal digits: + +```hcl +var.example ? 12 : "hello" +``` + +Relying on this automatic conversion behavior can be confusing for those who +are not familiar with Terraform's conversion rules though, so we recommend +being explicit using type conversion functions in any situation where there may +be some uncertainty about the expected result type. + +The following example is contrived because it would be easier to write the +constant `"12"` instead of the type conversion in this case, but shows how to +use [`tostring`](/language/functions/tostring) to explicitly convert a number to +a string. + +```hcl +var.example ? tostring(12) : "hello" +``` ## Error Messages From 2eb9118cd12d001ad62dd7c98ab0c9ddb48ee638 Mon Sep 17 00:00:00 2001 From: Tom Harvey Date: Wed, 20 Apr 2022 17:31:44 +0200 Subject: [PATCH 46/54] backend/remote-state/azure: defaulting the Azure Backend to use MSAL (#30891) * backend/remote-state/azure: defaulting the Azure Backend to use MSAL Fixes #30881 * backend/remote-state/azurerm: deprecating `use_microsoft_graph` --- .../backend/remote-state/azure/backend.go | 3 ++- internal/backend/testing.go | 3 ++- .../language/settings/backends/azurerm.mdx | 20 +++++++++---------- 3 files changed, 13 insertions(+), 13 deletions(-) diff --git a/internal/backend/remote-state/azure/backend.go b/internal/backend/remote-state/azure/backend.go index e126da47d3..5a8f15008c 100644 --- a/internal/backend/remote-state/azure/backend.go +++ b/internal/backend/remote-state/azure/backend.go @@ -145,8 +145,9 @@ func New() backend.Backend { "use_microsoft_graph": { Type: schema.TypeBool, Optional: true, + Deprecated: "This field now defaults to `true` and will be removed in v1.3 of Terraform Core due to the deprecation of ADAL by Microsoft.", Description: "Should Terraform obtain an MSAL auth token and use Microsoft Graph rather than Azure Active Directory?", - DefaultFunc: schema.EnvDefaultFunc("ARM_USE_MSGRAPH", false), + DefaultFunc: schema.EnvDefaultFunc("ARM_USE_MSGRAPH", true), }, }, } diff --git a/internal/backend/testing.go b/internal/backend/testing.go index b4a76879b0..844f95668d 100644 --- a/internal/backend/testing.go +++ b/internal/backend/testing.go @@ -41,7 +41,8 @@ func TestBackendConfig(t *testing.T, b Backend, c hcl.Body) Backend { newObj, valDiags := b.PrepareConfig(obj) diags = diags.Append(valDiags.InConfigBody(c, "")) - if len(diags) != 0 { + // it's valid for a Backend to have warnings (e.g. a Deprecation) as such we should only raise on errors + if diags.HasErrors() { t.Fatal(diags.ErrWithWarnings()) } diff --git a/website/docs/language/settings/backends/azurerm.mdx b/website/docs/language/settings/backends/azurerm.mdx index 0539f9ddfe..a2b91bc9f5 100644 --- a/website/docs/language/settings/backends/azurerm.mdx +++ b/website/docs/language/settings/backends/azurerm.mdx @@ -9,7 +9,7 @@ Stores the state as a Blob with the given Key within the Blob Container within [ This backend supports state locking and consistency checking with Azure Blob Storage native capabilities. --> **Note:** By default the Azure Backend uses ADAL for authentication which is deprecated in favour of MSAL - MSAL can be used by setting `use_microsoft_graph` to `true`. **The default for this will change in Terraform 1.2**, so that MSAL authentication is used by default. +-> **Note:** In Terraform 1.2 the Azure Backend uses MSAL (and Microsoft Graph) rather than ADAL (and Azure Active Directory Graph) for authentication by default - you can disable this by setting `use_microsoft_graph` to `false`. **This setting will be removed in Terraform 1.3, due to Microsoft's deprecation of ADAL**. ## Example Configuration @@ -219,15 +219,13 @@ When authenticating using the Managed Service Identity (MSI) - the following fie * `msi_endpoint` - (Optional) The path to a custom Managed Service Identity endpoint which is automatically determined if not specified. This can also be sourced from the `ARM_MSI_ENDPOINT` environment variable. -* - * `subscription_id` - (Optional) The Subscription ID in which the Storage Account exists. This can also be sourced from the `ARM_SUBSCRIPTION_ID` environment variable. * `tenant_id` - (Optional) The Tenant ID in which the Subscription exists. This can also be sourced from the `ARM_TENANT_ID` environment variable. -* `use_microsoft_graph` - (Optional) Should MSAL be used for authentication instead of ADAL, and should Microsoft Graph be used instead of Azure Active Directory Graph? Defaults to `false`. +* `use_microsoft_graph` - (Optional) Should MSAL be used for authentication instead of ADAL, and should Microsoft Graph be used instead of Azure Active Directory Graph? Defaults to `true`. --> **Note:** By default the Azure Backend uses ADAL for authentication which is deprecated in favour of MSAL - MSAL can be used by setting `use_microsoft_graph` to `true`. **The default for this will change in Terraform 1.2**, so that MSAL authentication is used by default. +-> **Note:** In Terraform 1.2 the Azure Backend uses MSAL (and Microsoft Graph) rather than ADAL (and Azure Active Directory Graph) for authentication by default - you can disable this by setting `use_microsoft_graph` to `false`. **This setting will be removed in Terraform 1.3, due to Microsoft's deprecation of ADAL**. * `use_msi` - (Optional) Should Managed Service Identity authentication be used? This can also be sourced from the `ARM_USE_MSI` environment variable. @@ -251,9 +249,9 @@ When authenticating using AzureAD Authentication - the following fields are also -> **Note:** When using AzureAD for Authentication to Storage you also need to ensure the `Storage Blob Data Owner` role is assigned. -* `use_microsoft_graph` - (Optional) Should MSAL be used for authentication instead of ADAL, and should Microsoft Graph be used instead of Azure Active Directory Graph? Defaults to `false`. +* `use_microsoft_graph` - (Optional) Should MSAL be used for authentication instead of ADAL, and should Microsoft Graph be used instead of Azure Active Directory Graph? Defaults to `true`. --> **Note:** By default the Azure Backend uses ADAL for authentication which is deprecated in favour of MSAL - MSAL can be used by setting `use_microsoft_graph` to `true`. **The default for this will change in Terraform 1.2**, so that MSAL authentication is used by default. +-> **Note:** In Terraform 1.2 the Azure Backend uses MSAL (and Microsoft Graph) rather than ADAL (and Azure Active Directory Graph) for authentication by default - you can disable this by setting `use_microsoft_graph` to `false`. **This setting will be removed in Terraform 1.3, due to Microsoft's deprecation of ADAL**. *** @@ -271,9 +269,9 @@ When authenticating using a Service Principal with a Client Certificate - the fo * `tenant_id` - (Optional) The Tenant ID in which the Subscription exists. This can also be sourced from the `ARM_TENANT_ID` environment variable. -* `use_microsoft_graph` - (Optional) Should MSAL be used for authentication instead of ADAL, and should Microsoft Graph be used instead of Azure Active Directory Graph? Defaults to `false`. +* `use_microsoft_graph` - (Optional) Should MSAL be used for authentication instead of ADAL, and should Microsoft Graph be used instead of Azure Active Directory Graph? Defaults to `true`. --> **Note:** By default the Azure Backend uses ADAL for authentication which is deprecated in favour of MSAL - MSAL can be used by setting `use_microsoft_graph` to `true`. **The default for this will change in Terraform 1.2**, so that MSAL authentication is used by default. +-> **Note:** In Terraform 1.2 the Azure Backend uses MSAL (and Microsoft Graph) rather than ADAL (and Azure Active Directory Graph) for authentication by default - you can disable this by setting `use_microsoft_graph` to `false`. **This setting will be removed in Terraform 1.3, due to Microsoft's deprecation of ADAL**. *** @@ -289,6 +287,6 @@ When authenticating using a Service Principal with a Client Secret - the followi * `tenant_id` - (Optional) The Tenant ID in which the Subscription exists. This can also be sourced from the `ARM_TENANT_ID` environment variable. -* `use_microsoft_graph` - (Optional) Should MSAL be used for authentication instead of ADAL, and should Microsoft Graph be used instead of Azure Active Directory Graph? Defaults to `false`. +* `use_microsoft_graph` - (Optional) Should MSAL be used for authentication instead of ADAL, and should Microsoft Graph be used instead of Azure Active Directory Graph? Defaults to `true`. --> **Note:** By default the Azure Backend uses ADAL for authentication which is deprecated in favour of MSAL - MSAL can be used by setting `use_microsoft_graph` to `true`. **The default for this will change in Terraform 1.2**, so that MSAL authentication is used by default. +-> **Note:** In Terraform 1.2 the Azure Backend uses MSAL (and Microsoft Graph) rather than ADAL (and Azure Active Directory Graph) for authentication by default - you can disable this by setting `use_microsoft_graph` to `false`. **This setting will be removed in Terraform 1.3, due to Microsoft's deprecation of ADAL**. From 0028a26cf81d6b9d7c036407426469597c017a02 Mon Sep 17 00:00:00 2001 From: Tom Harvey Date: Wed, 20 Apr 2022 17:37:08 +0200 Subject: [PATCH 47/54] updating to include #30891 --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 981fd2c0d3..f9063c55fb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,7 @@ ENHANCEMENTS: * Add `TF_HOSTNAME` environment variable fallback for `hostname` in the cloud configuration * `TF_WORKSPACE` can now be used to configure the `workspaces` attribute in your cloud configuration * When running on macOS, Terraform will now use platform APIs to validate certificates presented by TLS (HTTPS) servers. This may change exactly which root certificates Terraform will accept as valid. ([#30768](https://github.com/hashicorp/terraform/issues/30768)) +* The AzureRM Backend now defaults to using MSAL (and Microsoft Graph) rather than ADAL (and Azure Active Directory Graph) for authentication. ([#30891](https://github.com/hashicorp/terraform/issues/30891)) * Show remote host in error message for clarity when installation of provider fails ([#30810](https://github.com/hashicorp/terraform/issues/30810)) BUG FIXES: From 7a43db405c8da55f918b64b37aae59be2a8180b7 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 20 Apr 2022 11:46:37 -0400 Subject: [PATCH 48/54] Add link to operators page and all out other types --- .../language/expressions/custom-conditions.mdx | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 657849051b..71acb7d7bf 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -189,6 +189,16 @@ in a condition as long as the expression is valid and returns a boolean result. The following language features are particularly useful when writing condition expressions. +### Logical Operators + +Use the logical operators `&&` (AND), `||` (OR), and `!` (NOT) to combine multiple conditions together. + +```hcl + condition = var.name != "" && lower(var.name) == var.name +``` + +You can also use arithmetic (e.g. `a+b`), equality (eg., `a==b`) and comparison operators (e.g., `a Date: Wed, 20 Apr 2022 11:49:33 -0400 Subject: [PATCH 49/54] fix incorrect HCL syntax for example --- website/docs/language/data-sources/index.mdx | 2 +- website/docs/language/expressions/custom-conditions.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/language/data-sources/index.mdx b/website/docs/language/data-sources/index.mdx index 651f800b90..6cfcf04712 100644 --- a/website/docs/language/data-sources/index.mdx +++ b/website/docs/language/data-sources/index.mdx @@ -134,7 +134,7 @@ data "aws_ami" "example" { # The AMI ID must refer to an existing AMI that has the tag "nomad-server". postcondition { condition = self.tags["Component"] == "nomad-server" - error_message = error_message = "tags[\"Component\"] must be \"nomad-server\"." + error_message = "tags[\"Component\"] must be \"nomad-server\"." } } } diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 71acb7d7bf..3d599cbf7d 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -81,7 +81,7 @@ data "aws_ami" "example" { # The AMI ID must refer to an existing AMI that has the tag "nomad-server". postcondition { condition = self.tags["Component"] == "nomad-server" - error_message = "The selected AMI must be tagged with the Component value \"nomad-server\"." + error_message = "tags[\"Component\"] must be \"nomad-server\"." } } } From 3c7c5bbd21dc32fa650c2b3505c77315838421aa Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 20 Apr 2022 12:06:50 -0400 Subject: [PATCH 50/54] add links to function and expressions; move result types back to conditionals page (oops) --- .../language/expressions/conditionals.mdx | 32 +++++++++++ .../expressions/custom-conditions.mdx | 54 +++++-------------- 2 files changed, 46 insertions(+), 40 deletions(-) diff --git a/website/docs/language/expressions/conditionals.mdx b/website/docs/language/expressions/conditionals.mdx index fc241f9e67..90e08bbaaf 100644 --- a/website/docs/language/expressions/conditionals.mdx +++ b/website/docs/language/expressions/conditionals.mdx @@ -46,3 +46,35 @@ You can create conditions that produce custom error messages for several types o Custom conditions can help capture assumptions, helping future maintainers understand the configuration design and intent. They also return useful information about errors earlier and in context, helping consumers more easily diagnose issues in their configurations. Refer to [Custom Condition Checks](/language/expressions/custom-conditions#input-variable-validation) for details. + +## Result Types + +The two result values may be of any type, but they must both +be of the _same_ type so that Terraform can determine what type the whole +conditional expression will return without knowing the condition value. + +If the two result expressions don't produce the same type then Terraform will +attempt to find a type that they can both convert to, and make those +conversions automatically if so. + +For example, the following expression is valid and will always return a string, +because in Terraform all numbers can convert automatically to a string using +decimal digits: + +```hcl +var.example ? 12 : "hello" +``` + +Relying on this automatic conversion behavior can be confusing for those who +are not familiar with Terraform's conversion rules though, so we recommend +being explicit using type conversion functions in any situation where there may +be some uncertainty about the expected result type. + +The following example is contrived because it would be easier to write the +constant `"12"` instead of the type conversion in this case, but shows how to +use [`tostring`](/language/functions/tostring) to explicitly convert a number to +a string. + +```hcl +var.example ? tostring(12) : "hello" +``` diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 3d599cbf7d..72188b5b57 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -165,14 +165,20 @@ output "api_base_url" { You can often implement a validation check as either a postcondition of the resource producing the data or as a precondition of a resource or output value using the data. To decide which is most appropriate, consider whether the check is representing either an assumption or a guarantee. -**Assumption:** A condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. +#### Use Preconditions for Assumptions + +An assumption is a condition that must be true in order for the configuration of a particular resource to be usable. For example, an `aws_instance` configuration can have the assumption that the given AMI will always be configured for the `x86_64` CPU architecture. We recommend using preconditions for assumptions, so that future maintainers can find them close to the other expressions that rely on that condition. This lets them understand more about what that resource is intended to allow. -**Guarantee:** A characteristic or behavior of an object that the rest of the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. +#### Use Postconditions for Guarantees + +A guarantee is a characteristic or behavior of an object that the rest of the configuration should be able to rely on. For example, an `aws_instance` configuration can have the guarantee that an EC2 instance will be running in a network that assigns it a private DNS record. We recommend using postconditions for guarantees, so that future maintainers can find them close to the resource configuration that is responsible for implementing those guarantees. This lets them more easily determine which behaviors they should preserve when changing the configuration. +#### Additional Decision Factors + You should also consider the following factors. - Which resource or output value would be most helpful to report in the error message. Terraform will always report errors in the location where the condition was declared. @@ -197,11 +203,11 @@ Use the logical operators `&&` (AND), `||` (OR), and `!` (NOT) to combine multip condition = var.name != "" && lower(var.name) == var.name ``` -You can also use arithmetic (e.g. `a+b`), equality (eg., `a==b`) and comparison operators (e.g., `a Date: Fri, 15 Apr 2022 14:06:25 -0700 Subject: [PATCH 51/54] lang/funcs: type conversion functions can convert null values We had intended these functions to attempt to convert any given value, but there is a special behavior in the function system where functions must opt in to being able to handle dynamically-typed arguments so that we don't need to repeat the special case for that inside every function implementation. In this case we _do_ want to specially handle dynamically-typed values, because the keyword "null" in HCL produces cty.NullVal(cty.DynamicPseudoType) and we want the conversion function to convert it to a null of a more specific type. These conversion functions are already just a thin wrapper around the underlying type conversion functionality anyway, and that already supports converting dynamic-typed values in the expected way, so we can just opt in to allowing dynamically-typed values and let the conversion functionality do the expected work. Fixing this allows module authors to use type conversion functions to give additional type information to Terraform in situations that are too ambiguous to be handled automatically by the type inference/unification process. Previously tostring(null) was effectively a no-op, totally ignoring the author's request to treat the null as a string. --- internal/lang/funcs/conversion.go | 7 ++++--- internal/lang/funcs/conversion_test.go | 10 ++++++++++ 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/internal/lang/funcs/conversion.go b/internal/lang/funcs/conversion.go index 8eebb3a620..721606226e 100644 --- a/internal/lang/funcs/conversion.go +++ b/internal/lang/funcs/conversion.go @@ -30,9 +30,10 @@ func MakeToFunc(wantTy cty.Type) function.Function { // messages to be more appropriate for an explicit type // conversion, whereas the cty function system produces // messages aimed at _implicit_ type conversions. - Type: cty.DynamicPseudoType, - AllowNull: true, - AllowMarked: true, + Type: cty.DynamicPseudoType, + AllowNull: true, + AllowMarked: true, + AllowDynamicType: true, }, }, Type: func(args []cty.Value) (cty.Type, error) { diff --git a/internal/lang/funcs/conversion_test.go b/internal/lang/funcs/conversion_test.go index 40317ba134..9c3e7e9f74 100644 --- a/internal/lang/funcs/conversion_test.go +++ b/internal/lang/funcs/conversion_test.go @@ -33,6 +33,16 @@ func TestTo(t *testing.T) { cty.NullVal(cty.String), ``, }, + { + // This test case represents evaluating the expression tostring(null) + // from HCL, since null in HCL is cty.NullVal(cty.DynamicPseudoType). + // The result in that case should still be null, but a null specifically + // of type string. + cty.NullVal(cty.DynamicPseudoType), + cty.String, + cty.NullVal(cty.String), + ``, + }, { cty.StringVal("a").Mark("boop"), cty.String, From eb2724d37f4d4b7c055518a7358802e419337661 Mon Sep 17 00:00:00 2001 From: Martin Atkins Date: Wed, 20 Apr 2022 09:13:17 -0700 Subject: [PATCH 52/54] Update CHANGELOG.md --- CHANGELOG.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f9063c55fb..be60642f76 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,16 +31,17 @@ ENHANCEMENTS: * When running on macOS, Terraform will now use platform APIs to validate certificates presented by TLS (HTTPS) servers. This may change exactly which root certificates Terraform will accept as valid. ([#30768](https://github.com/hashicorp/terraform/issues/30768)) * The AzureRM Backend now defaults to using MSAL (and Microsoft Graph) rather than ADAL (and Azure Active Directory Graph) for authentication. ([#30891](https://github.com/hashicorp/terraform/issues/30891)) * Show remote host in error message for clarity when installation of provider fails ([#30810](https://github.com/hashicorp/terraform/issues/30810)) +* Terraform now prints a warning when adding an attribute to `ignore_changes` that is managed only by the provider (non-optional computed attribute). ([#30517](https://github.com/hashicorp/terraform/issues/30517)) BUG FIXES: * Terraform now handles type constraints, nullability, and custom variable validation properly for root module variables. Previously there was an order of operations problem where the nullability and custom variable validation were checked too early, prior to dealing with the type constraints, and thus that logic could potentially "see" an incorrectly-typed value in spite of the type constraint, leading to incorrect errors. ([#29959](https://github.com/hashicorp/terraform/issues/29959)) -* `terraform show -json`: JSON plan output now correctly maps aliased providers to their configurations, and includes the full provider source address alongside the short provider name. ([#30138](https://github.com/hashicorp/terraform/issues/30138)) -* Terraform now prints a warning when adding an attribute to `ignore_changes` that is managed only by the provider (non-optional computed attribute). ([#30517](https://github.com/hashicorp/terraform/issues/30517)) -* Terraform will prioritize local terraform variables over remote terraform variables in operations such as `import`, `plan`, `refresh` and `apply` for workspaces in local execution mode. This behavior applies to both `remote` backend and the `cloud` integration configuration. ([#29972](https://github.com/hashicorp/terraform/issues/29972)) -* Terraform now outputs an error when `cidrnetmask()` is called with an IPv6 address. ([#30703](https://github.com/hashicorp/terraform/issues/30703)) +* Applying the various type conversion functions like `tostring`, `tonumber`, etc to `null` will now return a null value of the intended type. For example, `tostring(null)` converts from a null value of an unknown type to a null value of string type. Terraform can often handle such conversions automatically when needed, but explicit annotations like this can help Terraform to understand author intent when inferring type conversions for complex-typed values. [GH-30879] +* Terraform now outputs an error when `cidrnetmask()` is called with an IPv6 address, as it was previously documented to do. ([#30703](https://github.com/hashicorp/terraform/issues/30703)) * When performing advanced state management with the `terraform state` commands, Terraform now checks the `required_version` field in the configuration before proceeding. ([#30511](https://github.com/hashicorp/terraform/pull/30511)) * When rendering a diff, Terraform now quotes the name of any object attribute whose string representation is not a valid identifier. ([#30766](https://github.com/hashicorp/terraform/issues/30766)) +* Terraform will prioritize local terraform variables over remote terraform variables in operations such as `import`, `plan`, `refresh` and `apply` for workspaces in local execution mode. This behavior applies to both `remote` backend and the `cloud` integration configuration. ([#29972](https://github.com/hashicorp/terraform/issues/29972)) +* `terraform show -json`: JSON plan output now correctly maps aliased providers to their configurations, and includes the full provider source address alongside the short provider name. ([#30138](https://github.com/hashicorp/terraform/issues/30138)) UPGRADE NOTES: From b1d9339368563b3e76e0b71fd200cafb02870853 Mon Sep 17 00:00:00 2001 From: Laura Pacilio <83350965+laurapacilio@users.noreply.github.com> Date: Wed, 20 Apr 2022 12:19:29 -0400 Subject: [PATCH 53/54] Final formatting nits --- .../expressions/custom-conditions.mdx | 26 +++++++------------ 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/website/docs/language/expressions/custom-conditions.mdx b/website/docs/language/expressions/custom-conditions.mdx index 72188b5b57..449831b411 100644 --- a/website/docs/language/expressions/custom-conditions.mdx +++ b/website/docs/language/expressions/custom-conditions.mdx @@ -21,11 +21,9 @@ This page explains the following: -> **Note:** Input variable validation is available in Terraform CLI v0.13.0 and later. -Add one or more `validation` blocks within the `variable` block to specify custom conditions. +Add one or more `validation` blocks within the `variable` block to specify custom conditions. Each validation requires a [`condition` argument](#condition-expressions), an expression that must use the value of the variable to return `true` if the value is valid, or `false` if it is invalid. The expression can refer only to the containing variable and must not produce errors. -Each validation requires a [`condition` argument](#condition-expressions), an expression that must use the value of the variable to return `true` if the value is valid, or `false` if it is invalid. The expression can refer only to the containing variable and must not produce errors. - -If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple `validation` blocks, Terraform returns error messages for all failed conditions. +If the condition evaluates to `false`, Terraform produces an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple validations, Terraform returns error messages for all failed conditions. The following example checks whether the AMI ID has valid syntax. @@ -67,11 +65,11 @@ Terraform checks a precondition _before_ evaluating the object it is associated ### Usage -Each precondition and postcondition requires a [`condition` argument](#condition-expressions). This is an expression that must return `true` if the conditition is fufilled or `false` if it is invalid. The expression can refer to any other objects in the same module, as long as the references do not create cyclic dependencies. Resource postconditions can also use the symbol `self` to refer to attributes of each instance of the resource where they are configured. For example, `self.private_dns` refers to the `private_dns` attribute of each instance of the containing resource. +Each precondition and postcondition requires a [`condition` argument](#condition-expressions). This is an expression that must return `true` if the conditition is fufilled or `false` if it is invalid. The expression can refer to any other objects in the same module, as long as the references do not create cyclic dependencies. Resource postconditions can also use the [`self` object](#self-object) to refer to attributes of each instance of the resource where they are configured. If the condition evaluates to `false`, Terraform will produce an [error message](#error-messages) that includes the result of the `error_message` expression. If you declare multiple preconditions or postconditions, Terraform returns error messages for all failed conditions. -The following example uses a postcondition to detect if the caller accidentally provided an AMI intended for the wrong system component. This might otherwise be detected only after booting the EC2 instance and noticing that the expected network service is not running. +The following example uses a postcondition to detect if the caller accidentally provided an AMI intended for the wrong system component. ``` hcl data "aws_ami" "example" { @@ -87,8 +85,6 @@ data "aws_ami" "example" { } ``` -You can add `precondition` and `postcondition` blocks in the following locations within your configuration. - #### Resources and Data Sources The `lifecycle` block inside a `resource` or `data` block can include both `precondition` and `postcondition` blocks. @@ -179,11 +175,11 @@ We recommend using postconditions for guarantees, so that future maintainers can #### Additional Decision Factors -You should also consider the following factors. +You should also consider the following questions when creating preconditions and postconditions. -- Which resource or output value would be most helpful to report in the error message. Terraform will always report errors in the location where the condition was declared. -- Which approach is more convenient. If a particular resource has many dependencies that all make an assumption about that resource, it can be pragmatic to declare that once as a post-condition of the resource, rather than declaring it many times as preconditions on each of the dependencies. -- Whether it is helpful to declare the same or similar conditions as both preconditions and postconditions. This can be useful if the postcondition is in a different module than the precondition because it lets the modules verify one another as they evolve independently. +- Which resource or output value would be most helpful to report in the error message? Terraform will always report errors in the location where the condition was declared. +- Which approach is more convenient? If a particular resource has many dependencies that all make an assumption about that resource, it can be pragmatic to declare that once as a post-condition of the resource, rather than declaring it many times as preconditions on each of the dependencies. +- Is it helpful to declare the same or similar conditions as both preconditions and postconditions? This can be useful if the postcondition is in a different module than the precondition because it lets the modules verify one another as they evolve independently. ## Condition Expressions @@ -191,9 +187,7 @@ You should also consider the following factors. Input variable validation, preconditions, and postconditions all require a `condition` argument. This is a boolean expression that should return `true` if the intended assumption or guarantee is fulfilled or `false` if it does not. You can use any of Terraform's built-in functions or language operators -in a condition as long as the expression is valid and returns a boolean result. - -The following language features are particularly useful when writing condition expressions. +in a condition as long as the expression is valid and returns a boolean result. The following language features are particularly useful when writing condition expressions. ### Logical Operators @@ -359,5 +353,5 @@ During the apply phase, a failed _precondition_ will prevent Terraform from implementing planned actions for the associated resource. However, a failed _postcondition_ will halt processing after Terraform has already implemented these actions. The failed postcondition prevents any further downstream actions that rely on the resource, but does not undo the actions Terraform has already taken. Terraform typically has less information during the initial creation of a -full configuration than when applying subsequent changes. Therefore, conditions checked during apply for initial creation may be checked during planning for subsequent updates. +full configuration than when applying subsequent changes. Therefore, Terraform may check conditions during apply for initial creation and then check them during planning for subsequent updates. From f63ef2b5ef0b44fe677d02f598d6610b7c880385 Mon Sep 17 00:00:00 2001 From: Sebastian Rivera Date: Wed, 20 Apr 2022 13:36:23 -0400 Subject: [PATCH 54/54] Rename cloud env vars to use TF_CLOUD prefix --- CHANGELOG.md | 4 ++-- internal/cloud/backend.go | 12 ++++++------ internal/cloud/backend_test.go | 24 ++++++++++++------------ internal/cloud/e2e/env_variables_test.go | 8 ++++---- website/docs/cli/cloud/settings.mdx | 6 +++--- 5 files changed, 27 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index be60642f76..c33c7825c4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,8 +25,8 @@ ENHANCEMENTS: * Error messages for preconditions, postconditions, and custom variable validations are now evaluated as expressions, allowing interpolation of relevant values into the output. ([#30613](https://github.com/hashicorp/terraform/issues/30613)) * There are some small improvements to the error and warning messages Terraform will emit in the case of invalid provider configuration passing between modules. There are no changes to which situations will produce errors and warnings, but the messages now include additional information intended to clarify what problem Terraform is describing and how to address it. ([#30639](https://github.com/hashicorp/terraform/issues/30639)) * When running `terraform plan`, only show external changes which may have contributed to the current plan ([#30486](https://github.com/hashicorp/terraform/issues/30486)) -* Add `TF_ORGANIZATION` environment variable fallback for `organization` in the cloud configuration -* Add `TF_HOSTNAME` environment variable fallback for `hostname` in the cloud configuration +* Add `TF_CLOUD_ORGANIZATION` environment variable fallback for `organization` in the cloud configuration +* Add `TF_CLOUD_HOSTNAME` environment variable fallback for `hostname` in the cloud configuration * `TF_WORKSPACE` can now be used to configure the `workspaces` attribute in your cloud configuration * When running on macOS, Terraform will now use platform APIs to validate certificates presented by TLS (HTTPS) servers. This may change exactly which root certificates Terraform will accept as valid. ([#30768](https://github.com/hashicorp/terraform/issues/30768)) * The AzureRM Backend now defaults to using MSAL (and Microsoft Graph) rather than ADAL (and Azure Active Directory Graph) for authentication. ([#30891](https://github.com/hashicorp/terraform/issues/30891)) diff --git a/internal/cloud/backend.go b/internal/cloud/backend.go index cdb59cf2d8..9a01af33f0 100644 --- a/internal/cloud/backend.go +++ b/internal/cloud/backend.go @@ -155,9 +155,9 @@ func (b *Cloud) PrepareConfig(obj cty.Value) (cty.Value, tfdiags.Diagnostics) { // check if organization is specified in the config. if val := obj.GetAttr("organization"); val.IsNull() || val.AsString() == "" { // organization is specified in the config but is invalid, so - // we'll fallback on TF_ORGANIZATION - if val := os.Getenv("TF_ORGANIZATION"); val == "" { - diags = diags.Append(missingConfigAttributeAndEnvVar("organization", "TF_ORGANIZATION")) + // we'll fallback on TF_CLOUD_ORGANIZATION + if val := os.Getenv("TF_CLOUD_ORGANIZATION"); val == "" { + diags = diags.Append(missingConfigAttributeAndEnvVar("organization", "TF_CLOUD_ORGANIZATION")) } } @@ -355,7 +355,7 @@ func (b *Cloud) setConfigurationFields(obj cty.Value) tfdiags.Diagnostics { var diags tfdiags.Diagnostics // Get the hostname. - b.hostname = os.Getenv("TF_HOSTNAME") + b.hostname = os.Getenv("TF_CLOUD_HOSTNAME") if val := obj.GetAttr("hostname"); !val.IsNull() && val.AsString() != "" { b.hostname = val.AsString() } else if b.hostname == "" { @@ -363,10 +363,10 @@ func (b *Cloud) setConfigurationFields(obj cty.Value) tfdiags.Diagnostics { } // We can have two options, setting the organization via the config - // or using TF_ORGANIZATION. Since PrepareConfig() validates that one of these + // or using TF_CLOUD_ORGANIZATION. Since PrepareConfig() validates that one of these // values must exist, we'll initially set it to the env var and override it if // specified in the configuration. - b.organization = os.Getenv("TF_ORGANIZATION") + b.organization = os.Getenv("TF_CLOUD_ORGANIZATION") // Check if the organization is present and valid in the config. if val := obj.GetAttr("organization"); !val.IsNull() && val.AsString() != "" { diff --git a/internal/cloud/backend_test.go b/internal/cloud/backend_test.go index f857c5cef5..33afc124d0 100644 --- a/internal/cloud/backend_test.go +++ b/internal/cloud/backend_test.go @@ -86,7 +86,7 @@ func TestCloud_PrepareConfig(t *testing.T) { "tags": cty.NullVal(cty.Set(cty.String)), }), }), - expectedErr: `Invalid or missing required argument: "organization" must be set in the cloud configuration or as an environment variable: TF_ORGANIZATION.`, + expectedErr: `Invalid or missing required argument: "organization" must be set in the cloud configuration or as an environment variable: TF_CLOUD_ORGANIZATION.`, }, "null workspace": { config: cty.ObjectVal(map[string]cty.Value{ @@ -161,7 +161,7 @@ func TestCloud_PrepareConfigWithEnvVars(t *testing.T) { }), }), vars: map[string]string{ - "TF_ORGANIZATION": "example-org", + "TF_CLOUD_ORGANIZATION": "example-org", }, }, "with no organization attribute or env var": { @@ -173,7 +173,7 @@ func TestCloud_PrepareConfigWithEnvVars(t *testing.T) { }), }), vars: map[string]string{}, - expectedErr: `Invalid or missing required argument: "organization" must be set in the cloud configuration or as an environment variable: TF_ORGANIZATION.`, + expectedErr: `Invalid or missing required argument: "organization" must be set in the cloud configuration or as an environment variable: TF_CLOUD_ORGANIZATION.`, }, "null workspace": { config: cty.ObjectVal(map[string]cty.Value{ @@ -190,8 +190,8 @@ func TestCloud_PrepareConfigWithEnvVars(t *testing.T) { "workspaces": cty.NullVal(cty.String), }), vars: map[string]string{ - "TF_ORGANIZATION": "hashicorp", - "TF_WORKSPACE": "my-workspace", + "TF_CLOUD_ORGANIZATION": "hashicorp", + "TF_WORKSPACE": "my-workspace", }, }, } @@ -242,7 +242,7 @@ func TestCloud_configWithEnvVars(t *testing.T) { }), }), vars: map[string]string{ - "TF_ORGANIZATION": "hashicorp", + "TF_CLOUD_ORGANIZATION": "hashicorp", }, expectedOrganization: "hashicorp", }, @@ -257,7 +257,7 @@ func TestCloud_configWithEnvVars(t *testing.T) { }), }), vars: map[string]string{ - "TF_ORGANIZATION": "we-should-not-see-this", + "TF_CLOUD_ORGANIZATION": "we-should-not-see-this", }, expectedOrganization: "hashicorp", }, @@ -272,7 +272,7 @@ func TestCloud_configWithEnvVars(t *testing.T) { }), }), vars: map[string]string{ - "TF_HOSTNAME": "private.hashicorp.engineering", + "TF_CLOUD_HOSTNAME": "private.hashicorp.engineering", }, expectedHostname: "private.hashicorp.engineering", }, @@ -287,7 +287,7 @@ func TestCloud_configWithEnvVars(t *testing.T) { }), }), vars: map[string]string{ - "TF_HOSTNAME": "mycool.tfe-host.io", + "TF_CLOUD_HOSTNAME": "mycool.tfe-host.io", }, expectedHostname: "private.hashicorp.engineering", }, @@ -386,9 +386,9 @@ func TestCloud_configWithEnvVars(t *testing.T) { "workspaces": cty.NullVal(cty.String), }), vars: map[string]string{ - "TF_ORGANIZATION": "mordor", - "TF_WORKSPACE": "mt-doom", - "TF_HOSTNAME": "mycool.tfe-host.io", + "TF_CLOUD_ORGANIZATION": "mordor", + "TF_WORKSPACE": "mt-doom", + "TF_CLOUD_HOSTNAME": "mycool.tfe-host.io", }, expectedOrganization: "mordor", expectedWorkspaceName: "mt-doom", diff --git a/internal/cloud/e2e/env_variables_test.go b/internal/cloud/e2e/env_variables_test.go index 3cf0096440..1044432bfd 100644 --- a/internal/cloud/e2e/env_variables_test.go +++ b/internal/cloud/e2e/env_variables_test.go @@ -17,7 +17,7 @@ func Test_cloud_organization_env_var(t *testing.T) { t.Cleanup(cleanup) cases := testCases{ - "with TF_ORGANIZATION set": { + "with TF_CLOUD_ORGANIZATION set": { operations: []operationSets{ { prep: func(t *testing.T, orgName, dir string) { @@ -50,7 +50,7 @@ func Test_cloud_organization_env_var(t *testing.T) { }, } - testRunner(t, cases, 0, fmt.Sprintf("TF_ORGANIZATION=%s", org.Name)) + testRunner(t, cases, 0, fmt.Sprintf("TF_CLOUD_ORGANIZATION=%s", org.Name)) } func Test_cloud_workspace_name_env_var(t *testing.T) { @@ -258,7 +258,7 @@ func Test_cloud_null_config(t *testing.T) { } testRunner(t, cases, 1, - fmt.Sprintf(`TF_ORGANIZATION=%s`, org.Name), - fmt.Sprintf(`TF_HOSTNAME=%s`, tfeHostname), + fmt.Sprintf(`TF_CLOUD_ORGANIZATION=%s`, org.Name), + fmt.Sprintf(`TF_CLOUD_HOSTNAME=%s`, tfeHostname), fmt.Sprintf(`TF_WORKSPACE=%s`, wk.Name)) } diff --git a/website/docs/cli/cloud/settings.mdx b/website/docs/cli/cloud/settings.mdx index ad40e15798..cdb1a4147b 100644 --- a/website/docs/cli/cloud/settings.mdx +++ b/website/docs/cli/cloud/settings.mdx @@ -89,12 +89,12 @@ You can use environment variables to configure one or more `cloud` block attribu Use the following environment variables to configure the `cloud` block: -- `TF_ORGANIZATION` - The name of the organization. Serves as a fallback for `organization` +- `TF_CLOUD_ORGANIZATION` - The name of the organization. Serves as a fallback for `organization` in the cloud configuration. If both are specified, the configuration takes precedence. -- `TF_HOSTNAME` - The hostname of a Terraform Enterprise installation. Serves as a fallback if `hostname` is not specified in the cloud configuration. If both are specified, the configuration takes precendence. +- `TF_CLOUD_HOSTNAME` - The hostname of a Terraform Enterprise installation. Serves as a fallback if `hostname` is not specified in the cloud configuration. If both are specified, the configuration takes precendence. -- `TF_WORKSPACE` - The name of a single Terraform Cloud workspace. If the `workspaces` attribute is not included in your configuration file, the `cloud` block interprets `TF_WORKSPACE` as the `name` value of the `workspaces` attribute. The workspace must exist in the organization specified in the configuration or `TF_ORGANIZATION`. If the `cloud` block uses tags, Terraform Cloud will return an error if the value of `TF_WORKSPACE` is not included in the set of tags. Refer to [TF_WORKSPACE](https://www.terraform.io/cli/config/environment-variables#tf_workspace) for more details. +- `TF_WORKSPACE` - The name of a single Terraform Cloud workspace. If the `workspaces` attribute is not included in your configuration file, the `cloud` block interprets `TF_WORKSPACE` as the `name` value of the `workspaces` attribute. The workspace must exist in the organization specified in the configuration or `TF_CLOUD_ORGANIZATION`. If the `cloud` block uses tags, Terraform Cloud will return an error if the value of `TF_WORKSPACE` is not included in the set of tags. Refer to [TF_WORKSPACE](https://www.terraform.io/cli/config/environment-variables#tf_workspace) for more details. ## Excluding Files from Upload with .terraformignore