2024-02-08 03:48:59 -06:00
// Copyright (c) The OpenTofu Authors
// SPDX-License-Identifier: MPL-2.0
// Copyright (c) 2023 HashiCorp, Inc.
2023-05-02 10:33:06 -05:00
// SPDX-License-Identifier: MPL-2.0
2023-09-20 07:16:53 -05:00
package tofu
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
import (
"fmt"
2022-02-07 13:46:31 -06:00
"strings"
2024-12-04 13:24:44 -06:00
"sync"
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
"testing"
2021-12-21 20:04:24 -06:00
"github.com/hashicorp/hcl/v2"
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
"github.com/zclconf/go-cty/cty"
2023-09-20 06:35:35 -05:00
"github.com/opentofu/opentofu/internal/addrs"
"github.com/opentofu/opentofu/internal/checks"
"github.com/opentofu/opentofu/internal/lang"
"github.com/opentofu/opentofu/internal/lang/marks"
"github.com/opentofu/opentofu/internal/tfdiags"
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
)
func TestPrepareFinalInputVariableValue ( t * testing . T ) {
// This is just a concise way to define a bunch of *configs.Variable
// objects to use in our tests below. We're only going to decode this
// config, not fully evaluate it.
cfgSrc := `
variable "nullable_required" {
}
variable "nullable_optional_default_string" {
default = "hello"
}
variable "nullable_optional_default_null" {
default = null
}
variable "constrained_string_nullable_required" {
type = string
}
variable "constrained_string_nullable_optional_default_string" {
type = string
default = "hello"
}
variable "constrained_string_nullable_optional_default_bool" {
type = string
default = true
}
variable "constrained_string_nullable_optional_default_null" {
type = string
default = null
}
variable "required" {
nullable = false
}
variable "optional_default_string" {
nullable = false
default = "hello"
}
variable "constrained_string_required" {
nullable = false
type = string
}
variable "constrained_string_optional_default_string" {
nullable = false
type = string
default = "hello"
}
variable "constrained_string_optional_default_bool" {
nullable = false
type = string
default = true
}
2022-02-18 04:46:56 -06:00
variable "constrained_string_sensitive_required" {
sensitive = true
nullable = false
type = string
}
2022-11-02 03:38:23 -05:00
variable "complex_type_with_nested_default_optional" {
type = set ( object ( {
name = string
schedules = set ( object ( {
name = string
cold_storage_after = optional ( number , 10 )
} ) )
} ) )
}
variable "complex_type_with_nested_complex_types" {
type = object ( {
name = string
nested_object = object ( {
name = string
value = optional ( string , "foo" )
} )
nested_object_with_default = optional ( object ( {
name = string
value = optional ( string , "bar" )
} ) , {
name = "nested_object_with_default"
} )
} )
}
2022-11-10 08:00:16 -06:00
// https://github.com/hashicorp/terraform/issues/32152
// This variable was originally added to test that optional attribute
// metadata is stripped from empty default collections. Essentially, you
// should be able to mix and match custom and default values for the
// optional_list attribute.
variable "complex_type_with_empty_default_and_nested_optional" {
type = list ( object ( {
name = string
optional_list = optional ( list ( object ( {
string = string
optional_string = optional ( string )
} ) ) , [ ] )
} ) )
}
// https://github.com/hashicorp/terraform/issues/32160#issuecomment-1302783910
// These variables were added to test the specific use case from this
// GitHub comment.
variable "empty_object_with_optional_nested_object_with_optional_bool" {
type = object ( {
thing = optional ( object ( {
flag = optional ( bool , false )
} ) )
} )
default = { }
}
variable "populated_object_with_optional_nested_object_with_optional_bool" {
type = object ( {
thing = optional ( object ( {
flag = optional ( bool , false )
} ) )
} )
default = {
thing = { }
}
}
variable "empty_object_with_default_nested_object_with_optional_bool" {
type = object ( {
thing = optional ( object ( {
flag = optional ( bool , false )
} ) , { } )
} )
default = { }
}
// https://github.com/hashicorp/terraform/issues/32160
// This variable was originally added to test that optional objects do
// get created containing only their defaults. Instead they should be
// left empty. We do not expect nested_object to be created just because
// optional_string has a default value.
variable "object_with_nested_object_with_required_and_optional_attributes" {
type = object ( {
nested_object = optional ( object ( {
string = string
optional_string = optional ( string , "optional" )
} ) )
} )
}
// https://github.com/hashicorp/terraform/issues/32157
// Similar to above, we want to see that merging combinations of the
// nested_object into a single collection doesn't crash because of
// inconsistent elements.
variable "list_with_nested_object_with_required_and_optional_attributes" {
type = list ( object ( {
nested_object = optional ( object ( {
string = string
optional_string = optional ( string , "optional" )
} ) )
} ) )
}
// https://github.com/hashicorp/terraform/issues/32109
// This variable was originally introduced to test the behaviour of
// the dynamic type constraint. You should be able to use the 'any'
// constraint and introduce empty, null, and populated values into the
// list.
variable "list_with_nested_list_of_any" {
type = list ( object ( {
a = string
b = optional ( list ( any ) )
} ) )
default = [
{
a = "a"
} ,
{
a = "b"
b = [ 1 ]
}
]
}
2023-01-31 05:37:24 -06:00
// https://github.com/hashicorp/terraform/issues/32396
// This variable was originally introduced to test the behaviour of the
// dynamic type constraint. You should be able to set primitive types in
// the list consistently.
variable "list_with_nested_collections_dynamic_with_default" {
type = list (
object ( {
name = optional ( string , "default" )
taints = optional ( list ( map ( any ) ) , [ ] )
} )
)
}
2023-03-09 07:29:58 -06:00
// https://github.com/hashicorp/terraform/issues/32752
// This variable was introduced to make sure the evaluation doesn't
// crash even when the types are wrong.
variable "invalid_nested_type" {
type = map (
object ( {
rules = map (
object ( {
destination_addresses = optional ( list ( string ) , [ ] )
} )
)
} )
)
default = { }
}
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
`
cfg := testModuleInline ( t , map [ string ] string {
"main.tf" : cfgSrc ,
} )
variableConfigs := cfg . Module . Variables
2021-12-21 20:04:24 -06:00
// Because we loaded our pseudo-module from a temporary file, the
// declaration source ranges will have unpredictable filenames. We'll
// fix that here just to make things easier below.
for _ , vc := range variableConfigs {
vc . DeclRange . Filename = "main.tf"
}
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
tests := [ ] struct {
varName string
given cty . Value
want cty . Value
wantErr string
} {
// nullable_required
{
"nullable_required" ,
cty . NilVal ,
cty . UnknownVal ( cty . DynamicPseudoType ) ,
` Required variable not set: The variable "nullable_required" is required, but is not set. ` ,
} ,
{
"nullable_required" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . NullVal ( cty . DynamicPseudoType ) ,
` ` , // "required" for a nullable variable means only that it must be set, even if it's set to null
} ,
{
"nullable_required" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"nullable_required" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// nullable_optional_default_string
{
"nullable_optional_default_string" ,
cty . NilVal ,
cty . StringVal ( "hello" ) , // the declared default value
` ` ,
} ,
{
"nullable_optional_default_string" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . NullVal ( cty . DynamicPseudoType ) , // nullable variables can be really set to null, masking the default
` ` ,
} ,
{
"nullable_optional_default_string" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"nullable_optional_default_string" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// nullable_optional_default_null
{
"nullable_optional_default_null" ,
cty . NilVal ,
cty . NullVal ( cty . DynamicPseudoType ) , // the declared default value
` ` ,
} ,
{
"nullable_optional_default_null" ,
cty . NullVal ( cty . String ) ,
cty . NullVal ( cty . String ) , // nullable variables can be really set to null, masking the default
` ` ,
} ,
{
"nullable_optional_default_null" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"nullable_optional_default_null" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// constrained_string_nullable_required
{
"constrained_string_nullable_required" ,
cty . NilVal ,
cty . UnknownVal ( cty . String ) ,
` Required variable not set: The variable "constrained_string_nullable_required" is required, but is not set. ` ,
} ,
{
"constrained_string_nullable_required" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . NullVal ( cty . String ) , // the null value still gets converted to match the type constraint
` ` , // "required" for a nullable variable means only that it must be set, even if it's set to null
} ,
{
"constrained_string_nullable_required" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"constrained_string_nullable_required" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// constrained_string_nullable_optional_default_string
{
"constrained_string_nullable_optional_default_string" ,
cty . NilVal ,
cty . StringVal ( "hello" ) , // the declared default value
` ` ,
} ,
{
"constrained_string_nullable_optional_default_string" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . NullVal ( cty . String ) , // nullable variables can be really set to null, masking the default
` ` ,
} ,
{
"constrained_string_nullable_optional_default_string" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"constrained_string_nullable_optional_default_string" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// constrained_string_nullable_optional_default_bool
{
"constrained_string_nullable_optional_default_bool" ,
cty . NilVal ,
cty . StringVal ( "true" ) , // the declared default value, automatically converted to match type constraint
` ` ,
} ,
{
"constrained_string_nullable_optional_default_bool" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . NullVal ( cty . String ) , // nullable variables can be really set to null, masking the default
` ` ,
} ,
{
"constrained_string_nullable_optional_default_bool" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"constrained_string_nullable_optional_default_bool" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// constrained_string_nullable_optional_default_null
{
"constrained_string_nullable_optional_default_null" ,
cty . NilVal ,
cty . NullVal ( cty . String ) ,
` ` ,
} ,
{
"constrained_string_nullable_optional_default_null" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . NullVal ( cty . String ) ,
` ` ,
} ,
{
"constrained_string_nullable_optional_default_null" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"constrained_string_nullable_optional_default_null" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// required
{
"required" ,
cty . NilVal ,
cty . UnknownVal ( cty . DynamicPseudoType ) ,
` Required variable not set: The variable "required" is required, but is not set. ` ,
} ,
{
"required" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . UnknownVal ( cty . DynamicPseudoType ) ,
2021-12-21 20:04:24 -06:00
` Required variable not set: Unsuitable value for var.required set from outside of the configuration: required variable may not be set to null. ` ,
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
} ,
{
"required" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"required" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// optional_default_string
{
"optional_default_string" ,
cty . NilVal ,
cty . StringVal ( "hello" ) , // the declared default value
` ` ,
} ,
{
"optional_default_string" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . StringVal ( "hello" ) , // the declared default value
` ` ,
} ,
{
"optional_default_string" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"optional_default_string" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// constrained_string_required
{
"constrained_string_required" ,
cty . NilVal ,
cty . UnknownVal ( cty . String ) ,
` Required variable not set: The variable "constrained_string_required" is required, but is not set. ` ,
} ,
{
"constrained_string_required" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . UnknownVal ( cty . String ) ,
2021-12-21 20:04:24 -06:00
` Required variable not set: Unsuitable value for var.constrained_string_required set from outside of the configuration: required variable may not be set to null. ` ,
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
} ,
{
"constrained_string_required" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"constrained_string_required" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// constrained_string_optional_default_string
{
"constrained_string_optional_default_string" ,
cty . NilVal ,
cty . StringVal ( "hello" ) , // the declared default value
` ` ,
} ,
{
"constrained_string_optional_default_string" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . StringVal ( "hello" ) , // the declared default value
` ` ,
} ,
{
"constrained_string_optional_default_string" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"constrained_string_optional_default_string" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
// constrained_string_optional_default_bool
{
"constrained_string_optional_default_bool" ,
cty . NilVal ,
cty . StringVal ( "true" ) , // the declared default value, automatically converted to match type constraint
` ` ,
} ,
{
"constrained_string_optional_default_bool" ,
cty . NullVal ( cty . DynamicPseudoType ) ,
cty . StringVal ( "true" ) , // the declared default value, automatically converted to match type constraint
` ` ,
} ,
{
"constrained_string_optional_default_bool" ,
cty . StringVal ( "ahoy" ) ,
cty . StringVal ( "ahoy" ) ,
` ` ,
} ,
{
"constrained_string_optional_default_bool" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
2023-01-31 05:37:24 -06:00
{
"list_with_nested_collections_dynamic_with_default" ,
cty . TupleVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "default" ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "complex" ) ,
"taints" : cty . ListVal ( [ ] cty . Value {
cty . MapVal ( map [ string ] cty . Value {
"key" : cty . StringVal ( "my_key" ) ,
"value" : cty . StringVal ( "my_value" ) ,
} ) ,
} ) ,
} ) ,
} ) ,
cty . ListVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "default" ) ,
"taints" : cty . ListValEmpty ( cty . Map ( cty . String ) ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "complex" ) ,
"taints" : cty . ListVal ( [ ] cty . Value {
cty . MapVal ( map [ string ] cty . Value {
"key" : cty . StringVal ( "my_key" ) ,
"value" : cty . StringVal ( "my_value" ) ,
} ) ,
} ) ,
} ) ,
} ) ,
` ` ,
} ,
2022-02-18 04:46:56 -06:00
2022-11-02 03:38:23 -05:00
// complex types
{
"complex_type_with_nested_default_optional" ,
cty . SetVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "test1" ) ,
"schedules" : cty . SetVal ( [ ] cty . Value {
cty . MapVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "daily" ) ,
} ) ,
} ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "test2" ) ,
"schedules" : cty . SetVal ( [ ] cty . Value {
cty . MapVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "daily" ) ,
} ) ,
cty . MapVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "weekly" ) ,
"cold_storage_after" : cty . StringVal ( "0" ) ,
} ) ,
} ) ,
} ) ,
} ) ,
cty . SetVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "test1" ) ,
"schedules" : cty . SetVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "daily" ) ,
"cold_storage_after" : cty . NumberIntVal ( 10 ) ,
} ) ,
} ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "test2" ) ,
"schedules" : cty . SetVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "daily" ) ,
"cold_storage_after" : cty . NumberIntVal ( 10 ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "weekly" ) ,
"cold_storage_after" : cty . NumberIntVal ( 0 ) ,
} ) ,
} ) ,
} ) ,
} ) ,
` ` ,
} ,
{
"complex_type_with_nested_complex_types" ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "object" ) ,
"nested_object" : cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "nested_object" ) ,
} ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "object" ) ,
"nested_object" : cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "nested_object" ) ,
"value" : cty . StringVal ( "foo" ) ,
} ) ,
"nested_object_with_default" : cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "nested_object_with_default" ) ,
"value" : cty . StringVal ( "bar" ) ,
} ) ,
} ) ,
` ` ,
} ,
2022-11-10 08:00:16 -06:00
{
"complex_type_with_empty_default_and_nested_optional" ,
cty . ListVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "abc" ) ,
"optional_list" : cty . ListVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"string" : cty . StringVal ( "child" ) ,
"optional_string" : cty . NullVal ( cty . String ) ,
} ) ,
} ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "def" ) ,
"optional_list" : cty . NullVal ( cty . List ( cty . Object ( map [ string ] cty . Type {
"string" : cty . String ,
"optional_string" : cty . String ,
} ) ) ) ,
} ) ,
} ) ,
cty . ListVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "abc" ) ,
"optional_list" : cty . ListVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"string" : cty . StringVal ( "child" ) ,
"optional_string" : cty . NullVal ( cty . String ) ,
} ) ,
} ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "def" ) ,
"optional_list" : cty . ListValEmpty ( cty . Object ( map [ string ] cty . Type {
"string" : cty . String ,
"optional_string" : cty . String ,
} ) ) ,
} ) ,
} ) ,
` ` ,
} ,
{
"object_with_nested_object_with_required_and_optional_attributes" ,
cty . EmptyObjectVal ,
cty . ObjectVal ( map [ string ] cty . Value {
"nested_object" : cty . NullVal ( cty . Object ( map [ string ] cty . Type {
"string" : cty . String ,
"optional_string" : cty . String ,
} ) ) ,
} ) ,
` ` ,
} ,
{
"empty_object_with_optional_nested_object_with_optional_bool" ,
cty . NilVal ,
cty . ObjectVal ( map [ string ] cty . Value {
"thing" : cty . NullVal ( cty . Object ( map [ string ] cty . Type {
"flag" : cty . Bool ,
} ) ) ,
} ) ,
` ` ,
} ,
{
"populated_object_with_optional_nested_object_with_optional_bool" ,
cty . NilVal ,
cty . ObjectVal ( map [ string ] cty . Value {
"thing" : cty . ObjectVal ( map [ string ] cty . Value {
"flag" : cty . False ,
} ) ,
} ) ,
` ` ,
} ,
{
"empty_object_with_default_nested_object_with_optional_bool" ,
cty . NilVal ,
cty . ObjectVal ( map [ string ] cty . Value {
"thing" : cty . ObjectVal ( map [ string ] cty . Value {
"flag" : cty . False ,
} ) ,
} ) ,
` ` ,
} ,
{
"list_with_nested_object_with_required_and_optional_attributes" ,
cty . ListVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"nested_object" : cty . ObjectVal ( map [ string ] cty . Value {
"string" : cty . StringVal ( "string" ) ,
"optional_string" : cty . NullVal ( cty . String ) ,
} ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"nested_object" : cty . NullVal ( cty . Object ( map [ string ] cty . Type {
"string" : cty . String ,
"optional_string" : cty . String ,
} ) ) ,
} ) ,
} ) ,
cty . ListVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"nested_object" : cty . ObjectVal ( map [ string ] cty . Value {
"string" : cty . StringVal ( "string" ) ,
"optional_string" : cty . StringVal ( "optional" ) ,
} ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"nested_object" : cty . NullVal ( cty . Object ( map [ string ] cty . Type {
"string" : cty . String ,
"optional_string" : cty . String ,
} ) ) ,
} ) ,
} ) ,
` ` ,
} ,
{
"list_with_nested_list_of_any" ,
cty . NilVal ,
cty . ListVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"a" : cty . StringVal ( "a" ) ,
"b" : cty . NullVal ( cty . List ( cty . Number ) ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"a" : cty . StringVal ( "b" ) ,
"b" : cty . ListVal ( [ ] cty . Value {
cty . NumberIntVal ( 1 ) ,
} ) ,
} ) ,
} ) ,
` ` ,
} ,
2023-01-31 05:37:24 -06:00
{
"list_with_nested_collections_dynamic_with_default" ,
cty . TupleVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "default" ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "complex" ) ,
"taints" : cty . ListVal ( [ ] cty . Value {
cty . MapVal ( map [ string ] cty . Value {
"key" : cty . StringVal ( "my_key" ) ,
"value" : cty . StringVal ( "my_value" ) ,
} ) ,
} ) ,
} ) ,
} ) ,
cty . ListVal ( [ ] cty . Value {
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "default" ) ,
"taints" : cty . ListValEmpty ( cty . Map ( cty . String ) ) ,
} ) ,
cty . ObjectVal ( map [ string ] cty . Value {
"name" : cty . StringVal ( "complex" ) ,
"taints" : cty . ListVal ( [ ] cty . Value {
cty . MapVal ( map [ string ] cty . Value {
"key" : cty . StringVal ( "my_key" ) ,
"value" : cty . StringVal ( "my_value" ) ,
} ) ,
} ) ,
} ) ,
} ) ,
` ` ,
} ,
2023-03-09 07:29:58 -06:00
{
"invalid_nested_type" ,
cty . MapVal ( map [ string ] cty . Value {
"mysql" : cty . ObjectVal ( map [ string ] cty . Value {
"rules" : cty . ObjectVal ( map [ string ] cty . Value {
"destination_addresses" : cty . ListVal ( [ ] cty . Value { cty . StringVal ( "192.168.0.1" ) } ) ,
} ) ,
} ) ,
} ) ,
cty . UnknownVal ( cty . Map ( cty . Object ( map [ string ] cty . Type {
"rules" : cty . Map ( cty . Object ( map [ string ] cty . Type {
"destination_addresses" : cty . List ( cty . String ) ,
} ) ) ,
} ) ) ) ,
` Invalid value for input variable: Unsuitable value for var.invalid_nested_type set from outside of the configuration: incorrect map element type: attribute "rules": element "destination_addresses": object required. ` ,
} ,
2022-11-02 03:38:23 -05:00
2022-02-18 04:46:56 -06:00
// sensitive
{
"constrained_string_sensitive_required" ,
cty . UnknownVal ( cty . String ) ,
cty . UnknownVal ( cty . String ) ,
` ` ,
} ,
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
}
for _ , test := range tests {
t . Run ( fmt . Sprintf ( "%s %#v" , test . varName , test . given ) , func ( t * testing . T ) {
varAddr := addrs . InputVariable { Name : test . varName } . Absolute ( addrs . RootModuleInstance )
varCfg := variableConfigs [ test . varName ]
if varCfg == nil {
t . Fatalf ( "invalid variable name %q" , test . varName )
}
t . Logf (
"test case\nvariable: %s\nconstraint: %#v\ndefault: %#v\nnullable: %#v\ngiven value: %#v" ,
varAddr ,
varCfg . Type ,
varCfg . Default ,
varCfg . Nullable ,
test . given ,
)
2021-12-21 20:04:24 -06:00
rawVal := & InputValue {
Value : test . given ,
SourceType : ValueFromCaller ,
}
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
got , diags := prepareFinalInputVariableValue (
2021-12-21 20:04:24 -06:00
varAddr , rawVal , varCfg ,
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
)
if test . wantErr != "" {
if ! diags . HasErrors ( ) {
t . Errorf ( "unexpected success\nwant error: %s" , test . wantErr )
} else if got , want := diags . Err ( ) . Error ( ) , test . wantErr ; got != want {
t . Errorf ( "wrong error\ngot: %s\nwant: %s" , got , want )
}
} else {
if diags . HasErrors ( ) {
t . Errorf ( "unexpected error\ngot: %s" , diags . Err ( ) . Error ( ) )
}
}
// NOTE: should still have returned some reasonable value even if there was an error
if ! test . want . RawEquals ( got ) {
t . Fatalf ( "wrong result\ngot: %#v\nwant: %#v" , got , test . want )
}
} )
}
2021-12-21 20:04:24 -06:00
t . Run ( "SourceType error message variants" , func ( t * testing . T ) {
tests := [ ] struct {
SourceType ValueSourceType
SourceRange tfdiags . SourceRange
WantTypeErr string
WantNullErr string
} {
{
ValueFromUnknown ,
tfdiags . SourceRange { } ,
` Invalid value for input variable: Unsuitable value for var.constrained_string_required set from outside of the configuration: string required. ` ,
` Required variable not set: Unsuitable value for var.constrained_string_required set from outside of the configuration: required variable may not be set to null. ` ,
} ,
{
ValueFromConfig ,
tfdiags . SourceRange {
Filename : "example.tf" ,
Start : tfdiags . SourcePos ( hcl . InitialPos ) ,
End : tfdiags . SourcePos ( hcl . InitialPos ) ,
} ,
` Invalid value for input variable: The given value is not suitable for var.constrained_string_required declared at main.tf:32,3-41: string required. ` ,
` Required variable not set: The given value is not suitable for var.constrained_string_required defined at main.tf:32,3-41: required variable may not be set to null. ` ,
} ,
{
ValueFromAutoFile ,
tfdiags . SourceRange {
Filename : "example.auto.tfvars" ,
Start : tfdiags . SourcePos ( hcl . InitialPos ) ,
End : tfdiags . SourcePos ( hcl . InitialPos ) ,
} ,
` Invalid value for input variable: The given value is not suitable for var.constrained_string_required declared at main.tf:32,3-41: string required. ` ,
` Required variable not set: The given value is not suitable for var.constrained_string_required defined at main.tf:32,3-41: required variable may not be set to null. ` ,
} ,
{
ValueFromNamedFile ,
tfdiags . SourceRange {
Filename : "example.tfvars" ,
Start : tfdiags . SourcePos ( hcl . InitialPos ) ,
End : tfdiags . SourcePos ( hcl . InitialPos ) ,
} ,
` Invalid value for input variable: The given value is not suitable for var.constrained_string_required declared at main.tf:32,3-41: string required. ` ,
` Required variable not set: The given value is not suitable for var.constrained_string_required defined at main.tf:32,3-41: required variable may not be set to null. ` ,
} ,
{
ValueFromCLIArg ,
tfdiags . SourceRange { } ,
` Invalid value for input variable: Unsuitable value for var.constrained_string_required set using -var="constrained_string_required=...": string required. ` ,
` Required variable not set: Unsuitable value for var.constrained_string_required set using -var="constrained_string_required=...": required variable may not be set to null. ` ,
} ,
{
ValueFromEnvVar ,
tfdiags . SourceRange { } ,
` Invalid value for input variable: Unsuitable value for var.constrained_string_required set using the TF_VAR_constrained_string_required environment variable: string required. ` ,
` Required variable not set: Unsuitable value for var.constrained_string_required set using the TF_VAR_constrained_string_required environment variable: required variable may not be set to null. ` ,
} ,
{
ValueFromInput ,
tfdiags . SourceRange { } ,
` Invalid value for input variable: Unsuitable value for var.constrained_string_required set using an interactive prompt: string required. ` ,
` Required variable not set: Unsuitable value for var.constrained_string_required set using an interactive prompt: required variable may not be set to null. ` ,
} ,
{
// NOTE: This isn't actually a realistic case for this particular
// function, because if we have a value coming from a plan then
// we must be in the apply step, and we shouldn't be able to
// get past the plan step if we have invalid variable values,
// and during planning we'll always have other source types.
ValueFromPlan ,
tfdiags . SourceRange { } ,
` Invalid value for input variable: Unsuitable value for var.constrained_string_required set from outside of the configuration: string required. ` ,
` Required variable not set: Unsuitable value for var.constrained_string_required set from outside of the configuration: required variable may not be set to null. ` ,
} ,
{
ValueFromCaller ,
tfdiags . SourceRange { } ,
` Invalid value for input variable: Unsuitable value for var.constrained_string_required set from outside of the configuration: string required. ` ,
` Required variable not set: Unsuitable value for var.constrained_string_required set from outside of the configuration: required variable may not be set to null. ` ,
} ,
}
for _ , test := range tests {
t . Run ( fmt . Sprintf ( "%s %s" , test . SourceType , test . SourceRange . StartString ( ) ) , func ( t * testing . T ) {
varAddr := addrs . InputVariable { Name : "constrained_string_required" } . Absolute ( addrs . RootModuleInstance )
varCfg := variableConfigs [ varAddr . Variable . Name ]
t . Run ( "type error" , func ( t * testing . T ) {
rawVal := & InputValue {
Value : cty . EmptyObjectVal ,
SourceType : test . SourceType ,
SourceRange : test . SourceRange ,
}
_ , diags := prepareFinalInputVariableValue (
varAddr , rawVal , varCfg ,
)
if ! diags . HasErrors ( ) {
t . Fatalf ( "unexpected success; want error" )
}
if got , want := diags . Err ( ) . Error ( ) , test . WantTypeErr ; got != want {
t . Errorf ( "wrong error\ngot: %s\nwant: %s" , got , want )
}
} )
t . Run ( "null error" , func ( t * testing . T ) {
rawVal := & InputValue {
Value : cty . NullVal ( cty . DynamicPseudoType ) ,
SourceType : test . SourceType ,
SourceRange : test . SourceRange ,
}
_ , diags := prepareFinalInputVariableValue (
varAddr , rawVal , varCfg ,
)
if ! diags . HasErrors ( ) {
t . Fatalf ( "unexpected success; want error" )
}
if got , want := diags . Err ( ) . Error ( ) , test . WantNullErr ; got != want {
t . Errorf ( "wrong error\ngot: %s\nwant: %s" , got , want )
}
} )
} )
}
} )
2022-02-18 04:46:56 -06:00
t . Run ( "SensitiveVariable error message variants, with source variants" , func ( t * testing . T ) {
tests := [ ] struct {
SourceType ValueSourceType
SourceRange tfdiags . SourceRange
WantTypeErr string
HideSubject bool
} {
{
ValueFromUnknown ,
tfdiags . SourceRange { } ,
"Invalid value for input variable: Unsuitable value for var.constrained_string_sensitive_required set from outside of the configuration: string required." ,
false ,
} ,
{
ValueFromConfig ,
tfdiags . SourceRange {
2022-05-13 16:09:38 -05:00
Filename : "example.tfvars" ,
2022-02-18 04:46:56 -06:00
Start : tfdiags . SourcePos ( hcl . InitialPos ) ,
End : tfdiags . SourcePos ( hcl . InitialPos ) ,
} ,
2022-05-13 16:09:38 -05:00
` Invalid value for input variable: The given value is not suitable for var.constrained_string_sensitive_required, which is sensitive: string required. Invalid value defined at example.tfvars:1,1-1. ` ,
2022-02-18 04:46:56 -06:00
true ,
} ,
}
for _ , test := range tests {
t . Run ( fmt . Sprintf ( "%s %s" , test . SourceType , test . SourceRange . StartString ( ) ) , func ( t * testing . T ) {
varAddr := addrs . InputVariable { Name : "constrained_string_sensitive_required" } . Absolute ( addrs . RootModuleInstance )
varCfg := variableConfigs [ varAddr . Variable . Name ]
t . Run ( "type error" , func ( t * testing . T ) {
rawVal := & InputValue {
Value : cty . EmptyObjectVal ,
SourceType : test . SourceType ,
SourceRange : test . SourceRange ,
}
_ , diags := prepareFinalInputVariableValue (
varAddr , rawVal , varCfg ,
)
if ! diags . HasErrors ( ) {
t . Fatalf ( "unexpected success; want error" )
}
if got , want := diags . Err ( ) . Error ( ) , test . WantTypeErr ; got != want {
t . Errorf ( "wrong error\ngot: %s\nwant: %s" , got , want )
}
2022-05-13 16:09:38 -05:00
if test . HideSubject {
if got , want := diags [ 0 ] . Source ( ) . Subject . StartString ( ) , test . SourceRange . StartString ( ) ; got == want {
t . Errorf ( "Subject start should have been hidden, but was %s" , got )
}
2022-02-18 04:46:56 -06:00
}
} )
} )
}
} )
core: Handle root and child module input variables consistently
Previously we had a significant discrepancy between these two situations:
we wrote the raw root module variables directly into the EvalContext and
then applied type conversions only at expression evaluation time, while
for child modules we converted and validated the values while visiting
the variable graph node and wrote only the _final_ value into the
EvalContext.
This confusion seems to have been the root cause for #29899, where
validation rules for root module variables were being applied at the wrong
point in the process, prior to type conversion.
To fix that bug and also make similar mistakes less likely in the future,
I've made the root module variable handling more like the child module
variable handling in the following ways:
- The "raw value" (exactly as given by the user) lives only in the graph
node representing the variable, which mirrors how the _expression_
for a child module variable lives in its graph node. This means that
the flow for the two is the same except that there's no expression
evaluation step for root module variables, because they arrive as
constant values from the caller.
- The set of variable values in the EvalContext is always only "final"
values, after type conversion is complete. That in turn means we no
longer need to do "just in time" conversion in
evaluationStateData.GetInputVariable, and can just return the value
exactly as stored, which is consistent with how we handle all other
references between objects.
This diff is noisier than I'd like because of how much it takes to wire
a new argument (the raw variable values) through to the plan graph builder,
but those changes are pretty mechanical and the interesting logic lives
inside the plan graph builder itself, in NodeRootVariable, and
the shared helper functions in eval_variable.go.
While here I also took the opportunity to fix a historical API wart in
EvalContext, where SetModuleCallArguments was built to take a set of
variable values all at once but our current caller always calls with only
one at a time. That is now just SetModuleCallArgument singular, to match
with the new SetRootModuleArgument to deal with root module variables.
2021-11-10 19:29:45 -06:00
}
2022-02-07 13:46:31 -06:00
// These tests cover the JSON syntax configuration edge case handling,
// the background of which is described in detail in comments in the
2023-09-26 12:09:27 -05:00
// evalVariableValidations function. Future versions of OpenTofu may
2022-02-07 13:46:31 -06:00
// be able to remove this behaviour altogether.
func TestEvalVariableValidations_jsonErrorMessageEdgeCase ( t * testing . T ) {
cfgSrc := ` {
"variable" : {
"valid" : {
"type" : "string" ,
"validation" : {
"condition" : "${var.valid != \"bar\"}" ,
"error_message" : "Valid template string ${var.valid}"
}
} ,
"invalid" : {
"type" : "string" ,
"validation" : {
"condition" : "${var.invalid != \"bar\"}" ,
"error_message" : "Invalid template string ${"
}
}
}
}
`
cfg := testModuleInline ( t , map [ string ] string {
"main.tf.json" : cfgSrc ,
} )
variableConfigs := cfg . Module . Variables
// Because we loaded our pseudo-module from a temporary file, the
// declaration source ranges will have unpredictable filenames. We'll
// fix that here just to make things easier below.
for _ , vc := range variableConfigs {
vc . DeclRange . Filename = "main.tf.json"
for _ , v := range vc . Validations {
v . DeclRange . Filename = "main.tf.json"
}
}
tests := [ ] struct {
varName string
given cty . Value
wantErr [ ] string
wantWarn [ ] string
2023-07-10 05:33:45 -05:00
status checks . Status
2022-02-07 13:46:31 -06:00
} {
// Valid variable validation declaration, assigned value which passes
// the condition generates no diagnostics.
{
varName : "valid" ,
given : cty . StringVal ( "foo" ) ,
2023-07-10 05:33:45 -05:00
status : checks . StatusPass ,
2022-02-07 13:46:31 -06:00
} ,
// Assigning a value which fails the condition generates an error
// message with the expression successfully evaluated.
{
varName : "valid" ,
given : cty . StringVal ( "bar" ) ,
wantErr : [ ] string {
"Invalid value for variable" ,
"Valid template string bar" ,
} ,
2023-07-10 05:33:45 -05:00
status : checks . StatusFail ,
2022-02-07 13:46:31 -06:00
} ,
2024-08-29 12:20:33 -05:00
// Invalid variable validation declaration due to an unparsable
2022-02-07 13:46:31 -06:00
// template string. Assigning a value which passes the condition
// results in a warning about the error message.
{
varName : "invalid" ,
given : cty . StringVal ( "foo" ) ,
wantWarn : [ ] string {
"Validation error message expression is invalid" ,
"Missing expression; Expected the start of an expression, but found the end of the file." ,
} ,
2023-07-10 05:33:45 -05:00
status : checks . StatusPass ,
2022-02-07 13:46:31 -06:00
} ,
// Assigning a value which fails the condition generates an error
// message including the configured string interpreted as a literal
// value, and the same warning diagnostic as above.
{
varName : "invalid" ,
given : cty . StringVal ( "bar" ) ,
wantErr : [ ] string {
"Invalid value for variable" ,
"Invalid template string ${" ,
} ,
wantWarn : [ ] string {
"Validation error message expression is invalid" ,
"Missing expression; Expected the start of an expression, but found the end of the file." ,
} ,
2023-07-10 05:33:45 -05:00
status : checks . StatusFail ,
2022-02-07 13:46:31 -06:00
} ,
}
for _ , test := range tests {
t . Run ( fmt . Sprintf ( "%s %#v" , test . varName , test . given ) , func ( t * testing . T ) {
varAddr := addrs . InputVariable { Name : test . varName } . Absolute ( addrs . RootModuleInstance )
varCfg := variableConfigs [ test . varName ]
if varCfg == nil {
t . Fatalf ( "invalid variable name %q" , test . varName )
}
// Build a mock context to allow the function under test to
// retrieve the variable value and evaluate the expressions
ctx := & MockEvalContext { }
// We need a minimal scope to allow basic functions to be passed to
// the HCL scope
2024-12-04 13:24:44 -06:00
ctx . EvaluationScopeScope = & lang . Scope {
Data : & evaluationStateData { Evaluator : & Evaluator {
Config : cfg ,
VariableValuesLock : & sync . Mutex { } ,
VariableValues : map [ string ] map [ string ] cty . Value { "" : {
test . varName : cty . UnknownVal ( cty . String ) ,
} } ,
} } ,
}
2022-02-07 13:46:31 -06:00
ctx . GetVariableValueFunc = func ( addr addrs . AbsInputVariableInstance ) cty . Value {
if got , want := addr . String ( ) , varAddr . String ( ) ; got != want {
t . Errorf ( "incorrect argument to GetVariableValue: got %s, want %s" , got , want )
}
return test . given
}
2023-07-10 05:33:45 -05:00
ctx . ChecksState = checks . NewState ( cfg )
ctx . ChecksState . ReportCheckableObjects ( varAddr . ConfigCheckable ( ) , addrs . MakeSet [ addrs . Checkable ] ( varAddr ) )
2022-02-07 13:46:31 -06:00
gotDiags := evalVariableValidations (
varAddr , varCfg , nil , ctx ,
)
2023-07-10 05:33:45 -05:00
if ctx . ChecksState . ObjectCheckStatus ( varAddr ) != test . status {
t . Errorf ( "expected check result %s but instead %s" , test . status , ctx . ChecksState . ObjectCheckStatus ( varAddr ) )
}
2022-02-07 13:46:31 -06:00
if len ( test . wantErr ) == 0 && len ( test . wantWarn ) == 0 {
if len ( gotDiags ) > 0 {
t . Errorf ( "no diags expected, got %s" , gotDiags . Err ( ) . Error ( ) )
}
} else {
wantErrs :
for _ , want := range test . wantErr {
for _ , diag := range gotDiags {
if diag . Severity ( ) != tfdiags . Error {
continue
}
desc := diag . Description ( )
if strings . Contains ( desc . Summary , want ) || strings . Contains ( desc . Detail , want ) {
continue wantErrs
}
}
t . Errorf ( "no error diagnostics found containing %q\ngot: %s" , want , gotDiags . Err ( ) . Error ( ) )
}
wantWarns :
for _ , want := range test . wantWarn {
for _ , diag := range gotDiags {
if diag . Severity ( ) != tfdiags . Warning {
continue
}
desc := diag . Description ( )
if strings . Contains ( desc . Summary , want ) || strings . Contains ( desc . Detail , want ) {
continue wantWarns
}
}
t . Errorf ( "no warning diagnostics found containing %q\ngot: %s" , want , gotDiags . Err ( ) . Error ( ) )
}
}
} )
}
}
2022-03-11 10:11:30 -06:00
func TestEvalVariableValidations_sensitiveValues ( t * testing . T ) {
cfgSrc := `
variable "foo" {
type = string
sensitive = true
default = "boop"
validation {
condition = length ( var . foo ) == 4
error_message = "Foo must be 4 characters, not ${length(var.foo)}"
}
}
variable "bar" {
type = string
sensitive = true
default = "boop"
validation {
condition = length ( var . bar ) == 4
error_message = "Bar must be 4 characters, not ${nonsensitive(length(var.bar))}."
}
}
`
cfg := testModuleInline ( t , map [ string ] string {
"main.tf" : cfgSrc ,
} )
variableConfigs := cfg . Module . Variables
// Because we loaded our pseudo-module from a temporary file, the
// declaration source ranges will have unpredictable filenames. We'll
// fix that here just to make things easier below.
for _ , vc := range variableConfigs {
vc . DeclRange . Filename = "main.tf"
for _ , v := range vc . Validations {
v . DeclRange . Filename = "main.tf"
}
}
tests := [ ] struct {
varName string
given cty . Value
wantErr [ ] string
2023-07-10 05:33:45 -05:00
status checks . Status
2022-03-11 10:11:30 -06:00
} {
// Validations pass on a sensitive variable with an error message which
// would generate a sensitive value
{
varName : "foo" ,
given : cty . StringVal ( "boop" ) ,
2023-07-10 05:33:45 -05:00
status : checks . StatusPass ,
2022-03-11 10:11:30 -06:00
} ,
// Assigning a value which fails the condition generates a sensitive
// error message, which is elided and generates another error
{
varName : "foo" ,
given : cty . StringVal ( "bap" ) ,
wantErr : [ ] string {
"Invalid value for variable" ,
"The error message included a sensitive value, so it will not be displayed." ,
"Error message refers to sensitive values" ,
} ,
2023-07-10 05:33:45 -05:00
status : checks . StatusFail ,
2022-03-11 10:11:30 -06:00
} ,
// Validations pass on a sensitive variable with a correctly defined
// error message
{
varName : "bar" ,
given : cty . StringVal ( "boop" ) ,
2023-07-10 05:33:45 -05:00
status : checks . StatusPass ,
2022-03-11 10:11:30 -06:00
} ,
// Assigning a value which fails the condition generates a nonsensitive
// error message, which is displayed
{
varName : "bar" ,
given : cty . StringVal ( "bap" ) ,
wantErr : [ ] string {
"Invalid value for variable" ,
"Bar must be 4 characters, not 3." ,
} ,
2023-07-10 05:33:45 -05:00
status : checks . StatusFail ,
2022-03-11 10:11:30 -06:00
} ,
}
for _ , test := range tests {
t . Run ( fmt . Sprintf ( "%s %#v" , test . varName , test . given ) , func ( t * testing . T ) {
varAddr := addrs . InputVariable { Name : test . varName } . Absolute ( addrs . RootModuleInstance )
varCfg := variableConfigs [ test . varName ]
if varCfg == nil {
t . Fatalf ( "invalid variable name %q" , test . varName )
}
// Build a mock context to allow the function under test to
// retrieve the variable value and evaluate the expressions
ctx := & MockEvalContext { }
// We need a minimal scope to allow basic functions to be passed to
// the HCL scope
2024-12-04 13:24:44 -06:00
ctx . EvaluationScopeScope = & lang . Scope {
Data : & evaluationStateData { Evaluator : & Evaluator {
Config : cfg ,
VariableValuesLock : & sync . Mutex { } ,
VariableValues : map [ string ] map [ string ] cty . Value { "" : {
test . varName : cty . UnknownVal ( cty . String ) ,
} } ,
} } ,
}
2022-03-11 10:11:30 -06:00
ctx . GetVariableValueFunc = func ( addr addrs . AbsInputVariableInstance ) cty . Value {
if got , want := addr . String ( ) , varAddr . String ( ) ; got != want {
t . Errorf ( "incorrect argument to GetVariableValue: got %s, want %s" , got , want )
}
2024-12-03 11:40:44 -06:00
// NOTE: This intentionally doesn't mark the result as sensitive,
// because BuiltinEvalContext.GetVariableValue doesn't either.
// It's the responsibility of downstream code to detect and handle
// configured sensitivity.
return test . given
2022-03-11 10:11:30 -06:00
}
2023-07-10 05:33:45 -05:00
ctx . ChecksState = checks . NewState ( cfg )
ctx . ChecksState . ReportCheckableObjects ( varAddr . ConfigCheckable ( ) , addrs . MakeSet [ addrs . Checkable ] ( varAddr ) )
2022-03-11 10:11:30 -06:00
gotDiags := evalVariableValidations (
varAddr , varCfg , nil , ctx ,
)
2023-07-10 05:33:45 -05:00
if ctx . ChecksState . ObjectCheckStatus ( varAddr ) != test . status {
t . Errorf ( "expected check result %s but instead %s" , test . status , ctx . ChecksState . ObjectCheckStatus ( varAddr ) )
}
2022-03-11 10:11:30 -06:00
if len ( test . wantErr ) == 0 {
if len ( gotDiags ) > 0 {
t . Errorf ( "no diags expected, got %s" , gotDiags . Err ( ) . Error ( ) )
}
} else {
wantErrs :
for _ , want := range test . wantErr {
for _ , diag := range gotDiags {
if diag . Severity ( ) != tfdiags . Error {
continue
}
desc := diag . Description ( )
if strings . Contains ( desc . Summary , want ) || strings . Contains ( desc . Detail , want ) {
continue wantErrs
}
}
t . Errorf ( "no error diagnostics found containing %q\ngot: %s" , want , gotDiags . Err ( ) . Error ( ) )
}
}
} )
}
}
2024-12-03 11:40:44 -06:00
func TestEvalVariableValidations_sensitiveValueDiagnostics ( t * testing . T ) {
// This test verifies that values for sensitive variables get captured
// into diagnostic messages with the sensitive mark intact, so that
// the values won't be disclosed in the UI.
// Earlier versions handled this incorrectly:
// https://github.com/opentofu/opentofu/issues/2219
cfgSrc := `
variable "foo" {
type = string
sensitive = true
validation {
condition = length ( var . foo ) == 8 # intentionally fails
error_message = "Foo must have 8 characters."
}
}
`
cfg := testModuleInline ( t , map [ string ] string {
"main.tf" : cfgSrc ,
} )
varAddr := addrs . InputVariable { Name : "foo" } . Absolute ( addrs . RootModuleInstance )
ctx := & MockEvalContext { }
2024-12-04 13:24:44 -06:00
ctx . EvaluationScopeScope = & lang . Scope {
Data : & evaluationStateData { Evaluator : & Evaluator {
Config : cfg ,
VariableValuesLock : & sync . Mutex { } ,
VariableValues : map [ string ] map [ string ] cty . Value { "" : {
varAddr . Variable . Name : cty . UnknownVal ( cty . String ) ,
} } ,
} } ,
}
2024-12-03 11:40:44 -06:00
ctx . GetVariableValueFunc = func ( addr addrs . AbsInputVariableInstance ) cty . Value {
if got , want := addr . String ( ) , varAddr . String ( ) ; got != want {
t . Errorf ( "incorrect argument to GetVariableValue: got %s, want %s" , got , want )
}
// NOTE: This intentionally doesn't mark the result as sensitive,
// because BuiltinEvalContext.GetVariableValueFunc doesn't either.
// It's the responsibility of downstream code to detect and handle
// configured sensitivity.
return cty . StringVal ( "boop" )
}
ctx . ChecksState = checks . NewState ( cfg )
ctx . ChecksState . ReportCheckableObjects ( varAddr . ConfigCheckable ( ) , addrs . MakeSet [ addrs . Checkable ] ( varAddr ) )
gotDiags := evalVariableValidations (
varAddr , cfg . Module . Variables [ "foo" ] , nil , ctx ,
)
if ! gotDiags . HasErrors ( ) {
t . Fatalf ( "unexpected success; want validation error" )
}
// The generated diagnostic(s) should all capture the evaluation context
// that was used to evaluate the condition, where the variable's value
// should be marked as sensitive so it won't get displayed in clear
// in the UI output.
// (The HasErrors check above guarantees that there's at least one
// diagnostic here for us to iterate over.)
for _ , diag := range gotDiags {
if diag . Severity ( ) != tfdiags . Error {
continue
}
fromExpr := diag . FromExpr ( )
if fromExpr == nil {
t . Fatalf ( "diagnostic does not have source expression information at all" )
}
allVarVals := fromExpr . EvalContext . Variables [ "var" ]
if allVarVals == cty . NilVal || ! allVarVals . Type ( ) . IsObjectType ( ) {
t . Fatalf ( "diagnostic did not capture an object value for the top-level symbol 'var'" )
}
gotVal := allVarVals . GetAttr ( "foo" )
if gotVal == cty . NilVal {
t . Fatalf ( "diagnostic did not capture a value for var.foo" )
}
if ! gotVal . HasMark ( marks . Sensitive ) {
t . Errorf ( "var.foo value is not marked as sensitive in diagnostic" )
}
}
}
