add override implementation for testing framework (#1499)

Signed-off-by: ollevche <ollevche@gmail.com>
Signed-off-by: Oleksandr Levchenkov <ollevche@gmail.com>
Co-authored-by: Janos <86970079+janosdebugs@users.noreply.github.com>
Co-authored-by: Ronny Orot <ronny.orot@gmail.com>
This commit is contained in:
Oleksandr Levchenkov 2024-06-06 13:20:41 +03:00 committed by GitHub
parent bfe5a4cd13
commit 64fb36dc54
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
38 changed files with 2437 additions and 36 deletions

View File

@ -3,20 +3,21 @@
UPGRADE NOTES:
NEW FEATURES:
* Added support for `override_resource`, `override_data` and `override_module` blocks in testing framework. ([1499](https://github.com/opentofu/opentofu/pull/1499))
ENHANCEMENTS:
* Added `tofu test -json` types to website Machine-Readable UI documentation ([1408](https://github.com/opentofu/opentofu/issues/1408))
* Added `tofu test -json` types to website Machine-Readable UI documentation. ([1408](https://github.com/opentofu/opentofu/issues/1408))
* Made `tofu plan` with `generate-config-out` flag replace JSON strings with `jsonencode` functions calls. ([#1595](https://github.com/opentofu/opentofu/pull/1595))
* Make state persistence interval configurable via `TF_STATE_PERSIST_INTERVAL` environment variable ([#1591](https://github.com/opentofu/opentofu/pull/1591))
* Improved performance of writing state files and reduced their size using compact json encoding. ([#1647](https://github.com/opentofu/opentofu/pull/1647))
* Allow to reference variable inside the `variables` block of a test file. ([1488](https://github.com/opentofu/opentofu/pull/1488))
BUG FIXES:
* Fixed crash in gcs backend when using certain commands ([#1618](https://github.com/opentofu/opentofu/pull/1618))
* Fix inmem backend crash due to missing struct field ([#1619](https://github.com/opentofu/opentofu/pull/1619))
* Fixed crash in gcs backend when using certain commands. ([#1618](https://github.com/opentofu/opentofu/pull/1618))
* Fixed inmem backend crash due to missing struct field. ([#1619](https://github.com/opentofu/opentofu/pull/1619))
* Added a check in the `tofu test` to validate that the names of test run blocks do not contain spaces. ([#1489](https://github.com/opentofu/opentofu/pull/1489))
* `tofu test` now supports accessing module outputs when the module has no resources. ([#1409](https://github.com/opentofu/opentofu/pull/1409))
* Fixed support for provider functions in tests ([#1603](https://github.com/opentofu/opentofu/pull/1603))
* Fixed support for provider functions in tests. ([#1603](https://github.com/opentofu/opentofu/pull/1603))
* Added a better error message on `for_each` block with sensitive value of unsuitable type. ([#1485](https://github.com/opentofu/opentofu/pull/1485))
* Fix race condition on locking in gcs backend ([#1342](https://github.com/opentofu/opentofu/pull/1342))
* Fix bug where provider functions were unusable in variables and outputs ([#1689](https://github.com/opentofu/opentofu/pull/1689))

View File

@ -177,6 +177,25 @@ func (m Module) configRemovableSigil() {
// Empty function so Module will fulfill the requirements of the removable interface
}
// ParseModule parses a module address from the given traversal,
// which has to contain only the module address with no resource/data/variable/etc.
// This function only supports module addresses without instance keys (as the
// returned Module struct doesn't support instance keys) and will return an
// error if it encounters one.
func ParseModule(traversal hcl.Traversal) (Module, tfdiags.Diagnostics) {
mod, remain, diags := parseModulePrefix(traversal)
if !diags.HasErrors() && len(remain) != 0 {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Module address expected",
Detail: "It's not allowed to reference anything other than module here.",
Subject: remain[0].SourceRange().Ptr(),
})
}
return mod, diags
}
// parseModulePrefix parses a module address from the given traversal,
// returning the module address and the remaining traversal.
// For example, if the input traversal is ["module","a","module","b",

View File

@ -8,6 +8,10 @@ package addrs
import (
"fmt"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/hclsyntax"
)
func TestModuleEqual_true(t *testing.T) {
@ -99,3 +103,76 @@ func BenchmarkModuleStringLong(b *testing.B) {
module.String()
}
}
func TestParseModule(t *testing.T) {
t.Parallel()
tests := []struct {
Input string
WantModule Module
WantErr string
}{
{
Input: "module.a",
WantModule: []string{"a"},
},
{
Input: "module.a.module.b",
WantModule: []string{"a", "b"},
},
{
Input: "module.a.module.b.c.d",
WantErr: "Module address expected: It's not allowed to reference anything other than module here.",
},
{
Input: "a.b.c.d",
WantErr: "Module address expected: It's not allowed to reference anything other than module here.",
},
{
Input: "module",
WantErr: `Invalid address operator: Prefix "module." must be followed by a module name.`,
},
{
Input: "module.a[0]",
WantErr: `Module instance address with keys is not allowed: Module address cannot be a module instance (e.g. "module.a[0]"), it must be a module instead (e.g. "module.a").`,
},
{
Input: `module.a["k"]`,
WantErr: `Module instance address with keys is not allowed: Module address cannot be a module instance (e.g. "module.a[0]"), it must be a module instead (e.g. "module.a").`,
},
}
for _, test := range tests {
test := test
t.Run(test.Input, func(t *testing.T) {
t.Parallel()
traversal, hclDiags := hclsyntax.ParseTraversalAbs([]byte(test.Input), "", hcl.InitialPos)
if hclDiags.HasErrors() {
t.Fatalf("Bug in tests: %s", hclDiags.Error())
}
mod, diags := ParseModule(traversal)
switch {
case test.WantErr != "":
if !diags.HasErrors() {
t.Fatalf("Unexpected success, wanted error: %s", test.WantErr)
}
gotErr := diags.Err().Error()
if gotErr != test.WantErr {
t.Fatalf("Mismatched error\nGot: %s\nWant: %s", gotErr, test.WantErr)
}
default:
if diags.HasErrors() {
t.Fatalf("Unexpected error: %s", diags.Err().Error())
}
if diff := cmp.Diff(test.WantModule, mod); diff != "" {
t.Fatalf("Mismatched result:\n%s", diff)
}
}
})
}
}

View File

@ -68,9 +68,8 @@ func ParseTarget(traversal hcl.Traversal) (*Target, tfdiags.Diagnostics) {
}
func parseResourceInstanceUnderModule(moduleAddr ModuleInstance, remain hcl.Traversal) (AbsResourceInstance, tfdiags.Diagnostics) {
// Note that this helper is used as part of both ParseTarget and
// ParseMoveEndpoint, so its error messages should be generic
// enough to suit both situations.
// Note that this helper is used as part of multiple public functions
// so its error messages should be generic enough to suit all the situations.
var diags tfdiags.Diagnostics

View File

@ -8,6 +8,9 @@ package addrs
import (
"fmt"
"strings"
"github.com/hashicorp/hcl/v2"
"github.com/opentofu/opentofu/internal/tfdiags"
)
// Resource is an address for a resource block within configuration, which
@ -391,6 +394,29 @@ type ConfigResource struct {
Resource Resource
}
// ParseConfigResource parses the module address from the given traversal
// and then parses the resource address from the leftover. Returning ConfigResource
// contains both module and resource addresses. ParseConfigResource doesn't support
// instance keys and will return an error if it encounters one.
func ParseConfigResource(traversal hcl.Traversal) (ConfigResource, tfdiags.Diagnostics) {
modulePath, remainTraversal, diags := parseModulePrefix(traversal)
if diags.HasErrors() {
return ConfigResource{}, diags
}
if len(remainTraversal) == 0 {
return ConfigResource{}, diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Module address is not allowed",
Detail: "Expected reference to either resource or data block. Provided reference appears to be a module.",
Subject: traversal.SourceRange().Ptr(),
})
}
configRes, moreDiags := parseResourceUnderModule(modulePath, remainTraversal)
return configRes, diags.Append(moreDiags)
}
// Resource returns the address of a particular resource within the module.
func (m Module) Resource(mode ResourceMode, typeName string, name string) ConfigResource {
return ConfigResource{

View File

@ -8,6 +8,10 @@ package addrs
import (
"fmt"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/hclsyntax"
)
func TestResourceEqual_true(t *testing.T) {
@ -350,3 +354,134 @@ func TestConfigResourceEqual_false(t *testing.T) {
})
}
}
func TestParseConfigResource(t *testing.T) {
t.Parallel()
tests := []struct {
Input string
WantConfigResource ConfigResource
WantErr string
}{
{
Input: "a.b",
WantConfigResource: ConfigResource{
Module: RootModule,
Resource: Resource{
Mode: ManagedResourceMode,
Type: "a",
Name: "b",
},
},
},
{
Input: "data.a.b",
WantConfigResource: ConfigResource{
Module: RootModule,
Resource: Resource{
Mode: DataResourceMode,
Type: "a",
Name: "b",
},
},
},
{
Input: "module.a.b.c",
WantConfigResource: ConfigResource{
Module: []string{"a"},
Resource: Resource{
Mode: ManagedResourceMode,
Type: "b",
Name: "c",
},
},
},
{
Input: "module.a.data.b.c",
WantConfigResource: ConfigResource{
Module: []string{"a"},
Resource: Resource{
Mode: DataResourceMode,
Type: "b",
Name: "c",
},
},
},
{
Input: "module.a.module.b.c.d",
WantConfigResource: ConfigResource{
Module: []string{"a", "b"},
Resource: Resource{
Mode: ManagedResourceMode,
Type: "c",
Name: "d",
},
},
},
{
Input: "module.a.module.b.data.c.d",
WantConfigResource: ConfigResource{
Module: []string{"a", "b"},
Resource: Resource{
Mode: DataResourceMode,
Type: "c",
Name: "d",
},
},
},
{
Input: "module.a.module.b",
WantErr: "Module address is not allowed: Expected reference to either resource or data block. Provided reference appears to be a module.",
},
{
Input: "module",
WantErr: `Invalid address operator: Prefix "module." must be followed by a module name.`,
},
{
Input: "module.a.module.b.c",
WantErr: "Invalid address: Resource specification must include a resource type and name.",
},
{
Input: "module.a.module.b.c.d[0]",
WantErr: `Resource instance address with keys is not allowed: Resource address cannot be a resource instance (e.g. "null_resource.a[0]"), it must be a resource instead (e.g. "null_resource.a").`,
},
{
Input: "module.a.module.b.data.c.d[0]",
WantErr: `Resource instance address with keys is not allowed: Resource address cannot be a resource instance (e.g. "null_resource.a[0]"), it must be a resource instead (e.g. "null_resource.a").`,
},
}
for _, test := range tests {
test := test
t.Run(test.Input, func(t *testing.T) {
t.Parallel()
traversal, hclDiags := hclsyntax.ParseTraversalAbs([]byte(test.Input), "", hcl.InitialPos)
if hclDiags.HasErrors() {
t.Fatalf("Bug in tests: %s", hclDiags.Error())
}
configRes, diags := ParseConfigResource(traversal)
switch {
case test.WantErr != "":
if !diags.HasErrors() {
t.Fatalf("Unexpected success, wanted error: %s", test.WantErr)
}
gotErr := diags.Err().Error()
if gotErr != test.WantErr {
t.Fatalf("Mismatched error\nGot: %s\nWant: %s", gotErr, test.WantErr)
}
default:
if diags.HasErrors() {
t.Fatalf("Unexpected error: %s", diags.Err().Error())
}
if diff := cmp.Diff(test.WantConfigResource, configRes); diff != "" {
t.Fatalf("Mismatched result:\n%s", diff)
}
}
})
}
}

View File

@ -51,3 +51,32 @@ func TestMultipleRunBlocks(t *testing.T) {
}
}
}
func TestOverrides(t *testing.T) {
// This test fetches "local" and "random" providers.
skipIfCannotAccessNetwork(t)
tf := e2e.NewBinary(t, tofuBin, filepath.Join("testdata", "overrides-in-tests"))
stdout, stderr, err := tf.Run("init")
if err != nil {
t.Errorf("unexpected error on 'init': %v", err)
}
if stderr != "" {
t.Errorf("unexpected stderr output on 'init':\n%s", stderr)
}
if stdout == "" {
t.Errorf("expected some output on 'init', got nothing")
}
stdout, stderr, err = tf.Run("test")
if err != nil {
t.Errorf("unexpected error on 'test': %v", err)
}
if stderr != "" {
t.Errorf("unexpected stderr output on 'test':\n%s", stderr)
}
if !strings.Contains(stdout, "11 passed, 0 failed") {
t.Errorf("output doesn't have expected success string:\n%s", stdout)
}
}

View File

@ -0,0 +1,13 @@
resource "local_file" "dont_create_me" {
filename = "${path.module}/dont_create_me.txt"
content = "101"
}
resource "local_file" "create_me" {
filename = "${path.module}/create_me.txt"
content = "101"
}
output "create_me_filename" {
value = "main.tf"
}

View File

@ -0,0 +1,59 @@
module "first" {
source = "./first"
}
module "second" {
source = "./second"
}
resource "local_file" "dont_create_me" {
filename = "${path.module}/dont_create_me.txt"
content = "101"
}
resource "local_file" "create_me" {
filename = "${path.module}/create_me.txt"
content = "101"
}
data "local_file" "second_mod_file" {
filename = module.first.create_me_filename
}
resource "random_integer" "count" {
count = 2
min = 1
max = 10
}
resource "random_integer" "for_each" {
for_each = {
"a": {
"min": 1
"max": 10
}
"b": {
"min": 20
"max": 30
}
}
min = each.value.min
max = each.value.max
}
module "rand_for_each" {
for_each = {
"a": 1
"b": 2
}
source = "./rand"
}
module "rand_count" {
count = 2
source = "./rand"
}

View File

@ -0,0 +1,223 @@
override_module {
target = module.second
}
override_resource {
target = local_file.dont_create_me
}
override_resource {
target = module.first.local_file.dont_create_me
}
run "check_root_overridden_res" {
assert {
condition = !fileexists("${path.module}/dont_create_me.txt")
error_message = "File 'dont_create_me.txt' must not be created in the root module"
}
}
run "check_root_overridden_res_twice" {
override_resource {
target = local_file.dont_create_me
values = {
file_permission = "0333"
}
}
assert {
condition = !fileexists("${path.module}/dont_create_me.txt") && local_file.dont_create_me.file_permission == "0333"
error_message = "File 'dont_create_me.txt' must not be created in the root module and its file_permission must be overridden"
}
}
run "check_root_data" {
assert {
condition = data.local_file.second_mod_file.content == file("main.tf")
error_message = "Content from the local_file data in the root module must be from real file"
}
}
run "check_root_overridden_data" {
override_data {
target = data.local_file.second_mod_file
values = {
content = "101"
}
}
assert {
condition = data.local_file.second_mod_file.content == "101"
error_message = "Content from the local_file data in the root module must be overridden"
}
}
run "check_overridden_module_output" {
override_module {
target = module.first
outputs = {
create_me_filename = "main.tftest.hcl"
}
}
assert {
condition = data.local_file.second_mod_file.content == file("main.tftest.hcl")
error_message = "Overridden module output is not used in the depending data resource"
}
}
run "check_first_module" {
assert {
condition = fileexists("${path.module}/first/create_me.txt")
error_message = "File 'create_me.txt' must be created in the first module"
}
}
run "check_first_module_overridden_res" {
assert {
condition = !fileexists("${path.module}/first/dont_create_me.txt")
error_message = "File 'dont_create_me.txt' must not be created in the first module"
}
}
run "check_second_module" {
assert {
condition = !fileexists("${path.module}/second/dont_create_me.txt")
error_message = "File 'dont_create_me.txt' must not be created in the second module"
}
}
run "check_third_module" {
assert {
condition = !fileexists("${path.module}/second/third/dont_create_me.txt")
error_message = "File 'dont_create_me.txt' must not be created in the third module"
}
}
override_resource {
target = random_integer.count
}
override_resource {
target = random_integer.for_each
}
override_module {
target = module.rand_count
}
override_module {
target = module.rand_for_each
}
run "check_for_each_n_count_mocked" {
assert {
condition = random_integer.count[0].result == 0
error_message = "Mocked random integer should be 0"
}
assert {
condition = random_integer.count[1].result == 0
error_message = "Mocked random integer should be 0"
}
assert {
condition = random_integer.for_each["a"].result == 0
error_message = "Mocked random integer should be 0"
}
assert {
condition = random_integer.for_each["b"].result == 0
error_message = "Mocked random integer should be 0"
}
assert {
condition = module.rand_count[0].random_integer == null
error_message = "Mocked random integer should be null"
}
assert {
condition = module.rand_count[1].random_integer == null
error_message = "Mocked random integer should be null"
}
assert {
condition = module.rand_for_each["a"].random_integer == null
error_message = "Mocked random integer should be null"
}
assert {
condition = module.rand_for_each["b"].random_integer == null
error_message = "Mocked random integer should be null"
}
}
run "check_for_each_n_count_overridden" {
override_resource {
target = random_integer.count
values = {
result = 101
}
}
assert {
condition = random_integer.count[0].result == 101
error_message = "Overridden random integer should be 101"
}
assert {
condition = random_integer.count[1].result == 101
error_message = "Overridden random integer should be 101"
}
override_resource {
target = random_integer.for_each
values = {
result = 101
}
}
assert {
condition = random_integer.for_each["a"].result == 101
error_message = "Overridden random integer should be 101"
}
assert {
condition = random_integer.for_each["b"].result == 101
error_message = "Overridden random integer should be 101"
}
override_module {
target = module.rand_count
outputs = {
random_integer = 101
}
}
assert {
condition = module.rand_count[0].random_integer == 101
error_message = "Mocked random integer should be 101"
}
assert {
condition = module.rand_count[1].random_integer == 101
error_message = "Mocked random integer should be 101"
}
override_module {
target = module.rand_for_each
outputs = {
random_integer = 101
}
}
assert {
condition = module.rand_for_each["a"].random_integer == 101
error_message = "Mocked random integer should be 101"
}
assert {
condition = module.rand_for_each["b"].random_integer == 101
error_message = "Mocked random integer should be 101"
}
}

View File

@ -0,0 +1,8 @@
resource "random_integer" "main" {
min = 1
max = 20
}
output "random_integer" {
value = random_integer.main.id
}

View File

@ -0,0 +1,8 @@
module "third" {
source = "./third"
}
resource "local_file" "dont_create_me" {
filename = "${path.module}/dont_create_me.txt"
content = "101"
}

View File

@ -0,0 +1,4 @@
resource "local_file" "dont_create_me" {
filename = "${path.module}/dont_create_me.txt"
content = "101"
}

View File

@ -479,6 +479,12 @@ func (runner *TestFileRunner) ExecuteTestRun(run *moduletest.Run, file *modulete
return state, false
}
run.Diagnostics = run.Diagnostics.Append(file.Config.Validate())
if run.Diagnostics.HasErrors() {
run.Status = moduletest.Error
return state, false
}
run.Diagnostics = run.Diagnostics.Append(run.Config.Validate())
if run.Diagnostics.HasErrors() {
run.Status = moduletest.Error

View File

@ -110,6 +110,9 @@ func (c *ValidateCommand) validate(dir, testDir string, noTests bool) tfdiags.Di
// We'll also do a quick validation of the OpenTofu test files. These live
// outside the OpenTofu graph so we have to do this separately.
for _, file := range cfg.Module.Tests {
diags = diags.Append(file.Validate())
for _, run := range file.Runs {
if run.Module != nil {

View File

@ -880,7 +880,37 @@ func (c *Config) CheckCoreVersionRequirements() hcl.Diagnostics {
func (c *Config) TransformForTest(run *TestRun, file *TestFile) (func(), hcl.Diagnostics) {
var diags hcl.Diagnostics
// Currently, we only need to override the provider settings.
// These transformation functions must be in sync of what is being transformed,
// currently all the functions operate on different fields of configuration.
transformFuncs := []func(*TestRun, *TestFile) (func(), hcl.Diagnostics){
c.transformProviderConfigsForTest,
c.transformOverriddenResourcesForTest,
c.transformOverriddenModulesForTest,
}
var resetFuncs []func()
// We call each function to transform the configuration
// and gather transformation diags as well as reset functions.
for _, f := range transformFuncs {
resetFunc, moreDiags := f(run, file)
diags = append(diags, moreDiags...)
resetFuncs = append(resetFuncs, resetFunc)
}
// Order of calls doesn't matter as long as transformation functions
// don't operate on the same set of fields.
return func() {
for _, f := range resetFuncs {
f()
}
}, diags
}
func (c *Config) transformProviderConfigsForTest(run *TestRun, file *TestFile) (func(), hcl.Diagnostics) {
var diags hcl.Diagnostics
// We need to override the provider settings.
//
// We can have a set of providers defined within the config, we can also
// have a set of providers defined within the test file. Then the run can
@ -955,8 +985,193 @@ func (c *Config) TransformForTest(run *TestRun, file *TestFile) (func(), hcl.Dia
}
c.Module.ProviderConfigs = next
return func() {
// Reset the original config within the returned function.
c.Module.ProviderConfigs = previous
}, diags
}
func (c *Config) transformOverriddenResourcesForTest(run *TestRun, file *TestFile) (func(), hcl.Diagnostics) {
resources, diags := mergeOverriddenResources(run.OverrideResources, file.OverrideResources)
// We want to pass override values to resources being overridden.
for _, overrideRes := range resources {
targetConfig := c.Root.Descendent(overrideRes.TargetParsed.Module)
if targetConfig == nil {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("Module not found: %v", overrideRes.TargetParsed.Module),
Detail: "Target points to resource in undefined module. Please, ensure module exists.",
Subject: overrideRes.Target.SourceRange().Ptr(),
})
continue
}
res := targetConfig.Module.ResourceByAddr(overrideRes.TargetParsed.Resource)
if res == nil {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("Resource not found: %v", overrideRes.TargetParsed),
Detail: "Target points to undefined resource. Please, ensure resource exists.",
Subject: overrideRes.Target.SourceRange().Ptr(),
})
continue
}
if res.Mode != overrideRes.Mode {
blockName, targetMode := blockNameOverrideResource, "data"
if overrideRes.Mode == addrs.DataResourceMode {
blockName, targetMode = blockNameOverrideData, "resource"
}
// It could be a warning, but for the sake of consistent UX let's make it an error
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("Unsupported `%v` target in `%v` block", targetMode, blockName),
Detail: fmt.Sprintf("Target `%v` is `%v` block itself and cannot be overridden with `%v`.",
overrideRes.TargetParsed, targetMode, blockName),
Subject: overrideRes.Target.SourceRange().Ptr(),
})
}
res.IsOverridden = true
res.OverrideValues = overrideRes.Values
}
return func() {
// Reset all the overridden resources.
for _, o := range run.OverrideResources {
m := c.Root.Descendent(o.TargetParsed.Module)
if m == nil {
continue
}
res := m.Module.ResourceByAddr(o.TargetParsed.Resource)
if res == nil {
continue
}
res.IsOverridden = false
res.OverrideValues = nil
}
}, diags
}
func (c *Config) transformOverriddenModulesForTest(run *TestRun, file *TestFile) (func(), hcl.Diagnostics) {
modules, diags := mergeOverriddenModules(run.OverrideModules, file.OverrideModules)
for _, overrideMod := range modules {
targetConfig := c.Root.Descendent(overrideMod.TargetParsed)
if targetConfig == nil {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("Module not found: %v", overrideMod.TargetParsed),
Detail: "Target points to an undefined module. Please, ensure module exists.",
Subject: overrideMod.Target.SourceRange().Ptr(),
})
continue
}
for overrideKey := range overrideMod.Outputs {
if _, ok := targetConfig.Module.Outputs[overrideKey]; !ok {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagWarning,
Summary: fmt.Sprintf("Output not found: %v", overrideKey),
Detail: "Specified output to override is not present in the module and will be ignored.",
Subject: overrideMod.Target.SourceRange().Ptr(),
})
}
}
targetConfig.Module.IsOverridden = true
for key, output := range targetConfig.Module.Outputs {
output.IsOverridden = true
// Override outputs are optional so it's okay to set IsOverridden with no OverrideValue.
if v, ok := overrideMod.Outputs[key]; ok {
output.OverrideValue = &v
}
}
}
return func() {
for _, overrideMod := range run.OverrideModules {
targetConfig := c.Root.Descendent(overrideMod.TargetParsed)
if targetConfig == nil {
continue
}
targetConfig.Module.IsOverridden = false
for _, output := range targetConfig.Module.Outputs {
output.IsOverridden = false
output.OverrideValue = nil
}
}
}, diags
}
func mergeOverriddenResources(runResources, fileResources []*OverrideResource) ([]*OverrideResource, hcl.Diagnostics) {
// resAddrsInRun is a unique set of resource addresses in run block.
// It's already validated for duplicates previously.
resAddrsInRun := make(map[string]struct{})
for _, r := range runResources {
resAddrsInRun[r.TargetParsed.String()] = struct{}{}
}
var diags hcl.Diagnostics
resources := runResources
for _, r := range fileResources {
addr := r.TargetParsed.String()
// Run and file override resources could have overlap
// so we warn user and proceed with the definition from the smaller scope.
if _, ok := resAddrsInRun[addr]; ok {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagWarning,
Summary: fmt.Sprintf("Multiple `%v` blocks for the same address", r.getBlockName()),
Detail: fmt.Sprintf("`%v` is overridden in both global file and local run blocks. The declaration in global file block will be ignored.", addr),
Subject: r.Target.SourceRange().Ptr(),
})
continue
}
resources = append(resources, r)
}
return resources, diags
}
func mergeOverriddenModules(runModules, fileModules []*OverrideModule) ([]*OverrideModule, hcl.Diagnostics) {
// modAddrsInRun is a unique set of module addresses in run block.
// It's already validated for duplicates previously.
modAddrsInRun := make(map[string]struct{})
for _, m := range runModules {
modAddrsInRun[m.TargetParsed.String()] = struct{}{}
}
var diags hcl.Diagnostics
modules := runModules
for _, m := range fileModules {
addr := m.TargetParsed.String()
// Run and file override modules could have overlap
// so we warn user and proceed with the definition from the smaller scope.
if _, ok := modAddrsInRun[addr]; ok {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagWarning,
Summary: "Multiple `override_module` blocks for the same address",
Detail: fmt.Sprintf("Module `%v` is overridden in both global file and local run blocks. The declaration in global file block will be ignored.", addr),
Subject: m.Target.SourceRange().Ptr(),
})
continue
}
modules = append(modules, m)
}
return modules, diags
}

View File

@ -0,0 +1,353 @@
package hcl2shim
import (
"fmt"
"math/rand"
"strings"
"github.com/opentofu/opentofu/internal/configs/configschema"
"github.com/opentofu/opentofu/internal/tfdiags"
"github.com/zclconf/go-cty/cty"
)
// ComposeMockValueBySchema composes mock value based on schema configuration. It uses
// configuration value as a baseline and populates null values with provided defaults.
// If the provided defaults doesn't contain needed fields, ComposeMockValueBySchema uses
// its own defaults. ComposeMockValueBySchema fails if schema contains dynamic types.
func ComposeMockValueBySchema(schema *configschema.Block, config cty.Value, defaults map[string]cty.Value) (
cty.Value, tfdiags.Diagnostics) {
return mockValueComposer{}.composeMockValueBySchema(schema, config, defaults)
}
type mockValueComposer struct {
getMockStringOverride func() string
}
func (mvc mockValueComposer) getMockString() string {
f := getRandomAlphaNumString
if mvc.getMockStringOverride != nil {
f = mvc.getMockStringOverride
}
return f()
}
func (mvc mockValueComposer) composeMockValueBySchema(schema *configschema.Block, config cty.Value, defaults map[string]cty.Value) (cty.Value, tfdiags.Diagnostics) {
var configMap map[string]cty.Value
var diags tfdiags.Diagnostics
if !config.IsNull() {
configMap = config.AsValueMap()
}
impliedTypes := schema.ImpliedType().AttributeTypes()
mockAttrs, moreDiags := mvc.composeMockValueForAttributes(schema, configMap, defaults)
diags = diags.Append(moreDiags)
if moreDiags.HasErrors() {
return cty.NilVal, diags
}
mockBlocks, moreDiags := mvc.composeMockValueForBlocks(schema, configMap, defaults)
diags = diags.Append(moreDiags)
if moreDiags.HasErrors() {
return cty.NilVal, diags
}
mockValues := mockAttrs
for k, v := range mockBlocks {
mockValues[k] = v
}
for k := range defaults {
if _, ok := impliedTypes[k]; !ok {
diags = diags.Append(tfdiags.WholeContainingBody(
tfdiags.Warning,
fmt.Sprintf("Ignored mock/override field `%v`", k),
"The field is unknown. Please, ensure it is a part of resource definition.",
))
}
}
return cty.ObjectVal(mockValues), diags
}
func (mvc mockValueComposer) composeMockValueForAttributes(schema *configschema.Block, configMap map[string]cty.Value, defaults map[string]cty.Value) (map[string]cty.Value, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
addPotentialDefaultsWarning := func(key, description string) {
if _, ok := defaults[key]; ok {
diags = diags.Append(tfdiags.WholeContainingBody(
tfdiags.Warning,
fmt.Sprintf("Ignored mock/override field `%v`", key),
description,
))
}
}
mockAttrs := make(map[string]cty.Value)
impliedTypes := schema.ImpliedType().AttributeTypes()
for k, attr := range schema.Attributes {
// If the value present in configuration - just use it.
if cv, ok := configMap[k]; ok && !cv.IsNull() {
mockAttrs[k] = cv
addPotentialDefaultsWarning(k, "The field is ignored since overriding configuration values is not allowed.")
continue
}
// Non-computed attributes can't be generated
// so we set them from configuration only.
if !attr.Computed {
mockAttrs[k] = cty.NullVal(attr.Type)
addPotentialDefaultsWarning(k, "The field is ignored since overriding non-computed fields is not allowed.")
continue
}
// If the attribute is computed and not configured,
// we use provided value from defaults.
if ov, ok := defaults[k]; ok {
typeConformanceErrs := ov.Type().TestConformance(attr.Type)
if len(typeConformanceErrs) == 0 {
mockAttrs[k] = ov
continue
}
for _, err := range typeConformanceErrs {
diags = diags.Append(tfdiags.WholeContainingBody(
tfdiags.Warning,
fmt.Sprintf("Ignored mock/override field `%v`", k),
fmt.Sprintf("Values provided for override / mock must match resource fields types: %v.", err),
))
}
}
// If there's no value in defaults, we generate our own.
v, ok := mvc.getMockValueByType(impliedTypes[k])
if !ok {
diags = diags.Append(tfdiags.WholeContainingBody(
tfdiags.Error,
"Failed to generate mock value",
fmt.Sprintf("Mock value cannot be generated for dynamic type. Please specify the `%v` field explicitly in the configuration.", k),
))
continue
}
mockAttrs[k] = v
}
return mockAttrs, diags
}
func (mvc mockValueComposer) composeMockValueForBlocks(schema *configschema.Block, configMap map[string]cty.Value, defaults map[string]cty.Value) (map[string]cty.Value, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
mockBlocks := make(map[string]cty.Value)
impliedTypes := schema.ImpliedType().AttributeTypes()
for k, block := range schema.BlockTypes {
// Checking if the config value really present for the block.
// It should be non-null and non-empty collection.
configVal, hasConfigVal := configMap[k]
if hasConfigVal && configVal.IsNull() {
hasConfigVal = false
}
if hasConfigVal && !configVal.IsKnown() {
hasConfigVal = false
}
if hasConfigVal && configVal.Type().IsCollectionType() && configVal.LengthInt() == 0 {
hasConfigVal = false
}
defaultVal, hasDefaultVal := defaults[k]
if hasDefaultVal && !defaultVal.Type().IsObjectType() {
hasDefaultVal = false
diags = diags.Append(tfdiags.WholeContainingBody(
tfdiags.Warning,
fmt.Sprintf("Ignored mock/override field `%v`", k),
fmt.Sprintf("Blocks can be overridden only by objects, got `%s`", defaultVal.Type().FriendlyName()),
))
}
// We must keep blocks the same as it defined in configuration,
// so provider response validation succeeds later.
if !hasConfigVal {
mockBlocks[k] = block.EmptyValue()
if hasDefaultVal {
diags = diags.Append(tfdiags.WholeContainingBody(
tfdiags.Warning,
fmt.Sprintf("Ignored mock/override field `%v`", k),
"Cannot overridde block value, because it's not present in configuration.",
))
}
continue
}
var blockDefaults map[string]cty.Value
if hasDefaultVal {
blockDefaults = defaultVal.AsValueMap()
}
v, moreDiags := mvc.getMockValueForBlock(impliedTypes[k], configVal, &block.Block, blockDefaults)
diags = append(diags, moreDiags...)
if moreDiags.HasErrors() {
return nil, diags
}
mockBlocks[k] = v
}
return mockBlocks, diags
}
// getMockValueForBlock uses an object from the defaults (overrides)
// to compose each value from the block's inner collection. It recursevily calls
// composeMockValueBySchema to proceed with all the inner attributes and blocks
// the same way so all the nested blocks follow the same logic.
func (mvc mockValueComposer) getMockValueForBlock(targetType cty.Type, configVal cty.Value, block *configschema.Block, defaults map[string]cty.Value) (cty.Value, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
switch {
case targetType.IsObjectType():
mockBlockVal, moreDiags := mvc.composeMockValueBySchema(block, configVal, defaults)
diags = diags.Append(moreDiags)
if moreDiags.HasErrors() {
return cty.NilVal, diags
}
return mockBlockVal, diags
case targetType.ListElementType() != nil || targetType.SetElementType() != nil:
var mockBlockVals []cty.Value
var iterator = configVal.ElementIterator()
for iterator.Next() {
_, blockConfigV := iterator.Element()
mockBlockVal, moreDiags := mvc.composeMockValueBySchema(block, blockConfigV, defaults)
diags = diags.Append(moreDiags)
if moreDiags.HasErrors() {
return cty.NilVal, diags
}
mockBlockVals = append(mockBlockVals, mockBlockVal)
}
if targetType.ListElementType() != nil {
return cty.ListVal(mockBlockVals), diags
} else {
return cty.SetVal(mockBlockVals), diags
}
case targetType.MapElementType() != nil:
var mockBlockVals = make(map[string]cty.Value)
var iterator = configVal.ElementIterator()
for iterator.Next() {
blockConfigK, blockConfigV := iterator.Element()
mockBlockVal, moreDiags := mvc.composeMockValueBySchema(block, blockConfigV, defaults)
diags = diags.Append(moreDiags)
if moreDiags.HasErrors() {
return cty.NilVal, diags
}
mockBlockVals[blockConfigK.AsString()] = mockBlockVal
}
return cty.MapVal(mockBlockVals), diags
default:
// Shouldn't happen as long as blocks are represented by lists / maps / sets / objs.
return cty.NilVal, diags.Append(tfdiags.WholeContainingBody(
tfdiags.Error,
fmt.Sprintf("Unexpected block type: %v", targetType.FriendlyName()),
"Failed to generate mock value for this block type. Please, report it as an issue at OpenTofu repository, since it's not expected.",
))
}
}
// getMockValueByType tries to generate mock cty.Value based on provided cty.Type.
// It will return non-ok response if it encounters dynamic type.
func (mvc mockValueComposer) getMockValueByType(t cty.Type) (cty.Value, bool) {
var v cty.Value
// just to be sure for cases when the logic below misses something
if t.HasDynamicTypes() {
return cty.Value{}, false
}
switch {
// primitives
case t.Equals(cty.Number):
v = cty.Zero
case t.Equals(cty.Bool):
v = cty.False
case t.Equals(cty.String):
v = cty.StringVal(mvc.getMockString())
// collections
case t.ListElementType() != nil:
v = cty.ListValEmpty(*t.ListElementType())
case t.MapElementType() != nil:
v = cty.MapValEmpty(*t.MapElementType())
case t.SetElementType() != nil:
v = cty.SetValEmpty(*t.SetElementType())
// structural
case t.IsObjectType():
objVals := make(map[string]cty.Value)
// populate the object with mock values
for k, at := range t.AttributeTypes() {
if t.AttributeOptional(k) {
continue
}
objV, ok := mvc.getMockValueByType(at)
if !ok {
return cty.Value{}, false
}
objVals[k] = objV
}
v = cty.ObjectVal(objVals)
case t.IsTupleType():
v = cty.EmptyTupleVal
// dynamically typed values are not supported
default:
return cty.Value{}, false
}
return v, true
}
func getRandomAlphaNumString() string {
const chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890"
const minLength, maxLength = 4, 16
length := rand.Intn(maxLength-minLength) + minLength //nolint:gosec // It doesn't need to be secure.
b := strings.Builder{}
b.Grow(length)
for i := 0; i < length; i++ {
b.WriteByte(chars[rand.Intn(len(chars))]) //nolint:gosec // It doesn't need to be secure.
}
return b.String()
}

View File

@ -0,0 +1,492 @@
package hcl2shim
import (
"strings"
"testing"
"github.com/opentofu/opentofu/internal/configs/configschema"
"github.com/zclconf/go-cty/cty"
)
func TestComposeMockValueBySchema(t *testing.T) {
t.Parallel()
tests := map[string]struct {
schema *configschema.Block
config cty.Value
defaults map[string]cty.Value
wantVal cty.Value
wantWarning bool
wantError bool
}{
"diff-props-in-root-attributes": {
schema: &configschema.Block{
Attributes: map[string]*configschema.Attribute{
"required-only": {
Type: cty.String,
Required: true,
Optional: false,
Computed: false,
Sensitive: false,
},
"required-computed": {
Type: cty.String,
Required: true,
Optional: false,
Computed: true,
Sensitive: false,
},
"optional": {
Type: cty.String,
Required: false,
Optional: true,
Computed: false,
Sensitive: false,
},
"optional-computed": {
Type: cty.String,
Required: false,
Optional: true,
Computed: true,
Sensitive: false,
},
"computed-only": {
Type: cty.String,
Required: false,
Optional: false,
Computed: true,
Sensitive: false,
},
"sensitive-optional": {
Type: cty.String,
Required: false,
Optional: true,
Computed: false,
Sensitive: true,
},
"sensitive-required": {
Type: cty.String,
Required: true,
Optional: false,
Computed: false,
Sensitive: true,
},
"sensitive-computed": {
Type: cty.String,
Required: true,
Optional: false,
Computed: true,
Sensitive: true,
},
},
},
config: cty.NilVal,
wantVal: cty.ObjectVal(map[string]cty.Value{
"required-only": cty.NullVal(cty.String),
"required-computed": cty.StringVal("aaaaaaaa"),
"optional": cty.NullVal(cty.String),
"optional-computed": cty.StringVal("aaaaaaaa"),
"computed-only": cty.StringVal("aaaaaaaa"),
"sensitive-optional": cty.NullVal(cty.String),
"sensitive-required": cty.NullVal(cty.String),
"sensitive-computed": cty.StringVal("aaaaaaaa"),
}),
},
"diff-props-in-single-block-attributes": {
schema: &configschema.Block{
BlockTypes: map[string]*configschema.NestedBlock{
"nested": {
Nesting: configschema.NestingSingle,
Block: configschema.Block{
Attributes: map[string]*configschema.Attribute{
"required-only": {
Type: cty.String,
Required: true,
Optional: false,
Computed: false,
Sensitive: false,
},
"required-computed": {
Type: cty.String,
Required: true,
Optional: false,
Computed: true,
Sensitive: false,
},
"optional": {
Type: cty.String,
Required: false,
Optional: true,
Computed: false,
Sensitive: false,
},
"optional-computed": {
Type: cty.String,
Required: false,
Optional: true,
Computed: true,
Sensitive: false,
},
"computed-only": {
Type: cty.String,
Required: false,
Optional: false,
Computed: true,
Sensitive: false,
},
"sensitive-optional": {
Type: cty.String,
Required: false,
Optional: true,
Computed: false,
Sensitive: true,
},
"sensitive-required": {
Type: cty.String,
Required: true,
Optional: false,
Computed: false,
Sensitive: true,
},
"sensitive-computed": {
Type: cty.String,
Required: true,
Optional: false,
Computed: true,
Sensitive: true,
},
},
},
},
},
},
config: cty.ObjectVal(map[string]cty.Value{
"nested": cty.ObjectVal(map[string]cty.Value{}),
}),
wantVal: cty.ObjectVal(map[string]cty.Value{
"nested": cty.ObjectVal(map[string]cty.Value{
"required-only": cty.NullVal(cty.String),
"required-computed": cty.StringVal("aaaaaaaa"),
"optional": cty.NullVal(cty.String),
"optional-computed": cty.StringVal("aaaaaaaa"),
"computed-only": cty.StringVal("aaaaaaaa"),
"sensitive-optional": cty.NullVal(cty.String),
"sensitive-required": cty.NullVal(cty.String),
"sensitive-computed": cty.StringVal("aaaaaaaa"),
}),
}),
},
"basic-group-block": {
schema: &configschema.Block{
BlockTypes: map[string]*configschema.NestedBlock{
"nested": {
Nesting: configschema.NestingGroup,
Block: configschema.Block{
Attributes: map[string]*configschema.Attribute{
"field": {
Type: cty.Number,
Computed: true,
},
},
},
},
},
},
config: cty.ObjectVal(map[string]cty.Value{
"nested": cty.ObjectVal(map[string]cty.Value{}),
}),
wantVal: cty.ObjectVal(map[string]cty.Value{
"nested": cty.ObjectVal(map[string]cty.Value{
"field": cty.NumberIntVal(0),
}),
}),
},
"basic-list-block": {
schema: &configschema.Block{
BlockTypes: map[string]*configschema.NestedBlock{
"nested": {
Nesting: configschema.NestingList,
Block: configschema.Block{
Attributes: map[string]*configschema.Attribute{
"field": {
Type: cty.Number,
Computed: true,
},
},
},
},
},
},
config: cty.ObjectVal(map[string]cty.Value{
"nested": cty.ListVal([]cty.Value{cty.ObjectVal(map[string]cty.Value{})}),
}),
wantVal: cty.ObjectVal(map[string]cty.Value{
"nested": cty.ListVal([]cty.Value{
cty.ObjectVal(map[string]cty.Value{
"field": cty.NumberIntVal(0),
}),
}),
}),
},
"basic-set-block": {
schema: &configschema.Block{
BlockTypes: map[string]*configschema.NestedBlock{
"nested": {
Nesting: configschema.NestingSet,
Block: configschema.Block{
Attributes: map[string]*configschema.Attribute{
"field": {
Type: cty.Number,
Computed: true,
},
},
},
},
},
},
config: cty.ObjectVal(map[string]cty.Value{
"nested": cty.SetVal([]cty.Value{cty.ObjectVal(map[string]cty.Value{})}),
}),
wantVal: cty.ObjectVal(map[string]cty.Value{
"nested": cty.SetVal([]cty.Value{
cty.ObjectVal(map[string]cty.Value{
"field": cty.NumberIntVal(0),
}),
}),
}),
},
"basic-map-block": {
schema: &configschema.Block{
BlockTypes: map[string]*configschema.NestedBlock{
"nested": {
Nesting: configschema.NestingMap,
Block: configschema.Block{
Attributes: map[string]*configschema.Attribute{
"field": {
Type: cty.Number,
Computed: true,
},
},
},
},
},
},
config: cty.ObjectVal(map[string]cty.Value{
"nested": cty.MapVal(map[string]cty.Value{
"somelabel": cty.ObjectVal(map[string]cty.Value{}),
}),
}),
wantVal: cty.ObjectVal(map[string]cty.Value{
"nested": cty.MapVal(map[string]cty.Value{
"somelabel": cty.ObjectVal(map[string]cty.Value{
"field": cty.NumberIntVal(0),
}),
}),
}),
},
"basic-mocked-attributes": {
schema: &configschema.Block{
Attributes: map[string]*configschema.Attribute{
"num": {
Type: cty.Number,
Computed: true,
Optional: true,
},
"str": {
Type: cty.String,
Computed: true,
Optional: true,
},
"bool": {
Type: cty.Bool,
Computed: true,
Optional: true,
},
"obj": {
Type: cty.Object(map[string]cty.Type{
"fieldNum": cty.Number,
"fieldStr": cty.String,
}),
Computed: true,
Optional: true,
},
"list": {
Type: cty.List(cty.String),
Computed: true,
Optional: true,
},
},
BlockTypes: map[string]*configschema.NestedBlock{
"nested": {
Nesting: configschema.NestingList,
Block: configschema.Block{
Attributes: map[string]*configschema.Attribute{
"num": {
Type: cty.Number,
Computed: true,
Optional: true,
},
"str": {
Type: cty.String,
Computed: true,
Optional: true,
},
"bool": {
Type: cty.Bool,
Computed: true,
Optional: true,
},
"obj": {
Type: cty.Object(map[string]cty.Type{
"fieldNum": cty.Number,
"fieldStr": cty.String,
}),
Computed: true,
Optional: true,
},
"list": {
Type: cty.List(cty.String),
Computed: true,
Optional: true,
},
},
},
},
},
},
config: cty.ObjectVal(map[string]cty.Value{
"nested": cty.ListVal([]cty.Value{cty.ObjectVal(map[string]cty.Value{})}),
}),
wantVal: cty.ObjectVal(map[string]cty.Value{
"num": cty.NumberIntVal(0),
"str": cty.StringVal("aaaaaaaa"),
"bool": cty.False,
"obj": cty.ObjectVal(map[string]cty.Value{
"fieldNum": cty.NumberIntVal(0),
"fieldStr": cty.StringVal("aaaaaaaa"),
}),
"list": cty.ListValEmpty(cty.String),
"nested": cty.ListVal([]cty.Value{
cty.ObjectVal(map[string]cty.Value{
"num": cty.NumberIntVal(0),
"str": cty.StringVal("aaaaaaaa"),
"bool": cty.False,
"obj": cty.ObjectVal(map[string]cty.Value{
"fieldNum": cty.NumberIntVal(0),
"fieldStr": cty.StringVal("aaaaaaaa"),
}),
"list": cty.ListValEmpty(cty.String),
}),
}),
}),
},
"source-priority": {
schema: &configschema.Block{
Attributes: map[string]*configschema.Attribute{
"useConfigValue": {
Type: cty.String,
Computed: true,
Optional: true,
},
"useDefaultsValue": {
Type: cty.String,
Computed: true,
Optional: true,
},
"generateMockValue": {
Type: cty.String,
Computed: true,
Optional: true,
},
},
BlockTypes: map[string]*configschema.NestedBlock{
"nested": {
Nesting: configschema.NestingList,
Block: configschema.Block{
Attributes: map[string]*configschema.Attribute{
"useConfigValue": {
Type: cty.String,
Computed: true,
Optional: true,
},
"useDefaultsValue": {
Type: cty.String,
Computed: true,
Optional: true,
},
"generateMockValue": {
Type: cty.String,
Computed: true,
Optional: true,
},
},
},
},
},
},
config: cty.ObjectVal(map[string]cty.Value{
"useConfigValue": cty.StringVal("iAmFromConfig"),
"nested": cty.ListVal([]cty.Value{
cty.ObjectVal(map[string]cty.Value{
"useConfigValue": cty.StringVal("iAmFromConfig"),
}),
}),
}),
defaults: map[string]cty.Value{
"useConfigValue": cty.StringVal("iAmFromDefaults"),
"useDefaultsValue": cty.StringVal("iAmFromDefaults"),
"nested": cty.ObjectVal(map[string]cty.Value{
"useConfigValue": cty.StringVal("iAmFromDefaults"),
"useDefaultsValue": cty.StringVal("iAmFromDefaults"),
}),
},
wantVal: cty.ObjectVal(map[string]cty.Value{
"useConfigValue": cty.StringVal("iAmFromConfig"),
"useDefaultsValue": cty.StringVal("iAmFromDefaults"),
"generateMockValue": cty.StringVal("aaaaaaaa"),
"nested": cty.ListVal([]cty.Value{
cty.ObjectVal(map[string]cty.Value{
"useConfigValue": cty.StringVal("iAmFromConfig"),
"useDefaultsValue": cty.StringVal("iAmFromDefaults"),
"generateMockValue": cty.StringVal("aaaaaaaa"),
}),
}),
}),
wantWarning: true, // ignored value in defaults
},
}
const mockStringLength = 8
mvc := mockValueComposer{
getMockStringOverride: func() string {
return strings.Repeat("a", mockStringLength)
},
}
for name, test := range tests {
test := test
t.Run(name, func(t *testing.T) {
t.Parallel()
gotVal, gotDiags := mvc.composeMockValueBySchema(test.schema, test.config, test.defaults)
switch {
case test.wantError && !gotDiags.HasErrors():
t.Fatalf("Expected error in diags, but none returned")
case !test.wantError && gotDiags.HasErrors():
t.Fatalf("Got unexpected error diags: %v", gotDiags.ErrWithWarnings())
case test.wantWarning && len(gotDiags) == 0:
t.Fatalf("Expected warning in diags, but none returned")
case !test.wantWarning && len(gotDiags) != 0:
t.Fatalf("Got unexpected diags: %v", gotDiags.ErrWithWarnings())
case !test.wantVal.RawEquals(gotVal):
t.Fatalf("Got unexpected value: %v", gotVal.GoString())
}
})
}
}

View File

@ -60,6 +60,10 @@ type Module struct {
Checks map[string]*Check
Tests map[string]*TestFile
// IsOverridden indicates if the module is being overridden. It's used in
// testing framework to not call the underlying module.
IsOverridden bool
}
// File describes the contents of a single configuration file.

View File

@ -406,6 +406,14 @@ type Output struct {
SensitiveSet bool
DeclRange hcl.Range
// IsOverridden indicates if the output is being overridden. It's used in
// testing framework to not evaluate expression and use OverrideValue instead.
IsOverridden bool
// OverrideValue is only valid if IsOverridden is set to true. The value
// should be used instead of evaluated expression. It's possible to have no
// OverrideValue even with IsOverridden is set to true.
OverrideValue *cty.Value
}
func decodeOutputBlock(block *hcl.Block, override bool) (*Output, hcl.Diagnostics) {

View File

@ -12,6 +12,8 @@ import (
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/hashicorp/hcl/v2/hclsyntax"
hcljson "github.com/hashicorp/hcl/v2/json"
"github.com/zclconf/go-cty/cty"
"github.com/opentofu/opentofu/internal/addrs"
"github.com/opentofu/opentofu/internal/configs/hcl2shim"
"github.com/opentofu/opentofu/internal/lang"
@ -49,6 +51,14 @@ type Resource struct {
// If this is nil, then this resource is essentially public.
Container Container
// IsOverridden indicates if the resource is being overridden. It's used in
// testing framework to not call the underlying provider.
IsOverridden bool
// OverrideValues are only valid if IsOverridden is set to true. The values
// should be used to compose mock provider response. It is possible to have
// zero-length OverrideValues even if IsOverridden is set to true.
OverrideValues map[string]cty.Value
DeclRange hcl.Range
TypeRange hcl.Range
}

View File

@ -11,6 +11,7 @@ import (
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/hashicorp/hcl/v2/hclsyntax"
"github.com/zclconf/go-cty/cty"
"github.com/opentofu/opentofu/internal/addrs"
"github.com/opentofu/opentofu/internal/getmodules"
@ -62,9 +63,30 @@ type TestFile struct {
// order.
Runs []*TestRun
// OverrideResources is a list of resources to be overridden with static values.
// Underlying providers shouldn't be called for overridden resources.
OverrideResources []*OverrideResource
// OverrideModules is a list of modules to be overridden with static values.
// Underlying modules shouldn't be called.
OverrideModules []*OverrideModule
VariablesDeclRange hcl.Range
}
// Validate does a very simple and cursory check across the file blocks to look
// for simple issues we can highlight early on. It doesn't validate nested run blocks.
func (file *TestFile) Validate() tfdiags.Diagnostics {
var diags tfdiags.Diagnostics
// It's not allowed to have multiple `override_resource`, `override_data` or `override_module` blocks
// declared globally in a file with the same target address so we want to ensure there's no such cases.
diags = diags.Append(checkForDuplicatedOverrideResources(file.OverrideResources))
diags = diags.Append(checkForDuplicatedOverrideModules(file.OverrideModules))
return diags
}
// TestRun represents a single run block within a test file.
//
// Each run block represents a single OpenTofu command to be executed and a set
@ -125,6 +147,14 @@ type TestRun struct {
// run.
ExpectFailures []hcl.Traversal
// OverrideResources is a list of resources to be overridden with static values.
// Underlying providers shouldn't be called for overridden resources.
OverrideResources []*OverrideResource
// OverrideModules is a list of modules to be overridden with static values.
// Underlying modules shouldn't be called.
OverrideModules []*OverrideModule
NameDeclRange hcl.Range
VariablesDeclRange hcl.Range
DeclRange hcl.Range
@ -135,8 +165,8 @@ type TestRun struct {
func (run *TestRun) Validate() tfdiags.Diagnostics {
var diags tfdiags.Diagnostics
// For now, we only want to make sure all the ExpectFailure references are
// the correct kind of reference.
// We want to make sure all the ExpectFailure references
// are the correct kind of reference.
for _, traversal := range run.ExpectFailures {
reference, refDiags := addrs.ParseRefFromTestingScope(traversal)
@ -160,6 +190,11 @@ func (run *TestRun) Validate() tfdiags.Diagnostics {
}
// It's not allowed to have multiple `override_resource`, `override_data` or `override_module` blocks
// inside a single run block with the same target address so we want to ensure there's no such cases.
diags = diags.Append(checkForDuplicatedOverrideResources(run.OverrideResources))
diags = diags.Append(checkForDuplicatedOverrideModules(run.OverrideModules))
return diags
}
@ -193,6 +228,51 @@ type TestRunOptions struct {
DeclRange hcl.Range
}
const (
blockNameOverrideResource = "override_resource"
blockNameOverrideData = "override_data"
)
// OverrideResource contains information about a resource or data block to be overridden.
type OverrideResource struct {
// Target references resource or data block to override.
Target hcl.Traversal
TargetParsed *addrs.ConfigResource
// Mode indicates if the Target is resource or data block.
Mode addrs.ResourceMode
// Values represents fields to use as defaults
// if they are not present in configuration.
Values map[string]cty.Value
}
func (r OverrideResource) getBlockName() string {
switch r.Mode {
case addrs.ManagedResourceMode:
return blockNameOverrideResource
case addrs.DataResourceMode:
return blockNameOverrideData
case addrs.InvalidResourceMode:
return "invalid"
default:
return "invalid"
}
}
const blockNameOverrideModule = "override_module"
// OverrideModule contains information about a module to be overridden.
type OverrideModule struct {
// Target references module call to override.
Target hcl.Traversal
TargetParsed addrs.Module
// Outputs represents fields to use instead
// of the real module call output.
Outputs map[string]cty.Value
}
func loadTestFile(body hcl.Body) (*TestFile, hcl.Diagnostics) {
var diags hcl.Diagnostics
@ -211,6 +291,7 @@ func loadTestFile(body hcl.Body) (*TestFile, hcl.Diagnostics) {
if !runDiags.HasErrors() {
tf.Runs = append(tf.Runs, run)
}
case "variables":
if tf.Variables != nil {
diags = append(diags, &hcl.Diagnostic{
@ -230,12 +311,35 @@ func loadTestFile(body hcl.Body) (*TestFile, hcl.Diagnostics) {
for _, v := range vars {
tf.Variables[v.Name] = v.Expr
}
case "provider":
provider, providerDiags := decodeProviderBlock(block)
diags = append(diags, providerDiags...)
if provider != nil {
tf.Providers[provider.moduleUniqueKey()] = provider
}
case blockNameOverrideResource:
overrideRes, overrideResDiags := decodeOverrideResourceBlock(block, addrs.ManagedResourceMode)
diags = append(diags, overrideResDiags...)
if !overrideResDiags.HasErrors() {
tf.OverrideResources = append(tf.OverrideResources, overrideRes)
}
case blockNameOverrideData:
overrideData, overrideDataDiags := decodeOverrideResourceBlock(block, addrs.DataResourceMode)
diags = append(diags, overrideDataDiags...)
if !overrideDataDiags.HasErrors() {
tf.OverrideResources = append(tf.OverrideResources, overrideData)
}
case blockNameOverrideModule:
overrideMod, overrideModDiags := decodeOverrideModuleBlock(block)
diags = append(diags, overrideModDiags...)
if !overrideModDiags.HasErrors() {
tf.OverrideModules = append(tf.OverrideModules, overrideMod)
}
}
}
@ -321,6 +425,27 @@ func decodeTestRunBlock(block *hcl.Block) (*TestRun, hcl.Diagnostics) {
if !moduleDiags.HasErrors() {
r.Module = module
}
case blockNameOverrideResource:
overrideRes, overrideResDiags := decodeOverrideResourceBlock(block, addrs.ManagedResourceMode)
diags = append(diags, overrideResDiags...)
if !overrideResDiags.HasErrors() {
r.OverrideResources = append(r.OverrideResources, overrideRes)
}
case blockNameOverrideData:
overrideData, overrideDataDiags := decodeOverrideResourceBlock(block, addrs.DataResourceMode)
diags = append(diags, overrideDataDiags...)
if !overrideDataDiags.HasErrors() {
r.OverrideResources = append(r.OverrideResources, overrideData)
}
case blockNameOverrideModule:
overrideMod, overrideModDiags := decodeOverrideModuleBlock(block)
diags = append(diags, overrideModDiags...)
if !overrideModDiags.HasErrors() {
r.OverrideModules = append(r.OverrideModules, overrideMod)
}
}
}
@ -537,6 +662,143 @@ func decodeTestRunOptionsBlock(block *hcl.Block) (*TestRunOptions, hcl.Diagnosti
return &opts, diags
}
func decodeOverrideResourceBlock(block *hcl.Block, mode addrs.ResourceMode) (*OverrideResource, hcl.Diagnostics) {
parseTarget := func(attr *hcl.Attribute) (hcl.Traversal, *addrs.ConfigResource, hcl.Diagnostics) {
traversal, traversalDiags := hcl.AbsTraversalForExpr(attr.Expr)
diags := traversalDiags
if traversalDiags.HasErrors() {
return nil, nil, diags
}
configRes, configResDiags := addrs.ParseConfigResource(traversal)
diags = append(diags, configResDiags.ToHCL()...)
if configResDiags.HasErrors() {
return nil, nil, diags
}
return traversal, &configRes, diags
}
res := &OverrideResource{
Mode: mode,
}
content, diags := block.Body.Content(overrideResourceBlockSchema)
if attr, exists := content.Attributes["target"]; exists {
target, parsed, moreDiags := parseTarget(attr)
res.Target, res.TargetParsed = target, parsed
diags = append(diags, moreDiags...)
}
if attr, exists := content.Attributes["values"]; exists {
v, moreDiags := parseObjectAttrWithNoVariables(attr)
res.Values, diags = v, append(diags, moreDiags...)
}
return res, diags
}
func decodeOverrideModuleBlock(block *hcl.Block) (*OverrideModule, hcl.Diagnostics) {
parseTarget := func(attr *hcl.Attribute) (hcl.Traversal, addrs.Module, hcl.Diagnostics) {
traversal, traversalDiags := hcl.AbsTraversalForExpr(attr.Expr)
diags := traversalDiags
if traversalDiags.HasErrors() {
return nil, nil, diags
}
target, targetDiags := addrs.ParseModule(traversal)
diags = append(diags, targetDiags.ToHCL()...)
if targetDiags.HasErrors() {
return nil, nil, diags
}
return traversal, target, diags
}
mod := &OverrideModule{}
content, diags := block.Body.Content(overrideModuleBlockSchema)
if attr, exists := content.Attributes["target"]; exists {
traversal, target, moreDiags := parseTarget(attr)
mod.Target, mod.TargetParsed = traversal, target
diags = append(diags, moreDiags...)
}
if attr, exists := content.Attributes["outputs"]; exists {
outputs, moreDiags := parseObjectAttrWithNoVariables(attr)
mod.Outputs, diags = outputs, append(diags, moreDiags...)
}
return mod, diags
}
func parseObjectAttrWithNoVariables(attr *hcl.Attribute) (map[string]cty.Value, hcl.Diagnostics) {
attrVal, valDiags := attr.Expr.Value(nil)
diags := valDiags
if valDiags.HasErrors() {
return nil, diags
}
if !attrVal.Type().IsObjectType() {
return nil, append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Object expected",
Detail: fmt.Sprintf("The attribute `%v` must be an object.", attr.Name),
Subject: attr.Range.Ptr(),
})
}
return attrVal.AsValueMap(), diags
}
func checkForDuplicatedOverrideResources(resources []*OverrideResource) hcl.Diagnostics {
var diags hcl.Diagnostics
overrideResources := make(map[string]struct{}, len(resources))
for _, res := range resources {
k := res.TargetParsed.String()
if _, ok := overrideResources[k]; ok {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("Duplicated `%v` block", res.getBlockName()),
Detail: fmt.Sprintf("It is not allowed to have multiple `%v` blocks with the same target: `%v`.", res.getBlockName(), res.TargetParsed),
Subject: res.Target.SourceRange().Ptr(),
})
continue
}
overrideResources[k] = struct{}{}
}
return diags
}
func checkForDuplicatedOverrideModules(modules []*OverrideModule) hcl.Diagnostics {
var diags hcl.Diagnostics
overrideModules := make(map[string]struct{}, len(modules))
for _, mod := range modules {
k := mod.TargetParsed.String()
if _, ok := overrideModules[k]; ok {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Duplicated `override_module` block",
Detail: fmt.Sprintf("It is not allowed to have multiple `override_module` blocks with the same target: `%v`.", mod.TargetParsed),
Subject: mod.Target.SourceRange().Ptr(),
})
continue
}
overrideModules[k] = struct{}{}
}
return diags
}
var testFileSchema = &hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
@ -550,6 +812,15 @@ var testFileSchema = &hcl.BodySchema{
{
Type: "variables",
},
{
Type: blockNameOverrideResource,
},
{
Type: blockNameOverrideData,
},
{
Type: blockNameOverrideModule,
},
},
}
@ -572,6 +843,15 @@ var testRunBlockSchema = &hcl.BodySchema{
{
Type: "module",
},
{
Type: blockNameOverrideResource,
},
{
Type: blockNameOverrideData,
},
{
Type: blockNameOverrideModule,
},
},
}
@ -590,3 +870,31 @@ var testRunModuleBlockSchema = &hcl.BodySchema{
{Name: "version"},
},
}
//nolint:gochecknoglobals // To follow existing code style.
var overrideResourceBlockSchema = &hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "target",
Required: true,
},
{
Name: "values",
Required: false,
},
},
}
//nolint:gochecknoglobals // To follow existing code style.
var overrideModuleBlockSchema = &hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "target",
Required: true,
},
{
Name: "outputs",
Required: false,
},
},
}

View File

@ -196,7 +196,9 @@ func (b *ApplyGraphBuilder) Steps() []GraphTransformer {
&CloseProviderTransformer{},
// close the root module
&CloseRootModuleTransformer{},
&CloseRootModuleTransformer{
RootConfig: b.Config,
},
// Perform the transitive reduction to make our graph a bit
// more understandable if possible (it usually is possible).

View File

@ -109,7 +109,9 @@ func (b *EvalGraphBuilder) Steps() []GraphTransformer {
&CloseProviderTransformer{},
// Close root module
&CloseRootModuleTransformer{},
&CloseRootModuleTransformer{
RootConfig: b.Config,
},
// Remove redundant edges to simplify the graph.
&TransitiveReductionTransformer{},

View File

@ -244,7 +244,9 @@ func (b *PlanGraphBuilder) Steps() []GraphTransformer {
&CloseProviderTransformer{},
// Close the root module
&CloseRootModuleTransformer{},
&CloseRootModuleTransformer{
RootConfig: b.Config,
},
// Perform the transitive reduction to make our graph a bit
// more understandable if possible (it usually is possible).

View File

@ -149,7 +149,8 @@ func (n *nodeExpandModule) Execute(ctx EvalContext, op walkOperation) (diags tfd
// The root module instance also closes any remaining provisioner plugins which
// do not have a lifecycle controlled by individual graph nodes.
type nodeCloseModule struct {
Addr addrs.Module
Addr addrs.Module
RootConfig *configs.Config
}
var (
@ -180,6 +181,19 @@ func (n *nodeCloseModule) Name() string {
return n.Addr.String() + " (close)"
}
func (n *nodeCloseModule) IsOverridden(addr addrs.Module) bool {
if n.RootConfig == nil {
return false
}
modConfig := n.RootConfig.Descendent(addr)
if modConfig == nil {
return false
}
return modConfig.Module.IsOverridden
}
func (n *nodeCloseModule) Execute(ctx EvalContext, op walkOperation) (diags tfdiags.Diagnostics) {
if !n.Addr.IsRoot() {
return
@ -202,8 +216,9 @@ func (n *nodeCloseModule) Execute(ctx EvalContext, op walkOperation) (diags tfdi
}
}
// empty child modules are always removed
if len(mod.Resources) == 0 && !mod.Addr.IsRoot() {
// empty non-root modules are removed normally,
// but if the module is being overridden, it should be kept
if len(mod.Resources) == 0 && !mod.Addr.IsRoot() && !n.IsOverridden(mod.Addr.Module()) {
delete(state.Modules, modKey)
}
}

View File

@ -46,7 +46,7 @@ func TestNodeCloseModuleExecute(t *testing.T) {
ctx := &MockEvalContext{
StateState: state.SyncWrapper(),
}
node := nodeCloseModule{addrs.Module{"child"}}
node := nodeCloseModule{Addr: addrs.Module{"child"}}
diags := node.Execute(ctx, walkApply)
if diags.HasErrors() {
t.Fatalf("unexpected error: %s", diags.Err())
@ -58,7 +58,7 @@ func TestNodeCloseModuleExecute(t *testing.T) {
}
// the root module should do all the module cleanup
node = nodeCloseModule{addrs.RootModule}
node = nodeCloseModule{Addr: addrs.RootModule}
diags = node.Execute(ctx, walkApply)
if diags.HasErrors() {
t.Fatalf("unexpected error: %s", diags.Err())
@ -77,7 +77,7 @@ func TestNodeCloseModuleExecute(t *testing.T) {
ctx := &MockEvalContext{
StateState: state.SyncWrapper(),
}
node := nodeCloseModule{addrs.Module{"child"}}
node := nodeCloseModule{Addr: addrs.Module{"child"}}
diags := node.Execute(ctx, walkImport)
if diags.HasErrors() {

View File

@ -344,11 +344,25 @@ func (n *NodeApplyableOutput) Execute(ctx EvalContext, op walkOperation) (diags
// If there was no change recorded, or the recorded change was not wholly
// known, then we need to re-evaluate the output
if !changeRecorded || !val.IsWhollyKnown() {
// This has to run before we have a state lock, since evaluation also
// reads the state
var evalDiags tfdiags.Diagnostics
val, evalDiags = ctx.EvaluateExpr(n.Config.Expr, cty.DynamicPseudoType, nil)
diags = diags.Append(evalDiags)
switch {
// If the module is not being overridden, we proceed normally
case !n.Config.IsOverridden:
// This has to run before we have a state lock, since evaluation also
// reads the state
var evalDiags tfdiags.Diagnostics
val, evalDiags = ctx.EvaluateExpr(n.Config.Expr, cty.DynamicPseudoType, nil)
diags = diags.Append(evalDiags)
// If the module is being overridden and we have a value to use,
// we just use it
case n.Config.OverrideValue != nil:
val = *n.Config.OverrideValue
// If the module is being overridden, but we don't have any value to use,
// we just set it to null
default:
val = cty.NilVal
}
// We'll handle errors below, after we have loaded the module.
// Outputs don't have a separate mode for validation, so validate

View File

@ -273,7 +273,7 @@ func (n *NodeAbstractResourceInstance) writeResourceInstanceStateDeposed(ctx Eva
// objects you are intending to write.
func (n *NodeAbstractResourceInstance) writeResourceInstanceStateImpl(ctx EvalContext, deposedKey states.DeposedKey, obj *states.ResourceInstanceObject, targetState phaseState) error {
absAddr := n.Addr
_, providerSchema, err := getProvider(ctx, n.ResolvedProvider)
_, providerSchema, err := n.getProvider(ctx, n.ResolvedProvider)
if err != nil {
return err
}
@ -416,7 +416,7 @@ func (n *NodeAbstractResourceInstance) planDestroy(ctx EvalContext, currentState
// operation.
nullVal := cty.NullVal(unmarkedPriorVal.Type())
provider, _, err := getProvider(ctx, n.ResolvedProvider)
provider, _, err := n.getProvider(ctx, n.ResolvedProvider)
if err != nil {
return plan, diags.Append(err)
}
@ -495,7 +495,7 @@ func (n *NodeAbstractResourceInstance) writeChange(ctx EvalContext, change *plan
return nil
}
_, providerSchema, err := getProvider(ctx, n.ResolvedProvider)
_, providerSchema, err := n.getProvider(ctx, n.ResolvedProvider)
if err != nil {
return err
}
@ -546,7 +546,7 @@ func (n *NodeAbstractResourceInstance) refresh(ctx EvalContext, deposedKey state
} else {
log.Printf("[TRACE] NodeAbstractResourceInstance.refresh for %s (deposed object %s)", absAddr, deposedKey)
}
provider, providerSchema, err := getProvider(ctx, n.ResolvedProvider)
provider, providerSchema, err := n.getProvider(ctx, n.ResolvedProvider)
if err != nil {
return state, diags.Append(err)
}
@ -681,7 +681,7 @@ func (n *NodeAbstractResourceInstance) plan(
var keyData instances.RepetitionData
resource := n.Addr.Resource.Resource
provider, providerSchema, err := getProvider(ctx, n.ResolvedProvider)
provider, providerSchema, err := n.getProviderWithPlannedChange(ctx, n.ResolvedProvider, plannedChange)
if err != nil {
return nil, nil, keyData, diags.Append(err)
}
@ -834,6 +834,7 @@ func (n *NodeAbstractResourceInstance) plan(
PriorPrivate: priorPrivate,
ProviderMeta: metaConfigVal,
})
diags = diags.Append(resp.Diagnostics.InConfigBody(config.Config, n.Addr.String()))
if diags.HasErrors() {
return nil, nil, keyData, diags
@ -1434,7 +1435,7 @@ func (n *NodeAbstractResourceInstance) readDataSource(ctx EvalContext, configVal
config := *n.Config
provider, providerSchema, err := getProvider(ctx, n.ResolvedProvider)
provider, providerSchema, err := n.getProvider(ctx, n.ResolvedProvider)
diags = diags.Append(err)
if diags.HasErrors() {
return newVal, diags
@ -1561,7 +1562,7 @@ func (n *NodeAbstractResourceInstance) providerMetas(ctx EvalContext) (cty.Value
var diags tfdiags.Diagnostics
metaConfigVal := cty.NullVal(cty.DynamicPseudoType)
_, providerSchema, err := getProvider(ctx, n.ResolvedProvider)
_, providerSchema, err := n.getProvider(ctx, n.ResolvedProvider)
if err != nil {
return metaConfigVal, diags.Append(err)
}
@ -1599,7 +1600,7 @@ func (n *NodeAbstractResourceInstance) planDataSource(ctx EvalContext, checkRule
var keyData instances.RepetitionData
var configVal cty.Value
_, providerSchema, err := getProvider(ctx, n.ResolvedProvider)
_, providerSchema, err := n.getProvider(ctx, n.ResolvedProvider)
if err != nil {
return nil, nil, keyData, diags.Append(err)
}
@ -1873,7 +1874,7 @@ func (n *NodeAbstractResourceInstance) applyDataSource(ctx EvalContext, planned
var diags tfdiags.Diagnostics
var keyData instances.RepetitionData
_, providerSchema, err := getProvider(ctx, n.ResolvedProvider)
_, providerSchema, err := n.getProvider(ctx, n.ResolvedProvider)
if err != nil {
return nil, keyData, diags.Append(err)
}
@ -2251,7 +2252,7 @@ func (n *NodeAbstractResourceInstance) apply(
return state, diags
}
provider, providerSchema, err := getProvider(ctx, n.ResolvedProvider)
provider, providerSchema, err := n.getProvider(ctx, n.ResolvedProvider)
if err != nil {
return nil, diags.Append(err)
}
@ -2340,6 +2341,7 @@ func (n *NodeAbstractResourceInstance) apply(
PlannedPrivate: change.Private,
ProviderMeta: metaConfigVal,
})
applyDiags := resp.Diagnostics
if applyConfig != nil {
applyDiags = applyDiags.InConfigBody(applyConfig.Config, n.Addr.String())
@ -2569,3 +2571,30 @@ func resourceInstancePrevRunAddr(ctx EvalContext, currentAddr addrs.AbsResourceI
table := ctx.MoveResults()
return table.OldAddr(currentAddr)
}
func (n *NodeAbstractResourceInstance) getProvider(ctx EvalContext, addr addrs.AbsProviderConfig) (providers.Interface, providers.ProviderSchema, error) {
return n.getProviderWithPlannedChange(ctx, addr, nil)
}
func (n *NodeAbstractResourceInstance) getProviderWithPlannedChange(ctx EvalContext, addr addrs.AbsProviderConfig, plannedChange *plans.ResourceInstanceChange) (providers.Interface, providers.ProviderSchema, error) {
underlyingProvider, schema, err := getProvider(ctx, addr)
if err != nil {
return nil, providers.ProviderSchema{}, err
}
if n.Config == nil || !n.Config.IsOverridden {
return underlyingProvider, schema, nil
}
providerForTest := providerForTest{
internal: underlyingProvider,
schema: schema,
overrideValues: n.Config.OverrideValues,
}
if plannedChange != nil {
providerForTest.plannedChange = &plannedChange.After
}
return providerForTest, schema, nil
}

View File

@ -0,0 +1,131 @@
// Copyright (c) The OpenTofu Authors
// SPDX-License-Identifier: MPL-2.0
// Copyright (c) 2023 HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package tofu
import (
"github.com/opentofu/opentofu/internal/addrs"
"github.com/opentofu/opentofu/internal/configs/hcl2shim"
"github.com/opentofu/opentofu/internal/providers"
"github.com/zclconf/go-cty/cty"
)
var _ providers.Interface = providerForTest{}
// providerForTest is a wrapper around real provider to allow certain resources to be overridden
// for testing framework. Currently, it's used in NodeAbstractResourceInstance only in a format
// of one time use. It handles overrideValues and plannedChange for a single resource instance
// (i.e. by certain address).
// TODO: providerForTest should be extended to handle mock providers implementation with per-type
// mocking. It will allow providerForTest to be used for both overrides and full mocking.
// In such scenario, overrideValues should be extended to handle per-type values and plannedChange
// should contain per PlanResourceChangeRequest cache to produce the same plan result
// for the same PlanResourceChangeRequest.
type providerForTest struct {
// It's not embedded to make it safer to extend providers.Interface
// without silently breaking providerForTest functionality.
internal providers.Interface
schema providers.ProviderSchema
overrideValues map[string]cty.Value
plannedChange *cty.Value
}
func (p providerForTest) ReadResource(r providers.ReadResourceRequest) providers.ReadResourceResponse {
var resp providers.ReadResourceResponse
resSchema, _ := p.schema.SchemaForResourceType(addrs.ManagedResourceMode, r.TypeName)
resp.NewState, resp.Diagnostics = hcl2shim.ComposeMockValueBySchema(resSchema, r.ProviderMeta, p.overrideValues)
return resp
}
func (p providerForTest) PlanResourceChange(r providers.PlanResourceChangeRequest) providers.PlanResourceChangeResponse {
if r.Config.IsNull() {
return providers.PlanResourceChangeResponse{
PlannedState: r.ProposedNewState, // null
}
}
if p.plannedChange != nil {
return providers.PlanResourceChangeResponse{
PlannedState: *p.plannedChange,
}
}
resSchema, _ := p.schema.SchemaForResourceType(addrs.ManagedResourceMode, r.TypeName)
var resp providers.PlanResourceChangeResponse
resp.PlannedState, resp.Diagnostics = hcl2shim.ComposeMockValueBySchema(resSchema, r.Config, p.overrideValues)
return resp
}
func (p providerForTest) ApplyResourceChange(r providers.ApplyResourceChangeRequest) providers.ApplyResourceChangeResponse {
return providers.ApplyResourceChangeResponse{
NewState: r.PlannedState,
}
}
func (p providerForTest) ReadDataSource(r providers.ReadDataSourceRequest) providers.ReadDataSourceResponse {
resSchema, _ := p.schema.SchemaForResourceType(addrs.DataResourceMode, r.TypeName)
var resp providers.ReadDataSourceResponse
resp.State, resp.Diagnostics = hcl2shim.ComposeMockValueBySchema(resSchema, r.Config, p.overrideValues)
return resp
}
// Calling the internal provider ensures providerForTest has the same behaviour as if
// it wasn't overridden. Some of these functions should be changed in the future to
// support mock_provider (e.g. ConfigureProvider should do nothing), mock_resource and
// mock_data. The only exception is ImportResourceState, which panics if called via providerForTest
// because importing is not supported in testing framework.
func (p providerForTest) GetProviderSchema() providers.GetProviderSchemaResponse {
return p.internal.GetProviderSchema()
}
func (p providerForTest) ValidateProviderConfig(r providers.ValidateProviderConfigRequest) providers.ValidateProviderConfigResponse {
return p.internal.ValidateProviderConfig(r)
}
func (p providerForTest) ValidateResourceConfig(r providers.ValidateResourceConfigRequest) providers.ValidateResourceConfigResponse {
return p.internal.ValidateResourceConfig(r)
}
func (p providerForTest) ValidateDataResourceConfig(r providers.ValidateDataResourceConfigRequest) providers.ValidateDataResourceConfigResponse {
return p.internal.ValidateDataResourceConfig(r)
}
func (p providerForTest) UpgradeResourceState(r providers.UpgradeResourceStateRequest) providers.UpgradeResourceStateResponse {
return p.internal.UpgradeResourceState(r)
}
func (p providerForTest) ConfigureProvider(r providers.ConfigureProviderRequest) providers.ConfigureProviderResponse {
return p.internal.ConfigureProvider(r)
}
func (p providerForTest) Stop() error {
return p.internal.Stop()
}
func (p providerForTest) GetFunctions() providers.GetFunctionsResponse {
return p.internal.GetFunctions()
}
func (p providerForTest) CallFunction(r providers.CallFunctionRequest) providers.CallFunctionResponse {
return p.internal.CallFunction(r)
}
func (p providerForTest) Close() error {
return p.internal.Close()
}
func (p providerForTest) ImportResourceState(providers.ImportResourceStateRequest) providers.ImportResourceStateResponse {
panic("Importing is not supported in testing context. providerForTest must not be used to call ImportResourceState")
}

View File

@ -71,6 +71,12 @@ func (t *ConfigTransformer) transform(g *Graph, config *configs.Config, generate
return nil
}
// If the module is being overridden, do nothing. We don't want to create anything
// from the underlying module.
if config.Module.IsOverridden {
return nil
}
// Add our resources
if err := t.transformSingle(g, config, generateConfigPath); err != nil {
return err

View File

@ -6,6 +6,7 @@
package tofu
import (
"github.com/opentofu/opentofu/internal/configs"
"github.com/opentofu/opentofu/internal/dag"
)
@ -62,11 +63,15 @@ func (n graphNodeRoot) Name() string {
}
// CloseRootModuleTransformer is a GraphTransformer that adds a root to the graph.
type CloseRootModuleTransformer struct{}
type CloseRootModuleTransformer struct {
RootConfig *configs.Config
}
func (t *CloseRootModuleTransformer) Transform(g *Graph) error {
// close the root module
closeRoot := &nodeCloseModule{}
closeRoot := &nodeCloseModule{
RootConfig: t.RootConfig,
}
g.Add(closeRoot)
// since this is closing the root module, make it depend on everything in

View File

@ -0,0 +1,13 @@
data "local_file" "bucket_name" {
filename = "bucket_name.txt"
}
output "name" {
value = data.local_file.bucket_name.content
}
output "tags" {
value = {
Environment = "Dev"
}
}

View File

@ -0,0 +1,12 @@
module "bucket_meta" {
source = "./bucket_meta"
}
provider "aws" {
region = "us-east-2"
}
resource "aws_s3_bucket" "test" {
bucket = module.bucket_meta.name
tags = module.bucket_meta.tags
}

View File

@ -0,0 +1,32 @@
// All the module configuration will be ignored for this
// module call. Instead, the `outputs` object will be used
// to populate module outputs.
override_module {
target = module.bucket_meta
outputs = {
name = "test"
tags = {
Environment = "Test Env"
}
}
}
// Test if the bucket name is correctly passed to the aws_s3_bucket
// resource from the module call.
run "test" {
// S3 bucket will not be created in AWS for this run,
// but it's available to use in both tests and configuration.
override_resource {
target = aws_s3_bucket.test
}
assert {
condition = aws_s3_bucket.test.bucket == "test"
error_message = "Incorrect bucket name: ${aws_s3_bucket.test.bucket}"
}
assert {
condition = aws_s3_bucket.test.tags["Environment"] == "Test Env"
error_message = "Incorrect `Environment` tag: ${aws_s3_bucket.test.tags["Environment"]}"
}
}

View File

@ -0,0 +1,11 @@
data "local_file" "bucket_name" {
filename = "bucket_name.txt"
}
provider "aws" {
region = "us-east-2"
}
resource "aws_s3_bucket" "test" {
bucket = data.local_file.bucket_name.content
}

View File

@ -0,0 +1,25 @@
// This data source will not be called for any run
// in this `.tftest.hcl` file. Instead, `values` object
// will be used to populate `content` attribute. Other
// attributes and blocks will be automatically generated.
override_data {
target = data.local_file.bucket_name
values = {
content = "test"
}
}
// Test if the bucket name is correctly passed to the aws_s3_bucket
// resource from the local file.
run "test" {
// S3 bucket will not be created in AWS for this run,
// but it's available to use in both tests and configuration.
override_resource {
target = aws_s3_bucket.test
}
assert {
condition = aws_s3_bucket.test.bucket == "test"
error_message = "Incorrect bucket name: ${aws_s3_bucket.test.bucket}"
}
}

View File

@ -27,6 +27,11 @@ import ExpectFailureVariablesMain from '!!raw-loader!./examples/expect_failures_
import ExpectFailureVariablesTest from '!!raw-loader!./examples/expect_failures_variables/main.tftest.hcl'
import ExpectFailureResourcesMain from '!!raw-loader!./examples/expect_failures_resources/main.tf'
import ExpectFailureResourcesTest from '!!raw-loader!./examples/expect_failures_resources/main.tftest.hcl'
import OverrideResourceMain from '!!raw-loader!./examples/override_resource/main.tf'
import OverrideResourceTest from '!!raw-loader!./examples/override_resource/main.tftest.hcl'
import OverrideModuleMain from '!!raw-loader!./examples/override_module/main.tf'
import OverrideModuleTest from '!!raw-loader!./examples/override_module/main.tftest.hcl'
import OverrideModuleBucketMeta from '!!raw-loader!./examples/override_module/bucket_meta/main.tf'
# Command: test
@ -127,6 +132,9 @@ A test file consists of:
* A **[`variables` block](#the-variables-and-runvariables-blocks)** (optional): define variables for all tests in the
current file.
* The **[`provider` blocks](#the-providers-block)** (optional): define the providers to be used for the tests.
* The **[`override_resource` block](#the-override_resource-and-override_data-blocks)** (optional): defines a resource to be overridden.
* The **[`override_data` block](#the-override_resource-and-override_data-blocks)** (optional): defines a data source to be overridden.
* The **[`override_module` block](#the-override_module-block)** (optional): defines a module call to be overridden.
### The `run` block
@ -144,6 +152,9 @@ A `run` block consists of the following elements:
| [`command`](#the-runcommand-setting-and-the-runplan_options-block) | `plan` or `apply` | Defines the command which OpenTofu will execute, `plan` or `apply`. Defaults to `apply`. |
| [`plan_options`](#the-runcommand-setting-and-the-runplan_options-block) | block | Options for the `plan` or `apply` operation. |
| [`providers`](#the-providers-block) | object | Aliases for providers. |
| [`override_resource`](#the-override_resource-and-override_data-blocks) | block | Defines a resource to be overridden for the run. |
| [`override_data`](#the-override_resource-and-override_data-blocks) | block | Defines a data source to be overridden for the run. |
| [`override_module`](#the-override_module-block) | block | Defines a module call to be overridden for the run. |
### The `run.assert` block
@ -374,3 +385,94 @@ of the file.
<CodeBlock language={"hcl"}>{ProviderAliasMain}</CodeBlock>
</TabItem>
</Tabs>
### The `override_resource` and `override_data` blocks
In some cases you may want to test your infrastructure with certain resources or data sources being overridden.
You can use the `override_resource` or `override_data` blocks to skip creation and retrieval of these resources or data sources using the real provider.
Instead, OpenTofu will automatically generate all computed attributes and blocks to be used in tests.
These blocks consist of the following elements:
| Name | Type | Description |
|:--------:|:------------:|------------------------------------------------------------------------------------------------------------------------|
| target | reference | Required. Address of the target resource or data source to be overridden. |
| values | object | Custom values for computed attributes and blocks to be used instead of automatically generated. |
You can use `override_resource` or `override_data` blocks for the whole test file or inside a single `run` block. The latter takes precedence if both specified for the same `target`.
In the example below, we test if the bucket name is correctly passed to the resource without actually creating it:
<Tabs>
<TabItem value={"test"} label={"main.tftest.hcl"} default>
<CodeBlock language={"hcl"}>{OverrideResourceTest}</CodeBlock>
</TabItem>
<TabItem value={"main"} label={"main.tf"}>
<CodeBlock language={"hcl"}>{OverrideResourceMain}</CodeBlock>
</TabItem>
</Tabs>
:::warning Limitation
You cannot use `override_resource` or `override_data` with a single instance of a resource or data source.
Each instance of a resource or data source must be overridden.
:::
#### Automatically generated values
Overriding resources and data sources requires OpenTofu to automatically generate computed attributes without calling respective providers.
When generating these values, OpenTofu cannot follow custom provider logic, so it uses simple rules based on value type:
| Attribute type | Generated value |
|:------------------:|---------------------------------------------------------------------|
| number | `0` |
| bool | `false` |
| string | A random alpha-numeric string. |
| list | An empty list. |
| map | An empty map. |
| set | An empty set. |
| object | An object with its fields populated by the same logic recursevily. |
| tuple | An empty tuple. |
:::tip Note
You can set custom values to use instead of automatically generated ones via `values` field in both `override_resource` and `override_data` blocks.
Keep in mind, it's only possible for computed attributes and configuration values cannot be changed.
:::
### The `override_module` block
In some cases you may want to test your infrastructure with certain module calls being overridden.
You can use the `override_module` block to ignore all the configuration provided by called module.
In this case, OpenTofu will use custom values specified in the `override_module` block as module outputs.
The block consist of the following elements:
| Name | Type | Description |
|:--------:|:------------:|------------------------------------------------------------------------------------------------------------------------|
| target | reference | Required. Address of the target module call to be overridden. |
| outputs | object | Values to be used as module call outputs. If an output is not specified, OpenTofu will set it to `null` by default. |
You can use `override_module` block for the whole test file or inside a single `run` block. The latter takes precedence if both specified for the same `target`.
In the example below, we test if the bucket name is correctly passed from the module without actually calling it:
<Tabs>
<TabItem value={"test"} label={"main.tftest.hcl"} default>
<CodeBlock language={"hcl"}>{OverrideModuleTest}</CodeBlock>
</TabItem>
<TabItem value={"main"} label={"main.tf"}>
<CodeBlock language={"hcl"}>{OverrideModuleMain}</CodeBlock>
</TabItem>
<TabItem value={"bucket_meta_main"} label={"bucket_meta/main.tf"}>
<CodeBlock language={"hcl"}>{OverrideModuleBucketMeta}</CodeBlock>
</TabItem>
</Tabs>
:::warning Limitation
You cannot use `override_module` with a single instance of a module call. Each instance of a module call must be overridden.
:::