opentofu/internal/configs/config_test.go

833 lines
29 KiB
Go
Raw Normal View History

// Copyright (c) The OpenTofu Authors
// SPDX-License-Identifier: MPL-2.0
// Copyright (c) 2023 HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
various: helpers for collecting necessary provider types Since schemas are required to interpret provider, resource, and provisioner attributes in configs, states, and plans, these helpers intend to make it easier to gather up the the necessary provider types in order to preload all of the needed schemas before beginning further processing. Config.ProviderTypes returns directly the list of provider types, since at this level further detail is not useful: we've not yet run the provider allocation algorithm, and so the only thing we can reliably extract here is provider types themselves. State.ProviderAddrs and Plan.ProviderAddrs each return a list of absolute provider addresses, which can then be turned into a list of provider types using the new helper providers.AddressedTypesAbs. Since we're already using configs.Config throughout core, this also updates the terraform.LoadSchemas helper to use Config.ProviderTypes to find the necessary providers, rather than implementing its own discovery logic. states.State is not yet plumbed in, so we cannot yet use State.ProviderAddrs to deal with the state but there's a TODO comment to remind us to update that in a later commit when we swap out terraform.State for states.State. A later commit will probably refactor this further so that we can easily obtain schema for the providers needed to interpret a plan too, but that is deferred here because further work is required to make core work with the new plan types first. At that point, terraform.LoadSchemas may become providers.LoadSchemas with a different interface that just accepts lists of provider and provisioner names that have been gathered by the caller using these new helpers.
2018-06-21 19:39:27 -05:00
package configs
import (
"bytes"
"fmt"
"os"
"path/filepath"
"strings"
various: helpers for collecting necessary provider types Since schemas are required to interpret provider, resource, and provisioner attributes in configs, states, and plans, these helpers intend to make it easier to gather up the the necessary provider types in order to preload all of the needed schemas before beginning further processing. Config.ProviderTypes returns directly the list of provider types, since at this level further detail is not useful: we've not yet run the provider allocation algorithm, and so the only thing we can reliably extract here is provider types themselves. State.ProviderAddrs and Plan.ProviderAddrs each return a list of absolute provider addresses, which can then be turned into a list of provider types using the new helper providers.AddressedTypesAbs. Since we're already using configs.Config throughout core, this also updates the terraform.LoadSchemas helper to use Config.ProviderTypes to find the necessary providers, rather than implementing its own discovery logic. states.State is not yet plumbed in, so we cannot yet use State.ProviderAddrs to deal with the state but there's a TODO comment to remind us to update that in a later commit when we swap out terraform.State for states.State. A later commit will probably refactor this further so that we can easily obtain schema for the providers needed to interpret a plan too, but that is deferred here because further work is required to make core work with the new plan types first. At that point, terraform.LoadSchemas may become providers.LoadSchemas with a different interface that just accepts lists of provider and provisioner names that have been gathered by the caller using these new helpers.
2018-06-21 19:39:27 -05:00
"testing"
"github.com/go-test/deep"
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 14:04:48 -05:00
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/hclparse"
"github.com/zclconf/go-cty/cty"
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
version "github.com/hashicorp/go-version"
"github.com/hashicorp/hcl/v2/hclsyntax"
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 14:04:48 -05:00
svchost "github.com/hashicorp/terraform-svchost"
"github.com/opentofu/opentofu/internal/addrs"
"github.com/opentofu/opentofu/internal/depsfile"
"github.com/opentofu/opentofu/internal/getproviders"
various: helpers for collecting necessary provider types Since schemas are required to interpret provider, resource, and provisioner attributes in configs, states, and plans, these helpers intend to make it easier to gather up the the necessary provider types in order to preload all of the needed schemas before beginning further processing. Config.ProviderTypes returns directly the list of provider types, since at this level further detail is not useful: we've not yet run the provider allocation algorithm, and so the only thing we can reliably extract here is provider types themselves. State.ProviderAddrs and Plan.ProviderAddrs each return a list of absolute provider addresses, which can then be turned into a list of provider types using the new helper providers.AddressedTypesAbs. Since we're already using configs.Config throughout core, this also updates the terraform.LoadSchemas helper to use Config.ProviderTypes to find the necessary providers, rather than implementing its own discovery logic. states.State is not yet plumbed in, so we cannot yet use State.ProviderAddrs to deal with the state but there's a TODO comment to remind us to update that in a later commit when we swap out terraform.State for states.State. A later commit will probably refactor this further so that we can easily obtain schema for the providers needed to interpret a plan too, but that is deferred here because further work is required to make core work with the new plan types first. At that point, terraform.LoadSchemas may become providers.LoadSchemas with a different interface that just accepts lists of provider and provisioner names that have been gathered by the caller using these new helpers.
2018-06-21 19:39:27 -05:00
)
func TestConfigProviderTypes(t *testing.T) {
// nil cfg should return an empty map
got := NewEmptyConfig().ProviderTypes()
if len(got) != 0 {
t.Fatal("expected empty result from empty config")
}
cfg, diags := testModuleConfigFromFile("testdata/valid-files/providers-explicit-implied.tf")
various: helpers for collecting necessary provider types Since schemas are required to interpret provider, resource, and provisioner attributes in configs, states, and plans, these helpers intend to make it easier to gather up the the necessary provider types in order to preload all of the needed schemas before beginning further processing. Config.ProviderTypes returns directly the list of provider types, since at this level further detail is not useful: we've not yet run the provider allocation algorithm, and so the only thing we can reliably extract here is provider types themselves. State.ProviderAddrs and Plan.ProviderAddrs each return a list of absolute provider addresses, which can then be turned into a list of provider types using the new helper providers.AddressedTypesAbs. Since we're already using configs.Config throughout core, this also updates the terraform.LoadSchemas helper to use Config.ProviderTypes to find the necessary providers, rather than implementing its own discovery logic. states.State is not yet plumbed in, so we cannot yet use State.ProviderAddrs to deal with the state but there's a TODO comment to remind us to update that in a later commit when we swap out terraform.State for states.State. A later commit will probably refactor this further so that we can easily obtain schema for the providers needed to interpret a plan too, but that is deferred here because further work is required to make core work with the new plan types first. At that point, terraform.LoadSchemas may become providers.LoadSchemas with a different interface that just accepts lists of provider and provisioner names that have been gathered by the caller using these new helpers.
2018-06-21 19:39:27 -05:00
if diags.HasErrors() {
t.Fatal(diags.Error())
}
got = cfg.ProviderTypes()
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
want := []addrs.Provider{
addrs.NewDefaultProvider("aws"),
addrs.NewDefaultProvider("local"),
addrs.NewDefaultProvider("null"),
addrs.NewDefaultProvider("template"),
addrs.NewDefaultProvider("test"),
various: helpers for collecting necessary provider types Since schemas are required to interpret provider, resource, and provisioner attributes in configs, states, and plans, these helpers intend to make it easier to gather up the the necessary provider types in order to preload all of the needed schemas before beginning further processing. Config.ProviderTypes returns directly the list of provider types, since at this level further detail is not useful: we've not yet run the provider allocation algorithm, and so the only thing we can reliably extract here is provider types themselves. State.ProviderAddrs and Plan.ProviderAddrs each return a list of absolute provider addresses, which can then be turned into a list of provider types using the new helper providers.AddressedTypesAbs. Since we're already using configs.Config throughout core, this also updates the terraform.LoadSchemas helper to use Config.ProviderTypes to find the necessary providers, rather than implementing its own discovery logic. states.State is not yet plumbed in, so we cannot yet use State.ProviderAddrs to deal with the state but there's a TODO comment to remind us to update that in a later commit when we swap out terraform.State for states.State. A later commit will probably refactor this further so that we can easily obtain schema for the providers needed to interpret a plan too, but that is deferred here because further work is required to make core work with the new plan types first. At that point, terraform.LoadSchemas may become providers.LoadSchemas with a different interface that just accepts lists of provider and provisioner names that have been gathered by the caller using these new helpers.
2018-06-21 19:39:27 -05:00
}
for _, problem := range deep.Equal(got, want) {
t.Error(problem)
}
}
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
func TestConfigProviderTypes_nested(t *testing.T) {
// basic test with a nil config
c := NewEmptyConfig()
got := c.ProviderTypes()
if len(got) != 0 {
t.Fatalf("wrong result!\ngot: %#v\nwant: nil\n", got)
}
// config with two provider sources, and one implicit (default) provider
cfg, diags := testNestedModuleConfigFromDir(t, "testdata/valid-modules/nested-providers-fqns")
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
if diags.HasErrors() {
t.Fatal(diags.Error())
}
got = cfg.ProviderTypes()
want := []addrs.Provider{
addrs.NewProvider(addrs.DefaultProviderRegistryHost, "bar", "test"),
addrs.NewProvider(addrs.DefaultProviderRegistryHost, "foo", "test"),
addrs.NewDefaultProvider("test"),
}
for _, problem := range deep.Equal(got, want) {
t.Error(problem)
}
}
func TestConfigResolveAbsProviderAddr(t *testing.T) {
cfg, diags := testModuleConfigFromDir("testdata/providers-explicit-fqn")
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
if diags.HasErrors() {
t.Fatal(diags.Error())
}
t.Run("already absolute", func(t *testing.T) {
addr := addrs.AbsProviderConfig{
Module: addrs.RootModule,
2020-10-05 07:33:49 -05:00
Provider: addrs.NewDefaultProvider("test"),
Alias: "boop",
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
}
got := cfg.ResolveAbsProviderAddr(addr, addrs.RootModule)
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
if got, want := got.String(), addr.String(); got != want {
t.Errorf("wrong result\ngot: %s\nwant: %s", got, want)
}
})
t.Run("local, implied mapping", func(t *testing.T) {
addr := addrs.LocalProviderConfig{
LocalName: "implied",
Alias: "boop",
}
got := cfg.ResolveAbsProviderAddr(addr, addrs.RootModule)
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
want := addrs.AbsProviderConfig{
Module: addrs.RootModule,
Provider: addrs.NewDefaultProvider("implied"),
Alias: "boop",
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
}
if got, want := got.String(), want.String(); got != want {
t.Errorf("wrong result\ngot: %s\nwant: %s", got, want)
}
})
t.Run("local, explicit mapping", func(t *testing.T) {
addr := addrs.LocalProviderConfig{
addrs: Stronger validation and normalization of provider namespace/type The provider FQN is becoming our primary identifier for a provider, so it's important that we are clear about the equality rules for these addresses and what characters are valid within them. We previously had a basic regex permitting ASCII letters and digits for validation and no normalization at all. We need to do at least case folding and UTF-8 normalization because these names will appear in file and directory names in case-insensitive filesystems and in repository names such as on GitHub. Since we're already using DNS-style normalization and validation rules for the hostname part, rather than defining an entirely new set of rules here we'll just treat the provider namespace and type as if they were single labels in a DNS name. Aside from some internal consistency, that also works out nicely because systems like GitHub use organization and repository names as part of hostnames (e.g. with GitHub Pages) and so tend to apply comparable constraints themselves. This introduces the possibility of names containing letters from alphabets other than the latin alphabet, and for latin letters with diacritics. That's consistent with our introduction of similar support for identifiers in the language in Terraform 0.12, and is intended to be more friendly to Terraform users throughout the world that might prefer to name their products using a different alphabet. This is also a further justification for using the DNS normalization rules: modern companies tend to choose product names that make good domain names, and now such names will be usable as Terraform provider names too.
2020-02-14 20:10:03 -06:00
LocalName: "foo-test", // this is explicitly set in the config
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
Alias: "boop",
}
got := cfg.ResolveAbsProviderAddr(addr, addrs.RootModule)
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
want := addrs.AbsProviderConfig{
Module: addrs.RootModule,
Provider: addrs.NewProvider(addrs.DefaultProviderRegistryHost, "foo", "test"),
Alias: "boop",
Initial steps towards AbsProviderConfig/LocalProviderConfig separation (#23978) * Introduce "Local" terminology for non-absolute provider config addresses In a future change AbsProviderConfig and LocalProviderConfig are going to become two entirely distinct types, rather than Abs embedding Local as written here. This naming change is in preparation for that subsequent work, which will also include introducing a new "ProviderConfig" type that is an interface that AbsProviderConfig and LocalProviderConfig both implement. This is intended to be largely just a naming change to get started, so we can deal with all of the messy renaming. However, this did also require a slight change in modeling where the Resource.DefaultProviderConfig method has become Resource.DefaultProvider returning a Provider address directly, because this method doesn't have enough information to construct a true and accurate LocalProviderConfig -- it would need to refer to the configuration to know what this module is calling the provider it has selected. In order to leave a trail to follow for subsequent work, all of the changes here are intended to ensure that remaining work will become obvious via compile-time errors when all of the following changes happen: - The concept of "legacy" provider addresses is removed from the addrs package, including removing addrs.NewLegacyProvider and addrs.Provider.LegacyString. - addrs.AbsProviderConfig stops having addrs.LocalProviderConfig embedded in it and has an addrs.Provider and a string alias directly instead. - The provider-schema-handling parts of Terraform core are updated to work with addrs.Provider to identify providers, rather than legacy strings. In particular, there are still several codepaths here making legacy provider address assumptions (in order to limit the scope of this change) but I've made sure each one is doing something that relies on at least one of the above changes not having been made yet. * addrs: ProviderConfig interface In a (very) few special situations in the main "terraform" package we need to make runtime decisions about whether a provider config is absolute or local. We currently do that by exploiting the fact that AbsProviderConfig has LocalProviderConfig nested inside of it and so in the local case we can just ignore the wrapping AbsProviderConfig and use the embedded value. In a future change we'll be moving away from that embedding and making these two types distinct in order to represent that mapping between them requires consulting a lookup table in the configuration, and so here we introduce a new interface type ProviderConfig that can represent either AbsProviderConfig or LocalProviderConfig decided dynamically at runtime. This also includes the Config.ResolveAbsProviderAddr method that will eventually be responsible for that local-to-absolute translation, so that callers with access to the configuration can normalize to an addrs.AbsProviderConfig given a non-nil addrs.ProviderConfig. That's currently unused because existing callers are still relying on the simplistic structural transform, but we'll switch them over in a later commit. * rename LocalType to LocalName Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
2020-01-31 07:23:07 -06:00
}
if got, want := got.String(), want.String(); got != want {
t.Errorf("wrong result\ngot: %s\nwant: %s", got, want)
}
})
}
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 14:04:48 -05:00
func TestConfigProviderRequirements(t *testing.T) {
cfg, diags := testNestedModuleConfigFromDir(t, "testdata/provider-reqs")
// TODO: Version Constraint Deprecation.
// Once we've removed the version argument from provider configuration
// blocks, this can go back to expected 0 diagnostics.
// assertNoDiagnostics(t, diags)
assertDiagnosticCount(t, diags, 1)
assertDiagnosticSummary(t, diags, "Version constraints inside provider configuration blocks are deprecated")
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 14:04:48 -05:00
tlsProvider := addrs.NewProvider(
addrs.DefaultProviderRegistryHost,
"hashicorp", "tls",
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 14:04:48 -05:00
)
happycloudProvider := addrs.NewProvider(
svchost.Hostname("tf.example.com"),
"awesomecorp", "happycloud",
)
nullProvider := addrs.NewDefaultProvider("null")
randomProvider := addrs.NewDefaultProvider("random")
impliedProvider := addrs.NewDefaultProvider("implied")
terraformProvider := addrs.NewBuiltInProvider("terraform")
configuredProvider := addrs.NewDefaultProvider("configured")
grandchildProvider := addrs.NewDefaultProvider("grandchild")
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 14:04:48 -05:00
got, diags := cfg.ProviderRequirements()
assertNoDiagnostics(t, diags)
want := getproviders.Requirements{
// the nullProvider constraints from the two modules are merged
nullProvider: getproviders.MustParseVersionConstraints("~> 2.0.0, 2.0.1"),
randomProvider: getproviders.MustParseVersionConstraints("~> 1.2.0"),
tlsProvider: getproviders.MustParseVersionConstraints("~> 3.0"),
configuredProvider: getproviders.MustParseVersionConstraints("~> 1.4"),
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 14:04:48 -05:00
impliedProvider: nil,
happycloudProvider: nil,
terraformProvider: nil,
grandchildProvider: nil,
internal/getproviders: A new shared model for provider requirements We've been using the models from the "moduledeps" package to represent our provider dependencies everywhere since the idea of provider dependencies was introduced in Terraform 0.10, but that model is not convenient to use for any use-case other than the "terraform providers" command that needs individual-module-level detail. To make things easier for new codepaths working with the new-style provider installer, here we introduce a new model type getproviders.Requirements which is based on the type the new installer was already taking as its input. We have new methods in the states, configs, and earlyconfig packages to produce values of this type, and a helper to merge Requirements together so we can combine config-derived and state-derived requirements together during installation. The advantage of this new model over the moduledeps one is that all of recursive module walking is done up front and we produce a simple, flat structure that is more convenient for the main use-cases of selecting providers for installation and then finding providers in the local cache to use them for other operations. This new model is _not_ suitable for implementing "terraform providers" because it does not retain module-specific requirement details. Therefore we will likely keep using moduledeps for "terraform providers" for now, and then possibly at a later time consider specializing the moduledeps logic for only what "terraform providers" needs, because it seems to be the only use-case that needs to retain that level of detail.
2020-03-26 14:04:48 -05:00
}
if diff := cmp.Diff(want, got); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
}
func TestConfigProviderRequirementsInclTests(t *testing.T) {
cfg, diags := testNestedModuleConfigFromDirWithTests(t, "testdata/provider-reqs-with-tests")
// TODO: Version Constraint Deprecation.
// Once we've removed the version argument from provider configuration
// blocks, this can go back to expected 0 diagnostics.
// assertNoDiagnostics(t, diags)
assertDiagnosticCount(t, diags, 1)
assertDiagnosticSummary(t, diags, "Version constraints inside provider configuration blocks are deprecated")
tlsProvider := addrs.NewProvider(
addrs.DefaultProviderRegistryHost,
"hashicorp", "tls",
)
nullProvider := addrs.NewDefaultProvider("null")
randomProvider := addrs.NewDefaultProvider("random")
impliedProvider := addrs.NewDefaultProvider("implied")
terraformProvider := addrs.NewBuiltInProvider("terraform")
configuredProvider := addrs.NewDefaultProvider("configured")
got, diags := cfg.ProviderRequirements()
assertNoDiagnostics(t, diags)
want := getproviders.Requirements{
// the nullProvider constraints from the two modules are merged
nullProvider: getproviders.MustParseVersionConstraints("~> 2.0.0"),
randomProvider: getproviders.MustParseVersionConstraints("~> 1.2.0"),
tlsProvider: getproviders.MustParseVersionConstraints("~> 3.0"),
configuredProvider: getproviders.MustParseVersionConstraints("~> 1.4"),
impliedProvider: nil,
terraformProvider: nil,
}
if diff := cmp.Diff(want, got); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
}
func TestConfigProviderRequirementsDuplicate(t *testing.T) {
_, diags := testNestedModuleConfigFromDir(t, "testdata/duplicate-local-name")
assertDiagnosticCount(t, diags, 3)
assertDiagnosticSummary(t, diags, "Duplicate required provider")
}
func TestConfigProviderRequirementsShallow(t *testing.T) {
cfg, diags := testNestedModuleConfigFromDir(t, "testdata/provider-reqs")
// TODO: Version Constraint Deprecation.
// Once we've removed the version argument from provider configuration
// blocks, this can go back to expected 0 diagnostics.
// assertNoDiagnostics(t, diags)
assertDiagnosticCount(t, diags, 1)
assertDiagnosticSummary(t, diags, "Version constraints inside provider configuration blocks are deprecated")
tlsProvider := addrs.NewProvider(
addrs.DefaultProviderRegistryHost,
"hashicorp", "tls",
)
nullProvider := addrs.NewDefaultProvider("null")
randomProvider := addrs.NewDefaultProvider("random")
impliedProvider := addrs.NewDefaultProvider("implied")
terraformProvider := addrs.NewBuiltInProvider("terraform")
configuredProvider := addrs.NewDefaultProvider("configured")
got, diags := cfg.ProviderRequirementsShallow()
assertNoDiagnostics(t, diags)
want := getproviders.Requirements{
// the nullProvider constraint is only from the root module
nullProvider: getproviders.MustParseVersionConstraints("~> 2.0.0"),
randomProvider: getproviders.MustParseVersionConstraints("~> 1.2.0"),
tlsProvider: getproviders.MustParseVersionConstraints("~> 3.0"),
configuredProvider: getproviders.MustParseVersionConstraints("~> 1.4"),
impliedProvider: nil,
terraformProvider: nil,
}
if diff := cmp.Diff(want, got); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
}
func TestConfigProviderRequirementsShallowInclTests(t *testing.T) {
cfg, diags := testNestedModuleConfigFromDirWithTests(t, "testdata/provider-reqs-with-tests")
// TODO: Version Constraint Deprecation.
// Once we've removed the version argument from provider configuration
// blocks, this can go back to expected 0 diagnostics.
// assertNoDiagnostics(t, diags)
assertDiagnosticCount(t, diags, 1)
assertDiagnosticSummary(t, diags, "Version constraints inside provider configuration blocks are deprecated")
tlsProvider := addrs.NewProvider(
addrs.DefaultProviderRegistryHost,
"hashicorp", "tls",
)
impliedProvider := addrs.NewDefaultProvider("implied")
terraformProvider := addrs.NewBuiltInProvider("terraform")
configuredProvider := addrs.NewDefaultProvider("configured")
got, diags := cfg.ProviderRequirementsShallow()
assertNoDiagnostics(t, diags)
want := getproviders.Requirements{
tlsProvider: getproviders.MustParseVersionConstraints("~> 3.0"),
configuredProvider: getproviders.MustParseVersionConstraints("~> 1.4"),
impliedProvider: nil,
terraformProvider: nil,
}
if diff := cmp.Diff(want, got); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
}
func TestConfigProviderRequirementsByModule(t *testing.T) {
cfg, diags := testNestedModuleConfigFromDir(t, "testdata/provider-reqs")
// TODO: Version Constraint Deprecation.
// Once we've removed the version argument from provider configuration
// blocks, this can go back to expected 0 diagnostics.
// assertNoDiagnostics(t, diags)
assertDiagnosticCount(t, diags, 1)
assertDiagnosticSummary(t, diags, "Version constraints inside provider configuration blocks are deprecated")
tlsProvider := addrs.NewProvider(
addrs.DefaultProviderRegistryHost,
"hashicorp", "tls",
)
happycloudProvider := addrs.NewProvider(
svchost.Hostname("tf.example.com"),
"awesomecorp", "happycloud",
)
nullProvider := addrs.NewDefaultProvider("null")
randomProvider := addrs.NewDefaultProvider("random")
impliedProvider := addrs.NewDefaultProvider("implied")
terraformProvider := addrs.NewBuiltInProvider("terraform")
configuredProvider := addrs.NewDefaultProvider("configured")
grandchildProvider := addrs.NewDefaultProvider("grandchild")
got, diags := cfg.ProviderRequirementsByModule()
assertNoDiagnostics(t, diags)
want := &ModuleRequirements{
Name: "",
Refactoring of module source addresses and module installation It's been a long while since we gave close attention to the codepaths for module source address parsing and external module package installation. Due to their age, these codepaths often diverged from our modern practices such as representing address types in the addrs package, and encapsulating package installation details only in a particular location. In particular, this refactor makes source address parsing a separate step from module installation, which therefore makes the result of that parsing available to other Terraform subsystems which work with the configuration representation objects. This also presented the opportunity to better encapsulate our use of go-getter into a new package "getmodules" (echoing "getproviders"), which is intended to be the only part of Terraform that directly interacts with go-getter. This is largely just a refactor of the existing functionality into a new code organization, but there is one notable change in behavior here: the source address parsing now happens during configuration loading rather than module installation, which may cause errors about invalid addresses to be returned in different situations than before. That counts as backward compatible because we only promise to remain compatible with configurations that are _valid_, which means that they can be initialized, planned, and applied without any errors. This doesn't introduce any new error cases, and instead just makes a pre-existing error case be detected earlier. Our module registry client is still using its own special module address type from registry/regsrc for now, with a small shim from the new addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also rework the registry client to work with the new address type, but this commit is already big enough as it is.
2021-05-27 21:24:59 -05:00
SourceAddr: nil,
SourceDir: "testdata/provider-reqs",
Requirements: getproviders.Requirements{
// Only the root module's version is present here
nullProvider: getproviders.MustParseVersionConstraints("~> 2.0.0"),
randomProvider: getproviders.MustParseVersionConstraints("~> 1.2.0"),
tlsProvider: getproviders.MustParseVersionConstraints("~> 3.0"),
configuredProvider: getproviders.MustParseVersionConstraints("~> 1.4"),
impliedProvider: nil,
terraformProvider: nil,
},
Children: map[string]*ModuleRequirements{
"kinder": {
Name: "kinder",
Refactoring of module source addresses and module installation It's been a long while since we gave close attention to the codepaths for module source address parsing and external module package installation. Due to their age, these codepaths often diverged from our modern practices such as representing address types in the addrs package, and encapsulating package installation details only in a particular location. In particular, this refactor makes source address parsing a separate step from module installation, which therefore makes the result of that parsing available to other Terraform subsystems which work with the configuration representation objects. This also presented the opportunity to better encapsulate our use of go-getter into a new package "getmodules" (echoing "getproviders"), which is intended to be the only part of Terraform that directly interacts with go-getter. This is largely just a refactor of the existing functionality into a new code organization, but there is one notable change in behavior here: the source address parsing now happens during configuration loading rather than module installation, which may cause errors about invalid addresses to be returned in different situations than before. That counts as backward compatible because we only promise to remain compatible with configurations that are _valid_, which means that they can be initialized, planned, and applied without any errors. This doesn't introduce any new error cases, and instead just makes a pre-existing error case be detected earlier. Our module registry client is still using its own special module address type from registry/regsrc for now, with a small shim from the new addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also rework the registry client to work with the new address type, but this commit is already big enough as it is.
2021-05-27 21:24:59 -05:00
SourceAddr: addrs.ModuleSourceLocal("./child"),
SourceDir: filepath.FromSlash("testdata/provider-reqs/child"),
Requirements: getproviders.Requirements{
nullProvider: getproviders.MustParseVersionConstraints("= 2.0.1"),
happycloudProvider: nil,
},
Children: map[string]*ModuleRequirements{
"nested": {
Name: "nested",
Refactoring of module source addresses and module installation It's been a long while since we gave close attention to the codepaths for module source address parsing and external module package installation. Due to their age, these codepaths often diverged from our modern practices such as representing address types in the addrs package, and encapsulating package installation details only in a particular location. In particular, this refactor makes source address parsing a separate step from module installation, which therefore makes the result of that parsing available to other Terraform subsystems which work with the configuration representation objects. This also presented the opportunity to better encapsulate our use of go-getter into a new package "getmodules" (echoing "getproviders"), which is intended to be the only part of Terraform that directly interacts with go-getter. This is largely just a refactor of the existing functionality into a new code organization, but there is one notable change in behavior here: the source address parsing now happens during configuration loading rather than module installation, which may cause errors about invalid addresses to be returned in different situations than before. That counts as backward compatible because we only promise to remain compatible with configurations that are _valid_, which means that they can be initialized, planned, and applied without any errors. This doesn't introduce any new error cases, and instead just makes a pre-existing error case be detected earlier. Our module registry client is still using its own special module address type from registry/regsrc for now, with a small shim from the new addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also rework the registry client to work with the new address type, but this commit is already big enough as it is.
2021-05-27 21:24:59 -05:00
SourceAddr: addrs.ModuleSourceLocal("./grandchild"),
SourceDir: filepath.FromSlash("testdata/provider-reqs/child/grandchild"),
Requirements: getproviders.Requirements{
grandchildProvider: nil,
},
Children: map[string]*ModuleRequirements{},
Tests: make(map[string]*TestFileModuleRequirements),
},
},
Tests: make(map[string]*TestFileModuleRequirements),
},
},
Tests: make(map[string]*TestFileModuleRequirements),
}
ignore := cmpopts.IgnoreUnexported(version.Constraint{}, cty.Value{}, hclsyntax.Body{})
if diff := cmp.Diff(want, got, ignore); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
}
func TestConfigProviderRequirementsByModuleInclTests(t *testing.T) {
cfg, diags := testNestedModuleConfigFromDirWithTests(t, "testdata/provider-reqs-with-tests")
// TODO: Version Constraint Deprecation.
// Once we've removed the version argument from provider configuration
// blocks, this can go back to expected 0 diagnostics.
// assertNoDiagnostics(t, diags)
assertDiagnosticCount(t, diags, 1)
assertDiagnosticSummary(t, diags, "Version constraints inside provider configuration blocks are deprecated")
tlsProvider := addrs.NewProvider(
addrs.DefaultProviderRegistryHost,
"hashicorp", "tls",
)
nullProvider := addrs.NewDefaultProvider("null")
randomProvider := addrs.NewDefaultProvider("random")
impliedProvider := addrs.NewDefaultProvider("implied")
terraformProvider := addrs.NewBuiltInProvider("terraform")
configuredProvider := addrs.NewDefaultProvider("configured")
got, diags := cfg.ProviderRequirementsByModule()
assertNoDiagnostics(t, diags)
want := &ModuleRequirements{
Name: "",
SourceAddr: nil,
SourceDir: "testdata/provider-reqs-with-tests",
Requirements: getproviders.Requirements{
// Only the root module's version is present here
tlsProvider: getproviders.MustParseVersionConstraints("~> 3.0"),
impliedProvider: nil,
terraformProvider: nil,
},
Children: make(map[string]*ModuleRequirements),
Tests: map[string]*TestFileModuleRequirements{
"provider-reqs-root.tftest.hcl": {
Requirements: getproviders.Requirements{
configuredProvider: getproviders.MustParseVersionConstraints("~> 1.4"),
},
Runs: map[string]*ModuleRequirements{
"setup": {
Name: "setup",
SourceAddr: addrs.ModuleSourceLocal("./setup"),
SourceDir: filepath.FromSlash("testdata/provider-reqs-with-tests/setup"),
Requirements: getproviders.Requirements{
nullProvider: getproviders.MustParseVersionConstraints("~> 2.0.0"),
randomProvider: getproviders.MustParseVersionConstraints("~> 1.2.0"),
},
Children: make(map[string]*ModuleRequirements),
Tests: make(map[string]*TestFileModuleRequirements),
},
},
},
},
}
ignore := cmpopts.IgnoreUnexported(version.Constraint{}, cty.Value{}, hclsyntax.Body{})
if diff := cmp.Diff(want, got, ignore); diff != "" {
t.Errorf("wrong result\n%s", diff)
}
}
backend/local: Check dependency lock consistency before any operations In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.
2021-09-29 19:31:43 -05:00
func TestVerifyDependencySelections(t *testing.T) {
cfg, diags := testNestedModuleConfigFromDir(t, "testdata/provider-reqs")
// TODO: Version Constraint Deprecation.
// Once we've removed the version argument from provider configuration
// blocks, this can go back to expected 0 diagnostics.
// assertNoDiagnostics(t, diags)
assertDiagnosticCount(t, diags, 1)
assertDiagnosticSummary(t, diags, "Version constraints inside provider configuration blocks are deprecated")
tlsProvider := addrs.NewProvider(
addrs.DefaultProviderRegistryHost,
"hashicorp", "tls",
backend/local: Check dependency lock consistency before any operations In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.
2021-09-29 19:31:43 -05:00
)
happycloudProvider := addrs.NewProvider(
svchost.Hostname("tf.example.com"),
"awesomecorp", "happycloud",
)
nullProvider := addrs.NewDefaultProvider("null")
randomProvider := addrs.NewDefaultProvider("random")
impliedProvider := addrs.NewDefaultProvider("implied")
configuredProvider := addrs.NewDefaultProvider("configured")
grandchildProvider := addrs.NewDefaultProvider("grandchild")
tests := map[string]struct {
PrepareLocks func(*depsfile.Locks)
WantErrs []string
}{
"empty locks": {
func(*depsfile.Locks) {
// Intentionally blank
},
[]string{
`provider registry.opentofu.org/hashicorp/configured: required by this configuration but no version is selected`,
`provider registry.opentofu.org/hashicorp/grandchild: required by this configuration but no version is selected`,
`provider registry.opentofu.org/hashicorp/implied: required by this configuration but no version is selected`,
`provider registry.opentofu.org/hashicorp/null: required by this configuration but no version is selected`,
`provider registry.opentofu.org/hashicorp/random: required by this configuration but no version is selected`,
`provider registry.opentofu.org/hashicorp/tls: required by this configuration but no version is selected`,
backend/local: Check dependency lock consistency before any operations In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.
2021-09-29 19:31:43 -05:00
`provider tf.example.com/awesomecorp/happycloud: required by this configuration but no version is selected`,
},
},
"suitable locks": {
func(locks *depsfile.Locks) {
locks.SetProvider(configuredProvider, getproviders.MustParseVersion("1.4.0"), nil, nil)
locks.SetProvider(grandchildProvider, getproviders.MustParseVersion("0.1.0"), nil, nil)
locks.SetProvider(impliedProvider, getproviders.MustParseVersion("0.2.0"), nil, nil)
locks.SetProvider(nullProvider, getproviders.MustParseVersion("2.0.1"), nil, nil)
locks.SetProvider(randomProvider, getproviders.MustParseVersion("1.2.2"), nil, nil)
locks.SetProvider(tlsProvider, getproviders.MustParseVersion("3.0.1"), nil, nil)
locks.SetProvider(happycloudProvider, getproviders.MustParseVersion("0.0.1"), nil, nil)
},
nil,
},
"null provider constraints changed": {
func(locks *depsfile.Locks) {
locks.SetProvider(configuredProvider, getproviders.MustParseVersion("1.4.0"), nil, nil)
locks.SetProvider(grandchildProvider, getproviders.MustParseVersion("0.1.0"), nil, nil)
locks.SetProvider(impliedProvider, getproviders.MustParseVersion("0.2.0"), nil, nil)
locks.SetProvider(nullProvider, getproviders.MustParseVersion("3.0.0"), nil, nil)
locks.SetProvider(randomProvider, getproviders.MustParseVersion("1.2.2"), nil, nil)
locks.SetProvider(tlsProvider, getproviders.MustParseVersion("3.0.1"), nil, nil)
locks.SetProvider(happycloudProvider, getproviders.MustParseVersion("0.0.1"), nil, nil)
},
[]string{
`provider registry.opentofu.org/hashicorp/null: locked version selection 3.0.0 doesn't match the updated version constraints "~> 2.0.0, 2.0.1"`,
backend/local: Check dependency lock consistency before any operations In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.
2021-09-29 19:31:43 -05:00
},
},
"null provider lock changed": {
func(locks *depsfile.Locks) {
// In this case, we set the lock file version constraints to
// match the configuration, and so our error message changes
// to not assume the configuration changed anymore.
locks.SetProvider(nullProvider, getproviders.MustParseVersion("3.0.0"), getproviders.MustParseVersionConstraints("~> 2.0.0, 2.0.1"), nil)
locks.SetProvider(configuredProvider, getproviders.MustParseVersion("1.4.0"), nil, nil)
locks.SetProvider(grandchildProvider, getproviders.MustParseVersion("0.1.0"), nil, nil)
locks.SetProvider(impliedProvider, getproviders.MustParseVersion("0.2.0"), nil, nil)
locks.SetProvider(randomProvider, getproviders.MustParseVersion("1.2.2"), nil, nil)
locks.SetProvider(tlsProvider, getproviders.MustParseVersion("3.0.1"), nil, nil)
locks.SetProvider(happycloudProvider, getproviders.MustParseVersion("0.0.1"), nil, nil)
},
[]string{
`provider registry.opentofu.org/hashicorp/null: version constraints "~> 2.0.0, 2.0.1" don't match the locked version selection 3.0.0`,
backend/local: Check dependency lock consistency before any operations In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.
2021-09-29 19:31:43 -05:00
},
},
"overridden provider": {
func(locks *depsfile.Locks) {
locks.SetProviderOverridden(happycloudProvider)
},
[]string{
// We still catch all of the other ones, because only happycloud was overridden
`provider registry.opentofu.org/hashicorp/configured: required by this configuration but no version is selected`,
`provider registry.opentofu.org/hashicorp/grandchild: required by this configuration but no version is selected`,
`provider registry.opentofu.org/hashicorp/implied: required by this configuration but no version is selected`,
`provider registry.opentofu.org/hashicorp/null: required by this configuration but no version is selected`,
`provider registry.opentofu.org/hashicorp/random: required by this configuration but no version is selected`,
`provider registry.opentofu.org/hashicorp/tls: required by this configuration but no version is selected`,
backend/local: Check dependency lock consistency before any operations In historical versions of Terraform the responsibility to check this was inside the terraform.NewContext function, along with various other assorted concerns that made that function particularly complicated. More recently, we reduced the responsibility of the "terraform" package only to instantiating particular named plugins, assuming that its caller is responsible for selecting appropriate versions of any providers that _are_ external. However, until this commit we were just assuming that "terraform init" had correctly selected appropriate plugins and recorded them in the lock file, and so nothing was dealing with the problem of ensuring that there haven't been any changes to the lock file or config since the most recent "terraform init" which would cause us to need to re-evaluate those decisions. Part of the game here is to slightly extend the role of the dependency locks object to also carry information about a subset of provider addresses whose lock entries we're intentionally disregarding as part of the various little edge-case features we have for overridding providers: dev_overrides, "unmanaged providers", and the testing overrides in our own unit tests. This is an in-memory-only annotation, never included in the serialized plan files on disk. I had originally intended to create a new package to encapsulate all of this plugin-selection logic, including both the version constraint checking here and also the handling of the provider factory functions, but as an interim step I've just made version constraint consistency checks the responsibility of the backend/local package, which means that we'll always catch problems as part of preparing for local operations, while not imposing these additional checks on commands that _don't_ run local operations, such as "terraform apply" when in remote operations mode.
2021-09-29 19:31:43 -05:00
},
},
}
for name, test := range tests {
t.Run(name, func(t *testing.T) {
depLocks := depsfile.NewLocks()
test.PrepareLocks(depLocks)
gotErrs := cfg.VerifyDependencySelections(depLocks)
var gotErrsStr []string
if gotErrs != nil {
gotErrsStr = make([]string, len(gotErrs))
for i, err := range gotErrs {
gotErrsStr[i] = err.Error()
}
}
if diff := cmp.Diff(test.WantErrs, gotErrsStr); diff != "" {
t.Errorf("wrong errors\n%s", diff)
}
})
}
}
func TestConfigProviderForConfigAddr(t *testing.T) {
cfg, diags := testModuleConfigFromDir("testdata/valid-modules/providers-fqns")
assertNoDiagnostics(t, diags)
got := cfg.ProviderForConfigAddr(addrs.NewDefaultLocalProviderConfig("foo-test"))
want := addrs.NewProvider(addrs.DefaultProviderRegistryHost, "foo", "test")
if !got.Equals(want) {
t.Errorf("wrong result\ngot: %s\nwant: %s", got, want)
}
// now check a provider that isn't in the configuration. It should return a DefaultProvider.
got = cfg.ProviderForConfigAddr(addrs.NewDefaultLocalProviderConfig("bar-test"))
want = addrs.NewDefaultProvider("bar-test")
if !got.Equals(want) {
t.Errorf("wrong result\ngot: %s\nwant: %s", got, want)
}
}
func TestConfigAddProviderRequirements(t *testing.T) {
cfg, diags := testModuleConfigFromFile("testdata/valid-files/providers-explicit-implied.tf")
assertNoDiagnostics(t, diags)
reqs := getproviders.Requirements{
addrs.NewDefaultProvider("null"): nil,
}
diags = cfg.addProviderRequirements(reqs, true, false)
assertNoDiagnostics(t, diags)
}
func TestConfigImportProviderClashesWithModules(t *testing.T) {
src, err := os.ReadFile("testdata/invalid-import-files/import-and-module-clash.tf")
if err != nil {
t.Fatal(err)
}
parser := testParser(map[string]string{
"main.tf": string(src),
})
_, diags := parser.LoadConfigFile("main.tf")
assertExactDiagnostics(t, diags, []string{
`main.tf:9,3-19: Invalid import provider argument; The provider argument can only be specified in import blocks that will generate configuration.
Use the providers argument within the module block to configure providers for all resources within a module, including imported resources.`,
})
}
func TestConfigImportProviderClashesWithResources(t *testing.T) {
cfg, diags := testModuleConfigFromFile("testdata/invalid-import-files/import-and-resource-clash.tf")
assertNoDiagnostics(t, diags)
diags = cfg.addProviderRequirements(getproviders.Requirements{}, true, false)
assertExactDiagnostics(t, diags, []string{
`testdata/invalid-import-files/import-and-resource-clash.tf:9,3-19: Invalid import provider argument; The provider argument can only be specified in import blocks that will generate configuration.
Use the provider argument in the target resource block to configure the provider for a resource with explicit provider configuration.`,
})
}
func TestConfigImportProviderWithNoResourceProvider(t *testing.T) {
cfg, diags := testModuleConfigFromFile("testdata/invalid-import-files/import-and-no-resource.tf")
assertNoDiagnostics(t, diags)
diags = cfg.addProviderRequirements(getproviders.Requirements{}, true, false)
assertExactDiagnostics(t, diags, []string{
`testdata/invalid-import-files/import-and-no-resource.tf:5,3-19: Invalid import provider argument; The provider argument can only be specified in import blocks that will generate configuration.
Use the provider argument in the target resource block to configure the provider for a resource with explicit provider configuration.`,
})
}
func TestTransformForTest(t *testing.T) {
str := func(providers map[string]string) string {
var buffer bytes.Buffer
for key, config := range providers {
buffer.WriteString(fmt.Sprintf("%s: %s\n", key, config))
}
return buffer.String()
}
convertToProviders := func(t *testing.T, contents map[string]string) map[string]*Provider {
t.Helper()
providers := make(map[string]*Provider)
for key, content := range contents {
parser := hclparse.NewParser()
file, diags := parser.ParseHCL([]byte(content), fmt.Sprintf("%s.hcl", key))
if diags.HasErrors() {
t.Fatal(diags.Error())
}
provider := &Provider{
Config: file.Body,
}
parts := strings.Split(key, ".")
provider.Name = parts[0]
if len(parts) > 1 {
provider.Alias = parts[1]
}
providers[key] = provider
}
return providers
}
validate := func(t *testing.T, msg string, expected map[string]string, actual map[string]*Provider) {
t.Helper()
converted := make(map[string]string)
for key, provider := range actual {
content, err := provider.Config.Content(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{Name: "source", Required: true},
},
})
if err != nil {
t.Fatal(err)
}
source, diags := content.Attributes["source"].Expr.Value(nil)
if diags.HasErrors() {
t.Fatal(diags.Error())
}
converted[key] = fmt.Sprintf("source = %q", source.AsString())
}
if diff := cmp.Diff(expected, converted); len(diff) > 0 {
t.Errorf("%s\nexpected:\n%s\nactual:\n%s\ndiff:\n%s", msg, str(expected), str(converted), diff)
}
}
tcs := map[string]struct {
configProviders map[string]string
fileProviders map[string]string
runProviders []PassedProviderConfig
expectedProviders map[string]string
expectedErrors []string
}{
"empty": {
configProviders: make(map[string]string),
expectedProviders: make(map[string]string),
},
"only providers in config": {
configProviders: map[string]string{
"foo": "source = \"config\"",
"bar": "source = \"config\"",
},
expectedProviders: map[string]string{
"foo": "source = \"config\"",
"bar": "source = \"config\"",
},
},
"only providers in test file": {
configProviders: make(map[string]string),
fileProviders: map[string]string{
"foo": "source = \"testfile\"",
"bar": "source = \"testfile\"",
},
expectedProviders: map[string]string{
"foo": "source = \"testfile\"",
"bar": "source = \"testfile\"",
},
},
"only providers in run block": {
configProviders: make(map[string]string),
runProviders: []PassedProviderConfig{
{
InChild: &ProviderConfigRef{
Name: "foo",
},
InParent: &ProviderConfigRef{
Name: "bar",
},
},
},
expectedProviders: make(map[string]string),
expectedErrors: []string{
":0,0-0: Missing provider definition for bar; This provider block references a provider definition that does not exist.",
},
},
"subset of providers in test file": {
configProviders: make(map[string]string),
fileProviders: map[string]string{
"bar": "source = \"testfile\"",
},
runProviders: []PassedProviderConfig{
{
InChild: &ProviderConfigRef{
Name: "foo",
},
InParent: &ProviderConfigRef{
Name: "bar",
},
},
},
expectedProviders: map[string]string{
"foo": "source = \"testfile\"",
},
},
"overrides providers in config": {
configProviders: map[string]string{
"foo": "source = \"config\"",
"bar": "source = \"config\"",
},
fileProviders: map[string]string{
"bar": "source = \"testfile\"",
},
expectedProviders: map[string]string{
"foo": "source = \"config\"",
"bar": "source = \"testfile\"",
},
},
"overrides subset of providers in config": {
configProviders: map[string]string{
"foo": "source = \"config\"",
"bar": "source = \"config\"",
},
fileProviders: map[string]string{
"foo": "source = \"testfile\"",
"bar": "source = \"testfile\"",
},
runProviders: []PassedProviderConfig{
{
InChild: &ProviderConfigRef{
Name: "bar",
},
InParent: &ProviderConfigRef{
Name: "bar",
},
},
},
expectedProviders: map[string]string{
"foo": "source = \"config\"",
"bar": "source = \"testfile\"",
},
},
"handles aliases": {
configProviders: map[string]string{
"foo.primary": "source = \"config\"",
"foo.secondary": "source = \"config\"",
},
fileProviders: map[string]string{
"foo": "source = \"testfile\"",
},
runProviders: []PassedProviderConfig{
{
InChild: &ProviderConfigRef{
Name: "foo.secondary",
},
InParent: &ProviderConfigRef{
Name: "foo",
},
},
},
expectedProviders: map[string]string{
"foo.primary": "source = \"config\"",
"foo.secondary": "source = \"testfile\"",
},
},
}
for name, tc := range tcs {
t.Run(name, func(t *testing.T) {
config := &Config{
Module: &Module{
ProviderConfigs: convertToProviders(t, tc.configProviders),
},
}
file := &TestFile{
Providers: convertToProviders(t, tc.fileProviders),
}
run := &TestRun{
Providers: tc.runProviders,
}
reset, diags := config.TransformForTest(run, file)
var actualErrs []string
for _, err := range diags.Errs() {
actualErrs = append(actualErrs, err.Error())
}
if diff := cmp.Diff(actualErrs, tc.expectedErrors, cmpopts.IgnoreUnexported()); len(diff) > 0 {
t.Errorf("unmatched errors\nexpected:\n%s\nactual:\n%s\ndiff:\n%s", strings.Join(tc.expectedErrors, "\n"), strings.Join(actualErrs, "\n"), diff)
}
validate(t, "after transform mismatch", tc.expectedProviders, config.Module.ProviderConfigs)
reset()
validate(t, "after reset mismatch", tc.configProviders, config.Module.ProviderConfigs)
})
}
}