2024-02-08 03:48:59 -06:00
// Copyright (c) The OpenTofu Authors
// SPDX-License-Identifier: MPL-2.0
// Copyright (c) 2023 HashiCorp, Inc.
2023-05-02 10:33:06 -05:00
// SPDX-License-Identifier: MPL-2.0
2023-09-20 07:16:53 -05:00
package tofu
2015-01-26 23:23:27 -06:00
import (
2022-09-27 18:25:04 -05:00
"fmt"
2015-02-08 19:58:02 -06:00
"log"
2020-10-18 09:01:48 -05:00
"strings"
2015-01-26 23:23:27 -06:00
2023-09-20 06:35:35 -05:00
"github.com/opentofu/opentofu/internal/logging"
"github.com/opentofu/opentofu/internal/tfdiags"
2015-01-26 23:23:27 -06:00
2023-09-20 06:35:35 -05:00
"github.com/opentofu/opentofu/internal/addrs"
2015-01-26 23:23:27 -06:00
2023-09-20 06:35:35 -05:00
"github.com/opentofu/opentofu/internal/dag"
terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-04-30 12:33:53 -05:00
)
2015-01-26 23:32:32 -06:00
2023-09-26 12:09:27 -05:00
// Graph represents the graph that OpenTofu uses to represent resources
2017-02-03 07:24:27 -06:00
// and their dependencies.
2015-01-26 23:23:27 -06:00
type Graph struct {
// Graph is the actual DAG. This is embedded so you can call the DAG
// methods directly.
2015-02-04 10:30:53 -06:00
dag . AcyclicGraph
2015-01-26 23:23:27 -06:00
// Path is the path in the module tree that this Graph represents.
terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-04-30 12:33:53 -05:00
Path addrs . ModuleInstance
2015-01-26 23:23:27 -06:00
}
2016-11-09 15:52:22 -06:00
func ( g * Graph ) DirectedGraph ( ) dag . Grapher {
return & g . AcyclicGraph
}
2015-02-07 15:29:55 -06:00
// Walk walks the graph with the given walker for callbacks. The graph
// will be walked with full parallelism, so the walker should expect
// to be called in concurrently.
terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-04-30 12:33:53 -05:00
func ( g * Graph ) Walk ( walker GraphWalker ) tfdiags . Diagnostics {
2015-02-10 10:58:14 -06:00
return g . walk ( walker )
2015-02-07 15:29:55 -06:00
}
terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-04-30 12:33:53 -05:00
func ( g * Graph ) walk ( walker GraphWalker ) tfdiags . Diagnostics {
2015-02-07 15:29:55 -06:00
// The callbacks for enter/exiting a graph
2020-03-17 13:02:46 -05:00
ctx := walker . EvalContext ( )
2015-02-07 15:29:55 -06:00
2024-03-26 06:41:16 -05:00
// We explicitly create the panicHandler before
// spawning many go routines for vertex evaluation
// to minimize the performance impact of capturing
// the stack trace.
panicHandler := logging . PanicHandlerWithTraceFn ( )
2015-02-08 16:00:13 -06:00
// Walk the graph.
2020-11-30 16:48:02 -06:00
walkFn := func ( v dag . Vertex ) ( diags tfdiags . Diagnostics ) {
2021-10-27 15:33:35 -05:00
// the walkFn is called asynchronously, and needs to be recovered
// separately in the case of a panic.
2024-03-26 06:41:16 -05:00
defer panicHandler ( )
2021-10-27 15:33:35 -05:00
terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-04-30 12:33:53 -05:00
log . Printf ( "[TRACE] vertex %q: starting visit (%T)" , dag . VertexName ( v ) , v )
2015-02-08 19:58:02 -06:00
2016-11-03 14:04:04 -05:00
defer func ( ) {
core: Functional-style API for terraform.Context
Previously terraform.Context was built in an unfortunate way where all of
the data was provided up front in terraform.NewContext and then mutated
directly by subsequent operations. That made the data flow hard to follow,
commonly leading to bugs, and also meant that we were forced to take
various actions too early in terraform.NewContext, rather than waiting
until a more appropriate time during an operation.
This (enormous) commit changes terraform.Context so that its fields are
broadly just unchanging data about the execution context (current
workspace name, available plugins, etc) whereas the main data Terraform
works with arrives via individual method arguments and is returned in
return values.
Specifically, this means that terraform.Context no longer "has-a" config,
state, and "planned changes", instead holding on to those only temporarily
during an operation. The caller is responsible for propagating the outcome
of one step into the next step so that the data flow between operations is
actually visible.
However, since that's a change to the main entry points in the "terraform"
package, this commit also touches every file in the codebase which
interacted with those APIs. Most of the noise here is in updating tests
to take the same actions using the new API style, but this also affects
the main-code callers in the backends and in the command package.
My goal here was to refactor without changing observable behavior, but in
practice there are a couple externally-visible behavior variations here
that seemed okay in service of the broader goal:
- The "terraform graph" command is no longer hooked directly into the
core graph builders, because that's no longer part of the public API.
However, I did include a couple new Context functions whose contract
is to produce a UI-oriented graph, and _for now_ those continue to
return the physical graph we use for those operations. There's no
exported API for generating the "validate" and "eval" graphs, because
neither is particularly interesting in its own right, and so
"terraform graph" no longer supports those graph types.
- terraform.NewContext no longer has the responsibility for collecting
all of the provider schemas up front. Instead, we wait until we need
them. However, that means that some of our error messages now have a
slightly different shape due to unwinding through a differently-shaped
call stack. As of this commit we also end up reloading the schemas
multiple times in some cases, which is functionally acceptable but
likely represents a performance regression. I intend to rework this to
use caching, but I'm saving that for a later commit because this one is
big enough already.
The proximal reason for this change is to resolve the chicken/egg problem
whereby there was previously no single point where we could apply "moved"
statements to the previous run state before creating a plan. With this
change in place, we can now do that as part of Context.Plan, prior to
forking the input state into the three separate state artifacts we use
during planning.
However, this is at least the third project in a row where the previous
API design led to piling more functionality into terraform.NewContext and
then working around the incorrect order of operations that produces, so
I intend that by paying the cost/risk of this large diff now we can in
turn reduce the cost/risk of future projects that relate to our main
workflow actions.
2021-08-24 14:06:38 -05:00
if diags . HasErrors ( ) {
for _ , diag := range diags {
if diag . Severity ( ) == tfdiags . Error {
desc := diag . Description ( )
log . Printf ( "[ERROR] vertex %q error: %s" , dag . VertexName ( v ) , desc . Summary )
}
}
log . Printf ( "[TRACE] vertex %q: visit complete, with errors" , dag . VertexName ( v ) )
} else {
log . Printf ( "[TRACE] vertex %q: visit complete" , dag . VertexName ( v ) )
}
2016-11-03 14:04:04 -05:00
} ( )
2015-05-01 16:19:32 -05:00
// vertexCtx is the context that we use when evaluating. This
// is normally the context of our graph but can be overridden
2020-03-05 15:13:54 -06:00
// with a GraphNodeModuleInstance impl.
2015-05-01 16:19:32 -05:00
vertexCtx := ctx
2020-03-17 13:02:46 -05:00
if pn , ok := v . ( GraphNodeModuleInstance ) ; ok {
terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-04-30 12:33:53 -05:00
vertexCtx = walker . EnterPath ( pn . Path ( ) )
2015-05-01 16:19:32 -05:00
defer walker . ExitPath ( pn . Path ( ) )
}
2020-09-04 13:03:45 -05:00
// If the node is exec-able, then execute it.
if ev , ok := v . ( GraphNodeExecutable ) ; ok {
diags = diags . Append ( walker . Execute ( vertexCtx , ev ) )
if diags . HasErrors ( ) {
return
}
}
2015-02-08 16:00:13 -06:00
// If the node is dynamically expanded, then expand it
if ev , ok := v . ( GraphNodeDynamicExpandable ) ; ok {
terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-04-30 12:33:53 -05:00
log . Printf ( "[TRACE] vertex %q: expanding dynamic subgraph" , dag . VertexName ( v ) )
2016-11-10 16:07:26 -06:00
2015-05-01 16:19:32 -05:00
g , err := ev . DynamicExpand ( vertexCtx )
2022-09-23 16:33:43 -05:00
diags = diags . Append ( err )
if diags . HasErrors ( ) {
log . Printf ( "[TRACE] vertex %q: failed expanding dynamic subgraph: %s" , dag . VertexName ( v ) , err )
2015-02-08 16:00:13 -06:00
return
}
2016-09-22 13:03:03 -05:00
if g != nil {
2022-09-27 18:25:04 -05:00
// The subgraph should always be valid, per our normal acyclic
// graph validation rules.
if err := g . Validate ( ) ; err != nil {
diags = diags . Append ( tfdiags . Sourceless (
tfdiags . Error ,
"Graph node has invalid dynamic subgraph" ,
2023-09-26 12:09:27 -05:00
fmt . Sprintf ( "The internal logic for %q generated an invalid dynamic subgraph: %s.\n\nThis is a bug in OpenTofu. Please report it!" , dag . VertexName ( v ) , err ) ,
2022-09-27 18:25:04 -05:00
) )
return
}
// If we passed validation then there is exactly one root node.
// That root node should always be "rootNode", the singleton
// root node value.
if n , err := g . Root ( ) ; err != nil || n != dag . Vertex ( rootNode ) {
diags = diags . Append ( tfdiags . Sourceless (
tfdiags . Error ,
"Graph node has invalid dynamic subgraph" ,
2023-09-26 12:09:27 -05:00
fmt . Sprintf ( "The internal logic for %q generated an invalid dynamic subgraph: the root node is %T, which is not a suitable root node type.\n\nThis is a bug in OpenTofu. Please report it!" , dag . VertexName ( v ) , n ) ,
2022-09-27 18:25:04 -05:00
) )
return
}
2016-09-22 13:03:03 -05:00
// Walk the subgraph
terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-04-30 12:33:53 -05:00
log . Printf ( "[TRACE] vertex %q: entering dynamic subgraph" , dag . VertexName ( v ) )
subDiags := g . walk ( walker )
diags = diags . Append ( subDiags )
if subDiags . HasErrors ( ) {
2020-10-18 09:01:48 -05:00
var errs [ ] string
for _ , d := range subDiags {
errs = append ( errs , d . Description ( ) . Summary )
}
log . Printf ( "[TRACE] vertex %q: dynamic subgraph encountered errors: %s" , dag . VertexName ( v ) , strings . Join ( errs , "," ) )
2016-09-22 13:03:03 -05:00
return
}
terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-04-30 12:33:53 -05:00
log . Printf ( "[TRACE] vertex %q: dynamic subgraph completed successfully" , dag . VertexName ( v ) )
} else {
log . Printf ( "[TRACE] vertex %q: produced no dynamic subgraph" , dag . VertexName ( v ) )
2015-02-08 19:10:54 -06:00
}
2015-02-08 16:00:13 -06:00
}
terraform: ugly huge change to weave in new HCL2-oriented types
Due to how deeply the configuration types go into Terraform Core, there
isn't a great way to switch out to HCL2 gradually. As a consequence, this
huge commit gets us from the old state to a _compilable_ new state, but
does not yet attempt to fix any tests and has a number of known missing
parts and bugs. We will continue to iterate on this in forthcoming
commits, heading back towards passing tests and making Terraform
fully-functional again.
The three main goals here are:
- Use the configuration models from the "configs" package instead of the
older models in the "config" package, which is now deprecated and
preserved only to help us write our migration tool.
- Do expression inspection and evaluation using the functionality of the
new "lang" package, instead of the Interpolator type and related
functionality in the main "terraform" package.
- Represent addresses of various objects using types in the addrs package,
rather than hand-constructed strings. This is not critical to support
the above, but was a big help during the implementation of these other
points since it made it much more explicit what kind of address is
expected in each context.
Since our new packages are built to accommodate some future planned
features that are not yet implemented (e.g. the "for_each" argument on
resources, "count"/"for_each" on modules), and since there's still a fair
amount of functionality still using old-style APIs, there is a moderate
amount of shimming here to connect new assumptions with old, hopefully in
a way that makes it easier to find and eliminate these shims later.
I apologize in advance to the person who inevitably just found this huge
commit while spelunking through the commit history.
2018-04-30 12:33:53 -05:00
return
2015-02-08 16:00:13 -06:00
}
2015-02-08 19:10:54 -06:00
return g . AcyclicGraph . Walk ( walkFn )
2015-02-07 15:29:55 -06:00
}