core: Functional-style API for terraform.Context
Previously terraform.Context was built in an unfortunate way where all of
the data was provided up front in terraform.NewContext and then mutated
directly by subsequent operations. That made the data flow hard to follow,
commonly leading to bugs, and also meant that we were forced to take
various actions too early in terraform.NewContext, rather than waiting
until a more appropriate time during an operation.
This (enormous) commit changes terraform.Context so that its fields are
broadly just unchanging data about the execution context (current
workspace name, available plugins, etc) whereas the main data Terraform
works with arrives via individual method arguments and is returned in
return values.
Specifically, this means that terraform.Context no longer "has-a" config,
state, and "planned changes", instead holding on to those only temporarily
during an operation. The caller is responsible for propagating the outcome
of one step into the next step so that the data flow between operations is
actually visible.
However, since that's a change to the main entry points in the "terraform"
package, this commit also touches every file in the codebase which
interacted with those APIs. Most of the noise here is in updating tests
to take the same actions using the new API style, but this also affects
the main-code callers in the backends and in the command package.
My goal here was to refactor without changing observable behavior, but in
practice there are a couple externally-visible behavior variations here
that seemed okay in service of the broader goal:
- The "terraform graph" command is no longer hooked directly into the
core graph builders, because that's no longer part of the public API.
However, I did include a couple new Context functions whose contract
is to produce a UI-oriented graph, and _for now_ those continue to
return the physical graph we use for those operations. There's no
exported API for generating the "validate" and "eval" graphs, because
neither is particularly interesting in its own right, and so
"terraform graph" no longer supports those graph types.
- terraform.NewContext no longer has the responsibility for collecting
all of the provider schemas up front. Instead, we wait until we need
them. However, that means that some of our error messages now have a
slightly different shape due to unwinding through a differently-shaped
call stack. As of this commit we also end up reloading the schemas
multiple times in some cases, which is functionally acceptable but
likely represents a performance regression. I intend to rework this to
use caching, but I'm saving that for a later commit because this one is
big enough already.
The proximal reason for this change is to resolve the chicken/egg problem
whereby there was previously no single point where we could apply "moved"
statements to the previous run state before creating a plan. With this
change in place, we can now do that as part of Context.Plan, prior to
forking the input state into the three separate state artifacts we use
during planning.
However, this is at least the third project in a row where the previous
API design led to piling more functionality into terraform.NewContext and
then working around the incorrect order of operations that produces, so
I intend that by paying the cost/risk of this large diff now we can in
turn reduce the cost/risk of future projects that relate to our main
workflow actions.
2021-08-24 14:06:38 -05:00
|
|
|
package terraform
|
|
|
|
|
|
|
|
import (
|
|
|
|
"fmt"
|
|
|
|
"log"
|
|
|
|
|
|
|
|
"github.com/hashicorp/terraform/internal/addrs"
|
|
|
|
"github.com/hashicorp/terraform/internal/configs"
|
|
|
|
"github.com/hashicorp/terraform/internal/plans"
|
|
|
|
"github.com/hashicorp/terraform/internal/states"
|
|
|
|
"github.com/hashicorp/terraform/internal/tfdiags"
|
|
|
|
"github.com/zclconf/go-cty/cty"
|
|
|
|
)
|
|
|
|
|
|
|
|
// Apply performs the actions described by the given Plan object and returns
|
|
|
|
// the resulting updated state.
|
|
|
|
//
|
|
|
|
// The given configuration *must* be the same configuration that was passed
|
|
|
|
// earlier to Context.Plan in order to create this plan.
|
|
|
|
//
|
|
|
|
// Even if the returned diagnostics contains errors, Apply always returns the
|
|
|
|
// resulting state which is likely to have been partially-updated.
|
|
|
|
func (c *Context) Apply(plan *plans.Plan, config *configs.Config) (*states.State, tfdiags.Diagnostics) {
|
|
|
|
defer c.acquireRun("apply")()
|
|
|
|
var diags tfdiags.Diagnostics
|
|
|
|
|
|
|
|
log.Printf("[DEBUG] Building and walking apply graph for %s plan", plan.UIMode)
|
|
|
|
|
2021-08-31 18:36:27 -05:00
|
|
|
graph, operation, moreDiags := c.applyGraph(plan, config, true)
|
core: Functional-style API for terraform.Context
Previously terraform.Context was built in an unfortunate way where all of
the data was provided up front in terraform.NewContext and then mutated
directly by subsequent operations. That made the data flow hard to follow,
commonly leading to bugs, and also meant that we were forced to take
various actions too early in terraform.NewContext, rather than waiting
until a more appropriate time during an operation.
This (enormous) commit changes terraform.Context so that its fields are
broadly just unchanging data about the execution context (current
workspace name, available plugins, etc) whereas the main data Terraform
works with arrives via individual method arguments and is returned in
return values.
Specifically, this means that terraform.Context no longer "has-a" config,
state, and "planned changes", instead holding on to those only temporarily
during an operation. The caller is responsible for propagating the outcome
of one step into the next step so that the data flow between operations is
actually visible.
However, since that's a change to the main entry points in the "terraform"
package, this commit also touches every file in the codebase which
interacted with those APIs. Most of the noise here is in updating tests
to take the same actions using the new API style, but this also affects
the main-code callers in the backends and in the command package.
My goal here was to refactor without changing observable behavior, but in
practice there are a couple externally-visible behavior variations here
that seemed okay in service of the broader goal:
- The "terraform graph" command is no longer hooked directly into the
core graph builders, because that's no longer part of the public API.
However, I did include a couple new Context functions whose contract
is to produce a UI-oriented graph, and _for now_ those continue to
return the physical graph we use for those operations. There's no
exported API for generating the "validate" and "eval" graphs, because
neither is particularly interesting in its own right, and so
"terraform graph" no longer supports those graph types.
- terraform.NewContext no longer has the responsibility for collecting
all of the provider schemas up front. Instead, we wait until we need
them. However, that means that some of our error messages now have a
slightly different shape due to unwinding through a differently-shaped
call stack. As of this commit we also end up reloading the schemas
multiple times in some cases, which is functionally acceptable but
likely represents a performance regression. I intend to rework this to
use caching, but I'm saving that for a later commit because this one is
big enough already.
The proximal reason for this change is to resolve the chicken/egg problem
whereby there was previously no single point where we could apply "moved"
statements to the previous run state before creating a plan. With this
change in place, we can now do that as part of Context.Plan, prior to
forking the input state into the three separate state artifacts we use
during planning.
However, this is at least the third project in a row where the previous
API design led to piling more functionality into terraform.NewContext and
then working around the incorrect order of operations that produces, so
I intend that by paying the cost/risk of this large diff now we can in
turn reduce the cost/risk of future projects that relate to our main
workflow actions.
2021-08-24 14:06:38 -05:00
|
|
|
if moreDiags.HasErrors() {
|
|
|
|
return nil, diags
|
|
|
|
}
|
|
|
|
|
|
|
|
variables := InputValues{}
|
|
|
|
for name, dyVal := range plan.VariableValues {
|
|
|
|
val, err := dyVal.Decode(cty.DynamicPseudoType)
|
|
|
|
if err != nil {
|
|
|
|
diags = diags.Append(tfdiags.Sourceless(
|
|
|
|
tfdiags.Error,
|
|
|
|
"Invalid variable value in plan",
|
|
|
|
fmt.Sprintf("Invalid value for variable %q recorded in plan file: %s.", name, err),
|
|
|
|
))
|
|
|
|
continue
|
|
|
|
}
|
|
|
|
|
|
|
|
variables[name] = &InputValue{
|
|
|
|
Value: val,
|
|
|
|
SourceType: ValueFromPlan,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
workingState := plan.PriorState.DeepCopy()
|
|
|
|
walker, walkDiags := c.walk(graph, operation, &graphWalkOpts{
|
|
|
|
Config: config,
|
|
|
|
InputState: workingState,
|
|
|
|
Changes: plan.Changes,
|
|
|
|
RootVariableValues: variables,
|
|
|
|
})
|
|
|
|
diags = diags.Append(walker.NonFatalDiagnostics)
|
|
|
|
diags = diags.Append(walkDiags)
|
|
|
|
|
|
|
|
newState := walker.State.Close()
|
|
|
|
if plan.UIMode == plans.DestroyMode && !diags.HasErrors() {
|
|
|
|
// NOTE: This is a vestigial violation of the rule that we mustn't
|
|
|
|
// use plan.UIMode to affect apply-time behavior.
|
|
|
|
// We ideally ought to just call newState.PruneResourceHusks
|
|
|
|
// unconditionally here, but we historically didn't and haven't yet
|
|
|
|
// verified that it'd be safe to do so.
|
|
|
|
newState.PruneResourceHusks()
|
|
|
|
}
|
|
|
|
|
|
|
|
if len(plan.TargetAddrs) > 0 {
|
|
|
|
diags = diags.Append(tfdiags.Sourceless(
|
|
|
|
tfdiags.Warning,
|
|
|
|
"Applied changes may be incomplete",
|
|
|
|
`The plan was created with the -target option in effect, so some changes requested in the configuration may have been ignored and the output values may not be fully updated. Run the following command to verify that no other changes are pending:
|
|
|
|
terraform plan
|
|
|
|
|
|
|
|
Note that the -target option is not suitable for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform specifically suggests to use it as part of an error message.`,
|
|
|
|
))
|
|
|
|
}
|
|
|
|
|
|
|
|
return newState, diags
|
|
|
|
}
|
|
|
|
|
2021-08-31 18:36:27 -05:00
|
|
|
func (c *Context) applyGraph(plan *plans.Plan, config *configs.Config, validate bool) (*Graph, walkOperation, tfdiags.Diagnostics) {
|
core: Functional-style API for terraform.Context
Previously terraform.Context was built in an unfortunate way where all of
the data was provided up front in terraform.NewContext and then mutated
directly by subsequent operations. That made the data flow hard to follow,
commonly leading to bugs, and also meant that we were forced to take
various actions too early in terraform.NewContext, rather than waiting
until a more appropriate time during an operation.
This (enormous) commit changes terraform.Context so that its fields are
broadly just unchanging data about the execution context (current
workspace name, available plugins, etc) whereas the main data Terraform
works with arrives via individual method arguments and is returned in
return values.
Specifically, this means that terraform.Context no longer "has-a" config,
state, and "planned changes", instead holding on to those only temporarily
during an operation. The caller is responsible for propagating the outcome
of one step into the next step so that the data flow between operations is
actually visible.
However, since that's a change to the main entry points in the "terraform"
package, this commit also touches every file in the codebase which
interacted with those APIs. Most of the noise here is in updating tests
to take the same actions using the new API style, but this also affects
the main-code callers in the backends and in the command package.
My goal here was to refactor without changing observable behavior, but in
practice there are a couple externally-visible behavior variations here
that seemed okay in service of the broader goal:
- The "terraform graph" command is no longer hooked directly into the
core graph builders, because that's no longer part of the public API.
However, I did include a couple new Context functions whose contract
is to produce a UI-oriented graph, and _for now_ those continue to
return the physical graph we use for those operations. There's no
exported API for generating the "validate" and "eval" graphs, because
neither is particularly interesting in its own right, and so
"terraform graph" no longer supports those graph types.
- terraform.NewContext no longer has the responsibility for collecting
all of the provider schemas up front. Instead, we wait until we need
them. However, that means that some of our error messages now have a
slightly different shape due to unwinding through a differently-shaped
call stack. As of this commit we also end up reloading the schemas
multiple times in some cases, which is functionally acceptable but
likely represents a performance regression. I intend to rework this to
use caching, but I'm saving that for a later commit because this one is
big enough already.
The proximal reason for this change is to resolve the chicken/egg problem
whereby there was previously no single point where we could apply "moved"
statements to the previous run state before creating a plan. With this
change in place, we can now do that as part of Context.Plan, prior to
forking the input state into the three separate state artifacts we use
during planning.
However, this is at least the third project in a row where the previous
API design led to piling more functionality into terraform.NewContext and
then working around the incorrect order of operations that produces, so
I intend that by paying the cost/risk of this large diff now we can in
turn reduce the cost/risk of future projects that relate to our main
workflow actions.
2021-08-24 14:06:38 -05:00
|
|
|
graph, diags := (&ApplyGraphBuilder{
|
|
|
|
Config: config,
|
|
|
|
Changes: plan.Changes,
|
|
|
|
State: plan.PriorState,
|
2021-08-31 12:58:05 -05:00
|
|
|
Plugins: c.plugins,
|
core: Functional-style API for terraform.Context
Previously terraform.Context was built in an unfortunate way where all of
the data was provided up front in terraform.NewContext and then mutated
directly by subsequent operations. That made the data flow hard to follow,
commonly leading to bugs, and also meant that we were forced to take
various actions too early in terraform.NewContext, rather than waiting
until a more appropriate time during an operation.
This (enormous) commit changes terraform.Context so that its fields are
broadly just unchanging data about the execution context (current
workspace name, available plugins, etc) whereas the main data Terraform
works with arrives via individual method arguments and is returned in
return values.
Specifically, this means that terraform.Context no longer "has-a" config,
state, and "planned changes", instead holding on to those only temporarily
during an operation. The caller is responsible for propagating the outcome
of one step into the next step so that the data flow between operations is
actually visible.
However, since that's a change to the main entry points in the "terraform"
package, this commit also touches every file in the codebase which
interacted with those APIs. Most of the noise here is in updating tests
to take the same actions using the new API style, but this also affects
the main-code callers in the backends and in the command package.
My goal here was to refactor without changing observable behavior, but in
practice there are a couple externally-visible behavior variations here
that seemed okay in service of the broader goal:
- The "terraform graph" command is no longer hooked directly into the
core graph builders, because that's no longer part of the public API.
However, I did include a couple new Context functions whose contract
is to produce a UI-oriented graph, and _for now_ those continue to
return the physical graph we use for those operations. There's no
exported API for generating the "validate" and "eval" graphs, because
neither is particularly interesting in its own right, and so
"terraform graph" no longer supports those graph types.
- terraform.NewContext no longer has the responsibility for collecting
all of the provider schemas up front. Instead, we wait until we need
them. However, that means that some of our error messages now have a
slightly different shape due to unwinding through a differently-shaped
call stack. As of this commit we also end up reloading the schemas
multiple times in some cases, which is functionally acceptable but
likely represents a performance regression. I intend to rework this to
use caching, but I'm saving that for a later commit because this one is
big enough already.
The proximal reason for this change is to resolve the chicken/egg problem
whereby there was previously no single point where we could apply "moved"
statements to the previous run state before creating a plan. With this
change in place, we can now do that as part of Context.Plan, prior to
forking the input state into the three separate state artifacts we use
during planning.
However, this is at least the third project in a row where the previous
API design led to piling more functionality into terraform.NewContext and
then working around the incorrect order of operations that produces, so
I intend that by paying the cost/risk of this large diff now we can in
turn reduce the cost/risk of future projects that relate to our main
workflow actions.
2021-08-24 14:06:38 -05:00
|
|
|
Targets: plan.TargetAddrs,
|
|
|
|
ForceReplace: plan.ForceReplaceAddrs,
|
|
|
|
Validate: validate,
|
|
|
|
}).Build(addrs.RootModuleInstance)
|
|
|
|
|
|
|
|
operation := walkApply
|
|
|
|
if plan.UIMode == plans.DestroyMode {
|
|
|
|
// NOTE: This is a vestigial violation of the rule that we mustn't
|
|
|
|
// use plan.UIMode to affect apply-time behavior. It's a design error
|
|
|
|
// if anything downstream switches behavior when operation is set
|
|
|
|
// to walkDestroy, but we've not yet fully audited that.
|
|
|
|
// TODO: Audit that and remove walkDestroy as an operation mode.
|
|
|
|
operation = walkDestroy
|
|
|
|
}
|
|
|
|
|
|
|
|
return graph, operation, diags
|
|
|
|
}
|
|
|
|
|
|
|
|
// ApplyGraphForUI is a last vestage of graphs in the public interface of
|
|
|
|
// Context (as opposed to graphs as an implementation detail) intended only for
|
|
|
|
// use by the "terraform graph" command when asked to render an apply-time
|
|
|
|
// graph.
|
|
|
|
//
|
|
|
|
// The result of this is intended only for rendering ot the user as a dot
|
|
|
|
// graph, and so may change in future in order to make the result more useful
|
|
|
|
// in that context, even if drifts away from the physical graph that Terraform
|
|
|
|
// Core currently uses as an implementation detail of planning.
|
|
|
|
func (c *Context) ApplyGraphForUI(plan *plans.Plan, config *configs.Config) (*Graph, tfdiags.Diagnostics) {
|
|
|
|
// For now though, this really is just the internal graph, confusing
|
|
|
|
// implementation details and all.
|
|
|
|
|
|
|
|
var diags tfdiags.Diagnostics
|
|
|
|
|
2021-08-31 18:36:27 -05:00
|
|
|
graph, _, moreDiags := c.applyGraph(plan, config, false)
|
core: Functional-style API for terraform.Context
Previously terraform.Context was built in an unfortunate way where all of
the data was provided up front in terraform.NewContext and then mutated
directly by subsequent operations. That made the data flow hard to follow,
commonly leading to bugs, and also meant that we were forced to take
various actions too early in terraform.NewContext, rather than waiting
until a more appropriate time during an operation.
This (enormous) commit changes terraform.Context so that its fields are
broadly just unchanging data about the execution context (current
workspace name, available plugins, etc) whereas the main data Terraform
works with arrives via individual method arguments and is returned in
return values.
Specifically, this means that terraform.Context no longer "has-a" config,
state, and "planned changes", instead holding on to those only temporarily
during an operation. The caller is responsible for propagating the outcome
of one step into the next step so that the data flow between operations is
actually visible.
However, since that's a change to the main entry points in the "terraform"
package, this commit also touches every file in the codebase which
interacted with those APIs. Most of the noise here is in updating tests
to take the same actions using the new API style, but this also affects
the main-code callers in the backends and in the command package.
My goal here was to refactor without changing observable behavior, but in
practice there are a couple externally-visible behavior variations here
that seemed okay in service of the broader goal:
- The "terraform graph" command is no longer hooked directly into the
core graph builders, because that's no longer part of the public API.
However, I did include a couple new Context functions whose contract
is to produce a UI-oriented graph, and _for now_ those continue to
return the physical graph we use for those operations. There's no
exported API for generating the "validate" and "eval" graphs, because
neither is particularly interesting in its own right, and so
"terraform graph" no longer supports those graph types.
- terraform.NewContext no longer has the responsibility for collecting
all of the provider schemas up front. Instead, we wait until we need
them. However, that means that some of our error messages now have a
slightly different shape due to unwinding through a differently-shaped
call stack. As of this commit we also end up reloading the schemas
multiple times in some cases, which is functionally acceptable but
likely represents a performance regression. I intend to rework this to
use caching, but I'm saving that for a later commit because this one is
big enough already.
The proximal reason for this change is to resolve the chicken/egg problem
whereby there was previously no single point where we could apply "moved"
statements to the previous run state before creating a plan. With this
change in place, we can now do that as part of Context.Plan, prior to
forking the input state into the three separate state artifacts we use
during planning.
However, this is at least the third project in a row where the previous
API design led to piling more functionality into terraform.NewContext and
then working around the incorrect order of operations that produces, so
I intend that by paying the cost/risk of this large diff now we can in
turn reduce the cost/risk of future projects that relate to our main
workflow actions.
2021-08-24 14:06:38 -05:00
|
|
|
diags = diags.Append(moreDiags)
|
|
|
|
return graph, diags
|
|
|
|
}
|