opentofu/internal/terraform/context_plan.go

687 lines
25 KiB
Go
Raw Normal View History

core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
package terraform
import (
core: Report a warning if any moves get blocked In most cases Terraform will be able to automatically fully resolve all of the pending move statements before creating a plan, but there are some edge cases where we can end up wanting to move one object to a location where another object is already declared. One relatively-obvious example is if someone uses "terraform state mv" in order to create a set of resource instance bindings that could never have arising in normal Terraform use. A less obvious example arises from the interactions between moves at different levels of granularity. If we are both moving a module to a new address and moving a resource into an instance of the new module at the same time, the old module might well have already had a resource of the same name and so the resource move will be unresolvable. In these situations Terraform will move the objects as far as possible, but because it's never valid for a move "from" address to still be declared in the configuration Terraform will inevitably always plan to destroy the objects that didn't find a final home. To give some additional explanation for that result, here we'll add a warning which describes what happened. This is not a particularly actionable warning because we don't really have enough information to guess what the user intended, but we do at least prompt that they might be able to use the "terraform state" family of subcommands to repair the ambiguous situation before planning, if they want a different result than what Terraform proposed.
2021-09-23 13:18:29 -05:00
"bytes"
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
"fmt"
"log"
core: Fail if a moved resource instance is excluded by -target Because "moved" blocks produce changes that span across more than one resource instance address at the same time, we need to take extra care with them during planning. The -target option allows for restricting Terraform's attention only to a subset of resources when planning, as an escape hatch to recover from bugs and mistakes. However, we need to avoid any situation where only one "side" of a move would be considered in a particular plan, because that'd create a new situation that would be otherwise unreachable and would be difficult to recover from. As a compromise then, we'll reject an attempt to create a targeted plan if the plan involves resolving a pending move and if the source address of that move is not included in the targets. Our error message offers the user two possible resolutions: to create an untargeted plan, thus allowing everything to resolve, or to add additional -target options to include just the existing resource instances that have pending moves to resolve. This compromise recognizes that it is possible -- though hopefully rare -- that a user could potentially both be recovering from a bug or mistake at the same time as processing a move, if e.g. the bug was fixed by upgrading a module and the new version includes a new "moved" block. In that edge case, it might be necessary to just add the one additional address to the targets rather than removing the targets altogether, if creating a normal untargeted plan is impossible due to whatever bug they're trying to recover from.
2021-09-15 17:54:16 -05:00
"sort"
"strings"
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
"github.com/zclconf/go-cty/cty"
"github.com/hashicorp/terraform/internal/addrs"
"github.com/hashicorp/terraform/internal/configs"
"github.com/hashicorp/terraform/internal/instances"
"github.com/hashicorp/terraform/internal/plans"
"github.com/hashicorp/terraform/internal/refactoring"
"github.com/hashicorp/terraform/internal/states"
"github.com/hashicorp/terraform/internal/tfdiags"
)
// PlanOpts are the various options that affect the details of how Terraform
// will build a plan.
type PlanOpts struct {
Mode plans.Mode
SkipRefresh bool
SetVariables InputValues
Targets []addrs.Targetable
ForceReplace []addrs.AbsResourceInstance
}
// Plan generates an execution plan for the given context, and returns the
// refreshed state.
//
// The execution plan encapsulates the context and can be stored
// in order to reinstantiate a context later for Apply.
//
// Plan also updates the diff of this context to be the diff generated
// by the plan, so Apply can be called after.
func (c *Context) Plan(config *configs.Config, prevRunState *states.State, opts *PlanOpts) (*plans.Plan, tfdiags.Diagnostics) {
defer c.acquireRun("plan")()
var diags tfdiags.Diagnostics
// Save the downstream functions from needing to deal with these broken situations.
// No real callers should rely on these, but we have a bunch of old and
// sloppy tests that don't always populate arguments properly.
if config == nil {
config = configs.NewEmptyConfig()
}
if prevRunState == nil {
prevRunState = states.NewState()
}
if opts == nil {
opts = &PlanOpts{
Mode: plans.NormalMode,
}
}
core: Simplify and centralize plugin availability checks Historically the responsibility for making sure that all of the available providers are of suitable versions and match the appropriate checksums has been split rather inexplicably over multiple different layers, with some of the checks happening as late as creating a terraform.Context. We're gradually iterating towards making that all be handled in one place, but in this step we're just cleaning up some old remnants from the main "terraform" package, which is now no longer responsible for any version or checksum verification and instead just assumes it's been provided with suitable factory functions by its caller. We do still have a pre-check here to make sure that we at least have a factory function for each plugin the configuration seems to depend on, because if we don't do that up front then it ends up getting caught instead deep inside the Terraform runtime, often inside a concurrent graph walk and thus it's not deterministic which codepath will happen to catch it on a particular run. As of this commit, this actually does leave some holes in our checks: the command package is using the dependency lock file to make sure we have exactly the provider packages we expect (exact versions and checksums), which is the most crucial part, but we don't yet have any spot where we make sure that the lock file is consistent with the current configuration, and we are no longer preserving the provider checksums as part of a saved plan. Both of those will come in subsequent commits. While it's unusual to have a series of commits that briefly subtracts functionality and then adds back in equivalent functionality later, the lock file checking is the only part that's crucial for security reasons, with everything else mainly just being to give better feedback when folks seem to be using Terraform incorrectly. The other bits are therefore mostly cosmetic and okay to be absent briefly as we work towards a better design that is clearer about where that responsibility belongs.
2021-09-29 17:51:09 -05:00
moreDiags := c.checkConfigDependencies(config)
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
diags = diags.Append(moreDiags)
core: Simplify and centralize plugin availability checks Historically the responsibility for making sure that all of the available providers are of suitable versions and match the appropriate checksums has been split rather inexplicably over multiple different layers, with some of the checks happening as late as creating a terraform.Context. We're gradually iterating towards making that all be handled in one place, but in this step we're just cleaning up some old remnants from the main "terraform" package, which is now no longer responsible for any version or checksum verification and instead just assumes it's been provided with suitable factory functions by its caller. We do still have a pre-check here to make sure that we at least have a factory function for each plugin the configuration seems to depend on, because if we don't do that up front then it ends up getting caught instead deep inside the Terraform runtime, often inside a concurrent graph walk and thus it's not deterministic which codepath will happen to catch it on a particular run. As of this commit, this actually does leave some holes in our checks: the command package is using the dependency lock file to make sure we have exactly the provider packages we expect (exact versions and checksums), which is the most crucial part, but we don't yet have any spot where we make sure that the lock file is consistent with the current configuration, and we are no longer preserving the provider checksums as part of a saved plan. Both of those will come in subsequent commits. While it's unusual to have a series of commits that briefly subtracts functionality and then adds back in equivalent functionality later, the lock file checking is the only part that's crucial for security reasons, with everything else mainly just being to give better feedback when folks seem to be using Terraform incorrectly. The other bits are therefore mostly cosmetic and okay to be absent briefly as we work towards a better design that is clearer about where that responsibility belongs.
2021-09-29 17:51:09 -05:00
// If required dependencies are not available then we'll bail early since
// otherwise we're likely to just see a bunch of other errors related to
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
// incompatibilities, which could be overwhelming for the user.
if diags.HasErrors() {
return nil, diags
}
switch opts.Mode {
case plans.NormalMode, plans.DestroyMode:
// OK
case plans.RefreshOnlyMode:
if opts.SkipRefresh {
// The CLI layer (and other similar callers) should prevent this
// combination of options.
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Incompatible plan options",
"Cannot skip refreshing in refresh-only mode. This is a bug in Terraform.",
))
return nil, diags
}
default:
// The CLI layer (and other similar callers) should not try to
// create a context for a mode that Terraform Core doesn't support.
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Unsupported plan mode",
fmt.Sprintf("Terraform Core doesn't know how to handle plan mode %s. This is a bug in Terraform.", opts.Mode),
))
return nil, diags
}
if len(opts.ForceReplace) > 0 && opts.Mode != plans.NormalMode {
// The other modes don't generate no-op or update actions that we might
// upgrade to be "replace", so doesn't make sense to combine those.
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Unsupported plan mode",
"Forcing resource instance replacement (with -replace=...) is allowed only in normal planning mode.",
))
return nil, diags
}
variables := mergeDefaultInputVariableValues(opts.SetVariables, config.Module.Variables)
// By the time we get here, we should have values defined for all of
// the root module variables, even if some of them are "unknown". It's the
// caller's responsibility to have already handled the decoding of these
// from the various ways the CLI allows them to be set and to produce
// user-friendly error messages if they are not all present, and so
// the error message from checkInputVariables should never be seen and
// includes language asking the user to report a bug.
varDiags := checkInputVariables(config.Module.Variables, variables)
diags = diags.Append(varDiags)
if len(opts.Targets) > 0 {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Warning,
"Resource targeting is in effect",
`You are creating a plan with the -target option, which means that the result of this plan may not represent all of the changes requested by the current configuration.
core: Fail if a moved resource instance is excluded by -target Because "moved" blocks produce changes that span across more than one resource instance address at the same time, we need to take extra care with them during planning. The -target option allows for restricting Terraform's attention only to a subset of resources when planning, as an escape hatch to recover from bugs and mistakes. However, we need to avoid any situation where only one "side" of a move would be considered in a particular plan, because that'd create a new situation that would be otherwise unreachable and would be difficult to recover from. As a compromise then, we'll reject an attempt to create a targeted plan if the plan involves resolving a pending move and if the source address of that move is not included in the targets. Our error message offers the user two possible resolutions: to create an untargeted plan, thus allowing everything to resolve, or to add additional -target options to include just the existing resource instances that have pending moves to resolve. This compromise recognizes that it is possible -- though hopefully rare -- that a user could potentially both be recovering from a bug or mistake at the same time as processing a move, if e.g. the bug was fixed by upgrading a module and the new version includes a new "moved" block. In that edge case, it might be necessary to just add the one additional address to the targets rather than removing the targets altogether, if creating a normal untargeted plan is impossible due to whatever bug they're trying to recover from.
2021-09-15 17:54:16 -05:00
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
The -target option is not for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when Terraform specifically suggests to use it as part of an error message.`,
))
}
var plan *plans.Plan
var planDiags tfdiags.Diagnostics
switch opts.Mode {
case plans.NormalMode:
plan, planDiags = c.plan(config, prevRunState, variables, opts)
case plans.DestroyMode:
plan, planDiags = c.destroyPlan(config, prevRunState, variables, opts)
case plans.RefreshOnlyMode:
plan, planDiags = c.refreshOnlyPlan(config, prevRunState, variables, opts)
default:
panic(fmt.Sprintf("unsupported plan mode %s", opts.Mode))
}
diags = diags.Append(planDiags)
if diags.HasErrors() {
return nil, diags
}
// convert the variables into the format expected for the plan
varVals := make(map[string]plans.DynamicValue, len(variables))
for k, iv := range variables {
// We use cty.DynamicPseudoType here so that we'll save both the
// value _and_ its dynamic type in the plan, so we can recover
// exactly the same value later.
dv, err := plans.NewDynamicValue(iv.Value, cty.DynamicPseudoType)
if err != nil {
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Failed to prepare variable value for plan",
fmt.Sprintf("The value for variable %q could not be serialized to store in the plan: %s.", k, err),
))
continue
}
varVals[k] = dv
}
// insert the run-specific data from the context into the plan; variables,
// targets and provider SHAs.
if plan != nil {
plan.VariableValues = varVals
plan.TargetAddrs = opts.Targets
} else if !diags.HasErrors() {
panic("nil plan but no errors")
}
return plan, diags
}
var DefaultPlanOpts = &PlanOpts{
Mode: plans.NormalMode,
}
func (c *Context) plan(config *configs.Config, prevRunState *states.State, rootVariables InputValues, opts *PlanOpts) (*plans.Plan, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
if opts.Mode != plans.NormalMode {
panic(fmt.Sprintf("called Context.plan with %s", opts.Mode))
}
plan, walkDiags := c.planWalk(config, prevRunState, rootVariables, opts)
diags = diags.Append(walkDiags)
if diags.HasErrors() {
return nil, diags
}
// The refreshed state ends up with some placeholder objects in it for
// objects pending creation. We only really care about those being in
// the working state, since that's what we're going to use when applying,
// so we'll prune them all here.
plan.PriorState.SyncWrapper().RemovePlannedResourceInstanceObjects()
return plan, diags
}
func (c *Context) refreshOnlyPlan(config *configs.Config, prevRunState *states.State, rootVariables InputValues, opts *PlanOpts) (*plans.Plan, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
if opts.Mode != plans.RefreshOnlyMode {
panic(fmt.Sprintf("called Context.refreshOnlyPlan with %s", opts.Mode))
}
plan, walkDiags := c.planWalk(config, prevRunState, rootVariables, opts)
diags = diags.Append(walkDiags)
if diags.HasErrors() {
return nil, diags
}
// If the graph builder and graph nodes correctly obeyed our directive
// to refresh only, the set of resource changes should always be empty.
// We'll safety-check that here so we can return a clear message about it,
// rather than probably just generating confusing output at the UI layer.
if len(plan.Changes.Resources) != 0 {
// Some extra context in the logs in case the user reports this message
// as a bug, as a starting point for debugging.
for _, rc := range plan.Changes.Resources {
if depKey := rc.DeposedKey; depKey == states.NotDeposed {
log.Printf("[DEBUG] Refresh-only plan includes %s change for %s", rc.Action, rc.Addr)
} else {
log.Printf("[DEBUG] Refresh-only plan includes %s change for %s deposed object %s", rc.Action, rc.Addr, depKey)
}
}
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Invalid refresh-only plan",
"Terraform generated planned resource changes in a refresh-only plan. This is a bug in Terraform.",
))
}
// Prune out any placeholder objects we put in the state to represent
// objects that would need to be created.
plan.PriorState.SyncWrapper().RemovePlannedResourceInstanceObjects()
return plan, diags
}
func (c *Context) destroyPlan(config *configs.Config, prevRunState *states.State, rootVariables InputValues, opts *PlanOpts) (*plans.Plan, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
pendingPlan := &plans.Plan{}
if opts.Mode != plans.DestroyMode {
panic(fmt.Sprintf("called Context.destroyPlan with %s", opts.Mode))
}
priorState := prevRunState
// A destroy plan starts by running Refresh to read any pending data
// sources, and remove missing managed resources. This is required because
// a "destroy plan" is only creating delete changes, and is essentially a
// local operation.
//
// NOTE: if skipRefresh _is_ set then we'll rely on the destroy-plan walk
// below to upgrade the prevRunState and priorState both to the latest
// resource type schemas, so NodePlanDestroyableResourceInstance.Execute
// must coordinate with this by taking that action only when c.skipRefresh
// _is_ set. This coupling between the two is unfortunate but necessary
// to work within our current structure.
if !opts.SkipRefresh {
log.Printf("[TRACE] Context.destroyPlan: calling Context.plan to get the effect of refreshing the prior state")
normalOpts := *opts
normalOpts.Mode = plans.NormalMode
refreshPlan, refreshDiags := c.plan(config, prevRunState, rootVariables, &normalOpts)
if refreshDiags.HasErrors() {
// NOTE: Normally we'd append diagnostics regardless of whether
// there are errors, just in case there are warnings we'd want to
// preserve, but we're intentionally _not_ doing that here because
// if the first plan succeeded then we'll be running another plan
// in DestroyMode below, and we don't want to double-up any
// warnings that both plan walks would generate.
// (This does mean we won't show any warnings that would've been
// unique to only this walk, but we're assuming here that if the
// warnings aren't also applicable to a destroy plan then we'd
// rather not show them here, because this non-destroy plan for
// refreshing is largely an implementation detail.)
diags = diags.Append(refreshDiags)
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
return nil, diags
}
// insert the refreshed state into the destroy plan result, and ignore
// the changes recorded from the refresh.
pendingPlan.PriorState = refreshPlan.PriorState.DeepCopy()
pendingPlan.PrevRunState = refreshPlan.PrevRunState.DeepCopy()
log.Printf("[TRACE] Context.destroyPlan: now _really_ creating a destroy plan")
// We'll use the refreshed state -- which is the "prior state" from
// the perspective of this "pending plan" -- as the starting state
// for our destroy-plan walk, so it can take into account if we
// detected during refreshing that anything was already deleted outside
// of Terraform.
priorState = pendingPlan.PriorState
}
destroyPlan, walkDiags := c.planWalk(config, priorState, rootVariables, opts)
diags = diags.Append(walkDiags)
if walkDiags.HasErrors() {
return nil, diags
}
if !opts.SkipRefresh {
// If we didn't skip refreshing then we want the previous run state
// prior state to be the one we originally fed into the c.plan call
// above, not the refreshed version we used for the destroy walk.
destroyPlan.PrevRunState = pendingPlan.PrevRunState
}
return destroyPlan, diags
}
func (c *Context) prePlanFindAndApplyMoves(config *configs.Config, prevRunState *states.State, targets []addrs.Targetable) ([]refactoring.MoveStatement, refactoring.MoveResults) {
core: refactoring.ImpliedMoveStatements replaces NodeCountBoundary Going back a long time we've had a special magic behavior which tries to recognize a situation where a module author either added or removed the "count" argument from a resource that already has instances, and to silently rename the zeroth or no-key instance so that we don't plan to destroy and recreate the associated object. Now we have a more general idea of "move statements", and specifically the idea of "implied" move statements which replicates the same heuristic we used to use for this behavior, we can treat this magic renaming rule as just another "move statement", special only in that Terraform generates it automatically rather than it being written out explicitly in the configuration. In return for wiring that in, we can now remove altogether the NodeCountBoundary graph node type and its associated graph transformer, CountBoundaryTransformer. We handle moves as a preprocessing step before building the plan graph, so we no longer need to include any special nodes in the graph to deal with that situation. The test updates here are mainly for the graph builders themselves, to acknowledge that indeed we're no longer inserting the NodeCountBoundary vertices. The vertices that NodeCountBoundary previously depended on now become dependencies of the special "root" vertex, although in many cases here we don't see that explicitly because of the transitive reduction algorithm, which notices when there's already an equivalent indirect dependency chain and removes the redundant edge. We already have plenty of test coverage for these "count boundary" cases in the context tests whose names start with TestContext2Plan_count and TestContext2Apply_resourceCount, all of which continued to pass here without any modification and so are not visible in the diff. The test functions particularly relevant to this situation are: - TestContext2Plan_countIncreaseFromNotSet - TestContext2Plan_countDecreaseToOne - TestContext2Plan_countOneIndex - TestContext2Apply_countDecreaseToOneCorrupted The last of those in particular deals with the situation where we have both a no-key instance _and_ a zero-key instance in the prior state, which is interesting here because to exercises an intentional interaction between refactoring.ImpliedMoveStatements and refactoring.ApplyMoves, where we intentionally generate an implied move statement that produces a collision and then expect ApplyMoves to deal with it in the same way as it would deal with all other collisions, and thus ensure we handle both the explicit and implied collisions in the same way. This does affect some UI-level tests, because a nice side-effect of this new treatment of this old feature is that we can now report explicitly in the UI that we're assigning new addresses to these objects, whereas before we just said nothing and hoped the user would just guess what had happened and why they therefore weren't seeing a diff. The backend/local plan tests actually had a pre-existing bug where they were using a state with a different instance key than the config called for but getting away with it because we'd previously silently fix it up. That's still fixed up, but now done with an explicit mention in the UI and so I made the state consistent with the configuration here so that the tests would be able to recognize _real_ differences where present, as opposed to the errant difference caused by that inconsistency.
2021-09-17 17:32:32 -05:00
explicitMoveStmts := refactoring.FindMoveStatements(config)
implicitMoveStmts := refactoring.ImpliedMoveStatements(config, prevRunState, explicitMoveStmts)
var moveStmts []refactoring.MoveStatement
if stmtsLen := len(explicitMoveStmts) + len(implicitMoveStmts); stmtsLen > 0 {
moveStmts = make([]refactoring.MoveStatement, 0, stmtsLen)
moveStmts = append(moveStmts, explicitMoveStmts...)
moveStmts = append(moveStmts, implicitMoveStmts...)
}
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
moveResults := refactoring.ApplyMoves(moveStmts, prevRunState)
core: Fail if a moved resource instance is excluded by -target Because "moved" blocks produce changes that span across more than one resource instance address at the same time, we need to take extra care with them during planning. The -target option allows for restricting Terraform's attention only to a subset of resources when planning, as an escape hatch to recover from bugs and mistakes. However, we need to avoid any situation where only one "side" of a move would be considered in a particular plan, because that'd create a new situation that would be otherwise unreachable and would be difficult to recover from. As a compromise then, we'll reject an attempt to create a targeted plan if the plan involves resolving a pending move and if the source address of that move is not included in the targets. Our error message offers the user two possible resolutions: to create an untargeted plan, thus allowing everything to resolve, or to add additional -target options to include just the existing resource instances that have pending moves to resolve. This compromise recognizes that it is possible -- though hopefully rare -- that a user could potentially both be recovering from a bug or mistake at the same time as processing a move, if e.g. the bug was fixed by upgrading a module and the new version includes a new "moved" block. In that edge case, it might be necessary to just add the one additional address to the targets rather than removing the targets altogether, if creating a normal untargeted plan is impossible due to whatever bug they're trying to recover from.
2021-09-15 17:54:16 -05:00
return moveStmts, moveResults
}
func (c *Context) prePlanVerifyTargetedMoves(moveResults refactoring.MoveResults, targets []addrs.Targetable) tfdiags.Diagnostics {
core: Fail if a moved resource instance is excluded by -target Because "moved" blocks produce changes that span across more than one resource instance address at the same time, we need to take extra care with them during planning. The -target option allows for restricting Terraform's attention only to a subset of resources when planning, as an escape hatch to recover from bugs and mistakes. However, we need to avoid any situation where only one "side" of a move would be considered in a particular plan, because that'd create a new situation that would be otherwise unreachable and would be difficult to recover from. As a compromise then, we'll reject an attempt to create a targeted plan if the plan involves resolving a pending move and if the source address of that move is not included in the targets. Our error message offers the user two possible resolutions: to create an untargeted plan, thus allowing everything to resolve, or to add additional -target options to include just the existing resource instances that have pending moves to resolve. This compromise recognizes that it is possible -- though hopefully rare -- that a user could potentially both be recovering from a bug or mistake at the same time as processing a move, if e.g. the bug was fixed by upgrading a module and the new version includes a new "moved" block. In that edge case, it might be necessary to just add the one additional address to the targets rather than removing the targets altogether, if creating a normal untargeted plan is impossible due to whatever bug they're trying to recover from.
2021-09-15 17:54:16 -05:00
if len(targets) < 1 {
return nil // the following only matters when targeting
}
var diags tfdiags.Diagnostics
var excluded []addrs.AbsResourceInstance
for _, result := range moveResults.Changes {
core: Fail if a moved resource instance is excluded by -target Because "moved" blocks produce changes that span across more than one resource instance address at the same time, we need to take extra care with them during planning. The -target option allows for restricting Terraform's attention only to a subset of resources when planning, as an escape hatch to recover from bugs and mistakes. However, we need to avoid any situation where only one "side" of a move would be considered in a particular plan, because that'd create a new situation that would be otherwise unreachable and would be difficult to recover from. As a compromise then, we'll reject an attempt to create a targeted plan if the plan involves resolving a pending move and if the source address of that move is not included in the targets. Our error message offers the user two possible resolutions: to create an untargeted plan, thus allowing everything to resolve, or to add additional -target options to include just the existing resource instances that have pending moves to resolve. This compromise recognizes that it is possible -- though hopefully rare -- that a user could potentially both be recovering from a bug or mistake at the same time as processing a move, if e.g. the bug was fixed by upgrading a module and the new version includes a new "moved" block. In that edge case, it might be necessary to just add the one additional address to the targets rather than removing the targets altogether, if creating a normal untargeted plan is impossible due to whatever bug they're trying to recover from.
2021-09-15 17:54:16 -05:00
fromMatchesTarget := false
toMatchesTarget := false
for _, targetAddr := range targets {
if targetAddr.TargetContains(result.From) {
fromMatchesTarget = true
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
}
core: Fail if a moved resource instance is excluded by -target Because "moved" blocks produce changes that span across more than one resource instance address at the same time, we need to take extra care with them during planning. The -target option allows for restricting Terraform's attention only to a subset of resources when planning, as an escape hatch to recover from bugs and mistakes. However, we need to avoid any situation where only one "side" of a move would be considered in a particular plan, because that'd create a new situation that would be otherwise unreachable and would be difficult to recover from. As a compromise then, we'll reject an attempt to create a targeted plan if the plan involves resolving a pending move and if the source address of that move is not included in the targets. Our error message offers the user two possible resolutions: to create an untargeted plan, thus allowing everything to resolve, or to add additional -target options to include just the existing resource instances that have pending moves to resolve. This compromise recognizes that it is possible -- though hopefully rare -- that a user could potentially both be recovering from a bug or mistake at the same time as processing a move, if e.g. the bug was fixed by upgrading a module and the new version includes a new "moved" block. In that edge case, it might be necessary to just add the one additional address to the targets rather than removing the targets altogether, if creating a normal untargeted plan is impossible due to whatever bug they're trying to recover from.
2021-09-15 17:54:16 -05:00
if targetAddr.TargetContains(result.To) {
toMatchesTarget = true
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
}
}
core: Fail if a moved resource instance is excluded by -target Because "moved" blocks produce changes that span across more than one resource instance address at the same time, we need to take extra care with them during planning. The -target option allows for restricting Terraform's attention only to a subset of resources when planning, as an escape hatch to recover from bugs and mistakes. However, we need to avoid any situation where only one "side" of a move would be considered in a particular plan, because that'd create a new situation that would be otherwise unreachable and would be difficult to recover from. As a compromise then, we'll reject an attempt to create a targeted plan if the plan involves resolving a pending move and if the source address of that move is not included in the targets. Our error message offers the user two possible resolutions: to create an untargeted plan, thus allowing everything to resolve, or to add additional -target options to include just the existing resource instances that have pending moves to resolve. This compromise recognizes that it is possible -- though hopefully rare -- that a user could potentially both be recovering from a bug or mistake at the same time as processing a move, if e.g. the bug was fixed by upgrading a module and the new version includes a new "moved" block. In that edge case, it might be necessary to just add the one additional address to the targets rather than removing the targets altogether, if creating a normal untargeted plan is impossible due to whatever bug they're trying to recover from.
2021-09-15 17:54:16 -05:00
if !fromMatchesTarget {
excluded = append(excluded, result.From)
}
if !toMatchesTarget {
excluded = append(excluded, result.To)
}
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
}
core: Fail if a moved resource instance is excluded by -target Because "moved" blocks produce changes that span across more than one resource instance address at the same time, we need to take extra care with them during planning. The -target option allows for restricting Terraform's attention only to a subset of resources when planning, as an escape hatch to recover from bugs and mistakes. However, we need to avoid any situation where only one "side" of a move would be considered in a particular plan, because that'd create a new situation that would be otherwise unreachable and would be difficult to recover from. As a compromise then, we'll reject an attempt to create a targeted plan if the plan involves resolving a pending move and if the source address of that move is not included in the targets. Our error message offers the user two possible resolutions: to create an untargeted plan, thus allowing everything to resolve, or to add additional -target options to include just the existing resource instances that have pending moves to resolve. This compromise recognizes that it is possible -- though hopefully rare -- that a user could potentially both be recovering from a bug or mistake at the same time as processing a move, if e.g. the bug was fixed by upgrading a module and the new version includes a new "moved" block. In that edge case, it might be necessary to just add the one additional address to the targets rather than removing the targets altogether, if creating a normal untargeted plan is impossible due to whatever bug they're trying to recover from.
2021-09-15 17:54:16 -05:00
if len(excluded) > 0 {
sort.Slice(excluded, func(i, j int) bool {
return excluded[i].Less(excluded[j])
})
var listBuf strings.Builder
var prevResourceAddr addrs.AbsResource
for _, instAddr := range excluded {
// Targeting generally ends up selecting whole resources rather
// than individual instances, because we don't factor in
// individual instances until DynamicExpand, so we're going to
// always show whole resource addresses here, excluding any
// instance keys. (This also neatly avoids dealing with the
// different quoting styles required for string instance keys
// on different shells, which is handy.)
//
// To avoid showing duplicates when we have multiple instances
// of the same resource, we'll remember the most recent
// resource we rendered in prevResource, which is sufficient
// because we sorted the list of instance addresses above, and
// our sort order always groups together instances of the same
// resource.
resourceAddr := instAddr.ContainingResource()
if resourceAddr.Equal(prevResourceAddr) {
continue
}
fmt.Fprintf(&listBuf, "\n -target=%q", resourceAddr.String())
prevResourceAddr = resourceAddr
}
diags = diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Moved resource instances excluded by targeting",
fmt.Sprintf(
"Resource instances in your current state have moved to new addresses in the latest configuration. Terraform must include those resource instances while planning in order to ensure a correct result, but your -target=... options to not fully cover all of those resource instances.\n\nTo create a valid plan, either remove your -target=... options altogether or add the following additional target options:%s\n\nNote that adding these options may include further additional resource instances in your plan, in order to respect object dependencies.",
listBuf.String(),
),
))
}
return diags
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
}
func (c *Context) postPlanValidateMoves(config *configs.Config, stmts []refactoring.MoveStatement, allInsts instances.Set) tfdiags.Diagnostics {
return refactoring.ValidateMoves(stmts, config, allInsts)
}
func (c *Context) planWalk(config *configs.Config, prevRunState *states.State, rootVariables InputValues, opts *PlanOpts) (*plans.Plan, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
log.Printf("[DEBUG] Building and walking plan graph for %s", opts.Mode)
prevRunState = prevRunState.DeepCopy() // don't modify the caller's object when we process the moves
moveStmts, moveResults := c.prePlanFindAndApplyMoves(config, prevRunState, opts.Targets)
core: Fail if a moved resource instance is excluded by -target Because "moved" blocks produce changes that span across more than one resource instance address at the same time, we need to take extra care with them during planning. The -target option allows for restricting Terraform's attention only to a subset of resources when planning, as an escape hatch to recover from bugs and mistakes. However, we need to avoid any situation where only one "side" of a move would be considered in a particular plan, because that'd create a new situation that would be otherwise unreachable and would be difficult to recover from. As a compromise then, we'll reject an attempt to create a targeted plan if the plan involves resolving a pending move and if the source address of that move is not included in the targets. Our error message offers the user two possible resolutions: to create an untargeted plan, thus allowing everything to resolve, or to add additional -target options to include just the existing resource instances that have pending moves to resolve. This compromise recognizes that it is possible -- though hopefully rare -- that a user could potentially both be recovering from a bug or mistake at the same time as processing a move, if e.g. the bug was fixed by upgrading a module and the new version includes a new "moved" block. In that edge case, it might be necessary to just add the one additional address to the targets rather than removing the targets altogether, if creating a normal untargeted plan is impossible due to whatever bug they're trying to recover from.
2021-09-15 17:54:16 -05:00
// If resource targeting is in effect then it might conflict with the
// move result.
diags = diags.Append(c.prePlanVerifyTargetedMoves(moveResults, opts.Targets))
if diags.HasErrors() {
// We'll return early here, because if we have any moved resource
// instances excluded by targeting then planning is likely to encounter
// strange problems that may lead to confusing error messages.
return nil, diags
}
core: Graph walk loads plugin schemas opportunistically Previously our graph walker expected to recieve a data structure containing schemas for all of the provider and provisioner plugins used in the configuration and state. That made sense back when terraform.NewContext was responsible for loading all of the schemas before taking any other action, but it no longer has that responsiblity. Instead, we'll now make sure that the "contextPlugins" object reaches all of the locations where we need schema -- many of which already had access to that object anyway -- and then load the needed schemas just in time. The contextPlugins object memoizes schema lookups, so we can safely call it many times with the same provider address or provisioner type name and know that it'll still only load each distinct plugin once per Context object. As of this commit, the Context.Schemas method is now a public interface only and not used by logic in the "terraform" package at all. However, that does leave us in a rather tenuous situation of relying on the fact that all practical users of terraform.Context end up calling "Schemas" at some point in order to verify that we have all of the expected versions of plugins. That's a non-obvious implicit dependency, and so in subsequent commits we'll gradually move all responsibility for verifying plugin versions into the caller of terraform.NewContext, which'll heal a long-standing architectural wart whereby the caller is responsible for installing and locating the plugin executables but not for verifying that what's installed is conforming to the current configuration and dependency lock file.
2021-08-31 19:53:03 -05:00
graph, walkOp, moreDiags := c.planGraph(config, prevRunState, opts, true)
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
diags = diags.Append(moreDiags)
if diags.HasErrors() {
return nil, diags
}
// If we get here then we should definitely have a non-nil "graph", which
// we can now walk.
changes := plans.NewChanges()
walker, walkDiags := c.walk(graph, walkOp, &graphWalkOpts{
Config: config,
InputState: prevRunState,
Changes: changes,
MoveResults: moveResults,
RootVariableValues: rootVariables,
})
diags = diags.Append(walker.NonFatalDiagnostics)
diags = diags.Append(walkDiags)
moveValidateDiags := c.postPlanValidateMoves(config, moveStmts, walker.InstanceExpander.AllInstances())
if moveValidateDiags.HasErrors() {
// If any of the move statements are invalid then those errors take
// precedence over any other errors because an incomplete move graph
// is quite likely to be the _cause_ of various errors. This oddity
// comes from the fact that we need to apply the moves before we
// actually validate them, because validation depends on the result
// of first trying to plan.
return nil, moveValidateDiags
}
diags = diags.Append(moveValidateDiags) // might just contain warnings
core: Report a warning if any moves get blocked In most cases Terraform will be able to automatically fully resolve all of the pending move statements before creating a plan, but there are some edge cases where we can end up wanting to move one object to a location where another object is already declared. One relatively-obvious example is if someone uses "terraform state mv" in order to create a set of resource instance bindings that could never have arising in normal Terraform use. A less obvious example arises from the interactions between moves at different levels of granularity. If we are both moving a module to a new address and moving a resource into an instance of the new module at the same time, the old module might well have already had a resource of the same name and so the resource move will be unresolvable. In these situations Terraform will move the objects as far as possible, but because it's never valid for a move "from" address to still be declared in the configuration Terraform will inevitably always plan to destroy the objects that didn't find a final home. To give some additional explanation for that result, here we'll add a warning which describes what happened. This is not a particularly actionable warning because we don't really have enough information to guess what the user intended, but we do at least prompt that they might be able to use the "terraform state" family of subcommands to repair the ambiguous situation before planning, if they want a different result than what Terraform proposed.
2021-09-23 13:18:29 -05:00
if len(moveResults.Blocked) > 0 && !diags.HasErrors() {
// If we had blocked moves and we're not going to be returning errors
// then we'll report the blockers as a warning. We do this only in the
// absense of errors because invalid move statements might well be
// the root cause of the blockers, and so better to give an actionable
// error message than a less-actionable warning.
diags = diags.Append(blockedMovesWarningDiag(moveResults))
}
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
prevRunState = walker.PrevRunState.Close()
priorState := walker.RefreshState.Close()
driftedResources, driftDiags := c.driftedResources(config, prevRunState, priorState, moveResults)
diags = diags.Append(driftDiags)
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
plan := &plans.Plan{
UIMode: opts.Mode,
Changes: changes,
DriftedResources: driftedResources,
PrevRunState: prevRunState,
PriorState: priorState,
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
// Other fields get populated by Context.Plan after we return
}
return plan, diags
}
core: Graph walk loads plugin schemas opportunistically Previously our graph walker expected to recieve a data structure containing schemas for all of the provider and provisioner plugins used in the configuration and state. That made sense back when terraform.NewContext was responsible for loading all of the schemas before taking any other action, but it no longer has that responsiblity. Instead, we'll now make sure that the "contextPlugins" object reaches all of the locations where we need schema -- many of which already had access to that object anyway -- and then load the needed schemas just in time. The contextPlugins object memoizes schema lookups, so we can safely call it many times with the same provider address or provisioner type name and know that it'll still only load each distinct plugin once per Context object. As of this commit, the Context.Schemas method is now a public interface only and not used by logic in the "terraform" package at all. However, that does leave us in a rather tenuous situation of relying on the fact that all practical users of terraform.Context end up calling "Schemas" at some point in order to verify that we have all of the expected versions of plugins. That's a non-obvious implicit dependency, and so in subsequent commits we'll gradually move all responsibility for verifying plugin versions into the caller of terraform.NewContext, which'll heal a long-standing architectural wart whereby the caller is responsible for installing and locating the plugin executables but not for verifying that what's installed is conforming to the current configuration and dependency lock file.
2021-08-31 19:53:03 -05:00
func (c *Context) planGraph(config *configs.Config, prevRunState *states.State, opts *PlanOpts, validate bool) (*Graph, walkOperation, tfdiags.Diagnostics) {
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
switch mode := opts.Mode; mode {
case plans.NormalMode:
graph, diags := (&PlanGraphBuilder{
Config: config,
State: prevRunState,
Plugins: c.plugins,
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
Targets: opts.Targets,
ForceReplace: opts.ForceReplace,
Validate: validate,
skipRefresh: opts.SkipRefresh,
}).Build(addrs.RootModuleInstance)
return graph, walkPlan, diags
case plans.RefreshOnlyMode:
graph, diags := (&PlanGraphBuilder{
Config: config,
State: prevRunState,
Plugins: c.plugins,
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
Targets: opts.Targets,
Validate: validate,
skipRefresh: opts.SkipRefresh,
skipPlanChanges: true, // this activates "refresh only" mode.
}).Build(addrs.RootModuleInstance)
return graph, walkPlan, diags
case plans.DestroyMode:
graph, diags := (&DestroyPlanGraphBuilder{
Config: config,
State: prevRunState,
Plugins: c.plugins,
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
Targets: opts.Targets,
Validate: validate,
skipRefresh: opts.SkipRefresh,
}).Build(addrs.RootModuleInstance)
return graph, walkPlanDestroy, diags
default:
// The above should cover all plans.Mode values
panic(fmt.Sprintf("unsupported plan mode %s", mode))
}
}
func (c *Context) driftedResources(config *configs.Config, oldState, newState *states.State, moves refactoring.MoveResults) ([]*plans.ResourceInstanceChangeSrc, tfdiags.Diagnostics) {
var diags tfdiags.Diagnostics
if newState.ManagedResourcesEqual(oldState) && len(moves.Changes) == 0 {
// Nothing to do, because we only detect and report drift for managed
// resource instances.
return nil, diags
}
schemas, schemaDiags := c.Schemas(config, newState)
diags = diags.Append(schemaDiags)
if diags.HasErrors() {
return nil, diags
}
var drs []*plans.ResourceInstanceChangeSrc
for _, ms := range oldState.Modules {
for _, rs := range ms.Resources {
if rs.Addr.Resource.Mode != addrs.ManagedResourceMode {
// Drift reporting is only for managed resources
continue
}
provider := rs.ProviderConfig.Provider
for key, oldIS := range rs.Instances {
if oldIS.Current == nil {
// Not interested in instances that only have deposed objects
continue
}
addr := rs.Addr.Instance(key)
// Previous run address defaults to the current address, but
// can differ if the resource moved before refreshing
prevRunAddr := addr
if move, ok := moves.Changes[addr.UniqueKey()]; ok {
prevRunAddr = move.From
}
newIS := newState.ResourceInstance(addr)
schema, _ := schemas.ResourceTypeConfig(
provider,
addr.Resource.Resource.Mode,
addr.Resource.Resource.Type,
)
if schema == nil {
// This should never happen, but just in case
return nil, diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Missing resource schema from provider",
fmt.Sprintf("No resource schema found for %s.", addr.Resource.Resource.Type),
))
}
ty := schema.ImpliedType()
oldObj, err := oldIS.Current.Decode(ty)
if err != nil {
// This should also never happen
return nil, diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Failed to decode resource from state",
fmt.Sprintf("Error decoding %q from previous state: %s", addr.String(), err),
))
}
var newObj *states.ResourceInstanceObject
if newIS != nil && newIS.Current != nil {
newObj, err = newIS.Current.Decode(ty)
if err != nil {
// This should also never happen
return nil, diags.Append(tfdiags.Sourceless(
tfdiags.Error,
"Failed to decode resource from state",
fmt.Sprintf("Error decoding %q from prior state: %s", addr.String(), err),
))
}
}
var oldVal, newVal cty.Value
oldVal = oldObj.Value
if newObj != nil {
newVal = newObj.Value
} else {
newVal = cty.NullVal(ty)
}
if oldVal.RawEquals(newVal) && addr.Equal(prevRunAddr) {
// No drift if the two values are semantically equivalent
// and no move has happened
continue
}
// We can detect three types of changes after refreshing state,
// only two of which are easily understood as "drift":
//
// - Resources which were deleted outside of Terraform;
// - Resources where the object value has changed outside of
// Terraform;
// - Resources which have been moved without other changes.
//
// All of these are returned as drift, to allow refresh-only plans
// to present a full set of changes which will be applied.
var action plans.Action
switch {
case newVal.IsNull():
action = plans.Delete
case !oldVal.RawEquals(newVal):
action = plans.Update
default:
action = plans.NoOp
}
change := &plans.ResourceInstanceChange{
Addr: addr,
PrevRunAddr: prevRunAddr,
ProviderAddr: rs.ProviderConfig,
Change: plans.Change{
Action: action,
Before: oldVal,
After: newVal,
},
}
changeSrc, err := change.Encode(ty)
if err != nil {
diags = diags.Append(err)
return nil, diags
}
drs = append(drs, changeSrc)
}
}
}
return drs, diags
}
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
// PlanGraphForUI is a last vestage of graphs in the public interface of Context
// (as opposed to graphs as an implementation detail) intended only for use
// by the "terraform graph" command when asked to render a plan-time graph.
//
// The result of this is intended only for rendering ot the user as a dot
// graph, and so may change in future in order to make the result more useful
// in that context, even if drifts away from the physical graph that Terraform
// Core currently uses as an implementation detail of planning.
func (c *Context) PlanGraphForUI(config *configs.Config, prevRunState *states.State, mode plans.Mode) (*Graph, tfdiags.Diagnostics) {
// For now though, this really is just the internal graph, confusing
// implementation details and all.
var diags tfdiags.Diagnostics
opts := &PlanOpts{Mode: mode}
core: Graph walk loads plugin schemas opportunistically Previously our graph walker expected to recieve a data structure containing schemas for all of the provider and provisioner plugins used in the configuration and state. That made sense back when terraform.NewContext was responsible for loading all of the schemas before taking any other action, but it no longer has that responsiblity. Instead, we'll now make sure that the "contextPlugins" object reaches all of the locations where we need schema -- many of which already had access to that object anyway -- and then load the needed schemas just in time. The contextPlugins object memoizes schema lookups, so we can safely call it many times with the same provider address or provisioner type name and know that it'll still only load each distinct plugin once per Context object. As of this commit, the Context.Schemas method is now a public interface only and not used by logic in the "terraform" package at all. However, that does leave us in a rather tenuous situation of relying on the fact that all practical users of terraform.Context end up calling "Schemas" at some point in order to verify that we have all of the expected versions of plugins. That's a non-obvious implicit dependency, and so in subsequent commits we'll gradually move all responsibility for verifying plugin versions into the caller of terraform.NewContext, which'll heal a long-standing architectural wart whereby the caller is responsible for installing and locating the plugin executables but not for verifying that what's installed is conforming to the current configuration and dependency lock file.
2021-08-31 19:53:03 -05:00
graph, _, moreDiags := c.planGraph(config, prevRunState, opts, false)
core: Functional-style API for terraform.Context Previously terraform.Context was built in an unfortunate way where all of the data was provided up front in terraform.NewContext and then mutated directly by subsequent operations. That made the data flow hard to follow, commonly leading to bugs, and also meant that we were forced to take various actions too early in terraform.NewContext, rather than waiting until a more appropriate time during an operation. This (enormous) commit changes terraform.Context so that its fields are broadly just unchanging data about the execution context (current workspace name, available plugins, etc) whereas the main data Terraform works with arrives via individual method arguments and is returned in return values. Specifically, this means that terraform.Context no longer "has-a" config, state, and "planned changes", instead holding on to those only temporarily during an operation. The caller is responsible for propagating the outcome of one step into the next step so that the data flow between operations is actually visible. However, since that's a change to the main entry points in the "terraform" package, this commit also touches every file in the codebase which interacted with those APIs. Most of the noise here is in updating tests to take the same actions using the new API style, but this also affects the main-code callers in the backends and in the command package. My goal here was to refactor without changing observable behavior, but in practice there are a couple externally-visible behavior variations here that seemed okay in service of the broader goal: - The "terraform graph" command is no longer hooked directly into the core graph builders, because that's no longer part of the public API. However, I did include a couple new Context functions whose contract is to produce a UI-oriented graph, and _for now_ those continue to return the physical graph we use for those operations. There's no exported API for generating the "validate" and "eval" graphs, because neither is particularly interesting in its own right, and so "terraform graph" no longer supports those graph types. - terraform.NewContext no longer has the responsibility for collecting all of the provider schemas up front. Instead, we wait until we need them. However, that means that some of our error messages now have a slightly different shape due to unwinding through a differently-shaped call stack. As of this commit we also end up reloading the schemas multiple times in some cases, which is functionally acceptable but likely represents a performance regression. I intend to rework this to use caching, but I'm saving that for a later commit because this one is big enough already. The proximal reason for this change is to resolve the chicken/egg problem whereby there was previously no single point where we could apply "moved" statements to the previous run state before creating a plan. With this change in place, we can now do that as part of Context.Plan, prior to forking the input state into the three separate state artifacts we use during planning. However, this is at least the third project in a row where the previous API design led to piling more functionality into terraform.NewContext and then working around the incorrect order of operations that produces, so I intend that by paying the cost/risk of this large diff now we can in turn reduce the cost/risk of future projects that relate to our main workflow actions.
2021-08-24 14:06:38 -05:00
diags = diags.Append(moreDiags)
return graph, diags
}
core: Report a warning if any moves get blocked In most cases Terraform will be able to automatically fully resolve all of the pending move statements before creating a plan, but there are some edge cases where we can end up wanting to move one object to a location where another object is already declared. One relatively-obvious example is if someone uses "terraform state mv" in order to create a set of resource instance bindings that could never have arising in normal Terraform use. A less obvious example arises from the interactions between moves at different levels of granularity. If we are both moving a module to a new address and moving a resource into an instance of the new module at the same time, the old module might well have already had a resource of the same name and so the resource move will be unresolvable. In these situations Terraform will move the objects as far as possible, but because it's never valid for a move "from" address to still be declared in the configuration Terraform will inevitably always plan to destroy the objects that didn't find a final home. To give some additional explanation for that result, here we'll add a warning which describes what happened. This is not a particularly actionable warning because we don't really have enough information to guess what the user intended, but we do at least prompt that they might be able to use the "terraform state" family of subcommands to repair the ambiguous situation before planning, if they want a different result than what Terraform proposed.
2021-09-23 13:18:29 -05:00
func blockedMovesWarningDiag(results refactoring.MoveResults) tfdiags.Diagnostic {
if len(results.Blocked) < 1 {
// Caller should check first
panic("request to render blocked moves warning without any blocked moves")
}
var itemsBuf bytes.Buffer
for _, blocked := range results.Blocked {
fmt.Fprintf(&itemsBuf, "\n - %s could not move to %s", blocked.Actual, blocked.Wanted)
}
return tfdiags.Sourceless(
tfdiags.Warning,
"Unresolved resource instance address changes",
fmt.Sprintf(
"Terraform tried to adjust resource instance addresses in the prior state based on change information recorded in the configuration, but some adjustments did not succeed due to existing objects already at the intended addresses:%s\n\nTerraform has planned to destroy these objects. If Terraform's proposed changes aren't appropriate, you must first resolve the conflicts using the \"terraform state\" subcommands and then create a new plan.",
itemsBuf.String(),
),
)
}