Although we don't promise that OpenTofu will work well with more than a
few hundred resource instances in a particular configuration, we know that
some people do use it that way and they sometimes find and share
opportunities for improving performance due to those opportunities being
easier to spot at large scale.
These two benchmarks aim to help us measure improvements that affect the
pure overhead of the core language runtime, without any real provider calls
and without the CLI layer's contributions, so that we can more easily
compare before and after of changes when we're either trying to understand
a reported performance regression or we're trying out potential performance
improvements.
As noted in the comments inline, the presence of these benchmarks is not
intended to represent a commitment to support using OpenTofu outside of its
design assumptions, but they will hopefully help us continue our current
best-effort approach to reducing overhead where we can do so without risk
to existing functionality, increasing maintenance overhead, or making it
harder to introduce new features over time.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
A lot of our testing utilities were written before the introduction of the
testing.TB interface that represents what *testing.T and *testing.B have
in common, and those that weren't have followed precedent from those that
were in directly expecting *testing.T.
Using testing.TB instead makes these helpers usable from both normal tests
and testing benchmark functions, since they only use methods that are
available in both contexts.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
Signed-off-by: Ilia Gogotchuri <ilia.gogotchuri0@gmail.com>
Co-authored-by: Christian Mesh <christianmesh1@gmail.com>
We're intending to gradually improve all of the existing functions that
fail these checks as a separate project from other work, because fixing
for these particular lint rules tends to be too invasive to be safe or
sensible to combine with other work.
Therefore we'll temporarily disable these lints from the main lint run
and add a separate .golangci-complexity.yml that we can use to track our
progress towards eliminating those lint failures without continuing to
litter the code with nolint comments in the meantime.
This also removes all of the existing nolint comments for these linters so
that we can start fresh and review each one as part of our improvement
project.
We'll re-enable these linters (and remove .golangci-complexity.yml) once
each example has either been rewritten to pass the checks or we've
concluded that further decomposition would hurt readability and so added
"nolint" comments back in so we can review whether our lint rules are too
strict once we've got a bunch of examples to consider together.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Using the word "iteration" to describe what count and for_each do tends to
confuse people because it sounds like explicit control flow rather than
just dynamically declaring multiple objects.
Elsewhere in the codebase we refer to this idea as "expansion" so this is
a rename for consistency with that and to remove the confusing terminology.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we made a very generic suggestion to use -target to split a
change into two parts as a workaround for the fact that count and for_each
must be known during planning. That works, but we didn't have enough
information available to tell the operator exactly what to target and so
anyone who is not an expert on the configuration they're working with tends
to get stuck unable to figure out exactly what they need to do.
The new -exclude option gives us an opportunity to do better here: we tend
to know for which object we're currently evaluating count or for_each, and
so we can mention that object directly in the error message when if we
recommend to use -exclude instead of -target.
Not all objects that support count/for_each will necessarily be directly
targetable, so we can still potentially recommend -target when we're
dealing with one of those objects. For example, as of this commit that
is true for for_each in a provider block, because there is not currently
any syntax for specifying a provider configuration as an addrs.Targetable.
Perhaps we'll introduce such a thing in the future, but that's outside the
scope of this change that's primarily focused on improving the messaging
for resource and module count/for_each.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Earlier commits arranged for each of our tofu.Context exported methods that
perform graph-based operations to take a context.Context from their
callers, and for the main callers in package command and package backend
to connect those contexts to the top-level context from "package main"
that can potentially have an OpenTelemetry span attached to it.
This propagates those contexts a little deeper into the guts of the
language runtime, getting it as far as the shared logic that drives a
graph walk.
The next step from here would be to change the interfaces
GraphNodeExecutable and GraphNodeDynamicExpandable so that their methods
both take a context.Context, but that would involve a big sprawling
update to every implementation of each of those interfaces and so
we'll save that for a later commit to keep this one relatively clean.
This commit also reaches the first point of ambiguity where our older
conventions call for "ctx" to be the variable name for a tofu.EvalContext
rather than a context.Context. Since "ctx context.Context" is a core idiom
in the Go community, we'll switch to using evalCtx as the variable name
for tofu.EvalContext both here and in our future commits that will
modify the two main graph walk interfaces that make extensive use of the
tofu.EvalContext interface.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Input now passes this through to the UI input implementation,
which was already written to handle cancellation. However, that
implementation currently handles interruption itself by directly watching
for the interrupt signal and so we remove the cancellation from the
context for now to avoid changing how interrupts are handled. Hopefully
in future we can remove the inline SIGINT handling from the UIInput
implementation and use its context-cancellation-handling instead.
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Validate doesn't yet do anything with its new context, but we'll
plumb this deeper in future.
Since the local backend's implementation of backend.Local.LocalRun calls
Validate on the given configuration before returning, it this also extends
that interface method to take a context, and so the various commands that
directly create "local runs" (rather than going through the backend
operation API) now all pass in a context derived from the one created
in "package main".
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Eval doesn't yet do anything with its new context, but we'll
plumb this deeper in future.
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Import doesn't yet do anything with its new context, but we'll
plumb this deeper in future.
OpenTofu has some historical situational private uses of context.Context
to handle the graceful shutdown behaviors. Those use context.Context as
a private implementation detail rather than public API, and so this commit
leaves them as-is and adds a new "primary context" alongside. Hopefully
in future refactoring we can simplify this to use the primary context also
as the primary cancellation signal, but that's too risky a change to bundle
in with this otherwise-mostly-harmless context plumbing.
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Refresh is really just a vestigal wrapper around Context.Plan, so
this just passes the given context through to Context.Plan which itself
currently ignores it.
OpenTofu has some historical situational private uses of context.Context
to handle the graceful shutdown behaviors. Those use context.Context as
a private implementation detail rather than public API, and so this commit
leaves them as-is and adds a new "primary context" alongside. Hopefully
in future refactoring we can simplify this to use the primary context also
as the primary cancellation signal, but that's too risky a change to bundle
in with this otherwise-mostly-harmless context plumbing.
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Apply does not yet do anything with its new context, but this gets
the context plumbed in enough that we should be able to pass values like
telemetry spans all the way from the top-level in future.
OpenTofu has some historical situational private uses of context.Context
to handle the graceful shutdown behaviors. Those use context.Context as
a private implementation detail rather than public API, and so this commit
leaves them as-is and adds a new "primary context" alongside. Hopefully
in future refactoring we can simplify this to use the primary context also
as the primary cancellation signal, but that's too risky a change to bundle
in with this otherwise-mostly-harmless context plumbing.
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Plan does not yet do anything with its new context, but this gets
the context plumbed in enough that we should be able to pass values like
telemetry spans all the way from the top-level in future.
OpenTofu has some historical situational private uses of context.Context
to handle the graceful shutdown behaviors. Those use context.Context as
a private implementation detail rather than public API, and so this commit
leaves them as-is and adds a new "primary context" alongside. Hopefully
in future refactoring we can simplify this to use the primary context also
as the primary cancellation signal, but that's too risky a change to bundle
in with this otherwise-mostly-harmless context plumbing.
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
