If the string to be tested is an unknown value that's been refined with
a prefix and the prefix we're being asked to test is in turn a prefix of
that known prefix then we can return a known answer despite the inputs
not being fully known.
There are also some other similar deductions we can make about other
combinations of inputs.
This extra analysis could be useful in a custom condition check that
requires a string with a particular prefix, since it can allow the
condition to fail even on partially-unknown input, thereby giving earlier
feedback about a problem.
cty's new "refinements" concept allows us to reduce the range of unknown
values from our functions. This initial changeset focuses only on
declaring which functions are guaranteed to return a non-null result,
which is a helpful baseline refinement because it allows "== null" and
"!= null" tests to produce known results even when the given value is
otherwise unknown.
This commit also includes some updates to test results that are now
refined based on cty's own built-in refinement behaviors, just as a
result of us having updated cty in the previous commit.
We inadvertently incorporated the new minor release of cty into the 1.4
branch, and that's introduced some more refined handling of unknown values
that is too much of a change to introduce in a patch release.
Therefore this reverts back to the previous minor release for the v1.4
series, and then we'll separately get the main branch ready to work
correctly with the new cty before Terraform v1.5.
This reverts just the upgrade and the corresponding test changes from
#32775, while retaining the HCL upgrade and the new test case it
introduced for that bug it was trying to fix. That new test is still
passing so it seems that the cty upgrade is not crucial to that fix.
This is a mostly mechanical refactor with a handful of changes which
are necessary due to the semantic difference between earlyconfig and
configs.
When parsing root and descendant modules in the module installer, we now
check the core version requirements inline. If the Terraform version is
incompatible, we drop any other module loader diagnostics. This ensures
that future language additions don't clutter the output and confuse the
user.
We also add two new checks during the module load process:
* Don't try to load a module with a `nil` source address. This is a
necessary change due to the move away from earlyconfig.
* Don't try to load a module with a blank name (i.e. `module ""`).
Because our module loading manifest uses the stringified module path
as its map key, this causes a collision with the root module, and a
later panic. This is the bug which triggered this refactor in the
first place.
* Add consolidated function description list
* Add function parameter descriptions
* Add descriptions to all functions
* Add sanity test for function descriptions
* Apply suggestions from code review
Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>
Co-authored-by: kmoe <5575356+kmoe@users.noreply.github.com>
This includes the fix for a bug in what Terraform calls the "yamldecode"
function, where it was not correctly handling any situation where the
decode result is a null value. It was previously returning an unknown
value in that case, whereas now it returns a null value as expected.
This is a complement to "timestamp" and "timeadd" which allows
establishing the ordering of two different timestamps while taking into
account their timezone offsets, which isn't otherwise possible using the
existing primitives in the Terraform language.
Go 1.19's "fmt" has some awareness of the new doc comment formatting
conventions and adjusts the presentation of the source comments to make
it clearer how godoc would interpret them. Therefore this commit includes
various updates made by "go fmt" to acheve that.
In line with our usual convention that we make stylistic/grammar/spelling
tweaks typically only when we're "in the area" changing something else
anyway, I also took this opportunity to review most of the comments that
this updated to see if there were any other opportunities to improve them.
We had intended these functions to attempt to convert any given value, but
there is a special behavior in the function system where functions must
opt in to being able to handle dynamically-typed arguments so that we
don't need to repeat the special case for that inside every function
implementation.
In this case we _do_ want to specially handle dynamically-typed values,
because the keyword "null" in HCL produces
cty.NullVal(cty.DynamicPseudoType) and we want the conversion function
to convert it to a null of a more specific type.
These conversion functions are already just a thin wrapper around the
underlying type conversion functionality anyway, and that already supports
converting dynamic-typed values in the expected way, so we can just opt
in to allowing dynamically-typed values and let the conversion
functionality do the expected work.
Fixing this allows module authors to use type conversion functions to
give additional type information to Terraform in situations that are too
ambiguous to be handled automatically by the type inference/unification
process. Previously tostring(null) was effectively a no-op, totally
ignoring the author's request to treat the null as a string.
Track individual instance drift rather than whole resources which
contributed to the plan. This will allow the output to be more precise,
and we can still use NoKey instances as a proxy for containing resources
when needed.
Convert a global reference to a specific AbsResource and attribute pair.
The hcl.Traversal is converted to a cty.Path at this point because plan
rendering is based on cty values.
The sum() function accepts a collection of values which must all convert
to numbers. It is valid for this to be a collection of string values
representing numbers.
Previously the function would panic if the first element of a collection
was a non-number type, as we didn't attempt to convert it to a number
before calling the cty `Add` method.
Our existing functionality for dealing with references generally only has
to concern itself with one level of references at a time, and only within
one module, because we use it to draw a dependency graph which then ends
up reflecting the broader context.
However, there are some situations where it's handy to be able to ask
questions about the indirect contributions to a particular expression in
the configuration, particularly for additional hints in the user interface
where we're just providing some extra context rather than changing
behavior.
This new "globalref" package therefore aims to be the home for algorithms
for use-cases like this. It introduces its own special "Reference" type
that wraps addrs.Reference to annotate it also with the usually-implied
context about where the references would be evaluated.
With that building block we can therefore ask questions whose answers
might involve discussing references in multiple packages at once, such as
"which resources directly or indirectly contribute to this expression?",
including indirect hops through input variables or output values which
would therefore change the evaluation context.
The current implementations of this are around mapping references onto the
static configuration expressions that they refer to, which is a pretty
broad and conservative approach that unfortunately therefore loses
accuracy when confronted with complex expressions that might take dynamic
actions on the contents of an object. My hunch is that this'll be good
enough to get some initial small use-cases solved, though there's plenty
room for improvement in accuracy.
It's somewhat ironic that this sort of "what is this value built from?"
question is the use-case I had in mind when I designed the "marks" feature
in cty, yet we've ended up putting it to an unexpected but still valid
use in Terraform for sensitivity analysis and our currently handling of
that isn't really tight enough to permit other concurrent uses of marks
for other use-cases. I expect we can address that later and so maybe we'll
try for a more accurate version of these analyses at a later date, but my
hunch is that this'll be good enough for us to still get some good use out
of it in the near future, particular related to helping understand where
unknown values came from and in tailoring our refresh results in plan
output to deemphasize detected changes that couldn't possibly have
contributed to the proposed plan.
This commit introduces a capsule type, `TypeType`, which is used to
extricate type information from the console-only `type` function. In
combination with the `TypeType` mark, this allows us to restrict the use
of this function to top-level display of a value's type. Any other use
of `type()` will result in an error diagnostic.
These instances of marks.Raw usage were semantically only testing the
properties of combining multiple marks. Testing this with an arbitrary
value for the mark is just as valid and clearer.
This is not currently gated by the experiment only because it is awkward
to do so in the context of evaluationStateData, which doesn't have any
concept of experiments at the moment.
Previously we were just returning a string representation of the file mode,
which spends more characters on the irrelevant permission bits that it
does on the directory entry type, and is presented in a Unix-centric
format that likely won't be familiar to the user of a Windows system.
Instead, we'll recognize a few specific directory entry types that seem
worth mentioning by name, and then use a generic message for the rest.
The original motivation here was actually to deal with the fact that our
tests for this function were previously not portable due to the error
message leaking system-specific permission detail that are not relevant
to the test. Rather than just directly addressing that portability
problem, I took the opportunity to improve the error messages at the same
time.
However, because of that initial focus there are only actually tests here
for the directory case. A test that tries to test any of these other file
modes would not be portable and in some cases would require superuser
access, so we'll just leave those cases untested for the moment since they
weren't tested before anyway, and so we've not _lost_ any test coverage
here.
Some function errors include values derived from arguments. This commit
is the result of a manual audit of these errors, which resulted in:
- Adding a helper function to redact sensitive values;
- Applying that helper function where errors include values derived from
possibly-sensitive arguments;
- Cleaning up other errors which need not include those values, or were
otherwise incorrect.
Running the tool this way ensures that we'll always run the version
selected by our go.mod file, rather than whatever happened to be available
in $GOPATH/bin on the system where we're running this.
This change caused some contexts to now be using a newer version of
staticcheck with additional checks, and so this commit also includes some
changes to quiet the new warnings without any change in overall behavior.
We can also rule out some attribute types as indicating something other
than the legacy SDK.
- Tuple types were not generated at all.
- There were no single objects types, the convention was to use a block
list or set of length 1.
- Maps of objects were not possible to generate, since named blocks were
not implemented.
- Nested collections were not supported, but when they were generated they
would have primitive types.
If structural types are being used, we can be assured that the legacy
SDK SchemaConfigModeAttr is not being used, and the fixup is not needed.
This prevents inadvertent mapping of blocks to structural attributes,
and allows us to skip the fixup overhead when possible.
Go 1.17 includes a breaking change to both net.ParseIP and net.ParseCIDR
functions to reject IPv4 address octets written with leading zeros.
Our use of these functions as part of the various CIDR functions in the
Terraform language doesn't have the same security concerns that the Go
team had in evaluating this change to the standard library, and so we
can't justify an exception to our v1.0 compatibility promises on the same
sort of security grounds that the Go team used to justify their
compatibility exception.
For that reason, we'll now use our own fork of the Go library functions
which has the new check disabled in order to preserve the prior behavior.
We're taking this path, rather than pre-normalizing the IP address before
calling into the standard library, because an additional normalization
layer would be entirely new code and additional complexity, whereas this
fork is relatively minor in terms of code size and avoids any significant
changes to our own calls to these functions.
Thanks to the Kubernetes team for their prior work on carving out a subset
of the "net" package for their similar backward-compatibility concern.
Our "ipaddr" package here is a lightly-modified fork of their fork, with
only the comments changed to talk about Terraform instead of Kubernetes.
This fork is not intended for use in any other future feature
implementations, because they wouldn't be subject to the same
compatibility constraints as our existing functions. We will use these
forked implementations for new callers only if consistency with the
behavior of the existing functions is a key requirement.
The previous name didn't fit with the naming scheme for addrs types:
The "Abs" prefix typically means that it's an addrs.ModuleInstance
combined with whatever type name appears after "Abs", but this is instead
a ModuleCallOutput combined with an InstanceKey, albeit structured the
other way around for convenience, and so the expected name for this would
be the suffix "Instance".
We don't have an "Abs" type corresponding with this one because it would
represent no additional information than AbsOutputValue.
