Commit Graph

162 Commits

Author SHA1 Message Date
Alisdair McDiarmid
c7568ccceb lang: Add test for jsonencode entity escaping
The encoding/json package escapes some HTML-specific characters to
prevent JSON from being misinterpreted when in the context of an HTML
document. Terraform 0.11 used this behaviour, and to preserve backwards
compatibility we are continuing to do so moving forward.

This commit adds an explicit test to document that this is intentional,
not a bug.
2020-09-04 09:18:52 -04:00
Kristin Laemmert
e639d813ee add test cases and remove no-longer-needed validation 2020-07-08 13:53:54 -04:00
Kristin Laemmert
384cfaacf9 lang/funcs: refactor Subnet and Host functions to support 64-bit systems 2020-07-08 10:58:06 -04:00
Martin Atkins
8f77bd344e lang/funcs: Filesystem functions hint about dynamically-generated files
The functions that interact with the filesystem are, by design, restricted
to reading files that are distributed as a static part of the
configuration, and cannot be used to interact with files that are
generated dynamically by resources in the configuration.

However, new users have often yet developed a correct mental model of how
Terraform execution works and are confused by the terse error messages
these functions return. As an interim step to help some of those users,
this just adds some more commentary to the error message which gives a
vague, generic directive to look to attributes of the resource that is
generating the file, which should (if it's designed well) export
attributes that allow the resulting file to be used effectively with
common patterns, such as checksums of the content of the generated file.

The error message here is not particularly attractive due to the
limitations of the context where it's being returned from, but I'm
accepting that here in the interest of keeping this change simple, so we
can give a hint about a case that seems to frequently generate new-user
questions. We may iterate further on the presentation of this message
later.
2020-06-24 09:02:38 -07:00
Alisdair McDiarmid
80ca4db54d lang: Add regression test for yamldecode bug 2020-06-17 13:05:14 -04:00
Alisdair McDiarmid
a4f5e04066 lang/funcs: Add support for OpenSSH RSA key format
Previously this function only supported the x509 RSA private key format.
More recent versions of OpenSSH default to generating a new PEM key
format, which this commit now supports using the x/crypto/ssh package.

Also improve the returned error messages for various invalid ciphertext
or invalid private key errors.
2020-06-03 10:50:38 -04:00
Daniel Dreier
0749e419de
Merge pull request #24484 from ctjhoa/fix_typo
fix typo in Base64DecodeFunc log
2020-05-04 16:11:47 -07:00
James Bardin
6c0f7703a6
Merge pull request #24697 from hashicorp/jbardin/get-module-data
Always return all module instances during evaluation
2020-04-22 09:49:45 -04:00
Noah Mercado
d4d8812afa
Feature: Sum Function (#24666)
The sum function takes a list or set of numbers and returns the sum of those
numbers.
2020-04-15 14:27:06 -04:00
James Bardin
42cee86ee2 remove GetModuleInstanceOutput
There is no codepath that can use this any longer, since we need to
evaluate the modules as whole objects.

This means we're going to have to live for now with invalid module
output references returning "object" errors rather that "module".
2020-04-14 14:49:10 -04:00
James Bardin
600d4c3e1f eval Data needs to operate on whole modules
In order to be able to use module values, and handle operations like
possibly invalid module indexes in conditional statements, whole modules
must always be returned during evaluation.
2020-04-12 10:50:31 -04:00
Camille TJHOA
d7f711f40e fix typo in Base64DecodeFunc log 2020-03-27 17:22:40 +00:00
Pam Selle
f738f85241 Create non-specific ModuleCallOutput 2020-03-26 13:29:38 -04:00
Alisdair McDiarmid
37006c5841 lang: Fix non-string key panics in map function
The map function assumed that the key arguments were strings, and would
panic if they were not.

After this commit, calling `map(1, 2)` will result in a map `{"1" = 1}`,
and calling `map(null, 1)` will result in a syntax error.

Fixes #23346, fixes #23043
2020-03-04 10:54:55 -05:00
James Bardin
0bd40fd496
Merge pull request #24265 from hashicorp/jbardin/cty-update
Update cty to v1.3.1 and migrate to cty stdlib function where applicable
2020-03-04 09:33:51 -05:00
James Bardin
d999d43483 remove old funcs code 2020-03-03 15:23:58 -05:00
James Bardin
d0d85f909a convert cty func calls to stdlib 2020-03-03 15:23:58 -05:00
Martin Atkins
67d95b97ce lang/funcs: templatefile requires valid variable names
Previously the templatefile function would permit any arbitrary string as
a variable name, but due to the HCL template syntax it would be impossible
to refer to one that isn't a valid HCL identifier without causing an
HCL syntax error.

The HCL syntax errors are correct, but don't really point to the root
cause of the problem. Instead, we'll pre-verify that the variable names
are valid before we even try to render the template, and given a
specialized error message that refers to the vars argument expression as
the problematic part, which will hopefully make the resolution path
clearer for a user encountering this situation.

The syntax error still remains for situations where all of the variable
names are correct but e.g. the user made a typo referring to one, which
makes sense because in that case the problem _is_ inside the template.
2020-02-25 10:19:46 -05:00
Martin Atkins
ec9f950b3f lang/funcs: Test actual error messages from templatefile
This function has a number of different error cases with hopefully-helpful
error messages for each, so it's good to test we're getting the error
message we were actually expecting in each case.
2020-02-25 10:19:46 -05:00
James Bardin
a765d69fb0
Merge pull request #24032 from hashicorp/jbardin/map-funcs
make the merge function more precise
2020-02-12 10:51:05 -05:00
James Goodhouse
25bfe7337b
lang: add setsubtract function (#23424)
* add setdifference and setsubtract functions and docs
* remove setdifference as it is not implemented correct in underlying lib

* Update setintersection.html.md
* Update setproduct.html.md
* Update setunion.html.md
2020-02-06 12:49:11 -05:00
James Bardin
529271e0be update merge docs to match behavior 2020-02-05 15:47:36 -05:00
James Bardin
f5bf9aa55d make the merge function more precise
This PR implements 2 changes to the merge function.
 - Rather than always defining the merge return type as dynamic, return
 a precise type when all argument types match, or all possible object
 attributes are known.
 - Always return a value containing all keys when the keys are known.
 This allows the use of merge output in for_each, even when keys are yet
 to be determined.
2020-02-05 13:46:08 -05:00
Martin Atkins
02576988c1 lang: "try" and "can" functions
These are intended to make it easier to work with arbitrary data
structures whose shape might not be known statically, such as the result
of jsondecode(...) or yamldecode(...) of data from a separate system.

For example, in an object value which has attributes that may or may not
be set we can concisely provide a fallback value to use when the attribute
isn't set:

    try(local.example.foo, "fallback-foo")

Using a "try to evaluate" model rather than explicit testing fits better
with the usual programming model of the Terraform language where values
are normally automatically converted to the necessary type where possible:
the given expression is subject to all of the same normal type conversions,
which avoids inadvertently creating a more restrictive evaluation model
as might happen if this were handled using checks like a hypothetical
isobject(...) function, etc.
2020-01-10 15:23:25 -08:00
George Christou
23fc68cc91
lang: Fix new trim* function mappings
Fixes #23413
2019-11-19 10:13:14 +00:00
George Christou
91100c003c lang/funcs: Add more trim* functions (#23016)
* lang/funcs: Add `trim*` functions
2019-11-18 08:31:44 -05:00
Parviz
6eaf1c23f9 lang/funcs: don't panic when transpose produces empty result 2019-11-08 09:40:39 -08:00
James Bardin
3839405d3d correctly evaluate self in for_each resources
The fallback type for GetResource from an EachMap is a cty.Object,
because resource schemas may contain dynamically typed attributes.
Check for an Object type in the evaluation of self, to use the proper
GetAttr method when extracting the value.
2019-10-29 12:44:42 -04:00
James Bardin
1ee851f256
Merge pull request #22846 from hashicorp/jbardin/evaluate-resource
Always evaluate resources in their entirety
2019-10-08 07:57:15 -04:00
James Bardin
bfce78064b lang/eval: more evalContext fixups
self references do not need to be added to `managedResource`, and in
fact that could cause issues later if self is allowed in contexts other
than managed resources.

Coalesce 2 cases in the Referenceable switch, be take the
ContainingResource address of an instance beforehand.
2019-10-07 18:13:20 -04:00
Martin Atkins
39e609d5fd vendor: switch to HCL 2.0 in the HCL repository
Previously we were using the experimental HCL 2 repository, but now we'll
shift over to the v2 import path within the main HCL repository as part of
actually releasing HCL 2.0 as stable.

This is a mechanical search/replace to the new import paths. It also
switches to the v2.0.0 release of HCL, which includes some new code that
Terraform didn't previously have but should not change any behavior that
matters for Terraform's purposes.

For the moment the experimental HCL2 repository is still an indirect
dependency via terraform-config-inspect, so it remains in our go.sum and
vendor directories for the moment. Because terraform-config-inspect uses
a much smaller subset of the HCL2 functionality, this does still manage
to prune the vendor directory a little. A subsequent release of
terraform-config-inspect should allow us to completely remove that old
repository in a future commit.
2019-10-02 15:10:21 -07:00
Martin Atkins
f84ab99b7d lang/funcs: cidrsubnets function
This is a companion to cidrsubnet that allows bulk-allocation of multiple
subnet addresses at once, with automatic numbering.

Unlike cidrsubnet, cidrsubnets allows each of the allocations to have a
different prefix length, and will pack the networks consecutively into the
given address space. cidrsubnets can potentially create more complicated
addressing schemes than cidrsubnet alone can, because it's able to take
into account the full set of requested prefix lengths rather than just
one at a time.
2019-09-20 15:58:01 -07:00
James Bardin
d4031918d1 parse resource refs correctly
Now that we only evaluate whole resources, we can parse resource refs
correct as the resource, rather than an unknown instance.
2019-09-19 11:46:09 -04:00
James Bardin
c49f976308 change lang eval to also only lookup resources
Continue only evaluating resource at a whole and push the indexing of
the resource down into the expression evaluation.

The exception here is that `self` must be an instance which must be
extracted from the resource. We now also add the entire resource to the
context, which was previously only partially populated with the self
referenced instance.
2019-09-19 11:46:09 -04:00
James Bardin
86bf674246 change GetResourceInstance to GetResource
In order to allow lazy evaluation of resource indexes, we can't index
resources immediately via GetResourceInstance. Change the evaluation to
always return whole Resources via GetResource, and index individual
instances during expression evaluation.

This will allow us to always check for invalid index errors rather than
returning an unknown value and ignoring it during apply.
2019-09-19 09:19:14 -04:00
Jeet Parekh
bcc69c05bb lang/funcs: parseint function 2019-09-17 15:33:22 -07:00
Pam Selle
f9ebae749c
Merge pull request #22707 from vsimon/spell
docs: Minor spelling and typo fixes
2019-09-06 11:46:01 -04:00
Vicken Simonian
853a0e0677 docs: Minor spelling and typo fixes 2019-09-05 10:08:34 -07:00
Brian Flad
19cf34114f
lang/funcs: Switch fileset() function glob implementation to github.com/bmatcuk/doublestar to support additional glob patterns
This allows the usage of the glob patterns `**` and `{alternative1,...}` to simplify Terraform configuration logic for more complex file matching.
2019-08-30 20:22:03 -04:00
Brian Flad
769f626a0e
lang/funcs: Remove homedir.Expand() and refactor path trimming with filepath.Rel() in fileset() function
Reference: https://github.com/hashicorp/terraform/pull/22621#pullrequestreview-282259385
2019-08-30 20:19:44 -04:00
Brian Flad
820b79d91d
lang/funcs: Clarification update to MakeFileSetFunc comment for filepath.ToSlash usage 2019-08-30 20:19:44 -04:00
Brian Flad
af7f6ef441
lang/funcs: Update fileset() function to include path as separate first argument, automatically trim the path argument from results, and ensure results are always canonical with forward slash path separators
Reference: https://github.com/hashicorp/terraform/pull/22523#pullrequestreview-279694703

These changes center around better function usability and consistency with other functions. The function has not yet been released, so these breaking changes can be applied safely.
2019-08-30 20:19:44 -04:00
Brian Flad
d48d9ed766
lang/funcs: Add fileset function
Reference: https://github.com/hashicorp/terraform/issues/16697

Enumerates a set of regular file names from a given glob pattern. Implemented via the Go stdlib `path/filepath.Glob()` functionality. Notably, stdlib does not support `**` or `{}` extended patterns. See also: https://github.com/golang/go/issues/11862

To support the extended glob patterns, it will require adding a dependency on a third party library or adding our own matching code.
2019-08-20 04:50:01 -04:00
Pam Selle
e38d588cfc
Merge pull request #22314 from hashicorp/pselle/double-dynamic
Fix panic on double dynamic blocks
2019-08-16 15:15:50 -04:00
Alex Pilon
e2eb70b737
remove satori/go.uuid in favor of google/uuid 2019-08-09 15:09:41 -04:00
Martin Atkins
135afaeb9c lang: "regex" and "regexall" functions
These existing upstream cty functions allow matching strings against
regular expression patterns, which can be useful if you need to consume
a non-standard string format that Terraform doesn't (and can't) have a
built-in function for.
2019-08-06 11:52:14 -07:00
Pam Selle
234c1c482a Add a comment 2019-08-02 16:15:18 -04:00
Pam Selle
cd83e6108d Fixes issue where attribute in nested is not of list type/is invalid and would panic 2019-08-02 16:08:16 -04:00
Kristin Laemmert
2b14a6b332
lang/funcs: lookup() can work with maps of lists, maps and objects (#22269)
* lang/funcs: lookup() can work with maps of lists, maps and objects

lookup() can already handle aribtrary objects of (whatever) and should
handle maps of (whatever) similarly.
2019-08-01 15:52:52 -04:00
Pam Selle
360068b3cb
Merge pull request #21922 from pselle/resource_for_each
Resource for_each
2019-07-26 11:41:56 -04:00