Commit Graph

312 Commits

Author SHA1 Message Date
Sander van Harmelen
b28f47055d backend/remote: extend mocks and add sentinel tests 2018-09-26 22:34:32 +02:00
Sander van Harmelen
2bd1040bbd backend/remote: extend mocks and add apply tests 2018-09-26 21:35:41 +02:00
Sander van Harmelen
9f9bbcb0e7 backend/remote: lots of improvements
This commit adds:

- support for `-lock-timeout`
- custom error message when a 404 is received
- canceling a pending run when TF is Ctrl-C’ed
- discard a run when the apply is not approved
2018-09-22 11:49:42 +02:00
Sander van Harmelen
621d589189 backend/remote: add support for the apply operation 2018-09-22 11:49:42 +02:00
Kristin Laemmert
813b5fd27f
Merge pull request #18825 from sean-/master-make-dev-drift
build: Updates required to bump golang version to 1.11
2018-09-17 09:00:04 -07:00
Sander van Harmelen
4aeb67b8ff backend/remote: handle empty results correctly
The pagination info of a list call that returns an empty list contains:

```go
CurrentPage: 1
TotalPages: 0
```

So checking if we have seen all pages using `CurrentPage == TotalPages` will not work and will result in an endless loop.

The tests are updated so they will fail (timeout after 1m) if this is handled incorreclty.
2018-09-17 16:22:54 +02:00
Sander van Harmelen
ea88daa499 backend/remote: add support for state locking 2018-09-10 19:49:53 +02:00
Sander van Harmelen
7c1da2ffaa
Merge pull request #18818 from hashicorp/b-associate-state
backend/remote: add the run ID to associate state
2018-09-10 19:13:53 +02:00
Sander van Harmelen
cd6d75bc03 backend/remote: add the run ID to associate state
If a run ID is available, we need to make sure we pass that when creating a new state version so the state will be properly associated with the run.
2018-09-09 21:30:42 +02:00
Sander van Harmelen
3cc58813f0 backend/remote: use a search query and use pagination
To prevent making unnecessary heavy calls to the backend, we should use a search query to limit the result.

But even if we use a search query, we should still use the pagination details to make sure we retrieved all items.
2018-09-09 21:29:50 +02:00
Sean Chittenden
d749420a25
Fix drift caused from gofmt when running make dev and go 1.11.
A fresh checkout of `origin/master` does not build atm using the `dev`
target because `master` has not been formatted using `gofmt` from Go
1.11 (tis has been the case for a while if you've been running devel).

None of the drift in question is especially new but now that Go 1.11
has been released and gofmt's formatting guidelines have been updated,
it would be *really* nice if the code in `master` reflected the current
tooling in order to avoid having to fight this drift locally.

* 8mo: https://github.com/hashicorp/terraform/blame/master/backend/remote-state/s3/backend_test.go#L260-L261
* 6mo: https://github.com/hashicorp/terraform/blame/master/builtin/provisioners/chef/linux_provisioner_test.go#L124
* 1yr: 7cfeffe36b/command/init.go (L75-L76)
* 12d: 7cfeffe36b/command/meta_backend_test.go (L1437)
* 2yr: 7cfeffe36b/helper/schema/resource_timeout_test.go (L26)
* 4yr: 7cfeffe36b/helper/schema/schema_test.go (L2059)
* 1yr: 7cfeffe36b/plugin/discovery/get_test.go (L151)
2018-09-09 10:18:08 -07:00
Sander van Harmelen
62ab84a319 backend/remote: compat changes for latest go-tfe 2018-09-08 11:47:18 +02:00
Sander van Harmelen
3cfc83e335 backend/remote: take working directories into account
In TFE you can configure a workspace to use a custom working directory. When determining which directory that needs to be uploaded to TFE, this working directory should be taken into account to make sure we are uploading the correct root directory for the workspace.
2018-08-31 20:35:16 +02:00
Sander van Harmelen
ce2869dced
Merge pull request #18760 from hashicorp/f-remote-backend
backend/migrations: migrate the default state
2018-08-30 11:16:25 +02:00
Sander van Harmelen
f410a5bb26 backend/migrations: migrate the default state
Certain backends (currently only the `remote` backend) do not support using both the default and named workspaces at the same time.

To make the migration easier for users that currently use both types of workspaces, this commit adds logic to ask the user for a new workspace name during the migration process.
2018-08-29 21:37:39 +02:00
Sander van Harmelen
77b9fad7f0 backend/manta: deprecate camelcase attribute name
This change is needed (next to making it consistent as its the only attribute name that uses camelcase) because passing that attribute objectName in the update `terraform_remote_state` resource, ended up giving me this error: https://github.com/hashicorp/terraform/blob/master/helper/schema/schema.go#L736
2018-08-29 20:22:04 +02:00
Sander van Harmelen
7049d973a9 Do not use the TFE_TOKEN env variable
Instead promote the use of shared credentials using the CLI Config File
2018-08-10 20:20:16 +02:00
Sander van Harmelen
83d5f4147b backend/remote: use schema max/min items options 2018-08-05 15:30:27 +02:00
Sander van Harmelen
7fb2d1b8de Implement the Enterprise enhanced remote backend 2018-08-03 22:22:55 +02:00
Sander van Harmelen
495d1ea350 Use New() instead of once.Do(b.init) 2018-08-03 11:29:11 +02:00
Sander van Harmelen
97d1c46602 Update the backend import names
It’s a purely cosmetic change, but I find it easier to read them like this.
2018-08-03 11:29:11 +02:00
Kristin Laemmert
85be12d783
cli: show workspace name in destroy confirmation (#18253)
* cli: show workspace name in destroy confirmation

If the workspace name is not "default", include it in the confirmation
message for `terraform destroy`.

Fixes #15480
2018-06-19 13:35:28 -07:00
stack72
df9b2913ae backend/manta: Add support for manta backend tiered path structure
This was already added to triton-go and is now making its way to
the manta backend

```
% acctests backend/remote-state/manta
=== RUN   TestBackend_impl
--- PASS: TestBackend_impl (0.00s)
=== RUN   TestBackend
--- PASS: TestBackend (27.36s)
=== RUN   TestBackendLocked
--- PASS: TestBackendLocked (16.24s)
=== RUN   TestRemoteClient_impl
--- PASS: TestRemoteClient_impl (0.00s)
=== RUN   TestRemoteClient
--- PASS: TestRemoteClient (3.40s)
=== RUN   TestRemoteClientLocks
--- PASS: TestRemoteClientLocks (7.17s)
PASS
ok  	github.com/hashicorp/terraform/backend/remote-state/manta
```
2018-04-03 17:32:10 +03:00
James Bardin
28c46d1a90 cleanup temp files from backend tests 2018-03-28 11:00:23 -04:00
stack72
a6f76aa0af backend/manta: Manta Backend was not dealing with a ResourceNotFound
Fixes: #17314

We now deal correctly with the creation of the state file - we were
not dealing well with a ResourceNotFound error

Now that this has been changed around, we try and create the statefile
and if there is an error, we look for an existing statefile - previously
this was not the order of operations
2018-03-12 17:27:57 +02:00
Martin Atkins
6aefa5835c Merge #17218: Add -auto-approve to "terraform destroy" for consistency 2018-03-08 17:42:15 -08:00
Paul Tyng
c868092d2d
Standardize http.Client creation with User-Agent 2018-02-28 12:09:50 -05:00
James Bardin
e9a76808df create clistate.Locker interface
Simplify the use of clistate.Lock by creating a clistate.Locker
instance, which stores the context of locking a state, to allow unlock
to be called without knowledge of how the state was locked.

This alows the backend code to bring the needed UI methods to the point
where the state is locked, and still unlock the state from an outer
scope.

Provide a NoopLocker as well, so that callers can always call Unlock
without verifying the status of the lock.

Add the StateLocker field to the backend.Operation, so that the state
lock can be carried between the different function scopes of the backend
code. This will allow the backend context to lock the state before it's
read, while allowing the different operations to unlock the state when
they complete.
2018-02-23 16:48:15 -05:00
James Bardin
8242c773b8 missed the local state backend 2018-02-20 22:09:54 -05:00
James Bardin
57cbcbe99d update remote state tests to use new test helpers 2018-02-20 21:05:58 -05:00
James Bardin
112c37458f report the proper lock id from a state lock error
Fix the now failing state unlock test by reporting the correct ID.

The ID used by GCS is the generation number of the info object, which
isn't known until the info is already written out. While we can't get
the correct ID from the info data for the error rmessage, we can update
it with the generation number after it's read.
2018-02-20 20:54:01 -05:00
James Bardin
1fee5ae172 update gcs to use the new backend tests 2018-02-20 20:36:08 -05:00
James Bardin
85d6b1d9cc add test for lock error and force-unlock
This adds a general test to verify that a remote state backend returns
the expected error type when it cannot lock a state. It then extracts
the ID reported in the error, and attempts to unlock the state using
that ID, which simulated the force-unlock scenario. This is a separate
test, since not all backends have persistent locks that can be unlocked
later.

We also split out the backend test to be called individually as needed.
2018-02-20 20:32:07 -05:00
James Bardin
ef8ed1e275 coalesce the backened interrupt code
Moves the nested select statements for backend operations into a single
function. The only difference in this part was that apply called
PersistState, which should be harmless regardless of the type of
operation being run.
2018-02-12 11:56:54 -05:00
James Bardin
7cba68326a always wait for a RunningOperation to return
If the user wishes to interrupt the running operation, only the first
interrupt was communicated to the operation by canceling the provided
context. A second interrupt would start the shutdown process, but not
communicate this to the running operation. This order of event could
cause partial writes of state.

What would happen is that once the command returns, the plugin system
would stop the provider processes. Once the provider processes dies, all
pending Eval operations would return return with an error, and quickly
cause the operation to complete. Since the backend code didn't know that
the process was shutting down imminently, it would continue by
attempting to write out the last known state. Under the right
conditions, the process would exit part way through the writing of the
state file.

Add Stop and Cancel CancelFuncs to the RunningOperation, to allow it to
easily differentiate between the two signals. The backend will then be
able to detect a shutdown and abort more gracefully.

In order to ensure that the backend is not in the process of writing the
state out, the command will always attempt to wait for the process to
complete after cancellation.
2018-02-12 11:56:03 -05:00
Laura Martin
6e1e614a56 Change -force to -auto-approve when destroying
Since an early version of Terraform, the `destroy` command has always
had the `-force` flag to allow an auto approval of the interactive
prompt. 0.11 introduced `-auto-approve` as default to `false` when using
the `apply` command.

The `-auto-approve` flag was introduced to reduce ambiguity of it's
function, but the `-force` flag was never updated for a destroy.

People often use wrappers when automating commands in Terraform, and the
inconsistency between `apply` and `destroy` means that additional logic
must be added to the wrappers to do similar functions. Both commands are
more or less able to run with similar syntax, and also heavily share
their code.

This commit updates the command in `destroy` to use the `-auto-approve` flag
making working with the Terraform CLI a more consistent experience.

We leave in `-force` in `destroy` for the time-being and flag it as
deprecated to ensure a safe switchover period.
2018-02-01 00:14:42 +00:00
James Bardin
a5ed7d0ae4
Merge pull request #17086 from hashicorp/jbardin/GH-17083
only split the prefix string once from the s3 key
2018-01-17 18:51:43 -05:00
stack72
e9476c6765 backend/manta: Update manta dependencies
Internally, triton-go has changed how it handles errors. We can now get rid of
checking strings for errors, and we have introduced an errors library that
wraps some of the major errors we encounter and test for
2018-01-17 11:34:02 -08:00
Masayuki Morita
f440dba137 Standardize on log level "WARN" rather than "WARNING" 2018-01-16 18:05:26 -08:00
James Bardin
43d000625d only split the prefix string once from the s3 key
Ensure that the prefix is only split off a single time when the
workspace_key_prefix is a substring of the workspace or key name.
2018-01-11 10:08:02 -05:00
Stefan Schmidt
c200c170ad Handle refresh errors. 2018-01-10 16:40:20 +01:00
Paddy
e4cdbd6c9f
Merge pull request #16936 from negz/gcskeys
Support 'customer supplied encryption keys' in the GCS backend
2018-01-09 01:17:35 -08:00
James Bardin
7c703f2ab2
Merge pull request #16932 from rv-jmaggio/master
Fixing issues with workspace_key_prefix for s3 backend
2018-01-05 13:24:40 -05:00
James Bardin
dd5882e5b2
Merge pull request #16992 from hashicorp/jbardin/go1.10-prep
go1.10 prep
2018-01-03 15:41:41 -05:00
goraxe
fb6b349e58 backend/s3: Allow forcing use of the S3 path-style API
This is useful when the underlying API is an S3-workalike service, rather than official S3.
2018-01-03 12:15:47 -08:00
Paul Stack
191cf283d5 backend/manta: Support Triton RBAC
Triton Manta allows an account other than the main triton account to be used via RBAC.

Here we expose the SDC_USER / TRITON_USER options to the backend so that a user can be specified.
2018-01-03 12:12:46 -08:00
James Bardin
4b49a323c3 go fmt
slight change to go fmt coming in 0.10
2017-12-26 13:26:38 -05:00
rv-jmaggio
b313ce80c4 Changing prefix for empty workspace prefix 2017-12-19 13:14:31 -05:00
Henry Bell
56357c0ab7 Fix typo ('depracted' -> 'deprecated') 2017-12-19 16:43:06 +00:00
rv-jmaggio
bef64cfe91 Fixing implementation for empty string and making acceptance test work 2017-12-19 09:31:53 -05:00