Commit Graph

199 Commits

Author SHA1 Message Date
James Bardin
979faa5dbe move artifactory remote state to backend 2018-10-16 18:23:14 -07:00
James Bardin
178eb6076e remove legacy remote file code
This code is not referenced at all
2018-10-16 18:22:37 -07:00
James Bardin
18ef072325 move legacy http remote state to a backend 2018-10-16 18:22:37 -07:00
James Bardin
3595bd3f81 remove dead legacy gcs code 2018-10-16 18:21:51 -07:00
James Bardin
fe527ec9d7 move legacy etcd remote state to a backend 2018-10-16 18:21:51 -07:00
Sander van Harmelen
7fb2d1b8de Implement the Enterprise enhanced remote backend 2018-08-03 22:22:55 +02:00
James Bardin
9bf9280c03
Merge pull request #17431 from oscr/use-seekstart
Use io.SeekStart instead of deprecated os.SEEK_SET
2018-04-04 15:27:24 -04:00
James Bardin
a84b4a669a use the open state file for refresh when possible
Only open a new file descriptor for RefreshState if we haven't written a
state, and don't have the correct state open already. This prevents
windows from failing to refresh a locked state.
2018-03-19 18:17:43 -04:00
James Bardin
e10a7917e6 add failing test for windows state locks
Refreshing a locked state on windows could return nil if the read path
was locked, no state was yet written, and the read path is the same as
the write path.

Add a test that locks then refreshes a newly initialized state struct.
2018-03-19 17:09:53 -04:00
Paul Tyng
c868092d2d
Standardize http.Client creation with User-Agent 2018-02-28 12:09:50 -05:00
Oscar Utbult
7434cd0a81 Use io.SeekStart instead of deprecated os.SEEK_SET 2018-02-25 23:11:40 +01:00
jwa
3d4642719f simulate a webdav backend 2018-01-19 02:30:21 +00:00
jwa
b3d432e6bc allow HTTP 201 & 204 when storing remote state
Apache mod_dav returns 201 (created) and 204 (no content) during PUTs;
treat these as valid responses when using Apache as a http backend.
2018-01-11 22:09:59 +00:00
stack72
1fd0f803e4 Migrate Manta Remote state to be a backend
This PR changes manta from being a legacy remote state client to a new backend type. This also includes creating a simple lock within manta

This PR also unifies the way the triton client is configured (the schema) and also uses the same env vars to set the backend up

It is important to note that if the remote state path does not exist, then the backend will create that path. This means the user doesn't need to fall into a chicken and egg situation of creating the directory in advance before interacting with it
2017-10-30 18:36:50 +02:00
Florian Forster
b09f121f86 state/remote: The "gcs" client has been superseeded by the "gcs" backend. 2017-10-27 16:52:21 -04:00
James Bardin
36b8be43e8 use the new version package
Update all references to the version values to use the new package.
The VersionString function was left in the terraform package
specifically for the aws provider, which is vendored. We can remove that
last call once the provider is updated.
2017-10-19 21:48:08 -04:00
Martin Atkins
c12d64f340 Use t.Helper() in our test helpers
Go 1.9 adds this new function which, when called, marks the caller as
being a "helper function". Helper function stack frames are then skipped
when trying to find a line of test code to blame for a test failure, so
that the code in the main test function appears in the test failure output
rather than a line within the helper function itself.

This covers many -- but probaly not all -- of our test helpers across
various packages.
2017-08-28 09:59:30 -07:00
Martin Atkins
2e0c1d07ae Merge #15793: Locking support in HTTP remote backend 2017-08-22 09:43:38 -07:00
Ross McFarland
d889ac38b0 Change remote/http store to update, more consistent with doc 2017-08-20 05:59:04 -07:00
Ross McFarland
510563b67f Fully test conf handling in httpFactory 2017-08-19 11:17:25 -07:00
Ross McFarland
69546c4b33 Pass at much more flexible remote/http backend
- Configurable Put (store) method, default POST to preserve behavior
- Configurable Lock method & address
- Configurable Unlock method & address

More thorough testing still needed, but this if functional
2017-08-19 10:31:47 -07:00
James Bardin
68da0390b7 remove legacy azure remote state code
The implementation has been moved to a backend.
2017-08-17 12:57:53 -04:00
Peter McAtominey
f9e8e54835 backend: convert Azure remote state to a backend
Added locking support via blob leasing (requires that an empty state is
created before any lock can be acquired.

Added support for "environments" in much the same way as the S3 backend.
2017-08-17 16:32:17 +01:00
Ross McFarland
6cdea5af5d Clean up code and make the state/http behavior more consistent 2017-08-13 09:49:08 -07:00
Ross McFarland
ce4d9fb3c2 Add tests for state/http with locking 2017-08-13 09:16:42 -07:00
Ross McFarland
1d38569c91 Add Lock/Unlock support to remote/http 2017-08-13 07:39:22 -07:00
James Bardin
32ae05c342 fix strict remote.State lineage check
We can't check lineage in the remote state instance, because we may need
to overwrite a state with a new lineage. Whil it's tempting to add an
optional interface for this, like OverwriteState(), optional interfaces
are never _really_ optional, and will have to be implemented by any
wrapper types as well.

Another solution may be to add a State.Supersedes field to indicate that
we intend to replace an existing state, but that may not be worth the
extra check either.
2017-08-01 19:34:22 -04:00
James Bardin
fba5decae5 update TestState helper
In practice, States must all implement the full interface, so checking
for each method set only leaves gaps where tests could be skipped.
Change the helper to only accept a full state.State implementation.

Add some Lineage, Version, and TFVersion checks to TestState to avoid
regressions.

Compare the copy test against the immediate State returnedm rather than
our previous "current" state.

Check that the states round-trip and still marhsal identically via
MarshalEqual.
2017-07-05 17:18:12 -04:00
Martin Atkins
4d53eaa6df state: more robust handling of state Serial
Previously we relied on a constellation of coincidences for everything to
work out correctly with state serials. In particular, callers needed to
be very careful about mutating states (or not) because many different bits
of code shared pointers to the same objects.

Here we move to a model where all of the state managers always use
distinct instances of state, copied when WriteState is called. This means
that they are truly a snapshot of the state as it was at that call, even
if the caller goes on mutating the state that was passed.

We also adjust the handling of serials so that the state managers ignore
any serials in incoming states and instead just treat each Persist as
the next version after what was most recently Refreshed.

(An exception exists for when nothing has been refreshed, e.g. because
we are writing a state to a location for the first time. In that case
we _do_ trust the caller, since the given state is either a new state
or it's a copy of something we're migrating from elsewhere with its
state and lineage intact.)

The intent here is to allow the rest of Terraform to not worry about
serials and state identity, and instead just treat the state as a mutable
structure. We'll just snapshot it occasionally, when WriteState is called,
and deal with serials _only_ at persist time.

This is intended as a more robust version of #15423, which was a quick
hotfix to an issue that resulted from our previous slopping handling
of state serials but arguably makes the problem worse by depending on
an additional coincidental behavior of the local backend's apply
implementation.
2017-07-05 12:34:30 -07:00
Gavin Williams
96e6bf89ec remote/swift: Remove un-used code 2017-06-27 21:18:43 +01:00
Gavin Williams
ee2e390f85 remote/swift: Migrate Swift remote state to a backend
Move the Swift State from a legacy remote state to an official backend.
Add `container` and `archive_container` configuration variables, and deprecate `path` and `archive_path` variables.

Future improvements: Add support for locking and environments.
2017-06-13 22:04:01 +01:00
James Bardin
808b504bcf rename openstack provider for swift remote state 2017-06-12 13:43:51 -04:00
Tom Harvey
13583b4b8b provider/azurerm: Upgrading to AutoRest v8 (#15006)
* Upgrading to AutoRest 8

* Upgrading to AutoRest v8

* Updating the Remote State to v8
2017-06-02 15:30:41 +01:00
Tom Harvey
df2875045d provider/azurerm: Upgrading to v10 of the Azure SDK (#14004)
provider/azurerm: Upgrading to v10.0.2 of the Azure SDK
2017-06-01 10:18:22 +01:00
James Bardin
4866f35645 add mutexes to Local, Backup, and InmemState 2017-05-25 11:20:52 -04:00
James Bardin
f0f2220abb add mutexes to remote.State 2017-05-25 11:20:52 -04:00
Gavin Williams
c63ad9c0f8 state/remote/swift: Support Openstack request logging (#13583)
* provider/openstack: Expose LogRoundTripper fields externally

* state/remote/swift: Add support for debugging Openstack calls using
OS_DEBUG env variable.

* provider/openstack: Update LogRoundTripper to log headers aswell as body.

* Add `RedactHeaders` function in order to redact sensitive http Headers.
Refactor `logRequest` and `logResponse` to use `RedactHeaders` func.
2017-04-15 17:11:28 +03:00
James Bardin
eeddc3f8ea add some nil checks for unexpected lock failures 2017-04-05 09:06:47 -04:00
James Bardin
af2e289212 remove Sleep from TestLockWithContext 2017-04-03 14:46:22 -04:00
James Bardin
d1460d8c82 test LockWithContext 2017-04-03 11:50:19 -04:00
James Bardin
93b1dd6323 give LockWithContext a little backoff
Backoff the Lock calls exponentially, to a reasonable limit.
2017-04-01 18:01:49 -04:00
James Bardin
3f0dcd1308 Have the clistate Lock use LockWithContext
- Have the ui Lock helper use state.LockWithContext.
- Rename the message package to clistate, since that's how it's imported
  everywhere.
- Use a more idiomatic placement of the Context in the LockWithContext
  args.
2017-04-01 17:09:20 -04:00
James Bardin
826771a830 add state.LockWithContext
LockWithContext will retry a lock until the context expires or is
cancelled. This will let us implement a `-lock-timeout` flag, and make
use of existing contexts when applicable.
2017-04-01 17:01:55 -04:00
James Bardin
75458a182d remove extra state.Locker assertions
All states are lockers, so get rid of extra asertions.
2017-04-01 17:01:45 -04:00
James Bardin
bf6384a163 All states are lockers
Since moving to the new backends, all states (except InmemState) are
Lockers. Add the methods to the State interface to remove a heap of
assertion checks.
2017-04-01 17:01:12 -04:00
James Bardin
9f5cf2b105 convert S3 remote state to a backend
Move the S3 State from a legacy remote state to an official backend.

This increases test coverage, uses a set schema for configuration, and
will allow new backend features to be implemented for the S3 state, e.g.
"environments".
2017-03-22 10:59:37 -04:00
Mitchell Hashimoto
1daff7a826
backend/consul: support "lock" option to disable locking
This adds a "lock" config (default true) to allow users to optionally
disable state locking with Consul. This is necessary if the token given
doesn't have session permission and is necessary for backwards
compatibility.
2017-03-14 17:59:10 -07:00
Ash Berlin
70de22253a Fix error message in Azure state backend (#12424)
The error led me to try adding `resource_group` but the code wanted `resource_group_name`. This fixes the error message to match the code.
2017-03-04 20:27:37 +02:00
Peter McAtominey
cbed463b44 state/azure: add environment option for non-public cloud usage (#12364) 2017-03-02 04:58:24 +00:00
Mitchell Hashimoto
942572b574
backend/init: add atlas, remove legacy atlas remote state 2017-03-01 13:29:16 -08:00