At the risk of a little bit of hidden spooky action at a distance, this
will slightly change the behavior of the "goimports check" to compare
against the base branch of a PR rather than to origin/main if we happen
to find one of the environment variables that GitHub Actions sets
automatically in its runners. This is targeting our "checks.yml" workflow
in particular.
The intention here is to avoid misreporting files that haven't actually
changed when a PR is targeting a branch other than the main branch, such
as directly targeting a historical release branch.
We'll still run against origin/main when we're not running in GitHub
Actions, since that's _typically_ the correct branch to use for new
work, even if it will eventually get backported to a release branch.
The previous implementation of this check tried to accumulate all of the
changed files into a single big string and then run goimports once with
all of them, but that approach ran into problems for changesets of a
certain (platform-specific) size due to limits on maximum command line
length.
This new version instead uses bash arrays and runs goimports separately
for each of the files which appear to have changed relative to the
base branch. This is likely to be slower to complete for changesets that
have many different changed files, but it's better for it to be slow than
to return an error and fail to check some of the files.
Because we maintain multiple versions of Terraform across different
release branches, we aim to avoid creating needless differences between
the branches to maximize the chance of successful automatic backporting.
Part of that policy is that we don't make cross-cutting changes to respond
to deprecation of functions in upstream packages and instead we respond
to them gradually over time when we'd be changing the nearby code anyway,
or when new work requires using the replacement APIs.
In recognition of that, this turns of the staticcheck rule that would
otherwise force us to resolve all deprecations before moving forward with
any other change.
* Add goimports as a check to the Github commit actions
* check diff against main instead of last commit
* goimports should fix the problems
* fix up conditionals and wildcard matching
* specify origin/main in diff
* fetch main branch when checkout
* back to origin main
This is intended to eventually replace the CircleCI-based checks we use
as part of the PR process in this repository. We're already using GitHub
Actions for various other processes in this repository, so this change is
motivated by consistency of having all of our automation running in the
same system and written in the same language.
This is not a complete replacement for our CircleCI workflow yet, and
probably won't ever be because the CircleCI workflow contains some steps
that are arguably redundant with other processes we follow elsewhere.
However, the CircleCI workflow remains for now and won't be removed until
we're satisfied that enough of it is replicated by this GitHub Actions
workflow.
Running the tool this way ensures that we'll always run the version
selected by our go.mod file, rather than whatever happened to be available
in $GOPATH/bin on the system where we're running this.
This change caused some contexts to now be using a newer version of
staticcheck with additional checks, and so this commit also includes some
changes to quiet the new warnings without any change in overall behavior.
For now, only check the JSON views package, since this was the instance
that most recently tripped us up. There are a few dozen failures
elsewhere in Terraform which would need to be addressed before expanding
this to other packages.
We have a few different .proto files in this repository that all need to
get recompiled into .pb.go files each time we change them, but we were
previously handling that with some scripts that just assumed that protoc
and the relevant plugins were already installed on the system somewhere,
at the right versions.
In practice we've been constantly flopping between different versions of
these tools due to folks having different versions installed in their
development environments. In particular, the state of the .pb.go files
in the prior commit wasn't reproducible by any single version of the tools
because they've all slightly diverged from one another.
In the interests of being more consistent here and avoiding accidental
inconsistencies, we'll now centralize the protocol buffer compile steps
all into a single tool that knows how to fetch and install the expected
versions of the various tools we need and then run those tools with the
right options to get a stable result.
If we want to upgrade to either a newer protoc or a newer protoc-gen-go
in future then we'll do that in a central location and update all of the
.pb.go files at the same time, so that we're always consistently tracking
the same version of protocol buffers everywhere.
While doing this I attempted to keep as close as possible to the toolchain
we'd most recently used, but since they were not consistent with each
other they've now all changed which version numbers they record at minimum,
and the planproto stub in particular now also has a slightly different
descriptor serialization but is otherwise offering the same API.
The revision field is only populated on dev builds so this means
most releases of Terraform have an empty "terraform_revision" field
in the JSON output. Since we recommend developers use go tooling
to `go build` this tool when developing, the revision is not useful
data and so it is removed.
Remove chef, habitat, puppet, and salt-masterless provsioners,
which follows their deprecation. Update the documentatin for these
provisioners to clarify that they have been removed from later versions
of Terraform. Adds the fmt Make target back and updates fmtcheck script
for correctness.
Just as with 336b352d6f, this refreshes our rate limit bypass cookie for
go.googlesource.com since the old one has apparently expired.
I did this by visiting https://go.googlesource.com/ and selecting
"Generate Password" from the top navigation. This cookie belongs to a
test account used by the Terraform team and should not be used by
non-Terraform codebases; please generate your own!
We're still using vendoring for now until we get _all_ of our tooling
updated, so the main idea here is to force use of the vendor directory
when running tests and building for development so we can quickly find
situations where we forget to run "go mod vendor".
We also setting GO111MODULE=off for installation of tools. Right now this
is the best way to install a tool in GOBIN without also interfering with
go.mod and go.sum, until a better pattern for managing tool dependencies
is devised by the Go team.
Finally, we run "go mod download" before launching "gox" in the main
build process, to prime the local module cache once so that the concurrent
"go build" processes won't race to populate it redundantly. This means
that we'll be producing final builds from the module cache rather than
from vendor as with everything else -- there's currently no way to tell
gox to use -mod=vendor -- but that should be fine in practice since
our go.sum file will ensure that we get the exact sources we expect in
the module cache before building.
This patch allows `build.sh` to be used with terraform plugins to
easily create cross-platform packages, using the same method as the
terraform Makefile:
```
mkdir scripts
curl https://raw.githubusercontent.com/hashicorp/terraform/master/scripts/build.sh -o scripts/build.sh
TF_RELEASE=1 sh -c "scripts/build.sh" # make bin
```
This launches Terraform inside a headless dlv configured to accept a
remote debugging process. It's configured this way so it can be easily
used from a debugger GUI integrated into an IDE/editor, but it can also
be used from the CLI by running the command it prints.
Using a remote debugger here is useful even when debugging with the CLI,
since it keeps Terraform's verbose and colorful output from interfering
with the debugger UI.
We have a generated cookie for googlesource.com so that we don't get so rate-limited when cloning Google-hosted Go libraries.
The previous credential was invalidated, so this is a newly-generated one. This credential does nothing except allow us to fetch git repositories from go.googlesource.com with a slightly-higher rate limit.
#15596 set things up with the intent that the docker image build process
would be handled by the automated build system on dockerhub, but after
merging we found that it's impossible to change the source git repository
for an existing dockerhub repository.
To get away from the limitations of dockerhub, we intend to eventually
automate these builds in a separate CI system. Here we add some scripts
that would drive such an automated process. It's split into multiple steps
to allow for situations where the new version should not be tagged as
the latest, and to make it easier and safer to test the build script
while doing development on it.
Since this automated process doesn't yet exist, a wrapper script
release.sh is included to help run a local, manual build and deploy
process in the mean time. The README.md in the docker-release dir here
contains details on the intended usage.
At the time of commit this file contains some things that are not yet true
due to the dockerhub configuration not having been updated. This therefore
remains aspirational until merged, after which the docherhub configuration
will be updated.
These are different than our "full" images because they include a binary
already released to releases.hashicorp.com, whereas the root Dockerfile
directly builds from the current work tree.
This particular Dockerfile is not intended to be run manually, but rather
exists only to drive the dockerhub automated build.
Since there is little left that isn't core, remove the distinction for
now to reduce confusion, since a "core" binary will mostly work except
for provisioners.
Instead of using a hardcoded version prerelease string, which makes release automation difficult, set the version prerelease string from an environment variable via the go linker tool during compile time.
The environment variable `TF_RELEASE` should only be set via the `make bin` target, and thus leaves the version prerelease string unset. Otherwise, when running a local compile of terraform via the `make dev` makefile target, the version prerelease string is set to `"dev"`, as usual.
This also requires some changes to both the circonus and postgresql providers, as they directly used the `VersionPrerelease` constant. We now simply call the `VersionString()` function, which returns the proper interpolated version string with the prerelease string populated correctly.
`TF_RELEASE` is unset:
```sh
$ make dev
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/05/22 10:38:19 Generated command/internal_plugin_list.go
==> Removing old directory...
==> Building...
Number of parallel builds: 3
--> linux/amd64: github.com/hashicorp/terraform
==> Results:
total 209M
-rwxr-xr-x 1 jake jake 209M May 22 10:39 terraform
$ terraform version
Terraform v0.9.6-dev (fd472e4a86500606b03c314f70d11f2bc4bc84e5+CHANGES)
```
`TF_RELEASE` is set (mimicking the `make bin` target):
```sh
$ TF_RELEASE=1 make dev
==> Checking that code complies with gofmt requirements...
go generate $(go list ./... | grep -v /terraform/vendor/)
2017/05/22 10:40:39 Generated command/internal_plugin_list.go
==> Removing old directory...
==> Building...
Number of parallel builds: 3
--> linux/amd64: github.com/hashicorp/terraform
==> Results:
total 121M
-rwxr-xr-x 1 jake jake 121M May 22 10:42 terraform
$ terraform version
Terraform v0.9.6
```
* core/providersplit: Split OPC Provider to separate repo
As we march towards Terraform 0.10.0, we are going to start building the
terraform providers as separate binaries - this will allow us to
continually release them. Before we go to 0.10.0, we need to be able to
continue building providers in the same manner, therefore, we have
hardcoded the path of the provider in the generate-plugins.go file
The interim solution will require us to vendor the opc provider and any
child dependencies, but when we get to 0.10.0, we will no longer have to
do this - the core will auto download the plugin binary. The plugin
package will have it's own dependencies vendored as well.
* core/providersplit: Removing the builtin version of OPC provider
* core/providersplit: Vendoring the OPC plugin
* core/providersplit: update internal plugin list
* core/providersplit: remove unused govendor item
