(cloudwatch) update aws-sdk-go to v1.5.8 (#6658)

* (cloudwatch) upgrade aws-sdk-go v1.5.8

* (cloudwatch) remove non-required aws-sdk-go part

* add go-ini
This commit is contained in:
Mitsuhiro Tanda
2016-11-22 21:38:11 +09:00
committed by Torkel Ödegaard
parent 50811c80ea
commit 51f80affe3
1008 changed files with 32404 additions and 766084 deletions

View File

@@ -1,11 +0,0 @@
dist
/doc
/doc-staging
.yardoc
Gemfile.lock
awstesting/integration/smoke/**/importmarker__.go
awstesting/integration/smoke/_test/
/vendor/bin/
/vendor/pkg/
/vendor/src/
/private/model/cli/gen-api/gen-api

View File

@@ -1,14 +0,0 @@
{
"PkgHandler": {
"Pattern": "/sdk-for-go/api/",
"StripPrefix": "/sdk-for-go/api",
"Include": ["/src/github.com/aws/aws-sdk-go/aws", "/src/github.com/aws/aws-sdk-go/service"],
"Exclude": ["/src/cmd", "/src/github.com/aws/aws-sdk-go/awstesting", "/src/github.com/aws/aws-sdk-go/awsmigrate"],
"IgnoredSuffixes": ["iface"]
},
"Github": {
"Tag": "master",
"Repo": "/aws/aws-sdk-go",
"UseGithub": true
}
}

View File

@@ -1,23 +0,0 @@
language: go
sudo: false
go:
- 1.4
- 1.5
- 1.6
- tip
# Use Go 1.5's vendoring experiment for 1.5 tests. 1.4 tests will use the tip of the dependencies repo.
env:
- GO15VENDOREXPERIMENT=1
install:
- make get-deps
script:
- make unit-with-race-cover
matrix:
allow_failures:
- go: tip

View File

@@ -1,7 +0,0 @@
--plugin go
-e doc-src/plugin/plugin.rb
-m markdown
-o doc/api
--title "AWS SDK for Go"
aws/**/*.go
service/**/*.go

204
vendor/github.com/aws/aws-sdk-go/CHANGELOG.md generated vendored Normal file
View File

@@ -0,0 +1,204 @@
Release v1.5.8 (2016-11-18)
===
Service Client Updates
---
`service/application-autoscaling`: Updates service API and documentation
`service/elasticmapreduce`: Updates service API and documentation
`service/elastictranscoder`: Updates service API, documentation, and examples
`service/gamelift`: Updates service API and documentation
`service/lambda`: Updates service API and documentation
Release v1.5.7 (2016-11-18)
===
Service Client Updates
---
`service/apigateway`: Updates service API and documentation
`service/meteringmarketplace`: Updates service API and documentation
`service/monitoring`: Updates service API and documentation
`service/sqs`: Updates service API, documentation, and examples
Release v1.5.6 (2016-11-16)
===
Service Client Updates
---
`service/route53`: Updates service API and documentation
`service/servicecatalog`: Updates service API and documentation
Release v1.5.5 (2016-11-15)
===
Service Client Updates
---
`service/ds`: Updates service API and documentation
`service/elasticache`: Updates service API and documentation
`service/kinesis`: Updates service API and documentation
Release v1.5.4 (2016-11-15)
===
Service Client Updates
---
`service/cognito-idp`: Updates service API and documentation
Release v1.5.3 (2016-11-11)
===
Service Client Updates
---
`service/cloudformation`: Updates service documentation and examples
`service/logs`: Updates service API and documentation
Release v1.5.2 (2016-11-03)
===
Service Client Updates
---
`service/directconnect`: Updates service API and documentation
Release v1.5.1 (2016-11-02)
===
Service Client Updates
---
`service/email`: Updates service API and documentation
Release v1.5.0 (2016-11-01)
===
Service Client Updates
---
`service/cloudformation`: Updates service API and documentation
`service/ecr`: Updates service paginators
SDK Feature Updates
---
* `private/model/api`: Add generated setters for API parameters (#918)
* Adds setters to the SDK's API parameter types, and are a convenience method that reduce the need to use `aws.String` and like utility.
Release v1.4.22 (2016-10-25)
===
Service Client Updates
---
* `service/elasticloadbalancingv2`: Updates service documentation.
* `service/autoscaling`: Updates service documentation.
Release v1.4.21 (2016-10-24)
===
Service Client Updates
---
* `service/sms`: AWS Server Migration Service (SMS) is an agentless service which makes it easier and faster for you to migrate thousands of on-premises workloads to AWS. AWS SMS allows you to automate, schedule, and track incremental replications of live server volumes, making it easier for you to coordinate large-scale server migrations.
* `service/ecs`: Updates documentation.
SDK Feature Updates
---
* `private/models/api`: Improve code generation of documentation.
Release v1.4.20 (2016-10-20)
===
Service Client Updates
---
* `service/budgets`: Adds new service, AWS Budgets.
* `service/waf`: Updates service documentation.
Release v1.4.19 (2016-10-18)
===
Service Client Updates
---
* `service/cloudfront`: Updates service API and documentation.
* Ability to use Amazon CloudFront to deliver your content both via IPv6 and IPv4 using HTTP/HTTPS.
* `service/configservice`: Update service API and documentation.
* `service/iot`: Updates service API and documentation.
* `service/kinesisanalytics`: Updates service API and documentation.
* Whenever Amazon Kinesis Analytics is not able to detect schema for the given streaming source on DiscoverInputSchema API, we would return the raw records that was sampled to detect the schema.
* `service/rds`: Updates service API and documentation.
* Amazon Aurora integrates with other AWS services to allow you to extend your Aurora DB cluster to utilize other capabilities in the AWS cloud. Permission to access other AWS services is granted by creating an IAM role with the necessary permissions, and then associating the role with your DB cluster.
SDK Feature Updates
---
* `service/dynamodb/dynamodbattribute`: Add UnmarshalListOfMaps #897
* Adds support for unmarshaling a list of maps. This is useful for unmarshaling the DynamoDB AttributeValue list of maps returned by APIs like Query and Scan.
Release v1.4.18 (2016-10-17)
===
Service Model Updates
---
* `service/route53`: Updates service API and documentation.
Release v1.4.17
===
Service Model Updates
---
* `service/acm`: Update service API, and documentation.
* This change allows users to import third-party SSL/TLS certificates into ACM.
* `service/elasticbeanstalk`: Update service API, documentation, and pagination.
* Elastic Beanstalk DescribeApplicationVersions API is being updated to support pagination.
* `service/gamelift`: Update service API, and documentation.
* New APIs to protect game developer resource (builds, alias, fleets, instances, game sessions and player sessions) against abuse.
SDK Features
---
* `service/s3`: Add support for accelerate with dualstack [#887](https://github.com/aws/aws-sdk-go/issues/887)
Release v1.4.16 (2016-10-13)
===
Service Model Updates
---
* `service/ecr`: Update Amazon EC2 Container Registry service model
* DescribeImages is a new api used to expose image metadata which today includes image size and image creation timestamp.
* `service/elasticache`: Update Amazon ElastiCache service model
* Elasticache is launching a new major engine release of Redis, 3.2 (providing stability updates and new command sets over 2.8), as well as ElasticSupport for enabling Redis Cluster in 3.2, which provides support for multiple node groups to horizontally scale data, as well as superior engine failover capabilities
SDK Bug Fixes
---
* `aws/session`: Skip shared config on read errors [#883](https://github.com/aws/aws-sdk-go/issues/883)
* `aws/signer/v4`: Add support for URL.EscapedPath to signer [#885](https://github.com/aws/aws-sdk-go/issues/885)
SDK Features
---
* `private/model/api`: Add docs for errors to API operations [#881](https://github.com/aws/aws-sdk-go/issues/881)
* `private/model/api`: Improve field and waiter doc strings [#879](https://github.com/aws/aws-sdk-go/issues/879)
* `service/dynamodb/dynamodbattribute`: Allow multiple struct tag elements [#886](https://github.com/aws/aws-sdk-go/issues/886)
* Add build tags to internal SDK tools [#880](https://github.com/aws/aws-sdk-go/issues/880)
Release v1.4.15 (2016-10-06)
===
Service Model Updates
---
* `service/cognitoidentityprovider`: Update Amazon Cognito Identity Provider service model
* `service/devicefarm`: Update AWS Device Farm documentation
* `service/opsworks`: Update AWS OpsWorks service model
* `service/s3`: Update Amazon Simple Storage Service model
* `service/waf`: Update AWS WAF service model
SDK Bug Fixes
---
* `aws/request`: Fix HTTP Request Body race condition [#874](https://github.com/aws/aws-sdk-go/issues/874)
SDK Feature Updates
---
* `aws/ec2metadata`: Add support for EC2 User Data [#872](https://github.com/aws/aws-sdk-go/issues/872)
* `aws/signer/v4`: Remove logic determining if request needs to be resigned [#876](https://github.com/aws/aws-sdk-go/issues/876)
Release v1.4.14 (2016-09-29)
===
* `service/ec2`: api, documentation, and paginators updates.
* `service/s3`: api and documentation updates.
Release v1.4.13 (2016-09-27)
===
* `service/codepipeline`: documentation updates.
* `service/cloudformation`: api and documentation updates.
* `service/kms`: documentation updates.
* `service/elasticfilesystem`: documentation updates.
* `service/snowball`: documentation updates.

View File

@@ -5,10 +5,13 @@ LINTIGNORESTUTTER='service/[^/]+/(api|service)\.go:.+(and that stutters)'
LINTIGNOREINFLECT='service/[^/]+/(api|service)\.go:.+method .+ should be '
LINTIGNOREINFLECTS3UPLOAD='service/s3/s3manager/upload\.go:.+struct field SSEKMSKeyId should be '
LINTIGNOREDEPS='vendor/.+\.go'
UNIT_TEST_TAGS="example codegen"
SDK_WITH_VENDOR_PKGS=$(shell go list ./... | grep -v "/vendor/src")
SDK_WITH_VENDOR_PKGS=$(shell go list -tags ${UNIT_TEST_TAGS} ./... | grep -v "/vendor/src")
SDK_ONLY_PKGS=$(shell go list ./... | grep -v "/vendor/")
SDK_UNIT_TEST_ONLY_PKGS=$(shell go list -tags ${UNIT_TEST_TAGS} ./... | grep -v "/vendor/")
SDK_GO_1_4=$(shell go version | grep "go1.4")
SDK_GO_1_5=$(shell go version | grep "go1.5")
SDK_GO_VERSION=$(shell go version | awk '''{print $$3}''' | tr -d '''\n''')
all: get-deps generate unit
@@ -50,11 +53,11 @@ build:
unit: get-deps-tests build verify
@echo "go test SDK and vendor packages"
@go test -tags $(SDK_ONLY_PKGS)
@go test -tags ${UNIT_TEST_TAGS} $(SDK_UNIT_TEST_ONLY_PKGS)
unit-with-race-cover: get-deps-tests build verify
@echo "go test SDK and vendor packages"
@go test -tags -race -cpu=1,2,4 $(SDK_ONLY_PKGS)
@go test -tags ${UNIT_TEST_TAGS} -race -cpu=1,2,4 $(SDK_UNIT_TEST_ONLY_PKGS)
integration: get-deps-tests integ-custom smoke-tests performance
@@ -101,20 +104,18 @@ verify: get-deps-verify lint vet
lint:
@echo "go lint SDK and vendor packages"
@lint=`if [ -z "${SDK_GO_1_4}" ]; then golint ./...; else echo "skipping golint"; fi`; \
@lint=`if [ \( -z "${SDK_GO_1_4}" \) -a \( -z "${SDK_GO_1_5}" \) ]; then golint ./...; else echo "skipping golint"; fi`; \
lint=`echo "$$lint" | grep -E -v -e ${LINTIGNOREDOT} -e ${LINTIGNOREDOC} -e ${LINTIGNORECONST} -e ${LINTIGNORESTUTTER} -e ${LINTIGNOREINFLECT} -e ${LINTIGNOREDEPS} -e ${LINTIGNOREINFLECTS3UPLOAD}`; \
echo "$$lint"; \
if [ "$$lint" != "" ] && [ "$$lint" != "skipping golint" ]; then exit 1; fi
SDK_BASE_FOLDERS=$(shell ls -d */ | grep -v vendor | grep -v awsmigrate)
ifneq (,$(findstring go1.5, ${SDK_GO_VERSION}))
GO_VET_CMD=go tool vet --all -shadow
ifneq (,$(findstring go1.4, ${SDK_GO_VERSION}))
GO_VET_CMD=echo skipping go vet, ${SDK_GO_VERSION}
else ifneq (,$(findstring go1.6, ${SDK_GO_VERSION}))
GO_VET_CMD=go tool vet --all -shadow -example=false
else ifneq (,$(findstring devel, ${SDK_GO_VERSION}))
GO_VET_CMD=go tool vet --all -shadow -tests=false
else
GO_VET_CMD=echo skipping go vet, ${SDK_GO_VERSION}
GO_VET_CMD=go tool vet --all -shadow
endif
vet:
@@ -129,10 +130,11 @@ get-deps-tests:
go get github.com/gucumber/gucumber/cmd/gucumber
go get github.com/stretchr/testify
go get github.com/smartystreets/goconvey
go get golang.org/x/net/html
get-deps-verify:
@echo "go get SDK verification utilities"
@if [ -z "${SDK_GO_1_4}" ]; then go get github.com/golang/lint/golint; else echo "skipped getting golint"; fi
@if [ \( -z "${SDK_GO_1_4}" \) -a \( -z "${SDK_GO_1_5}" \) ]; then go get github.com/golang/lint/golint; else echo "skipped getting golint"; fi
bench:
@echo "go bench SDK packages"
@@ -144,9 +146,12 @@ bench-protocol:
docs:
@echo "generate SDK docs"
rm -rf doc && bundle install && bundle exec yard
@# This env variable, DOCS, is for internal use
@if [ -n "$(AWS_DOC_GEN_TOOL)" ]; then echo "For internal use. Subject to change."; $(AWS_DOC_GEN_TOOL) `pwd`; fi
@if [ -z ${AWS_DOC_GEN_TOOL} ]; then\
rm -rf doc && bundle install && bundle exec yard;\
else\
$(AWS_DOC_GEN_TOOL) `pwd`;\
fi
api_info:
@go run private/model/cli/api-info/api-info.go

View File

@@ -17,7 +17,7 @@ If you are using Go 1.5 with the `GO15VENDOREXPERIMENT=1` vendoring flag, or 1.6
go get -u github.com/aws/aws-sdk-go
Otherwise if your Go environment does not have vendoring support enabled, or you do not want to include the vendored SDK's dependencies you can use the following command to retrieve the SDK and its non-testing dependencies using `go get`.
Otherwise if your Go environment does not have vendoring support enabled, or you do not want to include the vendored SDK's dependencies you can use the following command to retrieve the SDK and its non-testing dependencies using `go get`.
go get -u github.com/aws/aws-sdk-go/aws/...
go get -u github.com/aws/aws-sdk-go/service/...
@@ -25,7 +25,7 @@ Otherwise if your Go environment does not have vendoring support enabled, or you
If you're looking to retrieve just the SDK without any dependencies use the following command.
go get -d github.com/aws/aws-sdk-go/
These two processes will still include the `vendor` folder and it should be deleted if its not going to be used by your environment.
rm -rf $GOPATH/src/github.com/aws/aws-sdk-go/vendor
@@ -84,10 +84,15 @@ import (
)
func main() {
sess, err := session.NewSession()
if err != nil {
panic(err)
}
// Create an EC2 service object in the "us-west-2" region
// Note that you can also configure your region globally by
// exporting the AWS_REGION environment variable
svc := ec2.New(session.New(), &aws.Config{Region: aws.String("us-west-2")})
svc := ec2.New(sess, &aws.Config{Region: aws.String("us-west-2")})
// Call the DescribeInstances Operation
resp, err := svc.DescribeInstances(nil)

View File

@@ -3,6 +3,7 @@ package awsutil
import (
"io"
"reflect"
"time"
)
// Copy deeply copies a src structure to dst. Useful for copying request and
@@ -49,7 +50,14 @@ func rcopy(dst, src reflect.Value, root bool) {
} else {
e := src.Type().Elem()
if dst.CanSet() && !src.IsNil() {
dst.Set(reflect.New(e))
if _, ok := src.Interface().(*time.Time); !ok {
dst.Set(reflect.New(e))
} else {
tempValue := reflect.New(e)
tempValue.Elem().Set(src.Elem())
// Sets time.Time's unexported values
dst.Set(tempValue)
}
}
if src.Elem().IsValid() {
// Keep the current root state since the depth hasn't changed

View File

@@ -1,233 +0,0 @@
package awsutil_test
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"testing"
"github.com/aws/aws-sdk-go/aws/awsutil"
"github.com/stretchr/testify/assert"
)
func ExampleCopy() {
type Foo struct {
A int
B []*string
}
// Create the initial value
str1 := "hello"
str2 := "bye bye"
f1 := &Foo{A: 1, B: []*string{&str1, &str2}}
// Do the copy
var f2 Foo
awsutil.Copy(&f2, f1)
// Print the result
fmt.Println(awsutil.Prettify(f2))
// Output:
// {
// A: 1,
// B: ["hello","bye bye"]
// }
}
func TestCopy(t *testing.T) {
type Foo struct {
A int
B []*string
C map[string]*int
}
// Create the initial value
str1 := "hello"
str2 := "bye bye"
int1 := 1
int2 := 2
f1 := &Foo{
A: 1,
B: []*string{&str1, &str2},
C: map[string]*int{
"A": &int1,
"B": &int2,
},
}
// Do the copy
var f2 Foo
awsutil.Copy(&f2, f1)
// Values are equal
assert.Equal(t, f2.A, f1.A)
assert.Equal(t, f2.B, f1.B)
assert.Equal(t, f2.C, f1.C)
// But pointers are not!
str3 := "nothello"
int3 := 57
f2.A = 100
f2.B[0] = &str3
f2.C["B"] = &int3
assert.NotEqual(t, f2.A, f1.A)
assert.NotEqual(t, f2.B, f1.B)
assert.NotEqual(t, f2.C, f1.C)
}
func TestCopyNestedWithUnexported(t *testing.T) {
type Bar struct {
a int
B int
}
type Foo struct {
A string
B Bar
}
f1 := &Foo{A: "string", B: Bar{a: 1, B: 2}}
var f2 Foo
awsutil.Copy(&f2, f1)
// Values match
assert.Equal(t, f2.A, f1.A)
assert.NotEqual(t, f2.B, f1.B)
assert.NotEqual(t, f2.B.a, f1.B.a)
assert.Equal(t, f2.B.B, f2.B.B)
}
func TestCopyIgnoreNilMembers(t *testing.T) {
type Foo struct {
A *string
B []string
C map[string]string
}
f := &Foo{}
assert.Nil(t, f.A)
assert.Nil(t, f.B)
assert.Nil(t, f.C)
var f2 Foo
awsutil.Copy(&f2, f)
assert.Nil(t, f2.A)
assert.Nil(t, f2.B)
assert.Nil(t, f2.C)
fcopy := awsutil.CopyOf(f)
f3 := fcopy.(*Foo)
assert.Nil(t, f3.A)
assert.Nil(t, f3.B)
assert.Nil(t, f3.C)
}
func TestCopyPrimitive(t *testing.T) {
str := "hello"
var s string
awsutil.Copy(&s, &str)
assert.Equal(t, "hello", s)
}
func TestCopyNil(t *testing.T) {
var s string
awsutil.Copy(&s, nil)
assert.Equal(t, "", s)
}
func TestCopyReader(t *testing.T) {
var buf io.Reader = bytes.NewReader([]byte("hello world"))
var r io.Reader
awsutil.Copy(&r, buf)
b, err := ioutil.ReadAll(r)
assert.NoError(t, err)
assert.Equal(t, []byte("hello world"), b)
// empty bytes because this is not a deep copy
b, err = ioutil.ReadAll(buf)
assert.NoError(t, err)
assert.Equal(t, []byte(""), b)
}
func TestCopyDifferentStructs(t *testing.T) {
type SrcFoo struct {
A int
B []*string
C map[string]*int
SrcUnique string
SameNameDiffType int
unexportedPtr *int
ExportedPtr *int
}
type DstFoo struct {
A int
B []*string
C map[string]*int
DstUnique int
SameNameDiffType string
unexportedPtr *int
ExportedPtr *int
}
// Create the initial value
str1 := "hello"
str2 := "bye bye"
int1 := 1
int2 := 2
f1 := &SrcFoo{
A: 1,
B: []*string{&str1, &str2},
C: map[string]*int{
"A": &int1,
"B": &int2,
},
SrcUnique: "unique",
SameNameDiffType: 1,
unexportedPtr: &int1,
ExportedPtr: &int2,
}
// Do the copy
var f2 DstFoo
awsutil.Copy(&f2, f1)
// Values are equal
assert.Equal(t, f2.A, f1.A)
assert.Equal(t, f2.B, f1.B)
assert.Equal(t, f2.C, f1.C)
assert.Equal(t, "unique", f1.SrcUnique)
assert.Equal(t, 1, f1.SameNameDiffType)
assert.Equal(t, 0, f2.DstUnique)
assert.Equal(t, "", f2.SameNameDiffType)
assert.Equal(t, int1, *f1.unexportedPtr)
assert.Nil(t, f2.unexportedPtr)
assert.Equal(t, int2, *f1.ExportedPtr)
assert.Equal(t, int2, *f2.ExportedPtr)
}
func ExampleCopyOf() {
type Foo struct {
A int
B []*string
}
// Create the initial value
str1 := "hello"
str2 := "bye bye"
f1 := &Foo{A: 1, B: []*string{&str1, &str2}}
// Do the copy
v := awsutil.CopyOf(f1)
var f2 *Foo = v.(*Foo)
// Print the result
fmt.Println(awsutil.Prettify(f2))
// Output:
// {
// A: 1,
// B: ["hello","bye bye"]
// }
}

View File

@@ -1,29 +0,0 @@
package awsutil_test
import (
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awsutil"
"github.com/stretchr/testify/assert"
)
func TestDeepEqual(t *testing.T) {
cases := []struct {
a, b interface{}
equal bool
}{
{"a", "a", true},
{"a", "b", false},
{"a", aws.String(""), false},
{"a", nil, false},
{"a", aws.String("a"), true},
{(*bool)(nil), (*bool)(nil), true},
{(*bool)(nil), (*string)(nil), false},
{nil, nil, true},
}
for i, c := range cases {
assert.Equal(t, c.equal, awsutil.DeepEqual(c.a, c.b), "%d, a:%v b:%v, %t", i, c.a, c.b, c.equal)
}
}

View File

@@ -106,8 +106,8 @@ func rValuesAtPath(v interface{}, path string, createPath, caseSensitive, nilTer
if indexStar || index != nil {
nextvals = []reflect.Value{}
for _, value := range values {
value := reflect.Indirect(value)
for _, valItem := range values {
value := reflect.Indirect(valItem)
if value.Kind() != reflect.Slice {
continue
}

View File

@@ -1,142 +0,0 @@
package awsutil_test
import (
"testing"
"github.com/aws/aws-sdk-go/aws/awsutil"
"github.com/stretchr/testify/assert"
)
type Struct struct {
A []Struct
z []Struct
B *Struct
D *Struct
C string
E map[string]string
}
var data = Struct{
A: []Struct{{C: "value1"}, {C: "value2"}, {C: "value3"}},
z: []Struct{{C: "value1"}, {C: "value2"}, {C: "value3"}},
B: &Struct{B: &Struct{C: "terminal"}, D: &Struct{C: "terminal2"}},
C: "initial",
}
var data2 = Struct{A: []Struct{
{A: []Struct{{C: "1"}, {C: "1"}, {C: "1"}, {C: "1"}, {C: "1"}}},
{A: []Struct{{C: "2"}, {C: "2"}, {C: "2"}, {C: "2"}, {C: "2"}}},
}}
func TestValueAtPathSuccess(t *testing.T) {
var testCases = []struct {
expect []interface{}
data interface{}
path string
}{
{[]interface{}{"initial"}, data, "C"},
{[]interface{}{"value1"}, data, "A[0].C"},
{[]interface{}{"value2"}, data, "A[1].C"},
{[]interface{}{"value3"}, data, "A[2].C"},
{[]interface{}{"value3"}, data, "a[2].c"},
{[]interface{}{"value3"}, data, "A[-1].C"},
{[]interface{}{"value1", "value2", "value3"}, data, "A[].C"},
{[]interface{}{"terminal"}, data, "B . B . C"},
{[]interface{}{"initial"}, data, "A.D.X || C"},
{[]interface{}{"initial"}, data, "A[0].B || C"},
{[]interface{}{
Struct{A: []Struct{{C: "1"}, {C: "1"}, {C: "1"}, {C: "1"}, {C: "1"}}},
Struct{A: []Struct{{C: "2"}, {C: "2"}, {C: "2"}, {C: "2"}, {C: "2"}}},
}, data2, "A"},
}
for i, c := range testCases {
v, err := awsutil.ValuesAtPath(c.data, c.path)
assert.NoError(t, err, "case %d, expected no error, %s", i, c.path)
assert.Equal(t, c.expect, v, "case %d, %s", i, c.path)
}
}
func TestValueAtPathFailure(t *testing.T) {
var testCases = []struct {
expect []interface{}
errContains string
data interface{}
path string
}{
{nil, "", data, "C.x"},
{nil, "SyntaxError: Invalid token: tDot", data, ".x"},
{nil, "", data, "X.Y.Z"},
{nil, "", data, "A[100].C"},
{nil, "", data, "A[3].C"},
{nil, "", data, "B.B.C.Z"},
{nil, "", data, "z[-1].C"},
{nil, "", nil, "A.B.C"},
{[]interface{}{}, "", Struct{}, "A"},
{nil, "", data, "A[0].B.C"},
{nil, "", data, "D"},
}
for i, c := range testCases {
v, err := awsutil.ValuesAtPath(c.data, c.path)
if c.errContains != "" {
assert.Contains(t, err.Error(), c.errContains, "case %d, expected error, %s", i, c.path)
continue
} else {
assert.NoError(t, err, "case %d, expected no error, %s", i, c.path)
}
assert.Equal(t, c.expect, v, "case %d, %s", i, c.path)
}
}
func TestSetValueAtPathSuccess(t *testing.T) {
var s Struct
awsutil.SetValueAtPath(&s, "C", "test1")
awsutil.SetValueAtPath(&s, "B.B.C", "test2")
awsutil.SetValueAtPath(&s, "B.D.C", "test3")
assert.Equal(t, "test1", s.C)
assert.Equal(t, "test2", s.B.B.C)
assert.Equal(t, "test3", s.B.D.C)
awsutil.SetValueAtPath(&s, "B.*.C", "test0")
assert.Equal(t, "test0", s.B.B.C)
assert.Equal(t, "test0", s.B.D.C)
var s2 Struct
awsutil.SetValueAtPath(&s2, "b.b.c", "test0")
assert.Equal(t, "test0", s2.B.B.C)
awsutil.SetValueAtPath(&s2, "A", []Struct{{}})
assert.Equal(t, []Struct{{}}, s2.A)
str := "foo"
s3 := Struct{}
awsutil.SetValueAtPath(&s3, "b.b.c", str)
assert.Equal(t, "foo", s3.B.B.C)
s3 = Struct{B: &Struct{B: &Struct{C: str}}}
awsutil.SetValueAtPath(&s3, "b.b.c", nil)
assert.Equal(t, "", s3.B.B.C)
s3 = Struct{}
awsutil.SetValueAtPath(&s3, "b.b.c", nil)
assert.Equal(t, "", s3.B.B.C)
s3 = Struct{}
awsutil.SetValueAtPath(&s3, "b.b.c", &str)
assert.Equal(t, "foo", s3.B.B.C)
var s4 struct{ Name *string }
awsutil.SetValueAtPath(&s4, "Name", str)
assert.Equal(t, str, *s4.Name)
s4 = struct{ Name *string }{}
awsutil.SetValueAtPath(&s4, "Name", nil)
assert.Equal(t, (*string)(nil), s4.Name)
s4 = struct{ Name *string }{Name: &str}
awsutil.SetValueAtPath(&s4, "Name", nil)
assert.Equal(t, (*string)(nil), s4.Name)
s4 = struct{ Name *string }{}
awsutil.SetValueAtPath(&s4, "Name", &str)
assert.Equal(t, str, *s4.Name)
}

View File

@@ -61,6 +61,12 @@ func prettify(v reflect.Value, indent int, buf *bytes.Buffer) {
buf.WriteString("\n" + strings.Repeat(" ", indent) + "}")
case reflect.Slice:
strtype := v.Type().String()
if strtype == "[]uint8" {
fmt.Fprintf(buf, "<binary> len %d", v.Len())
break
}
nl, id, id2 := "", "", ""
if v.Len() > 3 {
nl, id, id2 = "\n", strings.Repeat(" ", indent), strings.Repeat(" ", indent+2)

View File

@@ -2,7 +2,6 @@ package client
import (
"fmt"
"io/ioutil"
"net/http/httputil"
"github.com/aws/aws-sdk-go/aws"
@@ -87,16 +86,24 @@ const logReqMsg = `DEBUG: Request %s/%s Details:
%s
-----------------------------------------------------`
const logReqErrMsg = `DEBUG ERROR: Request %s/%s:
---[ REQUEST DUMP ERROR ]-----------------------------
%s
-----------------------------------------------------`
func logRequest(r *request.Request) {
logBody := r.Config.LogLevel.Matches(aws.LogDebugWithHTTPBody)
dumpedBody, _ := httputil.DumpRequestOut(r.HTTPRequest, logBody)
dumpedBody, err := httputil.DumpRequestOut(r.HTTPRequest, logBody)
if err != nil {
r.Config.Logger.Log(fmt.Sprintf(logReqErrMsg, r.ClientInfo.ServiceName, r.Operation.Name, err))
return
}
if logBody {
// Reset the request body because dumpRequest will re-wrap the r.HTTPRequest's
// Body as a NoOpCloser and will not be reset after read by the HTTP
// client reader.
r.Body.Seek(r.BodyStart, 0)
r.HTTPRequest.Body = ioutil.NopCloser(r.Body)
r.ResetBody()
}
r.Config.Logger.Log(fmt.Sprintf(logReqMsg, r.ClientInfo.ServiceName, r.Operation.Name, string(dumpedBody)))
@@ -107,11 +114,21 @@ const logRespMsg = `DEBUG: Response %s/%s Details:
%s
-----------------------------------------------------`
const logRespErrMsg = `DEBUG ERROR: Response %s/%s:
---[ RESPONSE DUMP ERROR ]-----------------------------
%s
-----------------------------------------------------`
func logResponse(r *request.Request) {
var msg = "no response data"
if r.HTTPResponse != nil {
logBody := r.Config.LogLevel.Matches(aws.LogDebugWithHTTPBody)
dumpedBody, _ := httputil.DumpResponse(r.HTTPResponse, logBody)
dumpedBody, err := httputil.DumpResponse(r.HTTPResponse, logBody)
if err != nil {
r.Config.Logger.Log(fmt.Sprintf(logRespErrMsg, r.ClientInfo.ServiceName, r.Operation.Name, err))
return
}
msg = string(dumpedBody)
} else if r.Error != nil {
msg = r.Error.Error()

View File

@@ -137,9 +137,6 @@ type Config struct {
// accelerate enabled. If the bucket is not enabled for accelerate an error
// will be returned. The bucket name must be DNS compatible to also work
// with accelerate.
//
// Not compatible with UseDualStack requests will fail if both flags are
// specified.
S3UseAccelerate *bool
// Set this to `true` to disable the EC2Metadata client from overriding the
@@ -185,6 +182,19 @@ type Config struct {
// the delay of a request see the aws/client.DefaultRetryer and
// aws/request.Retryer.
SleepDelay func(time.Duration)
// DisableRestProtocolURICleaning will not clean the URL path when making rest protocol requests.
// Will default to false. This would only be used for empty directory names in s3 requests.
//
// Example:
// sess, err := session.NewSession(&aws.Config{DisableRestProtocolURICleaning: aws.Bool(true))
//
// svc := s3.New(sess)
// out, err := svc.GetObject(&s3.GetObjectInput {
// Bucket: aws.String("bucketname"),
// Key: aws.String("//foo//bar//moo"),
// })
DisableRestProtocolURICleaning *bool
}
// NewConfig returns a new Config pointer that can be chained with builder
@@ -406,6 +416,10 @@ func mergeInConfig(dst *Config, other *Config) {
if other.SleepDelay != nil {
dst.SleepDelay = other.SleepDelay
}
if other.DisableRestProtocolURICleaning != nil {
dst.DisableRestProtocolURICleaning = other.DisableRestProtocolURICleaning
}
}
// Copy will return a shallow copy of the Config object. If any additional

View File

@@ -1,86 +0,0 @@
package aws
import (
"net/http"
"reflect"
"testing"
"github.com/aws/aws-sdk-go/aws/credentials"
)
var testCredentials = credentials.NewStaticCredentials("AKID", "SECRET", "SESSION")
var copyTestConfig = Config{
Credentials: testCredentials,
Endpoint: String("CopyTestEndpoint"),
Region: String("COPY_TEST_AWS_REGION"),
DisableSSL: Bool(true),
HTTPClient: http.DefaultClient,
LogLevel: LogLevel(LogDebug),
Logger: NewDefaultLogger(),
MaxRetries: Int(3),
DisableParamValidation: Bool(true),
DisableComputeChecksums: Bool(true),
S3ForcePathStyle: Bool(true),
}
func TestCopy(t *testing.T) {
want := copyTestConfig
got := copyTestConfig.Copy()
if !reflect.DeepEqual(*got, want) {
t.Errorf("Copy() = %+v", got)
t.Errorf(" want %+v", want)
}
got.Region = String("other")
if got.Region == want.Region {
t.Errorf("Expect setting copy values not not reflect in source")
}
}
func TestCopyReturnsNewInstance(t *testing.T) {
want := copyTestConfig
got := copyTestConfig.Copy()
if got == &want {
t.Errorf("Copy() = %p; want different instance as source %p", got, &want)
}
}
var mergeTestZeroValueConfig = Config{}
var mergeTestConfig = Config{
Credentials: testCredentials,
Endpoint: String("MergeTestEndpoint"),
Region: String("MERGE_TEST_AWS_REGION"),
DisableSSL: Bool(true),
HTTPClient: http.DefaultClient,
LogLevel: LogLevel(LogDebug),
Logger: NewDefaultLogger(),
MaxRetries: Int(10),
DisableParamValidation: Bool(true),
DisableComputeChecksums: Bool(true),
S3ForcePathStyle: Bool(true),
}
var mergeTests = []struct {
cfg *Config
in *Config
want *Config
}{
{&Config{}, nil, &Config{}},
{&Config{}, &mergeTestZeroValueConfig, &Config{}},
{&Config{}, &mergeTestConfig, &mergeTestConfig},
}
func TestMerge(t *testing.T) {
for i, tt := range mergeTests {
got := tt.cfg.Copy()
got.MergeIn(tt.in)
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("Config %d %+v", i, tt.cfg)
t.Errorf(" Merge(%+v)", tt.in)
t.Errorf(" got %+v", got)
t.Errorf(" want %+v", tt.want)
}
}
}

View File

@@ -1,437 +0,0 @@
package aws
import (
"testing"
"time"
"github.com/stretchr/testify/assert"
)
var testCasesStringSlice = [][]string{
{"a", "b", "c", "d", "e"},
{"a", "b", "", "", "e"},
}
func TestStringSlice(t *testing.T) {
for idx, in := range testCasesStringSlice {
if in == nil {
continue
}
out := StringSlice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := StringValueSlice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesStringValueSlice = [][]*string{
{String("a"), String("b"), nil, String("c")},
}
func TestStringValueSlice(t *testing.T) {
for idx, in := range testCasesStringValueSlice {
if in == nil {
continue
}
out := StringValueSlice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
if in[i] == nil {
assert.Empty(t, out[i], "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, *(in[i]), out[i], "Unexpected value at idx %d", idx)
}
}
out2 := StringSlice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
for i := range out2 {
if in[i] == nil {
assert.Empty(t, *(out2[i]), "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, in[i], out2[i], "Unexpected value at idx %d", idx)
}
}
}
}
var testCasesStringMap = []map[string]string{
{"a": "1", "b": "2", "c": "3"},
}
func TestStringMap(t *testing.T) {
for idx, in := range testCasesStringMap {
if in == nil {
continue
}
out := StringMap(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := StringValueMap(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesBoolSlice = [][]bool{
{true, true, false, false},
}
func TestBoolSlice(t *testing.T) {
for idx, in := range testCasesBoolSlice {
if in == nil {
continue
}
out := BoolSlice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := BoolValueSlice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesBoolValueSlice = [][]*bool{}
func TestBoolValueSlice(t *testing.T) {
for idx, in := range testCasesBoolValueSlice {
if in == nil {
continue
}
out := BoolValueSlice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
if in[i] == nil {
assert.Empty(t, out[i], "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, *(in[i]), out[i], "Unexpected value at idx %d", idx)
}
}
out2 := BoolSlice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
for i := range out2 {
if in[i] == nil {
assert.Empty(t, *(out2[i]), "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, in[i], out2[i], "Unexpected value at idx %d", idx)
}
}
}
}
var testCasesBoolMap = []map[string]bool{
{"a": true, "b": false, "c": true},
}
func TestBoolMap(t *testing.T) {
for idx, in := range testCasesBoolMap {
if in == nil {
continue
}
out := BoolMap(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := BoolValueMap(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesIntSlice = [][]int{
{1, 2, 3, 4},
}
func TestIntSlice(t *testing.T) {
for idx, in := range testCasesIntSlice {
if in == nil {
continue
}
out := IntSlice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := IntValueSlice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesIntValueSlice = [][]*int{}
func TestIntValueSlice(t *testing.T) {
for idx, in := range testCasesIntValueSlice {
if in == nil {
continue
}
out := IntValueSlice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
if in[i] == nil {
assert.Empty(t, out[i], "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, *(in[i]), out[i], "Unexpected value at idx %d", idx)
}
}
out2 := IntSlice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
for i := range out2 {
if in[i] == nil {
assert.Empty(t, *(out2[i]), "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, in[i], out2[i], "Unexpected value at idx %d", idx)
}
}
}
}
var testCasesIntMap = []map[string]int{
{"a": 3, "b": 2, "c": 1},
}
func TestIntMap(t *testing.T) {
for idx, in := range testCasesIntMap {
if in == nil {
continue
}
out := IntMap(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := IntValueMap(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesInt64Slice = [][]int64{
{1, 2, 3, 4},
}
func TestInt64Slice(t *testing.T) {
for idx, in := range testCasesInt64Slice {
if in == nil {
continue
}
out := Int64Slice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := Int64ValueSlice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesInt64ValueSlice = [][]*int64{}
func TestInt64ValueSlice(t *testing.T) {
for idx, in := range testCasesInt64ValueSlice {
if in == nil {
continue
}
out := Int64ValueSlice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
if in[i] == nil {
assert.Empty(t, out[i], "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, *(in[i]), out[i], "Unexpected value at idx %d", idx)
}
}
out2 := Int64Slice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
for i := range out2 {
if in[i] == nil {
assert.Empty(t, *(out2[i]), "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, in[i], out2[i], "Unexpected value at idx %d", idx)
}
}
}
}
var testCasesInt64Map = []map[string]int64{
{"a": 3, "b": 2, "c": 1},
}
func TestInt64Map(t *testing.T) {
for idx, in := range testCasesInt64Map {
if in == nil {
continue
}
out := Int64Map(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := Int64ValueMap(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesFloat64Slice = [][]float64{
{1, 2, 3, 4},
}
func TestFloat64Slice(t *testing.T) {
for idx, in := range testCasesFloat64Slice {
if in == nil {
continue
}
out := Float64Slice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := Float64ValueSlice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesFloat64ValueSlice = [][]*float64{}
func TestFloat64ValueSlice(t *testing.T) {
for idx, in := range testCasesFloat64ValueSlice {
if in == nil {
continue
}
out := Float64ValueSlice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
if in[i] == nil {
assert.Empty(t, out[i], "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, *(in[i]), out[i], "Unexpected value at idx %d", idx)
}
}
out2 := Float64Slice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
for i := range out2 {
if in[i] == nil {
assert.Empty(t, *(out2[i]), "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, in[i], out2[i], "Unexpected value at idx %d", idx)
}
}
}
}
var testCasesFloat64Map = []map[string]float64{
{"a": 3, "b": 2, "c": 1},
}
func TestFloat64Map(t *testing.T) {
for idx, in := range testCasesFloat64Map {
if in == nil {
continue
}
out := Float64Map(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := Float64ValueMap(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesTimeSlice = [][]time.Time{
{time.Now(), time.Now().AddDate(100, 0, 0)},
}
func TestTimeSlice(t *testing.T) {
for idx, in := range testCasesTimeSlice {
if in == nil {
continue
}
out := TimeSlice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := TimeValueSlice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}
var testCasesTimeValueSlice = [][]*time.Time{}
func TestTimeValueSlice(t *testing.T) {
for idx, in := range testCasesTimeValueSlice {
if in == nil {
continue
}
out := TimeValueSlice(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
if in[i] == nil {
assert.Empty(t, out[i], "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, *(in[i]), out[i], "Unexpected value at idx %d", idx)
}
}
out2 := TimeSlice(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
for i := range out2 {
if in[i] == nil {
assert.Empty(t, *(out2[i]), "Unexpected value at idx %d", idx)
} else {
assert.Equal(t, in[i], out2[i], "Unexpected value at idx %d", idx)
}
}
}
}
var testCasesTimeMap = []map[string]time.Time{
{"a": time.Now().AddDate(-100, 0, 0), "b": time.Now()},
}
func TestTimeMap(t *testing.T) {
for idx, in := range testCasesTimeMap {
if in == nil {
continue
}
out := TimeMap(in)
assert.Len(t, out, len(in), "Unexpected len at idx %d", idx)
for i := range out {
assert.Equal(t, in[i], *(out[i]), "Unexpected value at idx %d", idx)
}
out2 := TimeValueMap(out)
assert.Len(t, out2, len(in), "Unexpected len at idx %d", idx)
assert.Equal(t, in, out2, "Unexpected value at idx %d", idx)
}
}

View File

@@ -10,9 +10,11 @@ import (
"regexp"
"runtime"
"strconv"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/request"
)
@@ -67,6 +69,34 @@ var SDKVersionUserAgentHandler = request.NamedHandler{
var reStatusCode = regexp.MustCompile(`^(\d{3})`)
// ValidateReqSigHandler is a request handler to ensure that the request's
// signature doesn't expire before it is sent. This can happen when a request
// is built and signed signficantly before it is sent. Or signficant delays
// occur whne retrying requests that would cause the signature to expire.
var ValidateReqSigHandler = request.NamedHandler{
Name: "core.ValidateReqSigHandler",
Fn: func(r *request.Request) {
// Unsigned requests are not signed
if r.Config.Credentials == credentials.AnonymousCredentials {
return
}
signedTime := r.Time
if !r.LastSignedAt.IsZero() {
signedTime = r.LastSignedAt
}
// 10 minutes to allow for some clock skew/delays in transmission.
// Would be improved with aws/aws-sdk-go#423
if signedTime.Add(10 * time.Minute).After(time.Now()) {
return
}
fmt.Println("request expired, resigning")
r.Sign()
},
}
// SendHandler is a request handler to send service request using HTTP client.
var SendHandler = request.NamedHandler{Name: "core.SendHandler", Fn: func(r *request.Request) {
var err error

View File

@@ -1,192 +0,0 @@
package corehandlers_test
import (
"bytes"
"fmt"
"io/ioutil"
"net/http"
"net/http/httptest"
"os"
"testing"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/corehandlers"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/awstesting"
"github.com/aws/aws-sdk-go/awstesting/unit"
"github.com/aws/aws-sdk-go/service/s3"
)
func TestValidateEndpointHandler(t *testing.T) {
os.Clearenv()
svc := awstesting.NewClient(aws.NewConfig().WithRegion("us-west-2"))
svc.Handlers.Clear()
svc.Handlers.Validate.PushBackNamed(corehandlers.ValidateEndpointHandler)
req := svc.NewRequest(&request.Operation{Name: "Operation"}, nil, nil)
err := req.Build()
assert.NoError(t, err)
}
func TestValidateEndpointHandlerErrorRegion(t *testing.T) {
os.Clearenv()
svc := awstesting.NewClient()
svc.Handlers.Clear()
svc.Handlers.Validate.PushBackNamed(corehandlers.ValidateEndpointHandler)
req := svc.NewRequest(&request.Operation{Name: "Operation"}, nil, nil)
err := req.Build()
assert.Error(t, err)
assert.Equal(t, aws.ErrMissingRegion, err)
}
type mockCredsProvider struct {
expired bool
retrieveCalled bool
}
func (m *mockCredsProvider) Retrieve() (credentials.Value, error) {
m.retrieveCalled = true
return credentials.Value{ProviderName: "mockCredsProvider"}, nil
}
func (m *mockCredsProvider) IsExpired() bool {
return m.expired
}
func TestAfterRetryRefreshCreds(t *testing.T) {
os.Clearenv()
credProvider := &mockCredsProvider{}
svc := awstesting.NewClient(&aws.Config{
Credentials: credentials.NewCredentials(credProvider),
MaxRetries: aws.Int(1),
})
svc.Handlers.Clear()
svc.Handlers.ValidateResponse.PushBack(func(r *request.Request) {
r.Error = awserr.New("UnknownError", "", nil)
r.HTTPResponse = &http.Response{StatusCode: 400, Body: ioutil.NopCloser(bytes.NewBuffer([]byte{}))}
})
svc.Handlers.UnmarshalError.PushBack(func(r *request.Request) {
r.Error = awserr.New("ExpiredTokenException", "", nil)
})
svc.Handlers.AfterRetry.PushBackNamed(corehandlers.AfterRetryHandler)
assert.True(t, svc.Config.Credentials.IsExpired(), "Expect to start out expired")
assert.False(t, credProvider.retrieveCalled)
req := svc.NewRequest(&request.Operation{Name: "Operation"}, nil, nil)
req.Send()
assert.True(t, svc.Config.Credentials.IsExpired())
assert.False(t, credProvider.retrieveCalled)
_, err := svc.Config.Credentials.Get()
assert.NoError(t, err)
assert.True(t, credProvider.retrieveCalled)
}
type testSendHandlerTransport struct{}
func (t *testSendHandlerTransport) RoundTrip(r *http.Request) (*http.Response, error) {
return nil, fmt.Errorf("mock error")
}
func TestSendHandlerError(t *testing.T) {
svc := awstesting.NewClient(&aws.Config{
HTTPClient: &http.Client{
Transport: &testSendHandlerTransport{},
},
})
svc.Handlers.Clear()
svc.Handlers.Send.PushBackNamed(corehandlers.SendHandler)
r := svc.NewRequest(&request.Operation{Name: "Operation"}, nil, nil)
r.Send()
assert.Error(t, r.Error)
assert.NotNil(t, r.HTTPResponse)
}
func setupContentLengthTestServer(t *testing.T, hasContentLength bool, contentLength int64) *httptest.Server {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, ok := r.Header["Content-Length"]
assert.Equal(t, hasContentLength, ok, "expect content length to be set, %t", hasContentLength)
assert.Equal(t, contentLength, r.ContentLength)
b, err := ioutil.ReadAll(r.Body)
assert.NoError(t, err)
r.Body.Close()
authHeader := r.Header.Get("Authorization")
if hasContentLength {
assert.Contains(t, authHeader, "content-length")
} else {
assert.NotContains(t, authHeader, "content-length")
}
assert.Equal(t, contentLength, int64(len(b)))
}))
return server
}
func TestBuildContentLength_ZeroBody(t *testing.T) {
server := setupContentLengthTestServer(t, false, 0)
svc := s3.New(unit.Session, &aws.Config{
Endpoint: aws.String(server.URL),
S3ForcePathStyle: aws.Bool(true),
DisableSSL: aws.Bool(true),
})
_, err := svc.GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucketname"),
Key: aws.String("keyname"),
})
assert.NoError(t, err)
}
func TestBuildContentLength_NegativeBody(t *testing.T) {
server := setupContentLengthTestServer(t, false, 0)
svc := s3.New(unit.Session, &aws.Config{
Endpoint: aws.String(server.URL),
S3ForcePathStyle: aws.Bool(true),
DisableSSL: aws.Bool(true),
})
req, _ := svc.GetObjectRequest(&s3.GetObjectInput{
Bucket: aws.String("bucketname"),
Key: aws.String("keyname"),
})
req.HTTPRequest.Header.Set("Content-Length", "-1")
assert.NoError(t, req.Send())
}
func TestBuildContentLength_WithBody(t *testing.T) {
server := setupContentLengthTestServer(t, true, 1024)
svc := s3.New(unit.Session, &aws.Config{
Endpoint: aws.String(server.URL),
S3ForcePathStyle: aws.Bool(true),
DisableSSL: aws.Bool(true),
})
_, err := svc.PutObject(&s3.PutObjectInput{
Bucket: aws.String("bucketname"),
Key: aws.String("keyname"),
Body: bytes.NewReader(make([]byte, 1024)),
})
assert.NoError(t, err)
}

View File

@@ -1,254 +0,0 @@
package corehandlers_test
import (
"fmt"
"testing"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/client"
"github.com/aws/aws-sdk-go/aws/client/metadata"
"github.com/aws/aws-sdk-go/aws/corehandlers"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/awstesting/unit"
"github.com/aws/aws-sdk-go/service/kinesis"
"github.com/stretchr/testify/require"
)
var testSvc = func() *client.Client {
s := &client.Client{
Config: aws.Config{},
ClientInfo: metadata.ClientInfo{
ServiceName: "mock-service",
APIVersion: "2015-01-01",
},
}
return s
}()
type StructShape struct {
_ struct{} `type:"structure"`
RequiredList []*ConditionalStructShape `required:"true"`
RequiredMap map[string]*ConditionalStructShape `required:"true"`
RequiredBool *bool `required:"true"`
OptionalStruct *ConditionalStructShape
hiddenParameter *string
}
func (s *StructShape) Validate() error {
invalidParams := request.ErrInvalidParams{Context: "StructShape"}
if s.RequiredList == nil {
invalidParams.Add(request.NewErrParamRequired("RequiredList"))
}
if s.RequiredMap == nil {
invalidParams.Add(request.NewErrParamRequired("RequiredMap"))
}
if s.RequiredBool == nil {
invalidParams.Add(request.NewErrParamRequired("RequiredBool"))
}
if s.RequiredList != nil {
for i, v := range s.RequiredList {
if v == nil {
continue
}
if err := v.Validate(); err != nil {
invalidParams.AddNested(fmt.Sprintf("%s[%v]", "RequiredList", i), err.(request.ErrInvalidParams))
}
}
}
if s.RequiredMap != nil {
for i, v := range s.RequiredMap {
if v == nil {
continue
}
if err := v.Validate(); err != nil {
invalidParams.AddNested(fmt.Sprintf("%s[%v]", "RequiredMap", i), err.(request.ErrInvalidParams))
}
}
}
if s.OptionalStruct != nil {
if err := s.OptionalStruct.Validate(); err != nil {
invalidParams.AddNested("OptionalStruct", err.(request.ErrInvalidParams))
}
}
if invalidParams.Len() > 0 {
return invalidParams
}
return nil
}
type ConditionalStructShape struct {
_ struct{} `type:"structure"`
Name *string `required:"true"`
}
func (s *ConditionalStructShape) Validate() error {
invalidParams := request.ErrInvalidParams{Context: "ConditionalStructShape"}
if s.Name == nil {
invalidParams.Add(request.NewErrParamRequired("Name"))
}
if invalidParams.Len() > 0 {
return invalidParams
}
return nil
}
func TestNoErrors(t *testing.T) {
input := &StructShape{
RequiredList: []*ConditionalStructShape{},
RequiredMap: map[string]*ConditionalStructShape{
"key1": {Name: aws.String("Name")},
"key2": {Name: aws.String("Name")},
},
RequiredBool: aws.Bool(true),
OptionalStruct: &ConditionalStructShape{Name: aws.String("Name")},
}
req := testSvc.NewRequest(&request.Operation{}, input, nil)
corehandlers.ValidateParametersHandler.Fn(req)
require.NoError(t, req.Error)
}
func TestMissingRequiredParameters(t *testing.T) {
input := &StructShape{}
req := testSvc.NewRequest(&request.Operation{}, input, nil)
corehandlers.ValidateParametersHandler.Fn(req)
require.Error(t, req.Error)
assert.Equal(t, "InvalidParameter", req.Error.(awserr.Error).Code())
assert.Equal(t, "3 validation error(s) found.", req.Error.(awserr.Error).Message())
errs := req.Error.(awserr.BatchedErrors).OrigErrs()
assert.Len(t, errs, 3)
assert.Equal(t, "ParamRequiredError: missing required field, StructShape.RequiredList.", errs[0].Error())
assert.Equal(t, "ParamRequiredError: missing required field, StructShape.RequiredMap.", errs[1].Error())
assert.Equal(t, "ParamRequiredError: missing required field, StructShape.RequiredBool.", errs[2].Error())
assert.Equal(t, "InvalidParameter: 3 validation error(s) found.\n- missing required field, StructShape.RequiredList.\n- missing required field, StructShape.RequiredMap.\n- missing required field, StructShape.RequiredBool.\n", req.Error.Error())
}
func TestNestedMissingRequiredParameters(t *testing.T) {
input := &StructShape{
RequiredList: []*ConditionalStructShape{{}},
RequiredMap: map[string]*ConditionalStructShape{
"key1": {Name: aws.String("Name")},
"key2": {},
},
RequiredBool: aws.Bool(true),
OptionalStruct: &ConditionalStructShape{},
}
req := testSvc.NewRequest(&request.Operation{}, input, nil)
corehandlers.ValidateParametersHandler.Fn(req)
require.Error(t, req.Error)
assert.Equal(t, "InvalidParameter", req.Error.(awserr.Error).Code())
assert.Equal(t, "3 validation error(s) found.", req.Error.(awserr.Error).Message())
errs := req.Error.(awserr.BatchedErrors).OrigErrs()
assert.Len(t, errs, 3)
assert.Equal(t, "ParamRequiredError: missing required field, StructShape.RequiredList[0].Name.", errs[0].Error())
assert.Equal(t, "ParamRequiredError: missing required field, StructShape.RequiredMap[key2].Name.", errs[1].Error())
assert.Equal(t, "ParamRequiredError: missing required field, StructShape.OptionalStruct.Name.", errs[2].Error())
}
type testInput struct {
StringField *string `min:"5"`
ListField []string `min:"3"`
MapField map[string]string `min:"4"`
}
func (s testInput) Validate() error {
invalidParams := request.ErrInvalidParams{Context: "testInput"}
if s.StringField != nil && len(*s.StringField) < 5 {
invalidParams.Add(request.NewErrParamMinLen("StringField", 5))
}
if s.ListField != nil && len(s.ListField) < 3 {
invalidParams.Add(request.NewErrParamMinLen("ListField", 3))
}
if s.MapField != nil && len(s.MapField) < 4 {
invalidParams.Add(request.NewErrParamMinLen("MapField", 4))
}
if invalidParams.Len() > 0 {
return invalidParams
}
return nil
}
var testsFieldMin = []struct {
err awserr.Error
in testInput
}{
{
err: func() awserr.Error {
invalidParams := request.ErrInvalidParams{Context: "testInput"}
invalidParams.Add(request.NewErrParamMinLen("StringField", 5))
return invalidParams
}(),
in: testInput{StringField: aws.String("abcd")},
},
{
err: func() awserr.Error {
invalidParams := request.ErrInvalidParams{Context: "testInput"}
invalidParams.Add(request.NewErrParamMinLen("StringField", 5))
invalidParams.Add(request.NewErrParamMinLen("ListField", 3))
return invalidParams
}(),
in: testInput{StringField: aws.String("abcd"), ListField: []string{"a", "b"}},
},
{
err: func() awserr.Error {
invalidParams := request.ErrInvalidParams{Context: "testInput"}
invalidParams.Add(request.NewErrParamMinLen("StringField", 5))
invalidParams.Add(request.NewErrParamMinLen("ListField", 3))
invalidParams.Add(request.NewErrParamMinLen("MapField", 4))
return invalidParams
}(),
in: testInput{StringField: aws.String("abcd"), ListField: []string{"a", "b"}, MapField: map[string]string{"a": "a", "b": "b"}},
},
{
err: nil,
in: testInput{StringField: aws.String("abcde"),
ListField: []string{"a", "b", "c"}, MapField: map[string]string{"a": "a", "b": "b", "c": "c", "d": "d"}},
},
}
func TestValidateFieldMinParameter(t *testing.T) {
for i, c := range testsFieldMin {
req := testSvc.NewRequest(&request.Operation{}, &c.in, nil)
corehandlers.ValidateParametersHandler.Fn(req)
assert.Equal(t, c.err, req.Error, "%d case failed", i)
}
}
func BenchmarkValidateAny(b *testing.B) {
input := &kinesis.PutRecordsInput{
StreamName: aws.String("stream"),
}
for i := 0; i < 100; i++ {
record := &kinesis.PutRecordsRequestEntry{
Data: make([]byte, 10000),
PartitionKey: aws.String("partition"),
}
input.Records = append(input.Records, record)
}
req, _ := kinesis.New(unit.Session).PutRecordsRequest(input)
b.ResetTimer()
for i := 0; i < b.N; i++ {
corehandlers.ValidateParametersHandler.Fn(req)
if err := req.Error; err != nil {
b.Fatalf("validation failed: %v", err)
}
}
}

View File

@@ -34,7 +34,7 @@ var (
//
// Example of ChainProvider to be used with an EnvProvider and EC2RoleProvider.
// In this example EnvProvider will first check if any credentials are available
// vai the environment variables. If there are none ChainProvider will check
// via the environment variables. If there are none ChainProvider will check
// the next Provider in the list, EC2RoleProvider in this case. If EC2RoleProvider
// does not return any credentials ChainProvider will return the error
// ErrNoValidProvidersFoundInChain

View File

@@ -1,154 +0,0 @@
package credentials
import (
"testing"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/stretchr/testify/assert"
)
type secondStubProvider struct {
creds Value
expired bool
err error
}
func (s *secondStubProvider) Retrieve() (Value, error) {
s.expired = false
s.creds.ProviderName = "secondStubProvider"
return s.creds, s.err
}
func (s *secondStubProvider) IsExpired() bool {
return s.expired
}
func TestChainProviderWithNames(t *testing.T) {
p := &ChainProvider{
Providers: []Provider{
&stubProvider{err: awserr.New("FirstError", "first provider error", nil)},
&stubProvider{err: awserr.New("SecondError", "second provider error", nil)},
&secondStubProvider{
creds: Value{
AccessKeyID: "AKIF",
SecretAccessKey: "NOSECRET",
SessionToken: "",
},
},
&stubProvider{
creds: Value{
AccessKeyID: "AKID",
SecretAccessKey: "SECRET",
SessionToken: "",
},
},
},
}
creds, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "secondStubProvider", creds.ProviderName, "Expect provider name to match")
// Also check credentials
assert.Equal(t, "AKIF", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "NOSECRET", creds.SecretAccessKey, "Expect secret access key to match")
assert.Empty(t, creds.SessionToken, "Expect session token to be empty")
}
func TestChainProviderGet(t *testing.T) {
p := &ChainProvider{
Providers: []Provider{
&stubProvider{err: awserr.New("FirstError", "first provider error", nil)},
&stubProvider{err: awserr.New("SecondError", "second provider error", nil)},
&stubProvider{
creds: Value{
AccessKeyID: "AKID",
SecretAccessKey: "SECRET",
SessionToken: "",
},
},
},
}
creds, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "AKID", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "SECRET", creds.SecretAccessKey, "Expect secret access key to match")
assert.Empty(t, creds.SessionToken, "Expect session token to be empty")
}
func TestChainProviderIsExpired(t *testing.T) {
stubProvider := &stubProvider{expired: true}
p := &ChainProvider{
Providers: []Provider{
stubProvider,
},
}
assert.True(t, p.IsExpired(), "Expect expired to be true before any Retrieve")
_, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.False(t, p.IsExpired(), "Expect not expired after retrieve")
stubProvider.expired = true
assert.True(t, p.IsExpired(), "Expect return of expired provider")
_, err = p.Retrieve()
assert.False(t, p.IsExpired(), "Expect not expired after retrieve")
}
func TestChainProviderWithNoProvider(t *testing.T) {
p := &ChainProvider{
Providers: []Provider{},
}
assert.True(t, p.IsExpired(), "Expect expired with no providers")
_, err := p.Retrieve()
assert.Equal(t,
ErrNoValidProvidersFoundInChain,
err,
"Expect no providers error returned")
}
func TestChainProviderWithNoValidProvider(t *testing.T) {
errs := []error{
awserr.New("FirstError", "first provider error", nil),
awserr.New("SecondError", "second provider error", nil),
}
p := &ChainProvider{
Providers: []Provider{
&stubProvider{err: errs[0]},
&stubProvider{err: errs[1]},
},
}
assert.True(t, p.IsExpired(), "Expect expired with no providers")
_, err := p.Retrieve()
assert.Equal(t,
ErrNoValidProvidersFoundInChain,
err,
"Expect no providers error returned")
}
func TestChainProviderWithNoValidProviderWithVerboseEnabled(t *testing.T) {
errs := []error{
awserr.New("FirstError", "first provider error", nil),
awserr.New("SecondError", "second provider error", nil),
}
p := &ChainProvider{
VerboseErrors: true,
Providers: []Provider{
&stubProvider{err: errs[0]},
&stubProvider{err: errs[1]},
},
}
assert.True(t, p.IsExpired(), "Expect expired with no providers")
_, err := p.Retrieve()
assert.Equal(t,
awserr.NewBatchError("NoCredentialProviders", "no valid providers in chain", errs),
err,
"Expect no providers error returned")
}

View File

@@ -1,73 +0,0 @@
package credentials
import (
"testing"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/stretchr/testify/assert"
)
type stubProvider struct {
creds Value
expired bool
err error
}
func (s *stubProvider) Retrieve() (Value, error) {
s.expired = false
s.creds.ProviderName = "stubProvider"
return s.creds, s.err
}
func (s *stubProvider) IsExpired() bool {
return s.expired
}
func TestCredentialsGet(t *testing.T) {
c := NewCredentials(&stubProvider{
creds: Value{
AccessKeyID: "AKID",
SecretAccessKey: "SECRET",
SessionToken: "",
},
expired: true,
})
creds, err := c.Get()
assert.Nil(t, err, "Expected no error")
assert.Equal(t, "AKID", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "SECRET", creds.SecretAccessKey, "Expect secret access key to match")
assert.Empty(t, creds.SessionToken, "Expect session token to be empty")
}
func TestCredentialsGetWithError(t *testing.T) {
c := NewCredentials(&stubProvider{err: awserr.New("provider error", "", nil), expired: true})
_, err := c.Get()
assert.Equal(t, "provider error", err.(awserr.Error).Code(), "Expected provider error")
}
func TestCredentialsExpire(t *testing.T) {
stub := &stubProvider{}
c := NewCredentials(stub)
stub.expired = false
assert.True(t, c.IsExpired(), "Expected to start out expired")
c.Expire()
assert.True(t, c.IsExpired(), "Expected to be expired")
c.forceRefresh = false
assert.False(t, c.IsExpired(), "Expected not to be expired")
stub.expired = true
assert.True(t, c.IsExpired(), "Expected to be expired")
}
func TestCredentialsGetWithProviderName(t *testing.T) {
stub := &stubProvider{}
c := NewCredentials(stub)
creds, err := c.Get()
assert.Nil(t, err, "Expected no error")
assert.Equal(t, creds.ProviderName, "stubProvider", "Expected provider name to match")
}

View File

@@ -111,7 +111,7 @@ func (m *EC2RoleProvider) Retrieve() (credentials.Value, error) {
}, nil
}
// A ec2RoleCredRespBody provides the shape for unmarshalling credential
// A ec2RoleCredRespBody provides the shape for unmarshaling credential
// request responses.
type ec2RoleCredRespBody struct {
// Success State

View File

@@ -1,159 +0,0 @@
package ec2rolecreds_test
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/awstesting/unit"
)
const credsRespTmpl = `{
"Code": "Success",
"Type": "AWS-HMAC",
"AccessKeyId" : "accessKey",
"SecretAccessKey" : "secret",
"Token" : "token",
"Expiration" : "%s",
"LastUpdated" : "2009-11-23T0:00:00Z"
}`
const credsFailRespTmpl = `{
"Code": "ErrorCode",
"Message": "ErrorMsg",
"LastUpdated": "2009-11-23T0:00:00Z"
}`
func initTestServer(expireOn string, failAssume bool) *httptest.Server {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/latest/meta-data/iam/security-credentials" {
fmt.Fprintln(w, "RoleName")
} else if r.URL.Path == "/latest/meta-data/iam/security-credentials/RoleName" {
if failAssume {
fmt.Fprintf(w, credsFailRespTmpl)
} else {
fmt.Fprintf(w, credsRespTmpl, expireOn)
}
} else {
http.Error(w, "bad request", http.StatusBadRequest)
}
}))
return server
}
func TestEC2RoleProvider(t *testing.T) {
server := initTestServer("2014-12-16T01:51:37Z", false)
defer server.Close()
p := &ec2rolecreds.EC2RoleProvider{
Client: ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")}),
}
creds, err := p.Retrieve()
assert.Nil(t, err, "Expect no error, %v", err)
assert.Equal(t, "accessKey", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "secret", creds.SecretAccessKey, "Expect secret access key to match")
assert.Equal(t, "token", creds.SessionToken, "Expect session token to match")
}
func TestEC2RoleProviderFailAssume(t *testing.T) {
server := initTestServer("2014-12-16T01:51:37Z", true)
defer server.Close()
p := &ec2rolecreds.EC2RoleProvider{
Client: ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")}),
}
creds, err := p.Retrieve()
assert.Error(t, err, "Expect error")
e := err.(awserr.Error)
assert.Equal(t, "ErrorCode", e.Code())
assert.Equal(t, "ErrorMsg", e.Message())
assert.Nil(t, e.OrigErr())
assert.Equal(t, "", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "", creds.SecretAccessKey, "Expect secret access key to match")
assert.Equal(t, "", creds.SessionToken, "Expect session token to match")
}
func TestEC2RoleProviderIsExpired(t *testing.T) {
server := initTestServer("2014-12-16T01:51:37Z", false)
defer server.Close()
p := &ec2rolecreds.EC2RoleProvider{
Client: ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")}),
}
p.CurrentTime = func() time.Time {
return time.Date(2014, 12, 15, 21, 26, 0, 0, time.UTC)
}
assert.True(t, p.IsExpired(), "Expect creds to be expired before retrieve.")
_, err := p.Retrieve()
assert.Nil(t, err, "Expect no error, %v", err)
assert.False(t, p.IsExpired(), "Expect creds to not be expired after retrieve.")
p.CurrentTime = func() time.Time {
return time.Date(3014, 12, 15, 21, 26, 0, 0, time.UTC)
}
assert.True(t, p.IsExpired(), "Expect creds to be expired.")
}
func TestEC2RoleProviderExpiryWindowIsExpired(t *testing.T) {
server := initTestServer("2014-12-16T01:51:37Z", false)
defer server.Close()
p := &ec2rolecreds.EC2RoleProvider{
Client: ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")}),
ExpiryWindow: time.Hour * 1,
}
p.CurrentTime = func() time.Time {
return time.Date(2014, 12, 15, 0, 51, 37, 0, time.UTC)
}
assert.True(t, p.IsExpired(), "Expect creds to be expired before retrieve.")
_, err := p.Retrieve()
assert.Nil(t, err, "Expect no error, %v", err)
assert.False(t, p.IsExpired(), "Expect creds to not be expired after retrieve.")
p.CurrentTime = func() time.Time {
return time.Date(2014, 12, 16, 0, 55, 37, 0, time.UTC)
}
assert.True(t, p.IsExpired(), "Expect creds to be expired.")
}
func BenchmarkEC3RoleProvider(b *testing.B) {
server := initTestServer("2014-12-16T01:51:37Z", false)
defer server.Close()
p := &ec2rolecreds.EC2RoleProvider{
Client: ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")}),
}
_, err := p.Retrieve()
if err != nil {
b.Fatal(err)
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
if _, err := p.Retrieve(); err != nil {
b.Fatal(err)
}
}
}

View File

@@ -1,111 +0,0 @@
package endpointcreds_test
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/credentials/endpointcreds"
"github.com/aws/aws-sdk-go/awstesting/unit"
"github.com/stretchr/testify/assert"
)
func TestRetrieveRefreshableCredentials(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, "/path/to/endpoint", r.URL.Path)
assert.Equal(t, "application/json", r.Header.Get("Accept"))
assert.Equal(t, "else", r.URL.Query().Get("something"))
encoder := json.NewEncoder(w)
err := encoder.Encode(map[string]interface{}{
"AccessKeyID": "AKID",
"SecretAccessKey": "SECRET",
"Token": "TOKEN",
"Expiration": time.Now().Add(1 * time.Hour),
})
if err != nil {
fmt.Println("failed to write out creds", err)
}
}))
client := endpointcreds.NewProviderClient(*unit.Session.Config,
unit.Session.Handlers,
server.URL+"/path/to/endpoint?something=else",
)
creds, err := client.Retrieve()
assert.NoError(t, err)
assert.Equal(t, "AKID", creds.AccessKeyID)
assert.Equal(t, "SECRET", creds.SecretAccessKey)
assert.Equal(t, "TOKEN", creds.SessionToken)
assert.False(t, client.IsExpired())
client.(*endpointcreds.Provider).CurrentTime = func() time.Time {
return time.Now().Add(2 * time.Hour)
}
assert.True(t, client.IsExpired())
}
func TestRetrieveStaticCredentials(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
encoder := json.NewEncoder(w)
err := encoder.Encode(map[string]interface{}{
"AccessKeyID": "AKID",
"SecretAccessKey": "SECRET",
})
if err != nil {
fmt.Println("failed to write out creds", err)
}
}))
client := endpointcreds.NewProviderClient(*unit.Session.Config, unit.Session.Handlers, server.URL)
creds, err := client.Retrieve()
assert.NoError(t, err)
assert.Equal(t, "AKID", creds.AccessKeyID)
assert.Equal(t, "SECRET", creds.SecretAccessKey)
assert.Empty(t, creds.SessionToken)
assert.False(t, client.IsExpired())
}
func TestFailedRetrieveCredentials(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(400)
encoder := json.NewEncoder(w)
err := encoder.Encode(map[string]interface{}{
"Code": "Error",
"Message": "Message",
})
if err != nil {
fmt.Println("failed to write error", err)
}
}))
client := endpointcreds.NewProviderClient(*unit.Session.Config, unit.Session.Handlers, server.URL)
creds, err := client.Retrieve()
assert.Error(t, err)
aerr := err.(awserr.Error)
assert.Equal(t, "CredentialsEndpointError", aerr.Code())
assert.Equal(t, "failed to load credentials", aerr.Message())
aerr = aerr.OrigErr().(awserr.Error)
assert.Equal(t, "Error", aerr.Code())
assert.Equal(t, "Message", aerr.Message())
assert.Empty(t, creds.AccessKeyID)
assert.Empty(t, creds.SecretAccessKey)
assert.Empty(t, creds.SessionToken)
assert.True(t, client.IsExpired())
}

View File

@@ -1,70 +0,0 @@
package credentials
import (
"github.com/stretchr/testify/assert"
"os"
"testing"
)
func TestEnvProviderRetrieve(t *testing.T) {
os.Clearenv()
os.Setenv("AWS_ACCESS_KEY_ID", "access")
os.Setenv("AWS_SECRET_ACCESS_KEY", "secret")
os.Setenv("AWS_SESSION_TOKEN", "token")
e := EnvProvider{}
creds, err := e.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "access", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "secret", creds.SecretAccessKey, "Expect secret access key to match")
assert.Equal(t, "token", creds.SessionToken, "Expect session token to match")
}
func TestEnvProviderIsExpired(t *testing.T) {
os.Clearenv()
os.Setenv("AWS_ACCESS_KEY_ID", "access")
os.Setenv("AWS_SECRET_ACCESS_KEY", "secret")
os.Setenv("AWS_SESSION_TOKEN", "token")
e := EnvProvider{}
assert.True(t, e.IsExpired(), "Expect creds to be expired before retrieve.")
_, err := e.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.False(t, e.IsExpired(), "Expect creds to not be expired after retrieve.")
}
func TestEnvProviderNoAccessKeyID(t *testing.T) {
os.Clearenv()
os.Setenv("AWS_SECRET_ACCESS_KEY", "secret")
e := EnvProvider{}
creds, err := e.Retrieve()
assert.Equal(t, ErrAccessKeyIDNotFound, err, "ErrAccessKeyIDNotFound expected, but was %#v error: %#v", creds, err)
}
func TestEnvProviderNoSecretAccessKey(t *testing.T) {
os.Clearenv()
os.Setenv("AWS_ACCESS_KEY_ID", "access")
e := EnvProvider{}
creds, err := e.Retrieve()
assert.Equal(t, ErrSecretAccessKeyNotFound, err, "ErrSecretAccessKeyNotFound expected, but was %#v error: %#v", creds, err)
}
func TestEnvProviderAlternateNames(t *testing.T) {
os.Clearenv()
os.Setenv("AWS_ACCESS_KEY", "access")
os.Setenv("AWS_SECRET_KEY", "secret")
e := EnvProvider{}
creds, err := e.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "access", creds.AccessKeyID, "Expected access key ID")
assert.Equal(t, "secret", creds.SecretAccessKey, "Expected secret access key")
assert.Empty(t, creds.SessionToken, "Expected no token")
}

View File

@@ -1,116 +0,0 @@
package credentials
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
)
func TestSharedCredentialsProvider(t *testing.T) {
os.Clearenv()
p := SharedCredentialsProvider{Filename: "example.ini", Profile: ""}
creds, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "accessKey", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "secret", creds.SecretAccessKey, "Expect secret access key to match")
assert.Equal(t, "token", creds.SessionToken, "Expect session token to match")
}
func TestSharedCredentialsProviderIsExpired(t *testing.T) {
os.Clearenv()
p := SharedCredentialsProvider{Filename: "example.ini", Profile: ""}
assert.True(t, p.IsExpired(), "Expect creds to be expired before retrieve")
_, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.False(t, p.IsExpired(), "Expect creds to not be expired after retrieve")
}
func TestSharedCredentialsProviderWithAWS_SHARED_CREDENTIALS_FILE(t *testing.T) {
os.Clearenv()
os.Setenv("AWS_SHARED_CREDENTIALS_FILE", "example.ini")
p := SharedCredentialsProvider{}
creds, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "accessKey", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "secret", creds.SecretAccessKey, "Expect secret access key to match")
assert.Equal(t, "token", creds.SessionToken, "Expect session token to match")
}
func TestSharedCredentialsProviderWithAWS_SHARED_CREDENTIALS_FILEAbsPath(t *testing.T) {
os.Clearenv()
wd, err := os.Getwd()
assert.NoError(t, err)
os.Setenv("AWS_SHARED_CREDENTIALS_FILE", filepath.Join(wd, "example.ini"))
p := SharedCredentialsProvider{}
creds, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "accessKey", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "secret", creds.SecretAccessKey, "Expect secret access key to match")
assert.Equal(t, "token", creds.SessionToken, "Expect session token to match")
}
func TestSharedCredentialsProviderWithAWS_PROFILE(t *testing.T) {
os.Clearenv()
os.Setenv("AWS_PROFILE", "no_token")
p := SharedCredentialsProvider{Filename: "example.ini", Profile: ""}
creds, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "accessKey", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "secret", creds.SecretAccessKey, "Expect secret access key to match")
assert.Empty(t, creds.SessionToken, "Expect no token")
}
func TestSharedCredentialsProviderWithoutTokenFromProfile(t *testing.T) {
os.Clearenv()
p := SharedCredentialsProvider{Filename: "example.ini", Profile: "no_token"}
creds, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "accessKey", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "secret", creds.SecretAccessKey, "Expect secret access key to match")
assert.Empty(t, creds.SessionToken, "Expect no token")
}
func TestSharedCredentialsProviderColonInCredFile(t *testing.T) {
os.Clearenv()
p := SharedCredentialsProvider{Filename: "example.ini", Profile: "with_colon"}
creds, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "accessKey", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "secret", creds.SecretAccessKey, "Expect secret access key to match")
assert.Empty(t, creds.SessionToken, "Expect no token")
}
func BenchmarkSharedCredentialsProvider(b *testing.B) {
os.Clearenv()
p := SharedCredentialsProvider{Filename: "example.ini", Profile: ""}
_, err := p.Retrieve()
if err != nil {
b.Fatal(err)
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
_, err := p.Retrieve()
if err != nil {
b.Fatal(err)
}
}
}

View File

@@ -1,34 +0,0 @@
package credentials
import (
"github.com/stretchr/testify/assert"
"testing"
)
func TestStaticProviderGet(t *testing.T) {
s := StaticProvider{
Value: Value{
AccessKeyID: "AKID",
SecretAccessKey: "SECRET",
SessionToken: "",
},
}
creds, err := s.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "AKID", creds.AccessKeyID, "Expect access key ID to match")
assert.Equal(t, "SECRET", creds.SecretAccessKey, "Expect secret access key to match")
assert.Empty(t, creds.SessionToken, "Expect no session token")
}
func TestStaticProviderIsExpired(t *testing.T) {
s := StaticProvider{
Value: Value{
AccessKeyID: "AKID",
SecretAccessKey: "SECRET",
SessionToken: "",
},
}
assert.False(t, s.IsExpired(), "Expect static credentials to never expire")
}

View File

@@ -1,56 +0,0 @@
package stscreds
import (
"testing"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/sts"
"github.com/stretchr/testify/assert"
)
type stubSTS struct {
}
func (s *stubSTS) AssumeRole(input *sts.AssumeRoleInput) (*sts.AssumeRoleOutput, error) {
expiry := time.Now().Add(60 * time.Minute)
return &sts.AssumeRoleOutput{
Credentials: &sts.Credentials{
// Just reflect the role arn to the provider.
AccessKeyId: input.RoleArn,
SecretAccessKey: aws.String("assumedSecretAccessKey"),
SessionToken: aws.String("assumedSessionToken"),
Expiration: &expiry,
},
}, nil
}
func TestAssumeRoleProvider(t *testing.T) {
stub := &stubSTS{}
p := &AssumeRoleProvider{
Client: stub,
RoleARN: "roleARN",
}
creds, err := p.Retrieve()
assert.Nil(t, err, "Expect no error")
assert.Equal(t, "roleARN", creds.AccessKeyID, "Expect access key ID to be reflected role ARN")
assert.Equal(t, "assumedSecretAccessKey", creds.SecretAccessKey, "Expect secret access key to match")
assert.Equal(t, "assumedSessionToken", creds.SessionToken, "Expect session token to match")
}
func BenchmarkAssumeRoleProvider(b *testing.B) {
stub := &stubSTS{}
p := &AssumeRoleProvider{
Client: stub,
RoleARN: "roleARN",
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
if _, err := p.Retrieve(); err != nil {
b.Fatal(err)
}
}
}

View File

@@ -72,6 +72,7 @@ func Handlers() request.Handlers {
handlers.Build.PushBackNamed(corehandlers.SDKVersionUserAgentHandler)
handlers.Build.AfterEachFn = request.HandlerListStopOnError
handlers.Sign.PushBackNamed(corehandlers.BuildContentLengthHandler)
handlers.Send.PushBackNamed(corehandlers.ValidateReqSigHandler)
handlers.Send.PushBackNamed(corehandlers.SendHandler)
handlers.AfterRetry.PushBackNamed(corehandlers.AfterRetryHandler)
handlers.ValidateResponse.PushBackNamed(corehandlers.ValidateResponseHandler)

View File

@@ -1,39 +0,0 @@
package defaults
import (
"fmt"
"os"
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds"
"github.com/aws/aws-sdk-go/aws/credentials/endpointcreds"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/stretchr/testify/assert"
)
func TestECSCredProvider(t *testing.T) {
defer os.Clearenv()
os.Setenv("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI", "/abc/123")
provider := RemoteCredProvider(aws.Config{}, request.Handlers{})
assert.NotNil(t, provider)
ecsProvider, ok := provider.(*endpointcreds.Provider)
assert.NotNil(t, ecsProvider)
assert.True(t, ok)
assert.Equal(t, fmt.Sprintf("http://169.254.170.2/abc/123"),
ecsProvider.Client.Endpoint)
}
func TestDefaultEC2RoleProvider(t *testing.T) {
provider := RemoteCredProvider(aws.Config{}, request.Handlers{})
assert.NotNil(t, provider)
ec2Provider, ok := provider.(*ec2rolecreds.EC2RoleProvider)
assert.NotNil(t, ec2Provider)
assert.True(t, ok)
}

View File

@@ -3,6 +3,7 @@ package ec2metadata
import (
"encoding/json"
"fmt"
"net/http"
"path"
"strings"
"time"
@@ -27,6 +28,27 @@ func (c *EC2Metadata) GetMetadata(p string) (string, error) {
return output.Content, req.Send()
}
// GetUserData returns the userdata that was configured for the service. If
// there is no user-data setup for the EC2 instance a "NotFoundError" error
// code will be returned.
func (c *EC2Metadata) GetUserData() (string, error) {
op := &request.Operation{
Name: "GetUserData",
HTTPMethod: "GET",
HTTPPath: path.Join("/", "user-data"),
}
output := &metadataOutput{}
req := c.NewRequest(op, nil, output)
req.Handlers.UnmarshalError.PushBack(func(r *request.Request) {
if r.HTTPResponse.StatusCode == http.StatusNotFound {
r.Error = awserr.New("NotFoundError", "user-data not found", r.Error)
}
})
return output.Content, req.Send()
}
// GetDynamicData uses the path provided to request information from the EC2
// instance metadata service for dynamic data. The content will be returned
// as a string, or error if the request failed.
@@ -111,7 +133,7 @@ func (c *EC2Metadata) Available() bool {
return true
}
// An EC2IAMInfo provides the shape for unmarshalling
// An EC2IAMInfo provides the shape for unmarshaling
// an IAM info from the metadata API
type EC2IAMInfo struct {
Code string
@@ -120,7 +142,7 @@ type EC2IAMInfo struct {
InstanceProfileID string
}
// An EC2InstanceIdentityDocument provides the shape for unmarshalling
// An EC2InstanceIdentityDocument provides the shape for unmarshaling
// an instance identity document
type EC2InstanceIdentityDocument struct {
DevpayProductCodes []string `json:"devpayProductCodes"`

View File

@@ -1,195 +0,0 @@
package ec2metadata_test
import (
"bytes"
"io/ioutil"
"net/http"
"net/http/httptest"
"path"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/awstesting/unit"
)
const instanceIdentityDocument = `{
"devpayProductCodes" : null,
"availabilityZone" : "us-east-1d",
"privateIp" : "10.158.112.84",
"version" : "2010-08-31",
"region" : "us-east-1",
"instanceId" : "i-1234567890abcdef0",
"billingProducts" : null,
"instanceType" : "t1.micro",
"accountId" : "123456789012",
"pendingTime" : "2015-11-19T16:32:11Z",
"imageId" : "ami-5fb8c835",
"kernelId" : "aki-919dcaf8",
"ramdiskId" : null,
"architecture" : "x86_64"
}`
const validIamInfo = `{
"Code" : "Success",
"LastUpdated" : "2016-03-17T12:27:32Z",
"InstanceProfileArn" : "arn:aws:iam::123456789012:instance-profile/my-instance-profile",
"InstanceProfileId" : "AIPAABCDEFGHIJKLMN123"
}`
const unsuccessfulIamInfo = `{
"Code" : "Failed",
"LastUpdated" : "2016-03-17T12:27:32Z",
"InstanceProfileArn" : "arn:aws:iam::123456789012:instance-profile/my-instance-profile",
"InstanceProfileId" : "AIPAABCDEFGHIJKLMN123"
}`
func initTestServer(path string, resp string) *httptest.Server {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.RequestURI != path {
http.Error(w, "not found", http.StatusNotFound)
return
}
w.Write([]byte(resp))
}))
}
func TestEndpoint(t *testing.T) {
c := ec2metadata.New(unit.Session)
op := &request.Operation{
Name: "GetMetadata",
HTTPMethod: "GET",
HTTPPath: path.Join("/", "meta-data", "testpath"),
}
req := c.NewRequest(op, nil, nil)
assert.Equal(t, "http://169.254.169.254/latest", req.ClientInfo.Endpoint)
assert.Equal(t, "http://169.254.169.254/latest/meta-data/testpath", req.HTTPRequest.URL.String())
}
func TestGetMetadata(t *testing.T) {
server := initTestServer(
"/latest/meta-data/some/path",
"success", // real response includes suffix
)
defer server.Close()
c := ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")})
resp, err := c.GetMetadata("some/path")
assert.NoError(t, err)
assert.Equal(t, "success", resp)
}
func TestGetRegion(t *testing.T) {
server := initTestServer(
"/latest/meta-data/placement/availability-zone",
"us-west-2a", // real response includes suffix
)
defer server.Close()
c := ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")})
region, err := c.Region()
assert.NoError(t, err)
assert.Equal(t, "us-west-2", region)
}
func TestMetadataAvailable(t *testing.T) {
server := initTestServer(
"/latest/meta-data/instance-id",
"instance-id",
)
defer server.Close()
c := ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")})
available := c.Available()
assert.True(t, available)
}
func TestMetadataIAMInfo_success(t *testing.T) {
server := initTestServer(
"/latest/meta-data/iam/info",
validIamInfo,
)
defer server.Close()
c := ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")})
iamInfo, err := c.IAMInfo()
assert.NoError(t, err)
assert.Equal(t, "Success", iamInfo.Code)
assert.Equal(t, "arn:aws:iam::123456789012:instance-profile/my-instance-profile", iamInfo.InstanceProfileArn)
assert.Equal(t, "AIPAABCDEFGHIJKLMN123", iamInfo.InstanceProfileID)
}
func TestMetadataIAMInfo_failure(t *testing.T) {
server := initTestServer(
"/latest/meta-data/iam/info",
unsuccessfulIamInfo,
)
defer server.Close()
c := ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")})
iamInfo, err := c.IAMInfo()
assert.NotNil(t, err)
assert.Equal(t, "", iamInfo.Code)
assert.Equal(t, "", iamInfo.InstanceProfileArn)
assert.Equal(t, "", iamInfo.InstanceProfileID)
}
func TestMetadataNotAvailable(t *testing.T) {
c := ec2metadata.New(unit.Session)
c.Handlers.Send.Clear()
c.Handlers.Send.PushBack(func(r *request.Request) {
r.HTTPResponse = &http.Response{
StatusCode: int(0),
Status: http.StatusText(int(0)),
Body: ioutil.NopCloser(bytes.NewReader([]byte{})),
}
r.Error = awserr.New("RequestError", "send request failed", nil)
r.Retryable = aws.Bool(true) // network errors are retryable
})
available := c.Available()
assert.False(t, available)
}
func TestMetadataErrorResponse(t *testing.T) {
c := ec2metadata.New(unit.Session)
c.Handlers.Send.Clear()
c.Handlers.Send.PushBack(func(r *request.Request) {
r.HTTPResponse = &http.Response{
StatusCode: http.StatusBadRequest,
Status: http.StatusText(http.StatusBadRequest),
Body: ioutil.NopCloser(strings.NewReader("error message text")),
}
r.Retryable = aws.Bool(false) // network errors are retryable
})
data, err := c.GetMetadata("uri/path")
assert.Empty(t, data)
assert.Contains(t, err.Error(), "error message text")
}
func TestEC2RoleProviderInstanceIdentity(t *testing.T) {
server := initTestServer(
"/latest/dynamic/instance-identity/document",
instanceIdentityDocument,
)
defer server.Close()
c := ec2metadata.New(unit.Session, &aws.Config{Endpoint: aws.String(server.URL + "/latest")})
doc, err := c.GetInstanceIdentityDocument()
assert.Nil(t, err, "Expect no error, %v", err)
assert.Equal(t, doc.AccountID, "123456789012")
assert.Equal(t, doc.AvailabilityZone, "us-east-1d")
assert.Equal(t, doc.Region, "us-east-1")
}

View File

@@ -1,78 +0,0 @@
package ec2metadata_test
import (
"net/http"
"net/http/httptest"
"sync"
"testing"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/awstesting/unit"
"github.com/stretchr/testify/assert"
)
func TestClientOverrideDefaultHTTPClientTimeout(t *testing.T) {
svc := ec2metadata.New(unit.Session)
assert.NotEqual(t, http.DefaultClient, svc.Config.HTTPClient)
assert.Equal(t, 5*time.Second, svc.Config.HTTPClient.Timeout)
}
func TestClientNotOverrideDefaultHTTPClientTimeout(t *testing.T) {
http.DefaultClient.Transport = &http.Transport{}
defer func() {
http.DefaultClient.Transport = nil
}()
svc := ec2metadata.New(unit.Session)
assert.Equal(t, http.DefaultClient, svc.Config.HTTPClient)
tr, ok := svc.Config.HTTPClient.Transport.(*http.Transport)
assert.True(t, ok)
assert.NotNil(t, tr)
assert.Nil(t, tr.Dial)
}
func TestClientDisableOverrideDefaultHTTPClientTimeout(t *testing.T) {
svc := ec2metadata.New(unit.Session, aws.NewConfig().WithEC2MetadataDisableTimeoutOverride(true))
assert.Equal(t, http.DefaultClient, svc.Config.HTTPClient)
}
func TestClientOverrideDefaultHTTPClientTimeoutRace(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("us-east-1a"))
}))
cfg := aws.NewConfig().WithEndpoint(server.URL)
runEC2MetadataClients(t, cfg, 100)
}
func TestClientOverrideDefaultHTTPClientTimeoutRaceWithTransport(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("us-east-1a"))
}))
cfg := aws.NewConfig().WithEndpoint(server.URL).WithHTTPClient(&http.Client{
Transport: http.DefaultTransport,
})
runEC2MetadataClients(t, cfg, 100)
}
func runEC2MetadataClients(t *testing.T, cfg *aws.Config, atOnce int) {
var wg sync.WaitGroup
wg.Add(atOnce)
for i := 0; i < atOnce; i++ {
go func() {
svc := ec2metadata.New(unit.Session, cfg)
_, err := svc.Region()
assert.NoError(t, err)
wg.Done()
}()
}
wg.Wait()
}

View File

@@ -1,87 +0,0 @@
package request_test
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/request"
)
func TestHandlerList(t *testing.T) {
s := ""
r := &request.Request{}
l := request.HandlerList{}
l.PushBack(func(r *request.Request) {
s += "a"
r.Data = s
})
l.Run(r)
assert.Equal(t, "a", s)
assert.Equal(t, "a", r.Data)
}
func TestMultipleHandlers(t *testing.T) {
r := &request.Request{}
l := request.HandlerList{}
l.PushBack(func(r *request.Request) { r.Data = nil })
l.PushFront(func(r *request.Request) { r.Data = aws.Bool(true) })
l.Run(r)
if r.Data != nil {
t.Error("Expected handler to execute")
}
}
func TestNamedHandlers(t *testing.T) {
l := request.HandlerList{}
named := request.NamedHandler{Name: "Name", Fn: func(r *request.Request) {}}
named2 := request.NamedHandler{Name: "NotName", Fn: func(r *request.Request) {}}
l.PushBackNamed(named)
l.PushBackNamed(named)
l.PushBackNamed(named2)
l.PushBack(func(r *request.Request) {})
assert.Equal(t, 4, l.Len())
l.Remove(named)
assert.Equal(t, 2, l.Len())
}
func TestLoggedHandlers(t *testing.T) {
expectedHandlers := []string{"name1", "name2"}
l := request.HandlerList{}
loggedHandlers := []string{}
l.AfterEachFn = request.HandlerListLogItem
cfg := aws.Config{Logger: aws.LoggerFunc(func(args ...interface{}) {
loggedHandlers = append(loggedHandlers, args[2].(string))
})}
named1 := request.NamedHandler{Name: "name1", Fn: func(r *request.Request) {}}
named2 := request.NamedHandler{Name: "name2", Fn: func(r *request.Request) {}}
l.PushBackNamed(named1)
l.PushBackNamed(named2)
l.Run(&request.Request{Config: cfg})
assert.Equal(t, expectedHandlers, loggedHandlers)
}
func TestStopHandlers(t *testing.T) {
l := request.HandlerList{}
stopAt := 1
l.AfterEachFn = func(item request.HandlerListRunItem) bool {
return item.Index != stopAt
}
called := 0
l.PushBackNamed(request.NamedHandler{Name: "name1", Fn: func(r *request.Request) {
called++
}})
l.PushBackNamed(request.NamedHandler{Name: "name2", Fn: func(r *request.Request) {
called++
}})
l.PushBackNamed(request.NamedHandler{Name: "name3", Fn: func(r *request.Request) {
assert.Fail(t, "third handler should not be called")
}})
l.Run(&request.Request{})
assert.Equal(t, 2, called, "Expect only two handlers to be called")
}

View File

@@ -1,5 +1,3 @@
// +build go1.5
package request
import (
@@ -9,20 +7,13 @@ import (
)
func copyHTTPRequest(r *http.Request, body io.ReadCloser) *http.Request {
req := &http.Request{
URL: &url.URL{},
Header: http.Header{},
Close: r.Close,
Body: body,
Host: r.Host,
Method: r.Method,
Proto: r.Proto,
ContentLength: r.ContentLength,
// Cancel will be deprecated in 1.7 and will be replaced with Context
Cancel: r.Cancel,
}
req := new(http.Request)
*req = *r
req.URL = &url.URL{}
*req.URL = *r.URL
req.Body = body
req.Header = http.Header{}
for k, v := range r.Header {
for _, vv := range v {
req.Header.Add(k, vv)

View File

@@ -1,31 +0,0 @@
// +build !go1.5
package request
import (
"io"
"net/http"
"net/url"
)
func copyHTTPRequest(r *http.Request, body io.ReadCloser) *http.Request {
req := &http.Request{
URL: &url.URL{},
Header: http.Header{},
Close: r.Close,
Body: body,
Host: r.Host,
Method: r.Method,
Proto: r.Proto,
ContentLength: r.ContentLength,
}
*req.URL = *r.URL
for k, v := range r.Header {
for _, vv := range v {
req.Header.Add(k, vv)
}
}
return req
}

View File

@@ -1,34 +0,0 @@
package request
import (
"bytes"
"io/ioutil"
"net/http"
"net/url"
"sync"
"testing"
)
func TestRequestCopyRace(t *testing.T) {
origReq := &http.Request{URL: &url.URL{}, Header: http.Header{}}
origReq.Header.Set("Header", "OrigValue")
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func() {
req := copyHTTPRequest(origReq, ioutil.NopCloser(&bytes.Buffer{}))
req.Header.Set("Header", "Value")
go func() {
req2 := copyHTTPRequest(req, ioutil.NopCloser(&bytes.Buffer{}))
req2.Header.Add("Header", "Value2")
}()
_ = req.Header.Get("Header")
wg.Done()
}()
_ = origReq.Header.Get("Header")
}
origReq.Header.Get("Header")
wg.Wait()
}

View File

@@ -1,37 +0,0 @@
// +build go1.5
package request_test
import (
"errors"
"strings"
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/awstesting/mock"
"github.com/stretchr/testify/assert"
)
func TestRequestCancelRetry(t *testing.T) {
c := make(chan struct{})
reqNum := 0
s := mock.NewMockClient(aws.NewConfig().WithMaxRetries(10))
s.Handlers.Validate.Clear()
s.Handlers.Unmarshal.Clear()
s.Handlers.UnmarshalMeta.Clear()
s.Handlers.UnmarshalError.Clear()
s.Handlers.Send.PushFront(func(r *request.Request) {
reqNum++
r.Error = errors.New("net/http: canceled")
})
out := &testData{}
r := s.NewRequest(&request.Operation{Name: "Operation"}, nil, out)
r.HTTPRequest.Cancel = c
close(c)
err := r.Send()
assert.True(t, strings.Contains(err.Error(), "canceled"))
assert.Equal(t, 1, reqNum)
}

View File

@@ -9,7 +9,7 @@ import (
// with retrying requests
type offsetReader struct {
buf io.ReadSeeker
lock sync.RWMutex
lock sync.Mutex
closed bool
}
@@ -21,7 +21,8 @@ func newOffsetReader(buf io.ReadSeeker, offset int64) *offsetReader {
return reader
}
// Close is a thread-safe close. Uses the write lock.
// Close will close the instance of the offset reader's access to
// the underlying io.ReadSeeker.
func (o *offsetReader) Close() error {
o.lock.Lock()
defer o.lock.Unlock()
@@ -29,10 +30,10 @@ func (o *offsetReader) Close() error {
return nil
}
// Read is a thread-safe read using a read lock.
// Read is a thread-safe read of the underlying io.ReadSeeker
func (o *offsetReader) Read(p []byte) (int, error) {
o.lock.RLock()
defer o.lock.RUnlock()
o.lock.Lock()
defer o.lock.Unlock()
if o.closed {
return 0, io.EOF
@@ -41,6 +42,14 @@ func (o *offsetReader) Read(p []byte) (int, error) {
return o.buf.Read(p)
}
// Seek is a thread-safe seeking operation.
func (o *offsetReader) Seek(offset int64, whence int) (int64, error) {
o.lock.Lock()
defer o.lock.Unlock()
return o.buf.Seek(offset, whence)
}
// CloseAndCopy will return a new offsetReader with a copy of the old buffer
// and close the old buffer.
func (o *offsetReader) CloseAndCopy(offset int64) *offsetReader {

View File

@@ -1,122 +0,0 @@
package request
import (
"bytes"
"io"
"math/rand"
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestOffsetReaderRead(t *testing.T) {
buf := []byte("testData")
reader := &offsetReader{buf: bytes.NewReader(buf)}
tempBuf := make([]byte, len(buf))
n, err := reader.Read(tempBuf)
assert.Equal(t, n, len(buf))
assert.Nil(t, err)
assert.Equal(t, buf, tempBuf)
}
func TestOffsetReaderClose(t *testing.T) {
buf := []byte("testData")
reader := &offsetReader{buf: bytes.NewReader(buf)}
err := reader.Close()
assert.Nil(t, err)
tempBuf := make([]byte, len(buf))
n, err := reader.Read(tempBuf)
assert.Equal(t, n, 0)
assert.Equal(t, err, io.EOF)
}
func TestOffsetReaderCloseAndCopy(t *testing.T) {
buf := []byte("testData")
tempBuf := make([]byte, len(buf))
reader := &offsetReader{buf: bytes.NewReader(buf)}
newReader := reader.CloseAndCopy(0)
n, err := reader.Read(tempBuf)
assert.Equal(t, n, 0)
assert.Equal(t, err, io.EOF)
n, err = newReader.Read(tempBuf)
assert.Equal(t, n, len(buf))
assert.Nil(t, err)
assert.Equal(t, buf, tempBuf)
}
func TestOffsetReaderCloseAndCopyOffset(t *testing.T) {
buf := []byte("testData")
tempBuf := make([]byte, len(buf))
reader := &offsetReader{buf: bytes.NewReader(buf)}
newReader := reader.CloseAndCopy(4)
n, err := newReader.Read(tempBuf)
assert.Equal(t, n, len(buf)-4)
assert.Nil(t, err)
expected := []byte{'D', 'a', 't', 'a', 0, 0, 0, 0}
assert.Equal(t, expected, tempBuf)
}
func TestOffsetReaderRace(t *testing.T) {
wg := sync.WaitGroup{}
f := func(reader *offsetReader) {
defer wg.Done()
var err error
buf := make([]byte, 1)
_, err = reader.Read(buf)
for err != io.EOF {
_, err = reader.Read(buf)
}
}
closeFn := func(reader *offsetReader) {
defer wg.Done()
time.Sleep(time.Duration(rand.Intn(20)+1) * time.Millisecond)
reader.Close()
}
for i := 0; i < 50; i++ {
reader := &offsetReader{buf: bytes.NewReader(make([]byte, 1024*1024))}
wg.Add(1)
go f(reader)
wg.Add(1)
go closeFn(reader)
}
wg.Wait()
}
func BenchmarkOffsetReader(b *testing.B) {
bufSize := 1024 * 1024 * 100
buf := make([]byte, bufSize)
reader := &offsetReader{buf: bytes.NewReader(buf)}
tempBuf := make([]byte, 1024)
for i := 0; i < b.N; i++ {
reader.Read(tempBuf)
}
}
func BenchmarkBytesReader(b *testing.B) {
bufSize := 1024 * 1024 * 100
buf := make([]byte, bufSize)
reader := bytes.NewReader(buf)
tempBuf := make([]byte, 1024)
for i := 0; i < b.N; i++ {
reader.Read(tempBuf)
}
}

View File

@@ -4,7 +4,6 @@ import (
"bytes"
"fmt"
"io"
"io/ioutil"
"net/http"
"net/url"
"reflect"
@@ -42,6 +41,12 @@ type Request struct {
LastSignedAt time.Time
built bool
// Need to persist an intermideant body betweend the input Body and HTTP
// request body because the HTTP Client's transport can maintain a reference
// to the HTTP request's body after the client has returned. This value is
// safe to use concurrently and rewraps the input Body for each HTTP request.
safeBody *offsetReader
}
// An Operation is the service API operation to be made.
@@ -135,8 +140,8 @@ func (r *Request) SetStringBody(s string) {
// SetReaderBody will set the request's body reader.
func (r *Request) SetReaderBody(reader io.ReadSeeker) {
r.HTTPRequest.Body = newOffsetReader(reader, 0)
r.Body = reader
r.ResetBody()
}
// Presign returns the request's signed URL. Error will be returned
@@ -220,6 +225,24 @@ func (r *Request) Sign() error {
return r.Error
}
// ResetBody rewinds the request body backto its starting position, and
// set's the HTTP Request body reference. When the body is read prior
// to being sent in the HTTP request it will need to be rewound.
func (r *Request) ResetBody() {
if r.safeBody != nil {
r.safeBody.Close()
}
r.safeBody = newOffsetReader(r.Body, r.BodyStart)
r.HTTPRequest.Body = r.safeBody
}
// GetBody will return an io.ReadSeeker of the Request's underlying
// input body with a concurrency safe wrapper.
func (r *Request) GetBody() io.ReadSeeker {
return r.safeBody
}
// Send will send the request returning error if errors are encountered.
//
// Send will sign the request prior to sending. All Send Handlers will
@@ -231,6 +254,8 @@ func (r *Request) Sign() error {
//
// readLoop() and getConn(req *Request, cm connectMethod)
// https://github.com/golang/go/blob/master/src/net/http/transport.go
//
// Send will not close the request.Request's body.
func (r *Request) Send() error {
for {
if aws.BoolValue(r.Retryable) {
@@ -239,21 +264,15 @@ func (r *Request) Send() error {
r.ClientInfo.ServiceName, r.Operation.Name, r.RetryCount))
}
var body io.ReadCloser
if reader, ok := r.HTTPRequest.Body.(*offsetReader); ok {
body = reader.CloseAndCopy(r.BodyStart)
} else {
if r.Config.Logger != nil {
r.Config.Logger.Log("Request body type has been overwritten. May cause race conditions")
}
r.Body.Seek(r.BodyStart, 0)
body = ioutil.NopCloser(r.Body)
}
// The previous http.Request will have a reference to the r.Body
// and the HTTP Client's Transport may still be reading from
// the request's body even though the Client's Do returned.
r.HTTPRequest = copyHTTPRequest(r.HTTPRequest, nil)
r.ResetBody()
r.HTTPRequest = copyHTTPRequest(r.HTTPRequest, body)
// Closing response body to ensure that no response body is leaked
// between retry attempts.
if r.HTTPResponse != nil && r.HTTPResponse.Body != nil {
// Closing response body. Since we are setting a new request to send off, this
// response will get squashed and leaked.
r.HTTPResponse.Body.Close()
}
}
@@ -281,7 +300,6 @@ func (r *Request) Send() error {
debugLogReqError(r, "Send Request", true, err)
continue
}
r.Handlers.UnmarshalMeta.Run(r)
r.Handlers.ValidateResponse.Run(r)
if r.Error != nil {

View File

@@ -1,33 +0,0 @@
// +build go1.6
package request_test
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/client"
"github.com/aws/aws-sdk-go/aws/client/metadata"
"github.com/aws/aws-sdk-go/aws/defaults"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/private/endpoints"
)
// go version 1.4 and 1.5 do not return an error. Version 1.5 will url encode
// the uri while 1.4 will not
func TestRequestInvalidEndpoint(t *testing.T) {
endpoint, _ := endpoints.NormalizeEndpoint("localhost:80 ", "test-service", "test-region", false, false)
r := request.New(
aws.Config{},
metadata.ClientInfo{Endpoint: endpoint},
defaults.Handlers(),
client.DefaultRetryer{},
&request.Operation{},
nil,
nil,
)
assert.Error(t, r.Error)
}

View File

@@ -1,455 +0,0 @@
package request_test
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/awstesting/unit"
"github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go/service/route53"
"github.com/aws/aws-sdk-go/service/s3"
)
// Use DynamoDB methods for simplicity
func TestPaginationQueryPage(t *testing.T) {
db := dynamodb.New(unit.Session)
tokens, pages, numPages, gotToEnd := []map[string]*dynamodb.AttributeValue{}, []map[string]*dynamodb.AttributeValue{}, 0, false
reqNum := 0
resps := []*dynamodb.QueryOutput{
{
LastEvaluatedKey: map[string]*dynamodb.AttributeValue{"key": {S: aws.String("key1")}},
Count: aws.Int64(1),
Items: []map[string]*dynamodb.AttributeValue{
{
"key": {S: aws.String("key1")},
},
},
},
{
LastEvaluatedKey: map[string]*dynamodb.AttributeValue{"key": {S: aws.String("key2")}},
Count: aws.Int64(1),
Items: []map[string]*dynamodb.AttributeValue{
{
"key": {S: aws.String("key2")},
},
},
},
{
LastEvaluatedKey: map[string]*dynamodb.AttributeValue{},
Count: aws.Int64(1),
Items: []map[string]*dynamodb.AttributeValue{
{
"key": {S: aws.String("key3")},
},
},
},
}
db.Handlers.Send.Clear() // mock sending
db.Handlers.Unmarshal.Clear()
db.Handlers.UnmarshalMeta.Clear()
db.Handlers.ValidateResponse.Clear()
db.Handlers.Build.PushBack(func(r *request.Request) {
in := r.Params.(*dynamodb.QueryInput)
if in == nil {
tokens = append(tokens, nil)
} else if len(in.ExclusiveStartKey) != 0 {
tokens = append(tokens, in.ExclusiveStartKey)
}
})
db.Handlers.Unmarshal.PushBack(func(r *request.Request) {
r.Data = resps[reqNum]
reqNum++
})
params := &dynamodb.QueryInput{
Limit: aws.Int64(2),
TableName: aws.String("tablename"),
}
err := db.QueryPages(params, func(p *dynamodb.QueryOutput, last bool) bool {
numPages++
for _, item := range p.Items {
pages = append(pages, item)
}
if last {
if gotToEnd {
assert.Fail(t, "last=true happened twice")
}
gotToEnd = true
}
return true
})
assert.Nil(t, err)
assert.Equal(t,
[]map[string]*dynamodb.AttributeValue{
{"key": {S: aws.String("key1")}},
{"key": {S: aws.String("key2")}},
}, tokens)
assert.Equal(t,
[]map[string]*dynamodb.AttributeValue{
{"key": {S: aws.String("key1")}},
{"key": {S: aws.String("key2")}},
{"key": {S: aws.String("key3")}},
}, pages)
assert.Equal(t, 3, numPages)
assert.True(t, gotToEnd)
assert.Nil(t, params.ExclusiveStartKey)
}
// Use DynamoDB methods for simplicity
func TestPagination(t *testing.T) {
db := dynamodb.New(unit.Session)
tokens, pages, numPages, gotToEnd := []string{}, []string{}, 0, false
reqNum := 0
resps := []*dynamodb.ListTablesOutput{
{TableNames: []*string{aws.String("Table1"), aws.String("Table2")}, LastEvaluatedTableName: aws.String("Table2")},
{TableNames: []*string{aws.String("Table3"), aws.String("Table4")}, LastEvaluatedTableName: aws.String("Table4")},
{TableNames: []*string{aws.String("Table5")}},
}
db.Handlers.Send.Clear() // mock sending
db.Handlers.Unmarshal.Clear()
db.Handlers.UnmarshalMeta.Clear()
db.Handlers.ValidateResponse.Clear()
db.Handlers.Build.PushBack(func(r *request.Request) {
in := r.Params.(*dynamodb.ListTablesInput)
if in == nil {
tokens = append(tokens, "")
} else if in.ExclusiveStartTableName != nil {
tokens = append(tokens, *in.ExclusiveStartTableName)
}
})
db.Handlers.Unmarshal.PushBack(func(r *request.Request) {
r.Data = resps[reqNum]
reqNum++
})
params := &dynamodb.ListTablesInput{Limit: aws.Int64(2)}
err := db.ListTablesPages(params, func(p *dynamodb.ListTablesOutput, last bool) bool {
numPages++
for _, t := range p.TableNames {
pages = append(pages, *t)
}
if last {
if gotToEnd {
assert.Fail(t, "last=true happened twice")
}
gotToEnd = true
}
return true
})
assert.Equal(t, []string{"Table2", "Table4"}, tokens)
assert.Equal(t, []string{"Table1", "Table2", "Table3", "Table4", "Table5"}, pages)
assert.Equal(t, 3, numPages)
assert.True(t, gotToEnd)
assert.Nil(t, err)
assert.Nil(t, params.ExclusiveStartTableName)
}
// Use DynamoDB methods for simplicity
func TestPaginationEachPage(t *testing.T) {
db := dynamodb.New(unit.Session)
tokens, pages, numPages, gotToEnd := []string{}, []string{}, 0, false
reqNum := 0
resps := []*dynamodb.ListTablesOutput{
{TableNames: []*string{aws.String("Table1"), aws.String("Table2")}, LastEvaluatedTableName: aws.String("Table2")},
{TableNames: []*string{aws.String("Table3"), aws.String("Table4")}, LastEvaluatedTableName: aws.String("Table4")},
{TableNames: []*string{aws.String("Table5")}},
}
db.Handlers.Send.Clear() // mock sending
db.Handlers.Unmarshal.Clear()
db.Handlers.UnmarshalMeta.Clear()
db.Handlers.ValidateResponse.Clear()
db.Handlers.Build.PushBack(func(r *request.Request) {
in := r.Params.(*dynamodb.ListTablesInput)
if in == nil {
tokens = append(tokens, "")
} else if in.ExclusiveStartTableName != nil {
tokens = append(tokens, *in.ExclusiveStartTableName)
}
})
db.Handlers.Unmarshal.PushBack(func(r *request.Request) {
r.Data = resps[reqNum]
reqNum++
})
params := &dynamodb.ListTablesInput{Limit: aws.Int64(2)}
req, _ := db.ListTablesRequest(params)
err := req.EachPage(func(p interface{}, last bool) bool {
numPages++
for _, t := range p.(*dynamodb.ListTablesOutput).TableNames {
pages = append(pages, *t)
}
if last {
if gotToEnd {
assert.Fail(t, "last=true happened twice")
}
gotToEnd = true
}
return true
})
assert.Equal(t, []string{"Table2", "Table4"}, tokens)
assert.Equal(t, []string{"Table1", "Table2", "Table3", "Table4", "Table5"}, pages)
assert.Equal(t, 3, numPages)
assert.True(t, gotToEnd)
assert.Nil(t, err)
}
// Use DynamoDB methods for simplicity
func TestPaginationEarlyExit(t *testing.T) {
db := dynamodb.New(unit.Session)
numPages, gotToEnd := 0, false
reqNum := 0
resps := []*dynamodb.ListTablesOutput{
{TableNames: []*string{aws.String("Table1"), aws.String("Table2")}, LastEvaluatedTableName: aws.String("Table2")},
{TableNames: []*string{aws.String("Table3"), aws.String("Table4")}, LastEvaluatedTableName: aws.String("Table4")},
{TableNames: []*string{aws.String("Table5")}},
}
db.Handlers.Send.Clear() // mock sending
db.Handlers.Unmarshal.Clear()
db.Handlers.UnmarshalMeta.Clear()
db.Handlers.ValidateResponse.Clear()
db.Handlers.Unmarshal.PushBack(func(r *request.Request) {
r.Data = resps[reqNum]
reqNum++
})
params := &dynamodb.ListTablesInput{Limit: aws.Int64(2)}
err := db.ListTablesPages(params, func(p *dynamodb.ListTablesOutput, last bool) bool {
numPages++
if numPages == 2 {
return false
}
if last {
if gotToEnd {
assert.Fail(t, "last=true happened twice")
}
gotToEnd = true
}
return true
})
assert.Equal(t, 2, numPages)
assert.False(t, gotToEnd)
assert.Nil(t, err)
}
func TestSkipPagination(t *testing.T) {
client := s3.New(unit.Session)
client.Handlers.Send.Clear() // mock sending
client.Handlers.Unmarshal.Clear()
client.Handlers.UnmarshalMeta.Clear()
client.Handlers.ValidateResponse.Clear()
client.Handlers.Unmarshal.PushBack(func(r *request.Request) {
r.Data = &s3.HeadBucketOutput{}
})
req, _ := client.HeadBucketRequest(&s3.HeadBucketInput{Bucket: aws.String("bucket")})
numPages, gotToEnd := 0, false
req.EachPage(func(p interface{}, last bool) bool {
numPages++
if last {
gotToEnd = true
}
return true
})
assert.Equal(t, 1, numPages)
assert.True(t, gotToEnd)
}
// Use S3 for simplicity
func TestPaginationTruncation(t *testing.T) {
client := s3.New(unit.Session)
reqNum := 0
resps := []*s3.ListObjectsOutput{
{IsTruncated: aws.Bool(true), Contents: []*s3.Object{{Key: aws.String("Key1")}}},
{IsTruncated: aws.Bool(true), Contents: []*s3.Object{{Key: aws.String("Key2")}}},
{IsTruncated: aws.Bool(false), Contents: []*s3.Object{{Key: aws.String("Key3")}}},
{IsTruncated: aws.Bool(true), Contents: []*s3.Object{{Key: aws.String("Key4")}}},
}
client.Handlers.Send.Clear() // mock sending
client.Handlers.Unmarshal.Clear()
client.Handlers.UnmarshalMeta.Clear()
client.Handlers.ValidateResponse.Clear()
client.Handlers.Unmarshal.PushBack(func(r *request.Request) {
r.Data = resps[reqNum]
reqNum++
})
params := &s3.ListObjectsInput{Bucket: aws.String("bucket")}
results := []string{}
err := client.ListObjectsPages(params, func(p *s3.ListObjectsOutput, last bool) bool {
results = append(results, *p.Contents[0].Key)
return true
})
assert.Equal(t, []string{"Key1", "Key2", "Key3"}, results)
assert.Nil(t, err)
// Try again without truncation token at all
reqNum = 0
resps[1].IsTruncated = nil
resps[2].IsTruncated = aws.Bool(true)
results = []string{}
err = client.ListObjectsPages(params, func(p *s3.ListObjectsOutput, last bool) bool {
results = append(results, *p.Contents[0].Key)
return true
})
assert.Equal(t, []string{"Key1", "Key2"}, results)
assert.Nil(t, err)
}
func TestPaginationNilToken(t *testing.T) {
client := route53.New(unit.Session)
reqNum := 0
resps := []*route53.ListResourceRecordSetsOutput{
{
ResourceRecordSets: []*route53.ResourceRecordSet{
{Name: aws.String("first.example.com.")},
},
IsTruncated: aws.Bool(true),
NextRecordName: aws.String("second.example.com."),
NextRecordType: aws.String("MX"),
NextRecordIdentifier: aws.String("second"),
MaxItems: aws.String("1"),
},
{
ResourceRecordSets: []*route53.ResourceRecordSet{
{Name: aws.String("second.example.com.")},
},
IsTruncated: aws.Bool(true),
NextRecordName: aws.String("third.example.com."),
NextRecordType: aws.String("MX"),
MaxItems: aws.String("1"),
},
{
ResourceRecordSets: []*route53.ResourceRecordSet{
{Name: aws.String("third.example.com.")},
},
IsTruncated: aws.Bool(false),
MaxItems: aws.String("1"),
},
}
client.Handlers.Send.Clear() // mock sending
client.Handlers.Unmarshal.Clear()
client.Handlers.UnmarshalMeta.Clear()
client.Handlers.ValidateResponse.Clear()
idents := []string{}
client.Handlers.Build.PushBack(func(r *request.Request) {
p := r.Params.(*route53.ListResourceRecordSetsInput)
idents = append(idents, aws.StringValue(p.StartRecordIdentifier))
})
client.Handlers.Unmarshal.PushBack(func(r *request.Request) {
r.Data = resps[reqNum]
reqNum++
})
params := &route53.ListResourceRecordSetsInput{
HostedZoneId: aws.String("id-zone"),
}
results := []string{}
err := client.ListResourceRecordSetsPages(params, func(p *route53.ListResourceRecordSetsOutput, last bool) bool {
results = append(results, *p.ResourceRecordSets[0].Name)
return true
})
assert.NoError(t, err)
assert.Equal(t, []string{"", "second", ""}, idents)
assert.Equal(t, []string{"first.example.com.", "second.example.com.", "third.example.com."}, results)
}
// Benchmarks
var benchResps = []*dynamodb.ListTablesOutput{
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE"), aws.String("NXT")}, LastEvaluatedTableName: aws.String("NXT")},
{TableNames: []*string{aws.String("TABLE")}},
}
var benchDb = func() *dynamodb.DynamoDB {
db := dynamodb.New(unit.Session)
db.Handlers.Send.Clear() // mock sending
db.Handlers.Unmarshal.Clear()
db.Handlers.UnmarshalMeta.Clear()
db.Handlers.ValidateResponse.Clear()
return db
}
func BenchmarkCodegenIterator(b *testing.B) {
reqNum := 0
db := benchDb()
db.Handlers.Unmarshal.PushBack(func(r *request.Request) {
r.Data = benchResps[reqNum]
reqNum++
})
input := &dynamodb.ListTablesInput{Limit: aws.Int64(2)}
iter := func(fn func(*dynamodb.ListTablesOutput, bool) bool) error {
page, _ := db.ListTablesRequest(input)
for ; page != nil; page = page.NextPage() {
page.Send()
out := page.Data.(*dynamodb.ListTablesOutput)
if result := fn(out, !page.HasNextPage()); page.Error != nil || !result {
return page.Error
}
}
return nil
}
for i := 0; i < b.N; i++ {
reqNum = 0
iter(func(p *dynamodb.ListTablesOutput, last bool) bool {
return true
})
}
}
func BenchmarkEachPageIterator(b *testing.B) {
reqNum := 0
db := benchDb()
db.Handlers.Unmarshal.PushBack(func(r *request.Request) {
r.Data = benchResps[reqNum]
reqNum++
})
input := &dynamodb.ListTablesInput{Limit: aws.Int64(2)}
for i := 0; i < b.N; i++ {
reqNum = 0
req, _ := db.ListTablesRequest(input)
req.EachPage(func(p interface{}, last bool) bool {
return true
})
}
}

View File

@@ -1,380 +0,0 @@
package request_test
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"io/ioutil"
"net/http"
"runtime"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/awstesting"
)
type testData struct {
Data string
}
func body(str string) io.ReadCloser {
return ioutil.NopCloser(bytes.NewReader([]byte(str)))
}
func unmarshal(req *request.Request) {
defer req.HTTPResponse.Body.Close()
if req.Data != nil {
json.NewDecoder(req.HTTPResponse.Body).Decode(req.Data)
}
return
}
func unmarshalError(req *request.Request) {
bodyBytes, err := ioutil.ReadAll(req.HTTPResponse.Body)
if err != nil {
req.Error = awserr.New("UnmarshaleError", req.HTTPResponse.Status, err)
return
}
if len(bodyBytes) == 0 {
req.Error = awserr.NewRequestFailure(
awserr.New("UnmarshaleError", req.HTTPResponse.Status, fmt.Errorf("empty body")),
req.HTTPResponse.StatusCode,
"",
)
return
}
var jsonErr jsonErrorResponse
if err := json.Unmarshal(bodyBytes, &jsonErr); err != nil {
req.Error = awserr.New("UnmarshaleError", "JSON unmarshal", err)
return
}
req.Error = awserr.NewRequestFailure(
awserr.New(jsonErr.Code, jsonErr.Message, nil),
req.HTTPResponse.StatusCode,
"",
)
}
type jsonErrorResponse struct {
Code string `json:"__type"`
Message string `json:"message"`
}
// test that retries occur for 5xx status codes
func TestRequestRecoverRetry5xx(t *testing.T) {
reqNum := 0
reqs := []http.Response{
{StatusCode: 500, Body: body(`{"__type":"UnknownError","message":"An error occurred."}`)},
{StatusCode: 501, Body: body(`{"__type":"UnknownError","message":"An error occurred."}`)},
{StatusCode: 200, Body: body(`{"data":"valid"}`)},
}
s := awstesting.NewClient(aws.NewConfig().WithMaxRetries(10))
s.Handlers.Validate.Clear()
s.Handlers.Unmarshal.PushBack(unmarshal)
s.Handlers.UnmarshalError.PushBack(unmarshalError)
s.Handlers.Send.Clear() // mock sending
s.Handlers.Send.PushBack(func(r *request.Request) {
r.HTTPResponse = &reqs[reqNum]
reqNum++
})
out := &testData{}
r := s.NewRequest(&request.Operation{Name: "Operation"}, nil, out)
err := r.Send()
assert.Nil(t, err)
assert.Equal(t, 2, int(r.RetryCount))
assert.Equal(t, "valid", out.Data)
}
// test that retries occur for 4xx status codes with a response type that can be retried - see `shouldRetry`
func TestRequestRecoverRetry4xxRetryable(t *testing.T) {
reqNum := 0
reqs := []http.Response{
{StatusCode: 400, Body: body(`{"__type":"Throttling","message":"Rate exceeded."}`)},
{StatusCode: 429, Body: body(`{"__type":"ProvisionedThroughputExceededException","message":"Rate exceeded."}`)},
{StatusCode: 200, Body: body(`{"data":"valid"}`)},
}
s := awstesting.NewClient(aws.NewConfig().WithMaxRetries(10))
s.Handlers.Validate.Clear()
s.Handlers.Unmarshal.PushBack(unmarshal)
s.Handlers.UnmarshalError.PushBack(unmarshalError)
s.Handlers.Send.Clear() // mock sending
s.Handlers.Send.PushBack(func(r *request.Request) {
r.HTTPResponse = &reqs[reqNum]
reqNum++
})
out := &testData{}
r := s.NewRequest(&request.Operation{Name: "Operation"}, nil, out)
err := r.Send()
assert.Nil(t, err)
assert.Equal(t, 2, int(r.RetryCount))
assert.Equal(t, "valid", out.Data)
}
// test that retries don't occur for 4xx status codes with a response type that can't be retried
func TestRequest4xxUnretryable(t *testing.T) {
s := awstesting.NewClient(aws.NewConfig().WithMaxRetries(10))
s.Handlers.Validate.Clear()
s.Handlers.Unmarshal.PushBack(unmarshal)
s.Handlers.UnmarshalError.PushBack(unmarshalError)
s.Handlers.Send.Clear() // mock sending
s.Handlers.Send.PushBack(func(r *request.Request) {
r.HTTPResponse = &http.Response{StatusCode: 401, Body: body(`{"__type":"SignatureDoesNotMatch","message":"Signature does not match."}`)}
})
out := &testData{}
r := s.NewRequest(&request.Operation{Name: "Operation"}, nil, out)
err := r.Send()
assert.NotNil(t, err)
if e, ok := err.(awserr.RequestFailure); ok {
assert.Equal(t, 401, e.StatusCode())
} else {
assert.Fail(t, "Expected error to be a service failure")
}
assert.Equal(t, "SignatureDoesNotMatch", err.(awserr.Error).Code())
assert.Equal(t, "Signature does not match.", err.(awserr.Error).Message())
assert.Equal(t, 0, int(r.RetryCount))
}
func TestRequestExhaustRetries(t *testing.T) {
delays := []time.Duration{}
sleepDelay := func(delay time.Duration) {
delays = append(delays, delay)
}
reqNum := 0
reqs := []http.Response{
{StatusCode: 500, Body: body(`{"__type":"UnknownError","message":"An error occurred."}`)},
{StatusCode: 500, Body: body(`{"__type":"UnknownError","message":"An error occurred."}`)},
{StatusCode: 500, Body: body(`{"__type":"UnknownError","message":"An error occurred."}`)},
{StatusCode: 500, Body: body(`{"__type":"UnknownError","message":"An error occurred."}`)},
}
s := awstesting.NewClient(aws.NewConfig().WithSleepDelay(sleepDelay))
s.Handlers.Validate.Clear()
s.Handlers.Unmarshal.PushBack(unmarshal)
s.Handlers.UnmarshalError.PushBack(unmarshalError)
s.Handlers.Send.Clear() // mock sending
s.Handlers.Send.PushBack(func(r *request.Request) {
r.HTTPResponse = &reqs[reqNum]
reqNum++
})
r := s.NewRequest(&request.Operation{Name: "Operation"}, nil, nil)
err := r.Send()
assert.NotNil(t, err)
if e, ok := err.(awserr.RequestFailure); ok {
assert.Equal(t, 500, e.StatusCode())
} else {
assert.Fail(t, "Expected error to be a service failure")
}
assert.Equal(t, "UnknownError", err.(awserr.Error).Code())
assert.Equal(t, "An error occurred.", err.(awserr.Error).Message())
assert.Equal(t, 3, int(r.RetryCount))
expectDelays := []struct{ min, max time.Duration }{{30, 59}, {60, 118}, {120, 236}}
for i, v := range delays {
min := expectDelays[i].min * time.Millisecond
max := expectDelays[i].max * time.Millisecond
assert.True(t, min <= v && v <= max,
"Expect delay to be within range, i:%d, v:%s, min:%s, max:%s", i, v, min, max)
}
}
// test that the request is retried after the credentials are expired.
func TestRequestRecoverExpiredCreds(t *testing.T) {
reqNum := 0
reqs := []http.Response{
{StatusCode: 400, Body: body(`{"__type":"ExpiredTokenException","message":"expired token"}`)},
{StatusCode: 200, Body: body(`{"data":"valid"}`)},
}
s := awstesting.NewClient(&aws.Config{MaxRetries: aws.Int(10), Credentials: credentials.NewStaticCredentials("AKID", "SECRET", "")})
s.Handlers.Validate.Clear()
s.Handlers.Unmarshal.PushBack(unmarshal)
s.Handlers.UnmarshalError.PushBack(unmarshalError)
credExpiredBeforeRetry := false
credExpiredAfterRetry := false
s.Handlers.AfterRetry.PushBack(func(r *request.Request) {
credExpiredAfterRetry = r.Config.Credentials.IsExpired()
})
s.Handlers.Sign.Clear()
s.Handlers.Sign.PushBack(func(r *request.Request) {
r.Config.Credentials.Get()
})
s.Handlers.Send.Clear() // mock sending
s.Handlers.Send.PushBack(func(r *request.Request) {
r.HTTPResponse = &reqs[reqNum]
reqNum++
})
out := &testData{}
r := s.NewRequest(&request.Operation{Name: "Operation"}, nil, out)
err := r.Send()
assert.Nil(t, err)
assert.False(t, credExpiredBeforeRetry, "Expect valid creds before retry check")
assert.True(t, credExpiredAfterRetry, "Expect expired creds after retry check")
assert.False(t, s.Config.Credentials.IsExpired(), "Expect valid creds after cred expired recovery")
assert.Equal(t, 1, int(r.RetryCount))
assert.Equal(t, "valid", out.Data)
}
func TestMakeAddtoUserAgentHandler(t *testing.T) {
fn := request.MakeAddToUserAgentHandler("name", "version", "extra1", "extra2")
r := &request.Request{HTTPRequest: &http.Request{Header: http.Header{}}}
r.HTTPRequest.Header.Set("User-Agent", "foo/bar")
fn(r)
assert.Equal(t, "foo/bar name/version (extra1; extra2)", r.HTTPRequest.Header.Get("User-Agent"))
}
func TestMakeAddtoUserAgentFreeFormHandler(t *testing.T) {
fn := request.MakeAddToUserAgentFreeFormHandler("name/version (extra1; extra2)")
r := &request.Request{HTTPRequest: &http.Request{Header: http.Header{}}}
r.HTTPRequest.Header.Set("User-Agent", "foo/bar")
fn(r)
assert.Equal(t, "foo/bar name/version (extra1; extra2)", r.HTTPRequest.Header.Get("User-Agent"))
}
func TestRequestUserAgent(t *testing.T) {
s := awstesting.NewClient(&aws.Config{Region: aws.String("us-east-1")})
// s.Handlers.Validate.Clear()
req := s.NewRequest(&request.Operation{Name: "Operation"}, nil, &testData{})
req.HTTPRequest.Header.Set("User-Agent", "foo/bar")
assert.NoError(t, req.Build())
expectUA := fmt.Sprintf("foo/bar %s/%s (%s; %s; %s)",
aws.SDKName, aws.SDKVersion, runtime.Version(), runtime.GOOS, runtime.GOARCH)
assert.Equal(t, expectUA, req.HTTPRequest.Header.Get("User-Agent"))
}
func TestRequestThrottleRetries(t *testing.T) {
delays := []time.Duration{}
sleepDelay := func(delay time.Duration) {
delays = append(delays, delay)
}
reqNum := 0
reqs := []http.Response{
{StatusCode: 500, Body: body(`{"__type":"Throttling","message":"An error occurred."}`)},
{StatusCode: 500, Body: body(`{"__type":"Throttling","message":"An error occurred."}`)},
{StatusCode: 500, Body: body(`{"__type":"Throttling","message":"An error occurred."}`)},
{StatusCode: 500, Body: body(`{"__type":"Throttling","message":"An error occurred."}`)},
}
s := awstesting.NewClient(aws.NewConfig().WithSleepDelay(sleepDelay))
s.Handlers.Validate.Clear()
s.Handlers.Unmarshal.PushBack(unmarshal)
s.Handlers.UnmarshalError.PushBack(unmarshalError)
s.Handlers.Send.Clear() // mock sending
s.Handlers.Send.PushBack(func(r *request.Request) {
r.HTTPResponse = &reqs[reqNum]
reqNum++
})
r := s.NewRequest(&request.Operation{Name: "Operation"}, nil, nil)
err := r.Send()
assert.NotNil(t, err)
if e, ok := err.(awserr.RequestFailure); ok {
assert.Equal(t, 500, e.StatusCode())
} else {
assert.Fail(t, "Expected error to be a service failure")
}
assert.Equal(t, "Throttling", err.(awserr.Error).Code())
assert.Equal(t, "An error occurred.", err.(awserr.Error).Message())
assert.Equal(t, 3, int(r.RetryCount))
expectDelays := []struct{ min, max time.Duration }{{500, 999}, {1000, 1998}, {2000, 3996}}
for i, v := range delays {
min := expectDelays[i].min * time.Millisecond
max := expectDelays[i].max * time.Millisecond
assert.True(t, min <= v && v <= max,
"Expect delay to be within range, i:%d, v:%s, min:%s, max:%s", i, v, min, max)
}
}
// test that retries occur for request timeouts when response.Body can be nil
func TestRequestRecoverTimeoutWithNilBody(t *testing.T) {
reqNum := 0
reqs := []*http.Response{
{StatusCode: 0, Body: nil}, // body can be nil when requests time out
{StatusCode: 200, Body: body(`{"data":"valid"}`)},
}
errors := []error{
errors.New("timeout"), nil,
}
s := awstesting.NewClient(aws.NewConfig().WithMaxRetries(10))
s.Handlers.Validate.Clear()
s.Handlers.Unmarshal.PushBack(unmarshal)
s.Handlers.UnmarshalError.PushBack(unmarshalError)
s.Handlers.AfterRetry.Clear() // force retry on all errors
s.Handlers.AfterRetry.PushBack(func(r *request.Request) {
if r.Error != nil {
r.Error = nil
r.Retryable = aws.Bool(true)
r.RetryCount++
}
})
s.Handlers.Send.Clear() // mock sending
s.Handlers.Send.PushBack(func(r *request.Request) {
r.HTTPResponse = reqs[reqNum]
r.Error = errors[reqNum]
reqNum++
})
out := &testData{}
r := s.NewRequest(&request.Operation{Name: "Operation"}, nil, out)
err := r.Send()
assert.Nil(t, err)
assert.Equal(t, 1, int(r.RetryCount))
assert.Equal(t, "valid", out.Data)
}
func TestRequestRecoverTimeoutWithNilResponse(t *testing.T) {
reqNum := 0
reqs := []*http.Response{
nil,
{StatusCode: 200, Body: body(`{"data":"valid"}`)},
}
errors := []error{
errors.New("timeout"),
nil,
}
s := awstesting.NewClient(aws.NewConfig().WithMaxRetries(10))
s.Handlers.Validate.Clear()
s.Handlers.Unmarshal.PushBack(unmarshal)
s.Handlers.UnmarshalError.PushBack(unmarshalError)
s.Handlers.AfterRetry.Clear() // force retry on all errors
s.Handlers.AfterRetry.PushBack(func(r *request.Request) {
if r.Error != nil {
r.Error = nil
r.Retryable = aws.Bool(true)
r.RetryCount++
}
})
s.Handlers.Send.Clear() // mock sending
s.Handlers.Send.PushBack(func(r *request.Request) {
r.HTTPResponse = reqs[reqNum]
r.Error = errors[reqNum]
reqNum++
})
out := &testData{}
r := s.NewRequest(&request.Operation{Name: "Operation"}, nil, out)
err := r.Send()
assert.Nil(t, err)
assert.Equal(t, 1, int(r.RetryCount))
assert.Equal(t, "valid", out.Data)
}

View File

@@ -1,16 +0,0 @@
package request
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws/awserr"
)
func TestRequestThrottling(t *testing.T) {
req := Request{}
req.Error = awserr.New("Throttling", "", nil)
assert.True(t, req.IsErrorThrottle())
}

View File

@@ -66,7 +66,7 @@ through code instead of being driven by environment variables only.
Use NewSessionWithOptions when you want to provide the config profile, or
override the shared config state (AWS_SDK_LOAD_CONFIG).
// Equivalent to session.New
// Equivalent to session.NewSession()
sess, err := session.NewSessionWithOptions(session.Options{})
// Specify profile to load for the session's config
@@ -85,12 +85,6 @@ override the shared config state (AWS_SDK_LOAD_CONFIG).
SharedConfigState: SharedConfigEnable,
})
Deprecated "New" function
The New session function has been deprecated because it does not provide good
way to return errors that occur when loading the configuration files and values.
Because of this, the NewWithError
Adding Handlers
You can add handlers to a session for processing HTTP requests. All service

View File

@@ -1,276 +0,0 @@
package session
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/stretchr/testify/assert"
)
func TestLoadEnvConfig_Creds(t *testing.T) {
env := stashEnv()
defer popEnv(env)
cases := []struct {
Env map[string]string
Val credentials.Value
}{
{
Env: map[string]string{
"AWS_ACCESS_KEY": "AKID",
},
Val: credentials.Value{},
},
{
Env: map[string]string{
"AWS_ACCESS_KEY_ID": "AKID",
},
Val: credentials.Value{},
},
{
Env: map[string]string{
"AWS_SECRET_KEY": "SECRET",
},
Val: credentials.Value{},
},
{
Env: map[string]string{
"AWS_SECRET_ACCESS_KEY": "SECRET",
},
Val: credentials.Value{},
},
{
Env: map[string]string{
"AWS_ACCESS_KEY_ID": "AKID",
"AWS_SECRET_ACCESS_KEY": "SECRET",
},
Val: credentials.Value{
AccessKeyID: "AKID", SecretAccessKey: "SECRET",
ProviderName: "EnvConfigCredentials",
},
},
{
Env: map[string]string{
"AWS_ACCESS_KEY": "AKID",
"AWS_SECRET_KEY": "SECRET",
},
Val: credentials.Value{
AccessKeyID: "AKID", SecretAccessKey: "SECRET",
ProviderName: "EnvConfigCredentials",
},
},
{
Env: map[string]string{
"AWS_ACCESS_KEY": "AKID",
"AWS_SECRET_KEY": "SECRET",
"AWS_SESSION_TOKEN": "TOKEN",
},
Val: credentials.Value{
AccessKeyID: "AKID", SecretAccessKey: "SECRET", SessionToken: "TOKEN",
ProviderName: "EnvConfigCredentials",
},
},
}
for _, c := range cases {
os.Clearenv()
for k, v := range c.Env {
os.Setenv(k, v)
}
cfg := loadEnvConfig()
assert.Equal(t, c.Val, cfg.Creds)
}
}
func TestLoadEnvConfig(t *testing.T) {
env := stashEnv()
defer popEnv(env)
cases := []struct {
Env map[string]string
Region, Profile string
UseSharedConfigCall bool
}{
{
Env: map[string]string{
"AWS_REGION": "region",
"AWS_PROFILE": "profile",
},
Region: "region", Profile: "profile",
},
{
Env: map[string]string{
"AWS_REGION": "region",
"AWS_DEFAULT_REGION": "default_region",
"AWS_PROFILE": "profile",
"AWS_DEFAULT_PROFILE": "default_profile",
},
Region: "region", Profile: "profile",
},
{
Env: map[string]string{
"AWS_REGION": "region",
"AWS_DEFAULT_REGION": "default_region",
"AWS_PROFILE": "profile",
"AWS_DEFAULT_PROFILE": "default_profile",
"AWS_SDK_LOAD_CONFIG": "1",
},
Region: "region", Profile: "profile",
},
{
Env: map[string]string{
"AWS_DEFAULT_REGION": "default_region",
"AWS_DEFAULT_PROFILE": "default_profile",
},
},
{
Env: map[string]string{
"AWS_DEFAULT_REGION": "default_region",
"AWS_DEFAULT_PROFILE": "default_profile",
"AWS_SDK_LOAD_CONFIG": "1",
},
Region: "default_region", Profile: "default_profile",
},
{
Env: map[string]string{
"AWS_REGION": "region",
"AWS_PROFILE": "profile",
},
Region: "region", Profile: "profile",
UseSharedConfigCall: true,
},
{
Env: map[string]string{
"AWS_REGION": "region",
"AWS_DEFAULT_REGION": "default_region",
"AWS_PROFILE": "profile",
"AWS_DEFAULT_PROFILE": "default_profile",
},
Region: "region", Profile: "profile",
UseSharedConfigCall: true,
},
{
Env: map[string]string{
"AWS_REGION": "region",
"AWS_DEFAULT_REGION": "default_region",
"AWS_PROFILE": "profile",
"AWS_DEFAULT_PROFILE": "default_profile",
"AWS_SDK_LOAD_CONFIG": "1",
},
Region: "region", Profile: "profile",
UseSharedConfigCall: true,
},
{
Env: map[string]string{
"AWS_DEFAULT_REGION": "default_region",
"AWS_DEFAULT_PROFILE": "default_profile",
},
Region: "default_region", Profile: "default_profile",
UseSharedConfigCall: true,
},
{
Env: map[string]string{
"AWS_DEFAULT_REGION": "default_region",
"AWS_DEFAULT_PROFILE": "default_profile",
"AWS_SDK_LOAD_CONFIG": "1",
},
Region: "default_region", Profile: "default_profile",
UseSharedConfigCall: true,
},
}
for _, c := range cases {
os.Clearenv()
for k, v := range c.Env {
os.Setenv(k, v)
}
var cfg envConfig
if c.UseSharedConfigCall {
cfg = loadSharedEnvConfig()
} else {
cfg = loadEnvConfig()
}
assert.Equal(t, c.Region, cfg.Region)
assert.Equal(t, c.Profile, cfg.Profile)
}
}
func TestSharedCredsFilename(t *testing.T) {
env := stashEnv()
defer popEnv(env)
os.Setenv("USERPROFILE", "profile_dir")
expect := filepath.Join("profile_dir", ".aws", "credentials")
name := sharedCredentialsFilename()
assert.Equal(t, expect, name)
os.Setenv("HOME", "home_dir")
expect = filepath.Join("home_dir", ".aws", "credentials")
name = sharedCredentialsFilename()
assert.Equal(t, expect, name)
expect = filepath.Join("path/to/credentials/file")
os.Setenv("AWS_SHARED_CREDENTIALS_FILE", expect)
name = sharedCredentialsFilename()
assert.Equal(t, expect, name)
}
func TestSharedConfigFilename(t *testing.T) {
env := stashEnv()
defer popEnv(env)
os.Setenv("USERPROFILE", "profile_dir")
expect := filepath.Join("profile_dir", ".aws", "config")
name := sharedConfigFilename()
assert.Equal(t, expect, name)
os.Setenv("HOME", "home_dir")
expect = filepath.Join("home_dir", ".aws", "config")
name = sharedConfigFilename()
assert.Equal(t, expect, name)
expect = filepath.Join("path/to/config/file")
os.Setenv("AWS_CONFIG_FILE", expect)
name = sharedConfigFilename()
assert.Equal(t, expect, name)
}
func TestSetEnvValue(t *testing.T) {
env := stashEnv()
defer popEnv(env)
os.Setenv("empty_key", "")
os.Setenv("second_key", "2")
os.Setenv("third_key", "3")
var dst string
setFromEnvVal(&dst, []string{
"empty_key", "first_key", "second_key", "third_key",
})
assert.Equal(t, "2", dst)
}
func stashEnv() []string {
env := os.Environ()
os.Clearenv()
return env
}
func popEnv(env []string) {
os.Clearenv()
for _, e := range env {
p := strings.SplitN(e, "=", 2)
os.Setenv(p[0], p[1])
}
}

View File

@@ -179,7 +179,12 @@ type Options struct {
// SharedConfigState: SharedConfigEnable,
// })
func NewSessionWithOptions(opts Options) (*Session, error) {
envCfg := loadEnvConfig()
var envCfg envConfig
if opts.SharedConfigState == SharedConfigEnable {
envCfg = loadSharedEnvConfig()
} else {
envCfg = loadEnvConfig()
}
if len(opts.Profile) > 0 {
envCfg.Profile = opts.Profile

View File

@@ -1,344 +0,0 @@
package session
import (
"bytes"
"fmt"
"net/http"
"net/http/httptest"
"os"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/defaults"
"github.com/aws/aws-sdk-go/service/s3"
)
func TestNewDefaultSession(t *testing.T) {
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
s := New(&aws.Config{Region: aws.String("region")})
assert.Equal(t, "region", *s.Config.Region)
assert.Equal(t, http.DefaultClient, s.Config.HTTPClient)
assert.NotNil(t, s.Config.Logger)
assert.Equal(t, aws.LogOff, *s.Config.LogLevel)
}
func TestNew_WithCustomCreds(t *testing.T) {
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
customCreds := credentials.NewStaticCredentials("AKID", "SECRET", "TOKEN")
s := New(&aws.Config{Credentials: customCreds})
assert.Equal(t, customCreds, s.Config.Credentials)
}
type mockLogger struct {
*bytes.Buffer
}
func (w mockLogger) Log(args ...interface{}) {
fmt.Fprintln(w, args...)
}
func TestNew_WithSessionLoadError(t *testing.T) {
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
os.Setenv("AWS_SDK_LOAD_CONFIG", "1")
os.Setenv("AWS_CONFIG_FILE", testConfigFilename)
os.Setenv("AWS_PROFILE", "assume_role_invalid_source_profile")
logger := bytes.Buffer{}
s := New(&aws.Config{Logger: &mockLogger{&logger}})
assert.NotNil(t, s)
svc := s3.New(s)
_, err := svc.ListBuckets(&s3.ListBucketsInput{})
assert.Error(t, err)
assert.Contains(t, logger.String(), "ERROR: failed to create session with AWS_SDK_LOAD_CONFIG enabled")
assert.Contains(t, err.Error(), SharedConfigAssumeRoleError{
RoleARN: "assume_role_invalid_source_profile_role_arn",
}.Error())
}
func TestSessionCopy(t *testing.T) {
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
os.Setenv("AWS_REGION", "orig_region")
s := Session{
Config: defaults.Config(),
Handlers: defaults.Handlers(),
}
newSess := s.Copy(&aws.Config{Region: aws.String("new_region")})
assert.Equal(t, "orig_region", *s.Config.Region)
assert.Equal(t, "new_region", *newSess.Config.Region)
}
func TestSessionClientConfig(t *testing.T) {
s, err := NewSession(&aws.Config{Region: aws.String("orig_region")})
assert.NoError(t, err)
cfg := s.ClientConfig("s3", &aws.Config{Region: aws.String("us-west-2")})
assert.Equal(t, "https://s3-us-west-2.amazonaws.com", cfg.Endpoint)
assert.Empty(t, cfg.SigningRegion)
assert.Equal(t, "us-west-2", *cfg.Config.Region)
}
func TestNewSession_NoCredentials(t *testing.T) {
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
s, err := NewSession()
assert.NoError(t, err)
assert.NotNil(t, s.Config.Credentials)
assert.NotEqual(t, credentials.AnonymousCredentials, s.Config.Credentials)
}
func TestNewSessionWithOptions_OverrideProfile(t *testing.T) {
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
os.Setenv("AWS_SDK_LOAD_CONFIG", "1")
os.Setenv("AWS_SHARED_CREDENTIALS_FILE", testConfigFilename)
os.Setenv("AWS_PROFILE", "other_profile")
s, err := NewSessionWithOptions(Options{
Profile: "full_profile",
})
assert.NoError(t, err)
assert.Equal(t, "full_profile_region", *s.Config.Region)
creds, err := s.Config.Credentials.Get()
assert.NoError(t, err)
assert.Equal(t, "full_profile_akid", creds.AccessKeyID)
assert.Equal(t, "full_profile_secret", creds.SecretAccessKey)
assert.Empty(t, creds.SessionToken)
assert.Contains(t, creds.ProviderName, "SharedConfigCredentials")
}
func TestNewSessionWithOptions_OverrideSharedConfigEnable(t *testing.T) {
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
os.Setenv("AWS_SDK_LOAD_CONFIG", "0")
os.Setenv("AWS_SHARED_CREDENTIALS_FILE", testConfigFilename)
os.Setenv("AWS_PROFILE", "full_profile")
s, err := NewSessionWithOptions(Options{
SharedConfigState: SharedConfigEnable,
})
assert.NoError(t, err)
assert.Equal(t, "full_profile_region", *s.Config.Region)
creds, err := s.Config.Credentials.Get()
assert.NoError(t, err)
assert.Equal(t, "full_profile_akid", creds.AccessKeyID)
assert.Equal(t, "full_profile_secret", creds.SecretAccessKey)
assert.Empty(t, creds.SessionToken)
assert.Contains(t, creds.ProviderName, "SharedConfigCredentials")
}
func TestNewSessionWithOptions_OverrideSharedConfigDisable(t *testing.T) {
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
os.Setenv("AWS_SDK_LOAD_CONFIG", "1")
os.Setenv("AWS_SHARED_CREDENTIALS_FILE", testConfigFilename)
os.Setenv("AWS_PROFILE", "full_profile")
s, err := NewSessionWithOptions(Options{
SharedConfigState: SharedConfigDisable,
})
assert.NoError(t, err)
assert.Empty(t, *s.Config.Region)
creds, err := s.Config.Credentials.Get()
assert.NoError(t, err)
assert.Equal(t, "full_profile_akid", creds.AccessKeyID)
assert.Equal(t, "full_profile_secret", creds.SecretAccessKey)
assert.Empty(t, creds.SessionToken)
assert.Contains(t, creds.ProviderName, "SharedConfigCredentials")
}
func TestNewSessionWithOptions_Overrides(t *testing.T) {
cases := []struct {
InEnvs map[string]string
InProfile string
OutRegion string
OutCreds credentials.Value
}{
{
InEnvs: map[string]string{
"AWS_SDK_LOAD_CONFIG": "0",
"AWS_SHARED_CREDENTIALS_FILE": testConfigFilename,
"AWS_PROFILE": "other_profile",
},
InProfile: "full_profile",
OutRegion: "full_profile_region",
OutCreds: credentials.Value{
AccessKeyID: "full_profile_akid",
SecretAccessKey: "full_profile_secret",
ProviderName: "SharedConfigCredentials",
},
},
{
InEnvs: map[string]string{
"AWS_SDK_LOAD_CONFIG": "0",
"AWS_SHARED_CREDENTIALS_FILE": testConfigFilename,
"AWS_REGION": "env_region",
"AWS_ACCESS_KEY": "env_akid",
"AWS_SECRET_ACCESS_KEY": "env_secret",
"AWS_PROFILE": "other_profile",
},
InProfile: "full_profile",
OutRegion: "env_region",
OutCreds: credentials.Value{
AccessKeyID: "env_akid",
SecretAccessKey: "env_secret",
ProviderName: "EnvConfigCredentials",
},
},
{
InEnvs: map[string]string{
"AWS_SDK_LOAD_CONFIG": "0",
"AWS_SHARED_CREDENTIALS_FILE": testConfigFilename,
"AWS_CONFIG_FILE": testConfigOtherFilename,
"AWS_PROFILE": "shared_profile",
},
InProfile: "config_file_load_order",
OutRegion: "shared_config_region",
OutCreds: credentials.Value{
AccessKeyID: "shared_config_akid",
SecretAccessKey: "shared_config_secret",
ProviderName: "SharedConfigCredentials",
},
},
}
for _, c := range cases {
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
for k, v := range c.InEnvs {
os.Setenv(k, v)
}
s, err := NewSessionWithOptions(Options{
Profile: c.InProfile,
SharedConfigState: SharedConfigEnable,
})
assert.NoError(t, err)
creds, err := s.Config.Credentials.Get()
assert.NoError(t, err)
assert.Equal(t, c.OutRegion, *s.Config.Region)
assert.Equal(t, c.OutCreds.AccessKeyID, creds.AccessKeyID)
assert.Equal(t, c.OutCreds.SecretAccessKey, creds.SecretAccessKey)
assert.Equal(t, c.OutCreds.SessionToken, creds.SessionToken)
assert.Contains(t, creds.ProviderName, c.OutCreds.ProviderName)
}
}
func TestSesisonAssumeRole(t *testing.T) {
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
os.Setenv("AWS_REGION", "us-east-1")
os.Setenv("AWS_SDK_LOAD_CONFIG", "1")
os.Setenv("AWS_SHARED_CREDENTIALS_FILE", testConfigFilename)
os.Setenv("AWS_PROFILE", "assume_role_w_creds")
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
const respMsg = `
<AssumeRoleResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
<AssumeRoleResult>
<AssumedRoleUser>
<Arn>arn:aws:sts::account_id:assumed-role/role/session_name</Arn>
<AssumedRoleId>AKID:session_name</AssumedRoleId>
</AssumedRoleUser>
<Credentials>
<AccessKeyId>AKID</AccessKeyId>
<SecretAccessKey>SECRET</SecretAccessKey>
<SessionToken>SESSION_TOKEN</SessionToken>
<Expiration>%s</Expiration>
</Credentials>
</AssumeRoleResult>
<ResponseMetadata>
<RequestId>request-id</RequestId>
</ResponseMetadata>
</AssumeRoleResponse>
`
w.Write([]byte(fmt.Sprintf(respMsg, time.Now().Add(15*time.Minute).Format("2006-01-02T15:04:05Z"))))
}))
s, err := NewSession(&aws.Config{Endpoint: aws.String(server.URL), DisableSSL: aws.Bool(true)})
creds, err := s.Config.Credentials.Get()
assert.NoError(t, err)
assert.Equal(t, "AKID", creds.AccessKeyID)
assert.Equal(t, "SECRET", creds.SecretAccessKey)
assert.Equal(t, "SESSION_TOKEN", creds.SessionToken)
assert.Contains(t, creds.ProviderName, "AssumeRoleProvider")
}
func TestSessionAssumeRole_DisableSharedConfig(t *testing.T) {
// Backwards compatibility with Shared config disabled
// assume role should not be built into the config.
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
os.Setenv("AWS_SDK_LOAD_CONFIG", "0")
os.Setenv("AWS_SHARED_CREDENTIALS_FILE", testConfigFilename)
os.Setenv("AWS_PROFILE", "assume_role_w_creds")
s, err := NewSession()
assert.NoError(t, err)
creds, err := s.Config.Credentials.Get()
assert.NoError(t, err)
assert.Equal(t, "assume_role_w_creds_akid", creds.AccessKeyID)
assert.Equal(t, "assume_role_w_creds_secret", creds.SecretAccessKey)
assert.Contains(t, creds.ProviderName, "SharedConfigCredentials")
}
func TestSessionAssumeRole_InvalidSourceProfile(t *testing.T) {
// Backwards compatibility with Shared config disabled
// assume role should not be built into the config.
oldEnv := initSessionTestEnv()
defer popEnv(oldEnv)
os.Setenv("AWS_SDK_LOAD_CONFIG", "1")
os.Setenv("AWS_SHARED_CREDENTIALS_FILE", testConfigFilename)
os.Setenv("AWS_PROFILE", "assume_role_invalid_source_profile")
s, err := NewSession()
assert.Error(t, err)
assert.Contains(t, err.Error(), "SharedConfigAssumeRoleError: failed to load assume role")
assert.Nil(t, s)
}
func initSessionTestEnv() (oldEnv []string) {
oldEnv = stashEnv()
os.Setenv("AWS_CONFIG_FILE", "file_not_exists")
os.Setenv("AWS_SHARED_CREDENTIALS_FILE", "file_not_exists")
return oldEnv
}

View File

@@ -2,7 +2,7 @@ package session
import (
"fmt"
"os"
"io/ioutil"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/credentials"
@@ -105,12 +105,13 @@ func loadSharedConfigIniFiles(filenames []string) ([]sharedConfigFile, error) {
files := make([]sharedConfigFile, 0, len(filenames))
for _, filename := range filenames {
if _, err := os.Stat(filename); os.IsNotExist(err) {
// Trim files from the list that don't exist.
b, err := ioutil.ReadFile(filename)
if err != nil {
// Skip files which can't be opened and read for whatever reason
continue
}
f, err := ini.Load(filename)
f, err := ini.Load(b)
if err != nil {
return nil, SharedConfigLoadError{Filename: filename}
}

View File

@@ -1,264 +0,0 @@
package session
import (
"fmt"
"path/filepath"
"testing"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/go-ini/ini"
"github.com/stretchr/testify/assert"
)
var (
testConfigFilename = filepath.Join("testdata", "shared_config")
testConfigOtherFilename = filepath.Join("testdata", "shared_config_other")
)
func TestLoadSharedConfig(t *testing.T) {
cases := []struct {
Filenames []string
Profile string
Expected sharedConfig
Err error
}{
{
Filenames: []string{"file_not_exists"},
Profile: "default",
},
{
Filenames: []string{testConfigFilename},
Expected: sharedConfig{
Region: "default_region",
},
},
{
Filenames: []string{testConfigOtherFilename, testConfigFilename},
Profile: "config_file_load_order",
Expected: sharedConfig{
Region: "shared_config_region",
Creds: credentials.Value{
AccessKeyID: "shared_config_akid",
SecretAccessKey: "shared_config_secret",
ProviderName: fmt.Sprintf("SharedConfigCredentials: %s", testConfigFilename),
},
},
},
{
Filenames: []string{testConfigFilename, testConfigOtherFilename},
Profile: "config_file_load_order",
Expected: sharedConfig{
Region: "shared_config_other_region",
Creds: credentials.Value{
AccessKeyID: "shared_config_other_akid",
SecretAccessKey: "shared_config_other_secret",
ProviderName: fmt.Sprintf("SharedConfigCredentials: %s", testConfigOtherFilename),
},
},
},
{
Filenames: []string{testConfigOtherFilename, testConfigFilename},
Profile: "assume_role",
Expected: sharedConfig{
AssumeRole: assumeRoleConfig{
RoleARN: "assume_role_role_arn",
SourceProfile: "complete_creds",
},
AssumeRoleSource: &sharedConfig{
Creds: credentials.Value{
AccessKeyID: "complete_creds_akid",
SecretAccessKey: "complete_creds_secret",
ProviderName: fmt.Sprintf("SharedConfigCredentials: %s", testConfigFilename),
},
},
},
},
{
Filenames: []string{testConfigOtherFilename, testConfigFilename},
Profile: "assume_role_invalid_source_profile",
Expected: sharedConfig{
AssumeRole: assumeRoleConfig{
RoleARN: "assume_role_invalid_source_profile_role_arn",
SourceProfile: "profile_not_exists",
},
},
Err: SharedConfigAssumeRoleError{RoleARN: "assume_role_invalid_source_profile_role_arn"},
},
{
Filenames: []string{testConfigOtherFilename, testConfigFilename},
Profile: "assume_role_w_creds",
Expected: sharedConfig{
Creds: credentials.Value{
AccessKeyID: "assume_role_w_creds_akid",
SecretAccessKey: "assume_role_w_creds_secret",
ProviderName: fmt.Sprintf("SharedConfigCredentials: %s", testConfigFilename),
},
AssumeRole: assumeRoleConfig{
RoleARN: "assume_role_w_creds_role_arn",
SourceProfile: "assume_role_w_creds",
ExternalID: "1234",
RoleSessionName: "assume_role_w_creds_session_name",
},
AssumeRoleSource: &sharedConfig{
Creds: credentials.Value{
AccessKeyID: "assume_role_w_creds_akid",
SecretAccessKey: "assume_role_w_creds_secret",
ProviderName: fmt.Sprintf("SharedConfigCredentials: %s", testConfigFilename),
},
},
},
},
{
Filenames: []string{testConfigOtherFilename, testConfigFilename},
Profile: "assume_role_wo_creds",
Expected: sharedConfig{
AssumeRole: assumeRoleConfig{
RoleARN: "assume_role_wo_creds_role_arn",
SourceProfile: "assume_role_wo_creds",
},
},
Err: SharedConfigAssumeRoleError{RoleARN: "assume_role_wo_creds_role_arn"},
},
{
Filenames: []string{filepath.Join("testdata", "shared_config_invalid_ini")},
Profile: "profile_name",
Err: SharedConfigLoadError{Filename: filepath.Join("testdata", "shared_config_invalid_ini")},
},
}
for i, c := range cases {
cfg, err := loadSharedConfig(c.Profile, c.Filenames)
if c.Err != nil {
assert.Contains(t, err.Error(), c.Err.Error(), "expected error, %d", i)
continue
}
assert.NoError(t, err, "unexpected error, %d", i)
assert.Equal(t, c.Expected, cfg, "not equal, %d", i)
}
}
func TestLoadSharedConfigFromFile(t *testing.T) {
filename := testConfigFilename
f, err := ini.Load(filename)
if err != nil {
t.Fatalf("failed to load test config file, %s, %v", filename, err)
}
iniFile := sharedConfigFile{IniData: f, Filename: filename}
cases := []struct {
Profile string
Expected sharedConfig
Err error
}{
{
Profile: "default",
Expected: sharedConfig{Region: "default_region"},
},
{
Profile: "alt_profile_name",
Expected: sharedConfig{Region: "alt_profile_name_region"},
},
{
Profile: "short_profile_name_first",
Expected: sharedConfig{Region: "short_profile_name_first_short"},
},
{
Profile: "partial_creds",
Expected: sharedConfig{},
},
{
Profile: "complete_creds",
Expected: sharedConfig{
Creds: credentials.Value{
AccessKeyID: "complete_creds_akid",
SecretAccessKey: "complete_creds_secret",
ProviderName: fmt.Sprintf("SharedConfigCredentials: %s", testConfigFilename),
},
},
},
{
Profile: "complete_creds_with_token",
Expected: sharedConfig{
Creds: credentials.Value{
AccessKeyID: "complete_creds_with_token_akid",
SecretAccessKey: "complete_creds_with_token_secret",
SessionToken: "complete_creds_with_token_token",
ProviderName: fmt.Sprintf("SharedConfigCredentials: %s", testConfigFilename),
},
},
},
{
Profile: "full_profile",
Expected: sharedConfig{
Creds: credentials.Value{
AccessKeyID: "full_profile_akid",
SecretAccessKey: "full_profile_secret",
ProviderName: fmt.Sprintf("SharedConfigCredentials: %s", testConfigFilename),
},
Region: "full_profile_region",
},
},
{
Profile: "partial_assume_role",
Expected: sharedConfig{},
},
{
Profile: "assume_role",
Expected: sharedConfig{
AssumeRole: assumeRoleConfig{
RoleARN: "assume_role_role_arn",
SourceProfile: "complete_creds",
},
},
},
{
Profile: "does_not_exists",
Err: SharedConfigProfileNotExistsError{Profile: "does_not_exists"},
},
}
for i, c := range cases {
cfg := sharedConfig{}
err := cfg.setFromIniFile(c.Profile, iniFile)
if c.Err != nil {
assert.Contains(t, err.Error(), c.Err.Error(), "expected error, %d", i)
continue
}
assert.NoError(t, err, "unexpected error, %d", i)
assert.Equal(t, c.Expected, cfg, "not equal, %d", i)
}
}
func TestLoadSharedConfigIniFiles(t *testing.T) {
cases := []struct {
Filenames []string
Expected []sharedConfigFile
}{
{
Filenames: []string{"not_exists", testConfigFilename},
Expected: []sharedConfigFile{
{Filename: testConfigFilename},
},
},
{
Filenames: []string{testConfigFilename, testConfigOtherFilename},
Expected: []sharedConfigFile{
{Filename: testConfigFilename},
{Filename: testConfigOtherFilename},
},
},
}
for i, c := range cases {
files, err := loadSharedConfigIniFiles(c.Filenames)
assert.NoError(t, err, "unexpected error, %d", i)
assert.Equal(t, len(c.Expected), len(files), "expected num files, %d", i)
for i, expectedFile := range c.Expected {
assert.Equal(t, expectedFile.Filename, files[i].Filename)
}
}
}

View File

@@ -1,77 +0,0 @@
package v4_test
import (
"net/http"
"net/url"
"testing"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/awstesting/unit"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/stretchr/testify/assert"
)
func TestPresignHandler(t *testing.T) {
svc := s3.New(unit.Session)
req, _ := svc.PutObjectRequest(&s3.PutObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("key"),
ContentDisposition: aws.String("a+b c$d"),
ACL: aws.String("public-read"),
})
req.Time = time.Unix(0, 0)
urlstr, err := req.Presign(5 * time.Minute)
assert.NoError(t, err)
expectedDate := "19700101T000000Z"
expectedHeaders := "content-disposition;host;x-amz-acl"
expectedSig := "b2754ba8ffeb74a40b94767017e24c4672107d6d5a894648d5d332ca61f5ffe4"
expectedCred := "AKID/19700101/mock-region/s3/aws4_request"
u, _ := url.Parse(urlstr)
urlQ := u.Query()
assert.Equal(t, expectedSig, urlQ.Get("X-Amz-Signature"))
assert.Equal(t, expectedCred, urlQ.Get("X-Amz-Credential"))
assert.Equal(t, expectedHeaders, urlQ.Get("X-Amz-SignedHeaders"))
assert.Equal(t, expectedDate, urlQ.Get("X-Amz-Date"))
assert.Equal(t, "300", urlQ.Get("X-Amz-Expires"))
assert.NotContains(t, urlstr, "+") // + encoded as %20
}
func TestPresignRequest(t *testing.T) {
svc := s3.New(unit.Session)
req, _ := svc.PutObjectRequest(&s3.PutObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("key"),
ContentDisposition: aws.String("a+b c$d"),
ACL: aws.String("public-read"),
})
req.Time = time.Unix(0, 0)
urlstr, headers, err := req.PresignRequest(5 * time.Minute)
assert.NoError(t, err)
expectedDate := "19700101T000000Z"
expectedHeaders := "content-disposition;host;x-amz-acl;x-amz-content-sha256"
expectedSig := "0d200ba61501d752acd06f39ef4dbe7d83ffd5ea15978dc3476dfc00b8eb574e"
expectedCred := "AKID/19700101/mock-region/s3/aws4_request"
expectedHeaderMap := http.Header{
"x-amz-acl": []string{"public-read"},
"content-disposition": []string{"a+b c$d"},
"x-amz-content-sha256": []string{"UNSIGNED-PAYLOAD"},
}
u, _ := url.Parse(urlstr)
urlQ := u.Query()
assert.Equal(t, expectedSig, urlQ.Get("X-Amz-Signature"))
assert.Equal(t, expectedCred, urlQ.Get("X-Amz-Credential"))
assert.Equal(t, expectedHeaders, urlQ.Get("X-Amz-SignedHeaders"))
assert.Equal(t, expectedDate, urlQ.Get("X-Amz-Date"))
assert.Equal(t, expectedHeaderMap, headers)
assert.Equal(t, "300", urlQ.Get("X-Amz-Expires"))
assert.NotContains(t, urlstr, "+") // + encoded as %20
}

View File

@@ -1,57 +0,0 @@
package v4
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestRuleCheckWhitelist(t *testing.T) {
w := whitelist{
mapRule{
"Cache-Control": struct{}{},
},
}
assert.True(t, w.IsValid("Cache-Control"))
assert.False(t, w.IsValid("Cache-"))
}
func TestRuleCheckBlacklist(t *testing.T) {
b := blacklist{
mapRule{
"Cache-Control": struct{}{},
},
}
assert.False(t, b.IsValid("Cache-Control"))
assert.True(t, b.IsValid("Cache-"))
}
func TestRuleCheckPattern(t *testing.T) {
p := patterns{"X-Amz-Meta-"}
assert.True(t, p.IsValid("X-Amz-Meta-"))
assert.True(t, p.IsValid("X-Amz-Meta-Star"))
assert.False(t, p.IsValid("Cache-"))
}
func TestRuleComplexWhitelist(t *testing.T) {
w := rules{
whitelist{
mapRule{
"Cache-Control": struct{}{},
},
},
patterns{"X-Amz-Meta-"},
}
r := rules{
inclusiveRules{patterns{"X-Amz-"}, blacklist{w}},
}
assert.True(t, r.IsValid("X-Amz-Blah"))
assert.False(t, r.IsValid("X-Amz-Meta-"))
assert.False(t, r.IsValid("X-Amz-Meta-Star"))
assert.False(t, r.IsValid("Cache-Control"))
}

View File

@@ -0,0 +1,24 @@
// +build go1.5
package v4
import (
"net/url"
"strings"
)
func getURIPath(u *url.URL) string {
var uri string
if len(u.Opaque) > 0 {
uri = "/" + strings.Join(strings.Split(u.Opaque, "/")[3:], "/")
} else {
uri = u.EscapedPath()
}
if len(uri) == 0 {
uri = "/"
}
return uri
}

View File

@@ -0,0 +1,24 @@
// +build !go1.5
package v4
import (
"net/url"
"strings"
)
func getURIPath(u *url.URL) string {
var uri string
if len(u.Opaque) > 0 {
uri = "/" + strings.Join(strings.Split(u.Opaque, "/")[3:], "/")
} else {
uri = u.Path
}
if len(uri) == 0 {
uri = "/"
}
return uri
}

View File

@@ -2,6 +2,48 @@
//
// Provides request signing for request that need to be signed with
// AWS V4 Signatures.
//
// Standalone Signer
//
// Generally using the signer outside of the SDK should not require any additional
// logic when using Go v1.5 or higher. The signer does this by taking advantage
// of the URL.EscapedPath method. If your request URI requires additional escaping
// you many need to use the URL.Opaque to define what the raw URI should be sent
// to the service as.
//
// The signer will first check the URL.Opaque field, and use its value if set.
// The signer does require the URL.Opaque field to be set in the form of:
//
// "//<hostname>/<path>"
//
// // e.g.
// "//example.com/some/path"
//
// The leading "//" and hostname are required or the URL.Opaque escaping will
// not work correctly.
//
// If URL.Opaque is not set the signer will fallback to the URL.EscapedPath()
// method and using the returned value. If you're using Go v1.4 you must set
// URL.Opaque if the URI path needs escaping. If URL.Opaque is not set with
// Go v1.5 the signer will fallback to URL.Path.
//
// AWS v4 signature validation requires that the canonical string's URI path
// element must be the URI escaped form of the HTTP request's path.
// http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
//
// The Go HTTP client will perform escaping automatically on the request. Some
// of these escaping may cause signature validation errors because the HTTP
// request differs from the URI path or query that the signature was generated.
// https://golang.org/pkg/net/url/#URL.EscapedPath
//
// Because of this, it is recommended that when using the signer outside of the
// SDK that explicitly escaping the request prior to being signed is preferable,
// and will help prevent signature validation errors. This can be done by setting
// the URL.Opaque or URL.RawPath. The SDK will use URL.Opaque first and then
// call URL.EscapedPath() if Opaque is not set.
//
// Test `TestStandaloneSign` provides a complete example of using the signer
// outside of the SDK and pre-escaping the URI path.
package v4
import (
@@ -11,6 +53,7 @@ import (
"encoding/hex"
"fmt"
"io"
"io/ioutil"
"net/http"
"net/url"
"sort"
@@ -119,6 +162,15 @@ type Signer struct {
// request's query string.
DisableHeaderHoisting bool
// Disables the automatic escaping of the URI path of the request for the
// siganture's canonical string's path. For services that do not need additional
// escaping then use this to disable the signer escaping the path.
//
// S3 is an example of a service that does not need additional escaping.
//
// http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html
DisableURIPathEscaping bool
// currentTimeFn returns the time value which represents the current time.
// This value should only be used for testing. If it is nil the default
// time.Now will be used.
@@ -150,6 +202,8 @@ type signingCtx struct {
ExpireTime time.Duration
SignedHeaderVals http.Header
DisableURIPathEscaping bool
credValues credentials.Value
isPresign bool
formattedTime string
@@ -175,6 +229,12 @@ type signingCtx struct {
// is not needed as the full request context will be captured by the http.Request
// value. It is included for reference though.
//
// Sign will set the request's Body to be the `body` parameter passed in. If
// the body is not already an io.ReadCloser, it will be wrapped within one. If
// a `nil` body parameter passed to Sign, the request's Body field will be
// also set to nil. Its important to note that this functionality will not
// change the request's ContentLength of the request.
//
// Sign differs from Presign in that it will sign the request using HTTP
// header values. This type of signing is intended for http.Request values that
// will not be shared, or are shared in a way the header values on the request
@@ -229,22 +289,22 @@ func (v4 Signer) signWithBody(r *http.Request, body io.ReadSeeker, service, regi
}
ctx := &signingCtx{
Request: r,
Body: body,
Query: r.URL.Query(),
Time: signTime,
ExpireTime: exp,
isPresign: exp != 0,
ServiceName: service,
Region: region,
Request: r,
Body: body,
Query: r.URL.Query(),
Time: signTime,
ExpireTime: exp,
isPresign: exp != 0,
ServiceName: service,
Region: region,
DisableURIPathEscaping: v4.DisableURIPathEscaping,
}
for key := range ctx.Query {
sort.Strings(ctx.Query[key])
}
if ctx.isRequestSigned() {
if !v4.Credentials.IsExpired() && currentTimeFn().Before(ctx.Time.Add(10*time.Minute)) {
// If the request is already signed, and the credentials have not
// expired, and the request is not too old ignore the signing request.
return ctx.SignedHeaderVals, nil
}
ctx.Time = currentTimeFn()
ctx.handlePresignRemoval()
}
@@ -258,6 +318,20 @@ func (v4 Signer) signWithBody(r *http.Request, body io.ReadSeeker, service, regi
ctx.assignAmzQueryValues()
ctx.build(v4.DisableHeaderHoisting)
// If the request is not presigned the body should be attached to it. This
// prevents the confusion of wanting to send a signed request without
// the body the request was signed for attached.
if !ctx.isPresign {
var reader io.ReadCloser
if body != nil {
var ok bool
if reader, ok = body.(io.ReadCloser); !ok {
reader = ioutil.NopCloser(body)
}
}
r.Body = reader
}
if v4.Debug.Matches(aws.LogDebugWithSigning) {
v4.logSigningInfo(ctx)
}
@@ -338,6 +412,10 @@ func signSDKRequestWithCurrTime(req *request.Request, curTimeFn func() time.Time
v4.Logger = req.Config.Logger
v4.DisableHeaderHoisting = req.NotHoist
v4.currentTimeFn = curTimeFn
if name == "s3" {
// S3 service should not have any escaping applied
v4.DisableURIPathEscaping = true
}
})
signingTime := req.Time
@@ -345,7 +423,9 @@ func signSDKRequestWithCurrTime(req *request.Request, curTimeFn func() time.Time
signingTime = req.LastSignedAt
}
signedHeaders, err := v4.signWithBody(req.HTTPRequest, req.Body, name, region, req.ExpireTime, signingTime)
signedHeaders, err := v4.signWithBody(req.HTTPRequest, req.GetBody(),
name, region, req.ExpireTime, signingTime,
)
if err != nil {
req.Error = err
req.SignedHeaderVals = nil
@@ -356,7 +436,7 @@ func signSDKRequestWithCurrTime(req *request.Request, curTimeFn func() time.Time
req.LastSignedAt = curTimeFn()
}
const logSignInfoMsg = `DEBUG: Request Signiture:
const logSignInfoMsg = `DEBUG: Request Signature:
---[ CANONICAL STRING ]-----------------------------
%s
---[ STRING TO SIGN ]--------------------------------
@@ -492,17 +572,10 @@ func (ctx *signingCtx) buildCanonicalHeaders(r rule, header http.Header) {
func (ctx *signingCtx) buildCanonicalString() {
ctx.Request.URL.RawQuery = strings.Replace(ctx.Query.Encode(), "+", "%20", -1)
uri := ctx.Request.URL.Opaque
if uri != "" {
uri = "/" + strings.Join(strings.Split(uri, "/")[3:], "/")
} else {
uri = ctx.Request.URL.Path
}
if uri == "" {
uri = "/"
}
if ctx.ServiceName != "s3" {
uri := getURIPath(ctx.Request.URL)
if !ctx.DisableURIPathEscaping {
uri = rest.EscapePath(uri, false)
}

View File

@@ -1,353 +0,0 @@
package v4
import (
"io"
"net/http"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/awstesting"
)
func TestStripExcessHeaders(t *testing.T) {
vals := []string{
"123",
"1 2 3",
" 1 2 3",
"1 2 3",
"1 23",
"1 2 3",
"1 2 ",
" 1 2 ",
}
expected := []string{
"123",
"1 2 3",
"1 2 3",
"1 2 3",
"1 23",
"1 2 3",
"1 2",
"1 2",
}
newVals := stripExcessSpaces(vals)
for i := 0; i < len(newVals); i++ {
assert.Equal(t, expected[i], newVals[i], "test: %d", i)
}
}
func buildRequest(serviceName, region, body string) (*http.Request, io.ReadSeeker) {
endpoint := "https://" + serviceName + "." + region + ".amazonaws.com"
reader := strings.NewReader(body)
req, _ := http.NewRequest("POST", endpoint, reader)
req.URL.Opaque = "//example.org/bucket/key-._~,!@#$%^&*()"
req.Header.Add("X-Amz-Target", "prefix.Operation")
req.Header.Add("Content-Type", "application/x-amz-json-1.0")
req.Header.Add("Content-Length", string(len(body)))
req.Header.Add("X-Amz-Meta-Other-Header", "some-value=!@#$%^&* (+)")
req.Header.Add("X-Amz-Meta-Other-Header_With_Underscore", "some-value=!@#$%^&* (+)")
req.Header.Add("X-amz-Meta-Other-Header_With_Underscore", "some-value=!@#$%^&* (+)")
return req, reader
}
func buildSigner() Signer {
return Signer{
Credentials: credentials.NewStaticCredentials("AKID", "SECRET", "SESSION"),
}
}
func removeWS(text string) string {
text = strings.Replace(text, " ", "", -1)
text = strings.Replace(text, "\n", "", -1)
text = strings.Replace(text, "\t", "", -1)
return text
}
func assertEqual(t *testing.T, expected, given string) {
if removeWS(expected) != removeWS(given) {
t.Errorf("\nExpected: %s\nGiven: %s", expected, given)
}
}
func TestPresignRequest(t *testing.T) {
req, body := buildRequest("dynamodb", "us-east-1", "{}")
signer := buildSigner()
signer.Presign(req, body, "dynamodb", "us-east-1", 300*time.Second, time.Unix(0, 0))
expectedDate := "19700101T000000Z"
expectedHeaders := "content-length;content-type;host;x-amz-meta-other-header;x-amz-meta-other-header_with_underscore"
expectedSig := "ea7856749041f727690c580569738282e99c79355fe0d8f125d3b5535d2ece83"
expectedCred := "AKID/19700101/us-east-1/dynamodb/aws4_request"
expectedTarget := "prefix.Operation"
q := req.URL.Query()
assert.Equal(t, expectedSig, q.Get("X-Amz-Signature"))
assert.Equal(t, expectedCred, q.Get("X-Amz-Credential"))
assert.Equal(t, expectedHeaders, q.Get("X-Amz-SignedHeaders"))
assert.Equal(t, expectedDate, q.Get("X-Amz-Date"))
assert.Empty(t, q.Get("X-Amz-Meta-Other-Header"))
assert.Equal(t, expectedTarget, q.Get("X-Amz-Target"))
}
func TestSignRequest(t *testing.T) {
req, body := buildRequest("dynamodb", "us-east-1", "{}")
signer := buildSigner()
signer.Sign(req, body, "dynamodb", "us-east-1", time.Unix(0, 0))
expectedDate := "19700101T000000Z"
expectedSig := "AWS4-HMAC-SHA256 Credential=AKID/19700101/us-east-1/dynamodb/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-meta-other-header;x-amz-meta-other-header_with_underscore;x-amz-security-token;x-amz-target, Signature=ea766cabd2ec977d955a3c2bae1ae54f4515d70752f2207618396f20aa85bd21"
q := req.Header
assert.Equal(t, expectedSig, q.Get("Authorization"))
assert.Equal(t, expectedDate, q.Get("X-Amz-Date"))
}
func TestSignBodyS3(t *testing.T) {
req, body := buildRequest("s3", "us-east-1", "hello")
signer := buildSigner()
signer.Sign(req, body, "s3", "us-east-1", time.Now())
hash := req.Header.Get("X-Amz-Content-Sha256")
assert.Equal(t, "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824", hash)
}
func TestSignBodyGlacier(t *testing.T) {
req, body := buildRequest("glacier", "us-east-1", "hello")
signer := buildSigner()
signer.Sign(req, body, "glacier", "us-east-1", time.Now())
hash := req.Header.Get("X-Amz-Content-Sha256")
assert.Equal(t, "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824", hash)
}
func TestPresignEmptyBodyS3(t *testing.T) {
req, body := buildRequest("s3", "us-east-1", "hello")
signer := buildSigner()
signer.Presign(req, body, "s3", "us-east-1", 5*time.Minute, time.Now())
hash := req.Header.Get("X-Amz-Content-Sha256")
assert.Equal(t, "UNSIGNED-PAYLOAD", hash)
}
func TestSignPrecomputedBodyChecksum(t *testing.T) {
req, body := buildRequest("dynamodb", "us-east-1", "hello")
req.Header.Set("X-Amz-Content-Sha256", "PRECOMPUTED")
signer := buildSigner()
signer.Sign(req, body, "dynamodb", "us-east-1", time.Now())
hash := req.Header.Get("X-Amz-Content-Sha256")
assert.Equal(t, "PRECOMPUTED", hash)
}
func TestAnonymousCredentials(t *testing.T) {
svc := awstesting.NewClient(&aws.Config{Credentials: credentials.AnonymousCredentials})
r := svc.NewRequest(
&request.Operation{
Name: "BatchGetItem",
HTTPMethod: "POST",
HTTPPath: "/",
},
nil,
nil,
)
SignSDKRequest(r)
urlQ := r.HTTPRequest.URL.Query()
assert.Empty(t, urlQ.Get("X-Amz-Signature"))
assert.Empty(t, urlQ.Get("X-Amz-Credential"))
assert.Empty(t, urlQ.Get("X-Amz-SignedHeaders"))
assert.Empty(t, urlQ.Get("X-Amz-Date"))
hQ := r.HTTPRequest.Header
assert.Empty(t, hQ.Get("Authorization"))
assert.Empty(t, hQ.Get("X-Amz-Date"))
}
func TestIgnoreResignRequestWithValidCreds(t *testing.T) {
svc := awstesting.NewClient(&aws.Config{
Credentials: credentials.NewStaticCredentials("AKID", "SECRET", "SESSION"),
Region: aws.String("us-west-2"),
})
r := svc.NewRequest(
&request.Operation{
Name: "BatchGetItem",
HTTPMethod: "POST",
HTTPPath: "/",
},
nil,
nil,
)
SignSDKRequest(r)
sig := r.HTTPRequest.Header.Get("Authorization")
SignSDKRequest(r)
assert.Equal(t, sig, r.HTTPRequest.Header.Get("Authorization"))
}
func TestIgnorePreResignRequestWithValidCreds(t *testing.T) {
svc := awstesting.NewClient(&aws.Config{
Credentials: credentials.NewStaticCredentials("AKID", "SECRET", "SESSION"),
Region: aws.String("us-west-2"),
})
r := svc.NewRequest(
&request.Operation{
Name: "BatchGetItem",
HTTPMethod: "POST",
HTTPPath: "/",
},
nil,
nil,
)
r.ExpireTime = time.Minute * 10
SignSDKRequest(r)
sig := r.HTTPRequest.Header.Get("X-Amz-Signature")
SignSDKRequest(r)
assert.Equal(t, sig, r.HTTPRequest.Header.Get("X-Amz-Signature"))
}
func TestResignRequestExpiredCreds(t *testing.T) {
creds := credentials.NewStaticCredentials("AKID", "SECRET", "SESSION")
svc := awstesting.NewClient(&aws.Config{Credentials: creds})
r := svc.NewRequest(
&request.Operation{
Name: "BatchGetItem",
HTTPMethod: "POST",
HTTPPath: "/",
},
nil,
nil,
)
SignSDKRequest(r)
querySig := r.HTTPRequest.Header.Get("Authorization")
var origSignedHeaders string
for _, p := range strings.Split(querySig, ", ") {
if strings.HasPrefix(p, "SignedHeaders=") {
origSignedHeaders = p[len("SignedHeaders="):]
break
}
}
assert.NotEmpty(t, origSignedHeaders)
assert.NotContains(t, origSignedHeaders, "authorization")
origSignedAt := r.LastSignedAt
creds.Expire()
signSDKRequestWithCurrTime(r, func() time.Time {
// Simulate one second has passed so that signature's date changes
// when it is resigned.
return time.Now().Add(1 * time.Second)
})
updatedQuerySig := r.HTTPRequest.Header.Get("Authorization")
assert.NotEqual(t, querySig, updatedQuerySig)
var updatedSignedHeaders string
for _, p := range strings.Split(updatedQuerySig, ", ") {
if strings.HasPrefix(p, "SignedHeaders=") {
updatedSignedHeaders = p[len("SignedHeaders="):]
break
}
}
assert.NotEmpty(t, updatedSignedHeaders)
assert.NotContains(t, updatedQuerySig, "authorization")
assert.NotEqual(t, origSignedAt, r.LastSignedAt)
}
func TestPreResignRequestExpiredCreds(t *testing.T) {
provider := &credentials.StaticProvider{Value: credentials.Value{
AccessKeyID: "AKID",
SecretAccessKey: "SECRET",
SessionToken: "SESSION",
}}
creds := credentials.NewCredentials(provider)
svc := awstesting.NewClient(&aws.Config{Credentials: creds})
r := svc.NewRequest(
&request.Operation{
Name: "BatchGetItem",
HTTPMethod: "POST",
HTTPPath: "/",
},
nil,
nil,
)
r.ExpireTime = time.Minute * 10
SignSDKRequest(r)
querySig := r.HTTPRequest.URL.Query().Get("X-Amz-Signature")
signedHeaders := r.HTTPRequest.URL.Query().Get("X-Amz-SignedHeaders")
assert.NotEmpty(t, signedHeaders)
origSignedAt := r.LastSignedAt
creds.Expire()
signSDKRequestWithCurrTime(r, func() time.Time {
// Simulate the request occurred 15 minutes in the past
return time.Now().Add(-48 * time.Hour)
})
assert.NotEqual(t, querySig, r.HTTPRequest.URL.Query().Get("X-Amz-Signature"))
resignedHeaders := r.HTTPRequest.URL.Query().Get("X-Amz-SignedHeaders")
assert.Equal(t, signedHeaders, resignedHeaders)
assert.NotContains(t, signedHeaders, "x-amz-signedHeaders")
assert.NotEqual(t, origSignedAt, r.LastSignedAt)
}
func TestResignRequestExpiredRequest(t *testing.T) {
creds := credentials.NewStaticCredentials("AKID", "SECRET", "SESSION")
svc := awstesting.NewClient(&aws.Config{Credentials: creds})
r := svc.NewRequest(
&request.Operation{
Name: "BatchGetItem",
HTTPMethod: "POST",
HTTPPath: "/",
},
nil,
nil,
)
SignSDKRequest(r)
querySig := r.HTTPRequest.Header.Get("Authorization")
origSignedAt := r.LastSignedAt
signSDKRequestWithCurrTime(r, func() time.Time {
// Simulate the request occurred 15 minutes in the past
return time.Now().Add(15 * time.Minute)
})
assert.NotEqual(t, querySig, r.HTTPRequest.Header.Get("Authorization"))
assert.NotEqual(t, origSignedAt, r.LastSignedAt)
}
func BenchmarkPresignRequest(b *testing.B) {
signer := buildSigner()
req, body := buildRequest("dynamodb", "us-east-1", "{}")
for i := 0; i < b.N; i++ {
signer.Presign(req, body, "dynamodb", "us-east-1", 300*time.Second, time.Now())
}
}
func BenchmarkSignRequest(b *testing.B) {
signer := buildSigner()
req, body := buildRequest("dynamodb", "us-east-1", "{}")
for i := 0; i < b.N; i++ {
signer.Sign(req, body, "dynamodb", "us-east-1", time.Now())
}
}
func BenchmarkStripExcessSpaces(b *testing.B) {
vals := []string{
`AWS4-HMAC-SHA256 Credential=AKIDFAKEIDFAKEID/20160628/us-west-2/s3/aws4_request, SignedHeaders=host;x-amz-date, Signature=1234567890abcdef1234567890abcdef1234567890abcdef`,
`123 321 123 321`,
` 123 321 123 321 `,
}
b.ResetTimer()
for i := 0; i < b.N; i++ {
stripExcessSpaces(vals)
}
}

View File

@@ -1,75 +0,0 @@
package aws
import (
"math/rand"
"testing"
"github.com/stretchr/testify/assert"
)
func TestWriteAtBuffer(t *testing.T) {
b := &WriteAtBuffer{}
n, err := b.WriteAt([]byte{1}, 0)
assert.NoError(t, err)
assert.Equal(t, 1, n)
n, err = b.WriteAt([]byte{1, 1, 1}, 5)
assert.NoError(t, err)
assert.Equal(t, 3, n)
n, err = b.WriteAt([]byte{2}, 1)
assert.NoError(t, err)
assert.Equal(t, 1, n)
n, err = b.WriteAt([]byte{3}, 2)
assert.NoError(t, err)
assert.Equal(t, 1, n)
assert.Equal(t, []byte{1, 2, 3, 0, 0, 1, 1, 1}, b.Bytes())
}
func BenchmarkWriteAtBuffer(b *testing.B) {
buf := &WriteAtBuffer{}
r := rand.New(rand.NewSource(1))
b.ResetTimer()
for i := 0; i < b.N; i++ {
to := r.Intn(10) * 4096
bs := make([]byte, to)
buf.WriteAt(bs, r.Int63n(10)*4096)
}
}
func BenchmarkWriteAtBufferOrderedWrites(b *testing.B) {
// test the performance of a WriteAtBuffer when written in an
// ordered fashion. This is similar to the behavior of the
// s3.Downloader, since downloads the first chunk of the file, then
// the second, and so on.
//
// This test simulates a 150MB file being written in 30 ordered 5MB chunks.
chunk := int64(5e6)
max := chunk * 30
// we'll write the same 5MB chunk every time
tmp := make([]byte, chunk)
for i := 0; i < b.N; i++ {
buf := &WriteAtBuffer{}
for i := int64(0); i < max; i += chunk {
buf.WriteAt(tmp, i)
}
}
}
func BenchmarkWriteAtBufferParallel(b *testing.B) {
buf := &WriteAtBuffer{}
r := rand.New(rand.NewSource(1))
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
to := r.Intn(10) * 4096
bs := make([]byte, to)
buf.WriteAt(bs, r.Int63n(10)*4096)
}
})
}

View File

@@ -5,4 +5,4 @@ package aws
const SDKName = "aws-sdk-go"
// SDKVersion is the version of this SDK
const SDKVersion = "1.4.1"
const SDKVersion = "1.5.8"

View File

@@ -1,19 +0,0 @@
{
"ImportPath": "github.com/aws/aws-sdk-go/awsmigrate/awsmigrate-renamer",
"GoVersion": "go1.6",
"GodepVersion": "v60",
"Deps": [
{
"ImportPath": "golang.org/x/tools/go/ast/astutil",
"Rev": "b75b3f5cd5d50fbb1fb88ce784d2e7cca17bba8a"
},
{
"ImportPath": "golang.org/x/tools/go/buildutil",
"Rev": "b75b3f5cd5d50fbb1fb88ce784d2e7cca17bba8a"
},
{
"ImportPath": "golang.org/x/tools/go/loader",
"Rev": "b75b3f5cd5d50fbb1fb88ce784d2e7cca17bba8a"
}
]
}

View File

@@ -1,5 +0,0 @@
This directory tree is generated automatically by godep.
Please do not edit.
See https://github.com/tools/godep for more information.

View File

@@ -1,200 +0,0 @@
// +build go1.5
package main
import (
"bytes"
"go/format"
"io"
"os"
"path/filepath"
"sort"
"strings"
"text/template"
"github.com/aws/aws-sdk-go/private/model/api"
)
type pkg struct {
oldAPI *api.API
newAPI *api.API
shapes map[string]*shapentry
operations map[string]*opentry
}
type shapentry struct {
oldShape *api.Shape
newShape *api.Shape
}
type opentry struct {
oldName string
newName string
}
type packageRenames struct {
Shapes map[string]string
Operations map[string]string
Fields map[string]string
}
var exportMap = map[string]*packageRenames{}
func generateRenames(w io.Writer) error {
tmpl, err := template.New("renames").Parse(t)
if err != nil {
return err
}
out := bytes.NewBuffer(nil)
if err = tmpl.Execute(out, exportMap); err != nil {
return err
}
b, err := format.Source(bytes.TrimSpace(out.Bytes()))
if err != nil {
return err
}
_, err = io.Copy(w, bytes.NewReader(b))
return err
}
const t = `
package rename
// THIS FILE IS AUTOMATICALLY GENERATED. DO NOT EDIT.
var renamedPackages = map[string]*packageRenames{
{{ range $key, $entry := . }}"{{ $key }}": &packageRenames{
operations: map[string]string{
{{ range $old, $new := $entry.Operations }}"{{ $old }}": "{{ $new }}",
{{ end }}
},
shapes: map[string]string{
{{ range $old, $new := $entry.Shapes }}"{{ $old }}": "{{ $new }}",
{{ end }}
},
fields: map[string]string{
{{ range $old, $new := $entry.Fields }}"{{ $old }}": "{{ $new }}",
{{ end }}
},
},
{{ end }}
}
`
func (p *pkg) buildRenames() {
pkgName := "github.com/aws/aws-sdk-go/service/" + p.oldAPI.PackageName()
if exportMap[pkgName] == nil {
exportMap[pkgName] = &packageRenames{map[string]string{}, map[string]string{}, map[string]string{}}
}
ifacename := "github.com/aws/aws-sdk-go/service/" + p.oldAPI.PackageName() + "/" +
p.oldAPI.InterfacePackageName()
if exportMap[ifacename] == nil {
exportMap[ifacename] = &packageRenames{map[string]string{}, map[string]string{}, map[string]string{}}
}
for _, entry := range p.operations {
if entry.oldName != entry.newName {
pkgNames := []string{pkgName, ifacename}
for _, p := range pkgNames {
exportMap[p].Operations[entry.oldName] = entry.newName
exportMap[p].Operations[entry.oldName+"Request"] = entry.newName + "Request"
exportMap[p].Operations[entry.oldName+"Pages"] = entry.newName + "Pages"
}
}
}
for _, entry := range p.shapes {
if entry.oldShape.Type == "structure" {
if entry.oldShape.ShapeName != entry.newShape.ShapeName {
exportMap[pkgName].Shapes[entry.oldShape.ShapeName] = entry.newShape.ShapeName
}
for _, n := range entry.oldShape.MemberNames() {
for _, m := range entry.newShape.MemberNames() {
if n != m && strings.ToLower(n) == strings.ToLower(m) {
exportMap[pkgName].Fields[n] = m
}
}
}
}
}
}
func load(file string) *pkg {
p := &pkg{&api.API{}, &api.API{}, map[string]*shapentry{}, map[string]*opentry{}}
p.oldAPI.Attach(file)
p.oldAPI.Setup()
p.newAPI.Attach(file)
p.newAPI.Setup()
for _, name := range p.oldAPI.OperationNames() {
p.operations[strings.ToLower(name)] = &opentry{oldName: name}
}
for _, name := range p.newAPI.OperationNames() {
p.operations[strings.ToLower(name)].newName = name
}
for _, shape := range p.oldAPI.ShapeList() {
p.shapes[strings.ToLower(shape.ShapeName)] = &shapentry{oldShape: shape}
}
for _, shape := range p.newAPI.ShapeList() {
if _, ok := p.shapes[strings.ToLower(shape.ShapeName)]; !ok {
panic("missing shape " + shape.ShapeName)
}
p.shapes[strings.ToLower(shape.ShapeName)].newShape = shape
}
return p
}
var excludeServices = map[string]struct{}{
"simpledb": {},
"importexport": {},
}
func main() {
files, _ := filepath.Glob("../../apis/*/*/api-2.json")
sort.Strings(files)
// Remove old API versions from list
m := map[string]bool{}
for i := range files {
idx := len(files) - 1 - i
parts := strings.Split(files[idx], string(filepath.Separator))
svc := parts[len(parts)-3] // service name is 2nd-to-last component
if m[svc] {
files[idx] = "" // wipe this one out if we already saw the service
}
m[svc] = true
}
for i := range files {
file := files[i]
if file == "" { // empty file
continue
}
if g := load(file); g != nil {
if _, ok := excludeServices[g.oldAPI.PackageName()]; !ok {
g.buildRenames()
}
}
}
outfile, err := os.Create("rename/renames.go")
if err != nil {
panic(err)
}
if err := generateRenames(outfile); err != nil {
panic(err)
}
}

View File

@@ -1,116 +0,0 @@
// +build go1.5
package rename
import (
"bytes"
"flag"
"fmt"
"go/format"
"go/parser"
"go/token"
"go/types"
"io/ioutil"
"golang.org/x/tools/go/loader"
)
var dryRun = flag.Bool("dryrun", false, "Dry run")
var verbose = flag.Bool("verbose", false, "Verbose")
type packageRenames struct {
operations map[string]string
shapes map[string]string
fields map[string]string
}
type renamer struct {
*loader.Program
files map[*token.File]bool
}
// ParsePathsFromArgs parses arguments from command line and looks at import
// paths to rename objects.
func ParsePathsFromArgs() {
flag.Parse()
for _, dir := range flag.Args() {
var conf loader.Config
conf.ParserMode = parser.ParseComments
conf.ImportWithTests(dir)
prog, err := conf.Load()
if err != nil {
panic(err)
}
r := renamer{prog, map[*token.File]bool{}}
r.parse()
if !*dryRun {
r.write()
}
}
}
func (r *renamer) dryInfo() string {
if *dryRun {
return "[DRY-RUN]"
}
return "[!]"
}
func (r *renamer) printf(msg string, args ...interface{}) {
if *verbose {
fmt.Printf(msg, args...)
}
}
func (r *renamer) parse() {
for _, pkg := range r.InitialPackages() {
r.parseUses(pkg)
}
}
func (r *renamer) write() {
for _, pkg := range r.InitialPackages() {
for _, f := range pkg.Files {
tokenFile := r.Fset.File(f.Pos())
if r.files[tokenFile] {
var buf bytes.Buffer
format.Node(&buf, r.Fset, f)
if err := ioutil.WriteFile(tokenFile.Name(), buf.Bytes(), 0644); err != nil {
panic(err)
}
}
}
}
}
func (r *renamer) parseUses(pkg *loader.PackageInfo) {
for k, v := range pkg.Uses {
if v.Pkg() != nil {
pkgPath := v.Pkg().Path()
if renames, ok := renamedPackages[pkgPath]; ok {
name := k.Name
switch t := v.(type) {
case *types.Func:
if newName, ok := renames.operations[t.Name()]; ok && newName != name {
r.printf("%s Rename [OPERATION]: %q -> %q\n", r.dryInfo(), name, newName)
r.files[r.Fset.File(k.Pos())] = true
k.Name = newName
}
case *types.TypeName:
if newName, ok := renames.shapes[name]; ok && newName != name {
r.printf("%s Rename [SHAPE]: %q -> %q\n", r.dryInfo(), t.Name(), newName)
r.files[r.Fset.File(k.Pos())] = true
k.Name = newName
}
case *types.Var:
if newName, ok := renames.fields[name]; ok && newName != name {
r.printf("%s Rename [FIELD]: %q -> %q\n", r.dryInfo(), t.Name(), newName)
r.files[r.Fset.File(k.Pos())] = true
k.Name = newName
}
}
}
}
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,45 +0,0 @@
// +build go1.5
package main
//go:generate go run gen/gen.go
import (
"os"
"os/exec"
"path/filepath"
"strings"
"github.com/aws/aws-sdk-go/awsmigrate/awsmigrate-renamer/rename"
)
var safeTag = "4e554f77f00d527b452c68a46f2e68595284121b"
func main() {
gopath := os.Getenv("GOPATH")
if gopath == "" {
panic("GOPATH not set!")
}
gopath = strings.Split(gopath, ":")[0]
// change directory to SDK
err := os.Chdir(filepath.Join(gopath, "src", "github.com", "aws", "aws-sdk-go"))
if err != nil {
panic("Cannot find SDK repository")
}
// store orig HEAD
head, err := exec.Command("git", "rev-parse", "--abbrev-ref", "HEAD").Output()
if err != nil {
panic("Cannot find SDK repository")
}
origHEAD := strings.Trim(string(head), " \r\n")
// checkout to safe tag and run conversion
exec.Command("git", "checkout", safeTag).Run()
defer func() {
exec.Command("git", "checkout", origHEAD).Run()
}()
rename.ParsePathsFromArgs()
}

View File

@@ -1,27 +0,0 @@
Copyright (c) 2009 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

View File

@@ -1,22 +0,0 @@
Additional IP Rights Grant (Patents)
"This implementation" means the copyrightable works distributed by
Google as part of the Go project.
Google hereby grants to You a perpetual, worldwide, non-exclusive,
no-charge, royalty-free, irrevocable (except as stated in this section)
patent license to make, have made, use, offer to sell, sell, import,
transfer and otherwise run, modify and propagate the contents of this
implementation of Go, where such license applies only to those patent
claims, both currently owned or controlled by Google and acquired in
the future, licensable by Google that are necessarily infringed by this
implementation of Go. This grant does not include claims that would be
infringed only as a consequence of further modification of this
implementation. If you or your agent or exclusive licensee institute or
order or agree to the institution of patent litigation against any
entity (including a cross-claim or counterclaim in a lawsuit) alleging
that this implementation of Go or any code incorporated within this
implementation of Go constitutes direct or contributory patent
infringement, or inducement of patent infringement, then any patent
rights granted to you under this License for this implementation of Go
shall terminate as of the date such litigation is filed.

View File

@@ -1,624 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package astutil
// This file defines utilities for working with source positions.
import (
"fmt"
"go/ast"
"go/token"
"sort"
)
// PathEnclosingInterval returns the node that encloses the source
// interval [start, end), and all its ancestors up to the AST root.
//
// The definition of "enclosing" used by this function considers
// additional whitespace abutting a node to be enclosed by it.
// In this example:
//
// z := x + y // add them
// <-A->
// <----B----->
//
// the ast.BinaryExpr(+) node is considered to enclose interval B
// even though its [Pos()..End()) is actually only interval A.
// This behaviour makes user interfaces more tolerant of imperfect
// input.
//
// This function treats tokens as nodes, though they are not included
// in the result. e.g. PathEnclosingInterval("+") returns the
// enclosing ast.BinaryExpr("x + y").
//
// If start==end, the 1-char interval following start is used instead.
//
// The 'exact' result is true if the interval contains only path[0]
// and perhaps some adjacent whitespace. It is false if the interval
// overlaps multiple children of path[0], or if it contains only
// interior whitespace of path[0].
// In this example:
//
// z := x + y // add them
// <--C--> <---E-->
// ^
// D
//
// intervals C, D and E are inexact. C is contained by the
// z-assignment statement, because it spans three of its children (:=,
// x, +). So too is the 1-char interval D, because it contains only
// interior whitespace of the assignment. E is considered interior
// whitespace of the BlockStmt containing the assignment.
//
// Precondition: [start, end) both lie within the same file as root.
// TODO(adonovan): return (nil, false) in this case and remove precond.
// Requires FileSet; see loader.tokenFileContainsPos.
//
// Postcondition: path is never nil; it always contains at least 'root'.
//
func PathEnclosingInterval(root *ast.File, start, end token.Pos) (path []ast.Node, exact bool) {
// fmt.Printf("EnclosingInterval %d %d\n", start, end) // debugging
// Precondition: node.[Pos..End) and adjoining whitespace contain [start, end).
var visit func(node ast.Node) bool
visit = func(node ast.Node) bool {
path = append(path, node)
nodePos := node.Pos()
nodeEnd := node.End()
// fmt.Printf("visit(%T, %d, %d)\n", node, nodePos, nodeEnd) // debugging
// Intersect [start, end) with interval of node.
if start < nodePos {
start = nodePos
}
if end > nodeEnd {
end = nodeEnd
}
// Find sole child that contains [start, end).
children := childrenOf(node)
l := len(children)
for i, child := range children {
// [childPos, childEnd) is unaugmented interval of child.
childPos := child.Pos()
childEnd := child.End()
// [augPos, augEnd) is whitespace-augmented interval of child.
augPos := childPos
augEnd := childEnd
if i > 0 {
augPos = children[i-1].End() // start of preceding whitespace
}
if i < l-1 {
nextChildPos := children[i+1].Pos()
// Does [start, end) lie between child and next child?
if start >= augEnd && end <= nextChildPos {
return false // inexact match
}
augEnd = nextChildPos // end of following whitespace
}
// fmt.Printf("\tchild %d: [%d..%d)\tcontains interval [%d..%d)?\n",
// i, augPos, augEnd, start, end) // debugging
// Does augmented child strictly contain [start, end)?
if augPos <= start && end <= augEnd {
_, isToken := child.(tokenNode)
return isToken || visit(child)
}
// Does [start, end) overlap multiple children?
// i.e. left-augmented child contains start
// but LR-augmented child does not contain end.
if start < childEnd && end > augEnd {
break
}
}
// No single child contained [start, end),
// so node is the result. Is it exact?
// (It's tempting to put this condition before the
// child loop, but it gives the wrong result in the
// case where a node (e.g. ExprStmt) and its sole
// child have equal intervals.)
if start == nodePos && end == nodeEnd {
return true // exact match
}
return false // inexact: overlaps multiple children
}
if start > end {
start, end = end, start
}
if start < root.End() && end > root.Pos() {
if start == end {
end = start + 1 // empty interval => interval of size 1
}
exact = visit(root)
// Reverse the path:
for i, l := 0, len(path); i < l/2; i++ {
path[i], path[l-1-i] = path[l-1-i], path[i]
}
} else {
// Selection lies within whitespace preceding the
// first (or following the last) declaration in the file.
// The result nonetheless always includes the ast.File.
path = append(path, root)
}
return
}
// tokenNode is a dummy implementation of ast.Node for a single token.
// They are used transiently by PathEnclosingInterval but never escape
// this package.
//
type tokenNode struct {
pos token.Pos
end token.Pos
}
func (n tokenNode) Pos() token.Pos {
return n.pos
}
func (n tokenNode) End() token.Pos {
return n.end
}
func tok(pos token.Pos, len int) ast.Node {
return tokenNode{pos, pos + token.Pos(len)}
}
// childrenOf returns the direct non-nil children of ast.Node n.
// It may include fake ast.Node implementations for bare tokens.
// it is not safe to call (e.g.) ast.Walk on such nodes.
//
func childrenOf(n ast.Node) []ast.Node {
var children []ast.Node
// First add nodes for all true subtrees.
ast.Inspect(n, func(node ast.Node) bool {
if node == n { // push n
return true // recur
}
if node != nil { // push child
children = append(children, node)
}
return false // no recursion
})
// Then add fake Nodes for bare tokens.
switch n := n.(type) {
case *ast.ArrayType:
children = append(children,
tok(n.Lbrack, len("[")),
tok(n.Elt.End(), len("]")))
case *ast.AssignStmt:
children = append(children,
tok(n.TokPos, len(n.Tok.String())))
case *ast.BasicLit:
children = append(children,
tok(n.ValuePos, len(n.Value)))
case *ast.BinaryExpr:
children = append(children, tok(n.OpPos, len(n.Op.String())))
case *ast.BlockStmt:
children = append(children,
tok(n.Lbrace, len("{")),
tok(n.Rbrace, len("}")))
case *ast.BranchStmt:
children = append(children,
tok(n.TokPos, len(n.Tok.String())))
case *ast.CallExpr:
children = append(children,
tok(n.Lparen, len("(")),
tok(n.Rparen, len(")")))
if n.Ellipsis != 0 {
children = append(children, tok(n.Ellipsis, len("...")))
}
case *ast.CaseClause:
if n.List == nil {
children = append(children,
tok(n.Case, len("default")))
} else {
children = append(children,
tok(n.Case, len("case")))
}
children = append(children, tok(n.Colon, len(":")))
case *ast.ChanType:
switch n.Dir {
case ast.RECV:
children = append(children, tok(n.Begin, len("<-chan")))
case ast.SEND:
children = append(children, tok(n.Begin, len("chan<-")))
case ast.RECV | ast.SEND:
children = append(children, tok(n.Begin, len("chan")))
}
case *ast.CommClause:
if n.Comm == nil {
children = append(children,
tok(n.Case, len("default")))
} else {
children = append(children,
tok(n.Case, len("case")))
}
children = append(children, tok(n.Colon, len(":")))
case *ast.Comment:
// nop
case *ast.CommentGroup:
// nop
case *ast.CompositeLit:
children = append(children,
tok(n.Lbrace, len("{")),
tok(n.Rbrace, len("{")))
case *ast.DeclStmt:
// nop
case *ast.DeferStmt:
children = append(children,
tok(n.Defer, len("defer")))
case *ast.Ellipsis:
children = append(children,
tok(n.Ellipsis, len("...")))
case *ast.EmptyStmt:
// nop
case *ast.ExprStmt:
// nop
case *ast.Field:
// TODO(adonovan): Field.{Doc,Comment,Tag}?
case *ast.FieldList:
children = append(children,
tok(n.Opening, len("(")),
tok(n.Closing, len(")")))
case *ast.File:
// TODO test: Doc
children = append(children,
tok(n.Package, len("package")))
case *ast.ForStmt:
children = append(children,
tok(n.For, len("for")))
case *ast.FuncDecl:
// TODO(adonovan): FuncDecl.Comment?
// Uniquely, FuncDecl breaks the invariant that
// preorder traversal yields tokens in lexical order:
// in fact, FuncDecl.Recv precedes FuncDecl.Type.Func.
//
// As a workaround, we inline the case for FuncType
// here and order things correctly.
//
children = nil // discard ast.Walk(FuncDecl) info subtrees
children = append(children, tok(n.Type.Func, len("func")))
if n.Recv != nil {
children = append(children, n.Recv)
}
children = append(children, n.Name)
if n.Type.Params != nil {
children = append(children, n.Type.Params)
}
if n.Type.Results != nil {
children = append(children, n.Type.Results)
}
if n.Body != nil {
children = append(children, n.Body)
}
case *ast.FuncLit:
// nop
case *ast.FuncType:
if n.Func != 0 {
children = append(children,
tok(n.Func, len("func")))
}
case *ast.GenDecl:
children = append(children,
tok(n.TokPos, len(n.Tok.String())))
if n.Lparen != 0 {
children = append(children,
tok(n.Lparen, len("(")),
tok(n.Rparen, len(")")))
}
case *ast.GoStmt:
children = append(children,
tok(n.Go, len("go")))
case *ast.Ident:
children = append(children,
tok(n.NamePos, len(n.Name)))
case *ast.IfStmt:
children = append(children,
tok(n.If, len("if")))
case *ast.ImportSpec:
// TODO(adonovan): ImportSpec.{Doc,EndPos}?
case *ast.IncDecStmt:
children = append(children,
tok(n.TokPos, len(n.Tok.String())))
case *ast.IndexExpr:
children = append(children,
tok(n.Lbrack, len("{")),
tok(n.Rbrack, len("}")))
case *ast.InterfaceType:
children = append(children,
tok(n.Interface, len("interface")))
case *ast.KeyValueExpr:
children = append(children,
tok(n.Colon, len(":")))
case *ast.LabeledStmt:
children = append(children,
tok(n.Colon, len(":")))
case *ast.MapType:
children = append(children,
tok(n.Map, len("map")))
case *ast.ParenExpr:
children = append(children,
tok(n.Lparen, len("(")),
tok(n.Rparen, len(")")))
case *ast.RangeStmt:
children = append(children,
tok(n.For, len("for")),
tok(n.TokPos, len(n.Tok.String())))
case *ast.ReturnStmt:
children = append(children,
tok(n.Return, len("return")))
case *ast.SelectStmt:
children = append(children,
tok(n.Select, len("select")))
case *ast.SelectorExpr:
// nop
case *ast.SendStmt:
children = append(children,
tok(n.Arrow, len("<-")))
case *ast.SliceExpr:
children = append(children,
tok(n.Lbrack, len("[")),
tok(n.Rbrack, len("]")))
case *ast.StarExpr:
children = append(children, tok(n.Star, len("*")))
case *ast.StructType:
children = append(children, tok(n.Struct, len("struct")))
case *ast.SwitchStmt:
children = append(children, tok(n.Switch, len("switch")))
case *ast.TypeAssertExpr:
children = append(children,
tok(n.Lparen-1, len(".")),
tok(n.Lparen, len("(")),
tok(n.Rparen, len(")")))
case *ast.TypeSpec:
// TODO(adonovan): TypeSpec.{Doc,Comment}?
case *ast.TypeSwitchStmt:
children = append(children, tok(n.Switch, len("switch")))
case *ast.UnaryExpr:
children = append(children, tok(n.OpPos, len(n.Op.String())))
case *ast.ValueSpec:
// TODO(adonovan): ValueSpec.{Doc,Comment}?
case *ast.BadDecl, *ast.BadExpr, *ast.BadStmt:
// nop
}
// TODO(adonovan): opt: merge the logic of ast.Inspect() into
// the switch above so we can make interleaved callbacks for
// both Nodes and Tokens in the right order and avoid the need
// to sort.
sort.Sort(byPos(children))
return children
}
type byPos []ast.Node
func (sl byPos) Len() int {
return len(sl)
}
func (sl byPos) Less(i, j int) bool {
return sl[i].Pos() < sl[j].Pos()
}
func (sl byPos) Swap(i, j int) {
sl[i], sl[j] = sl[j], sl[i]
}
// NodeDescription returns a description of the concrete type of n suitable
// for a user interface.
//
// TODO(adonovan): in some cases (e.g. Field, FieldList, Ident,
// StarExpr) we could be much more specific given the path to the AST
// root. Perhaps we should do that.
//
func NodeDescription(n ast.Node) string {
switch n := n.(type) {
case *ast.ArrayType:
return "array type"
case *ast.AssignStmt:
return "assignment"
case *ast.BadDecl:
return "bad declaration"
case *ast.BadExpr:
return "bad expression"
case *ast.BadStmt:
return "bad statement"
case *ast.BasicLit:
return "basic literal"
case *ast.BinaryExpr:
return fmt.Sprintf("binary %s operation", n.Op)
case *ast.BlockStmt:
return "block"
case *ast.BranchStmt:
switch n.Tok {
case token.BREAK:
return "break statement"
case token.CONTINUE:
return "continue statement"
case token.GOTO:
return "goto statement"
case token.FALLTHROUGH:
return "fall-through statement"
}
case *ast.CallExpr:
return "function call (or conversion)"
case *ast.CaseClause:
return "case clause"
case *ast.ChanType:
return "channel type"
case *ast.CommClause:
return "communication clause"
case *ast.Comment:
return "comment"
case *ast.CommentGroup:
return "comment group"
case *ast.CompositeLit:
return "composite literal"
case *ast.DeclStmt:
return NodeDescription(n.Decl) + " statement"
case *ast.DeferStmt:
return "defer statement"
case *ast.Ellipsis:
return "ellipsis"
case *ast.EmptyStmt:
return "empty statement"
case *ast.ExprStmt:
return "expression statement"
case *ast.Field:
// Can be any of these:
// struct {x, y int} -- struct field(s)
// struct {T} -- anon struct field
// interface {I} -- interface embedding
// interface {f()} -- interface method
// func (A) func(B) C -- receiver, param(s), result(s)
return "field/method/parameter"
case *ast.FieldList:
return "field/method/parameter list"
case *ast.File:
return "source file"
case *ast.ForStmt:
return "for loop"
case *ast.FuncDecl:
return "function declaration"
case *ast.FuncLit:
return "function literal"
case *ast.FuncType:
return "function type"
case *ast.GenDecl:
switch n.Tok {
case token.IMPORT:
return "import declaration"
case token.CONST:
return "constant declaration"
case token.TYPE:
return "type declaration"
case token.VAR:
return "variable declaration"
}
case *ast.GoStmt:
return "go statement"
case *ast.Ident:
return "identifier"
case *ast.IfStmt:
return "if statement"
case *ast.ImportSpec:
return "import specification"
case *ast.IncDecStmt:
if n.Tok == token.INC {
return "increment statement"
}
return "decrement statement"
case *ast.IndexExpr:
return "index expression"
case *ast.InterfaceType:
return "interface type"
case *ast.KeyValueExpr:
return "key/value association"
case *ast.LabeledStmt:
return "statement label"
case *ast.MapType:
return "map type"
case *ast.Package:
return "package"
case *ast.ParenExpr:
return "parenthesized " + NodeDescription(n.X)
case *ast.RangeStmt:
return "range loop"
case *ast.ReturnStmt:
return "return statement"
case *ast.SelectStmt:
return "select statement"
case *ast.SelectorExpr:
return "selector"
case *ast.SendStmt:
return "channel send"
case *ast.SliceExpr:
return "slice expression"
case *ast.StarExpr:
return "*-operation" // load/store expr or pointer type
case *ast.StructType:
return "struct type"
case *ast.SwitchStmt:
return "switch statement"
case *ast.TypeAssertExpr:
return "type assertion"
case *ast.TypeSpec:
return "type specification"
case *ast.TypeSwitchStmt:
return "type switch"
case *ast.UnaryExpr:
return fmt.Sprintf("unary %s operation", n.Op)
case *ast.ValueSpec:
return "value specification"
}
panic(fmt.Sprintf("unexpected node type: %T", n))
}

View File

@@ -1,400 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package astutil contains common utilities for working with the Go AST.
package astutil
import (
"fmt"
"go/ast"
"go/token"
"strconv"
"strings"
)
// AddImport adds the import path to the file f, if absent.
func AddImport(fset *token.FileSet, f *ast.File, ipath string) (added bool) {
return AddNamedImport(fset, f, "", ipath)
}
// AddNamedImport adds the import path to the file f, if absent.
// If name is not empty, it is used to rename the import.
//
// For example, calling
// AddNamedImport(fset, f, "pathpkg", "path")
// adds
// import pathpkg "path"
func AddNamedImport(fset *token.FileSet, f *ast.File, name, ipath string) (added bool) {
if imports(f, ipath) {
return false
}
newImport := &ast.ImportSpec{
Path: &ast.BasicLit{
Kind: token.STRING,
Value: strconv.Quote(ipath),
},
}
if name != "" {
newImport.Name = &ast.Ident{Name: name}
}
// Find an import decl to add to.
// The goal is to find an existing import
// whose import path has the longest shared
// prefix with ipath.
var (
bestMatch = -1 // length of longest shared prefix
lastImport = -1 // index in f.Decls of the file's final import decl
impDecl *ast.GenDecl // import decl containing the best match
impIndex = -1 // spec index in impDecl containing the best match
)
for i, decl := range f.Decls {
gen, ok := decl.(*ast.GenDecl)
if ok && gen.Tok == token.IMPORT {
lastImport = i
// Do not add to import "C", to avoid disrupting the
// association with its doc comment, breaking cgo.
if declImports(gen, "C") {
continue
}
// Match an empty import decl if that's all that is available.
if len(gen.Specs) == 0 && bestMatch == -1 {
impDecl = gen
}
// Compute longest shared prefix with imports in this group.
for j, spec := range gen.Specs {
impspec := spec.(*ast.ImportSpec)
n := matchLen(importPath(impspec), ipath)
if n > bestMatch {
bestMatch = n
impDecl = gen
impIndex = j
}
}
}
}
// If no import decl found, add one after the last import.
if impDecl == nil {
impDecl = &ast.GenDecl{
Tok: token.IMPORT,
}
if lastImport >= 0 {
impDecl.TokPos = f.Decls[lastImport].End()
} else {
// There are no existing imports.
// Our new import goes after the package declaration and after
// the comment, if any, that starts on the same line as the
// package declaration.
impDecl.TokPos = f.Package
file := fset.File(f.Package)
pkgLine := file.Line(f.Package)
for _, c := range f.Comments {
if file.Line(c.Pos()) > pkgLine {
break
}
impDecl.TokPos = c.End()
}
}
f.Decls = append(f.Decls, nil)
copy(f.Decls[lastImport+2:], f.Decls[lastImport+1:])
f.Decls[lastImport+1] = impDecl
}
// Insert new import at insertAt.
insertAt := 0
if impIndex >= 0 {
// insert after the found import
insertAt = impIndex + 1
}
impDecl.Specs = append(impDecl.Specs, nil)
copy(impDecl.Specs[insertAt+1:], impDecl.Specs[insertAt:])
impDecl.Specs[insertAt] = newImport
pos := impDecl.Pos()
if insertAt > 0 {
// If there is a comment after an existing import, preserve the comment
// position by adding the new import after the comment.
if spec, ok := impDecl.Specs[insertAt-1].(*ast.ImportSpec); ok && spec.Comment != nil {
pos = spec.Comment.End()
} else {
// Assign same position as the previous import,
// so that the sorter sees it as being in the same block.
pos = impDecl.Specs[insertAt-1].Pos()
}
}
if newImport.Name != nil {
newImport.Name.NamePos = pos
}
newImport.Path.ValuePos = pos
newImport.EndPos = pos
// Clean up parens. impDecl contains at least one spec.
if len(impDecl.Specs) == 1 {
// Remove unneeded parens.
impDecl.Lparen = token.NoPos
} else if !impDecl.Lparen.IsValid() {
// impDecl needs parens added.
impDecl.Lparen = impDecl.Specs[0].Pos()
}
f.Imports = append(f.Imports, newImport)
if len(f.Decls) <= 1 {
return true
}
// Merge all the import declarations into the first one.
var first *ast.GenDecl
for i, decl := range f.Decls {
gen, ok := decl.(*ast.GenDecl)
if !ok || gen.Tok != token.IMPORT || declImports(gen, "C") {
continue
}
if first == nil {
first = gen
continue // Don't touch the first one.
}
// Move the imports of the other import declaration to the first one.
for _, spec := range gen.Specs {
spec.(*ast.ImportSpec).Path.ValuePos = first.Pos()
first.Specs = append(first.Specs, spec)
}
f.Decls = append(f.Decls[:i], f.Decls[i+1:]...)
}
return true
}
// DeleteImport deletes the import path from the file f, if present.
func DeleteImport(fset *token.FileSet, f *ast.File, path string) (deleted bool) {
return DeleteNamedImport(fset, f, "", path)
}
// DeleteNamedImport deletes the import with the given name and path from the file f, if present.
func DeleteNamedImport(fset *token.FileSet, f *ast.File, name, path string) (deleted bool) {
var delspecs []*ast.ImportSpec
// Find the import nodes that import path, if any.
for i := 0; i < len(f.Decls); i++ {
decl := f.Decls[i]
gen, ok := decl.(*ast.GenDecl)
if !ok || gen.Tok != token.IMPORT {
continue
}
for j := 0; j < len(gen.Specs); j++ {
spec := gen.Specs[j]
impspec := spec.(*ast.ImportSpec)
if impspec.Name == nil && name != "" {
continue
}
if impspec.Name != nil && impspec.Name.Name != name {
continue
}
if importPath(impspec) != path {
continue
}
// We found an import spec that imports path.
// Delete it.
delspecs = append(delspecs, impspec)
deleted = true
copy(gen.Specs[j:], gen.Specs[j+1:])
gen.Specs = gen.Specs[:len(gen.Specs)-1]
// If this was the last import spec in this decl,
// delete the decl, too.
if len(gen.Specs) == 0 {
copy(f.Decls[i:], f.Decls[i+1:])
f.Decls = f.Decls[:len(f.Decls)-1]
i--
break
} else if len(gen.Specs) == 1 {
gen.Lparen = token.NoPos // drop parens
}
if j > 0 {
lastImpspec := gen.Specs[j-1].(*ast.ImportSpec)
lastLine := fset.Position(lastImpspec.Path.ValuePos).Line
line := fset.Position(impspec.Path.ValuePos).Line
// We deleted an entry but now there may be
// a blank line-sized hole where the import was.
if line-lastLine > 1 {
// There was a blank line immediately preceding the deleted import,
// so there's no need to close the hole.
// Do nothing.
} else {
// There was no blank line. Close the hole.
fset.File(gen.Rparen).MergeLine(line)
}
}
j--
}
}
// Delete them from f.Imports.
for i := 0; i < len(f.Imports); i++ {
imp := f.Imports[i]
for j, del := range delspecs {
if imp == del {
copy(f.Imports[i:], f.Imports[i+1:])
f.Imports = f.Imports[:len(f.Imports)-1]
copy(delspecs[j:], delspecs[j+1:])
delspecs = delspecs[:len(delspecs)-1]
i--
break
}
}
}
if len(delspecs) > 0 {
panic(fmt.Sprintf("deleted specs from Decls but not Imports: %v", delspecs))
}
return
}
// RewriteImport rewrites any import of path oldPath to path newPath.
func RewriteImport(fset *token.FileSet, f *ast.File, oldPath, newPath string) (rewrote bool) {
for _, imp := range f.Imports {
if importPath(imp) == oldPath {
rewrote = true
// record old End, because the default is to compute
// it using the length of imp.Path.Value.
imp.EndPos = imp.End()
imp.Path.Value = strconv.Quote(newPath)
}
}
return
}
// UsesImport reports whether a given import is used.
func UsesImport(f *ast.File, path string) (used bool) {
spec := importSpec(f, path)
if spec == nil {
return
}
name := spec.Name.String()
switch name {
case "<nil>":
// If the package name is not explicitly specified,
// make an educated guess. This is not guaranteed to be correct.
lastSlash := strings.LastIndex(path, "/")
if lastSlash == -1 {
name = path
} else {
name = path[lastSlash+1:]
}
case "_", ".":
// Not sure if this import is used - err on the side of caution.
return true
}
ast.Walk(visitFn(func(n ast.Node) {
sel, ok := n.(*ast.SelectorExpr)
if ok && isTopName(sel.X, name) {
used = true
}
}), f)
return
}
type visitFn func(node ast.Node)
func (fn visitFn) Visit(node ast.Node) ast.Visitor {
fn(node)
return fn
}
// imports returns true if f imports path.
func imports(f *ast.File, path string) bool {
return importSpec(f, path) != nil
}
// importSpec returns the import spec if f imports path,
// or nil otherwise.
func importSpec(f *ast.File, path string) *ast.ImportSpec {
for _, s := range f.Imports {
if importPath(s) == path {
return s
}
}
return nil
}
// importPath returns the unquoted import path of s,
// or "" if the path is not properly quoted.
func importPath(s *ast.ImportSpec) string {
t, err := strconv.Unquote(s.Path.Value)
if err == nil {
return t
}
return ""
}
// declImports reports whether gen contains an import of path.
func declImports(gen *ast.GenDecl, path string) bool {
if gen.Tok != token.IMPORT {
return false
}
for _, spec := range gen.Specs {
impspec := spec.(*ast.ImportSpec)
if importPath(impspec) == path {
return true
}
}
return false
}
// matchLen returns the length of the longest path segment prefix shared by x and y.
func matchLen(x, y string) int {
n := 0
for i := 0; i < len(x) && i < len(y) && x[i] == y[i]; i++ {
if x[i] == '/' {
n++
}
}
return n
}
// isTopName returns true if n is a top-level unresolved identifier with the given name.
func isTopName(n ast.Expr, name string) bool {
id, ok := n.(*ast.Ident)
return ok && id.Name == name && id.Obj == nil
}
// Imports returns the file imports grouped by paragraph.
func Imports(fset *token.FileSet, f *ast.File) [][]*ast.ImportSpec {
var groups [][]*ast.ImportSpec
for _, decl := range f.Decls {
genDecl, ok := decl.(*ast.GenDecl)
if !ok || genDecl.Tok != token.IMPORT {
break
}
group := []*ast.ImportSpec{}
var lastLine int
for _, spec := range genDecl.Specs {
importSpec := spec.(*ast.ImportSpec)
pos := importSpec.Path.ValuePos
line := fset.Position(pos).Line
if lastLine > 0 && pos > 0 && line-lastLine > 1 {
groups = append(groups, group)
group = []*ast.ImportSpec{}
}
group = append(group, importSpec)
lastLine = line
}
groups = append(groups, group)
}
return groups
}

View File

@@ -1,14 +0,0 @@
package astutil
import "go/ast"
// Unparen returns e with any enclosing parentheses stripped.
func Unparen(e ast.Expr) ast.Expr {
for {
p, ok := e.(*ast.ParenExpr)
if !ok {
return e
}
e = p.X
}
}

View File

@@ -1,195 +0,0 @@
// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package buildutil provides utilities related to the go/build
// package in the standard library.
//
// All I/O is done via the build.Context file system interface, which must
// be concurrency-safe.
package buildutil
import (
"go/build"
"os"
"path/filepath"
"sort"
"strings"
"sync"
)
// AllPackages returns the package path of each Go package in any source
// directory of the specified build context (e.g. $GOROOT or an element
// of $GOPATH). Errors are ignored. The results are sorted.
// All package paths are canonical, and thus may contain "/vendor/".
//
// The result may include import paths for directories that contain no
// *.go files, such as "archive" (in $GOROOT/src).
//
// All I/O is done via the build.Context file system interface,
// which must be concurrency-safe.
//
func AllPackages(ctxt *build.Context) []string {
var list []string
ForEachPackage(ctxt, func(pkg string, _ error) {
list = append(list, pkg)
})
sort.Strings(list)
return list
}
// ForEachPackage calls the found function with the package path of
// each Go package it finds in any source directory of the specified
// build context (e.g. $GOROOT or an element of $GOPATH).
// All package paths are canonical, and thus may contain "/vendor/".
//
// If the package directory exists but could not be read, the second
// argument to the found function provides the error.
//
// All I/O is done via the build.Context file system interface,
// which must be concurrency-safe.
//
func ForEachPackage(ctxt *build.Context, found func(importPath string, err error)) {
ch := make(chan item)
var wg sync.WaitGroup
for _, root := range ctxt.SrcDirs() {
root := root
wg.Add(1)
go func() {
allPackages(ctxt, root, ch)
wg.Done()
}()
}
go func() {
wg.Wait()
close(ch)
}()
// All calls to found occur in the caller's goroutine.
for i := range ch {
found(i.importPath, i.err)
}
}
type item struct {
importPath string
err error // (optional)
}
// We use a process-wide counting semaphore to limit
// the number of parallel calls to ReadDir.
var ioLimit = make(chan bool, 20)
func allPackages(ctxt *build.Context, root string, ch chan<- item) {
root = filepath.Clean(root) + string(os.PathSeparator)
var wg sync.WaitGroup
var walkDir func(dir string)
walkDir = func(dir string) {
// Avoid .foo, _foo, and testdata directory trees.
base := filepath.Base(dir)
if base == "" || base[0] == '.' || base[0] == '_' || base == "testdata" {
return
}
pkg := filepath.ToSlash(strings.TrimPrefix(dir, root))
// Prune search if we encounter any of these import paths.
switch pkg {
case "builtin":
return
}
ioLimit <- true
files, err := ReadDir(ctxt, dir)
<-ioLimit
if pkg != "" || err != nil {
ch <- item{pkg, err}
}
for _, fi := range files {
fi := fi
if fi.IsDir() {
wg.Add(1)
go func() {
walkDir(filepath.Join(dir, fi.Name()))
wg.Done()
}()
}
}
}
walkDir(root)
wg.Wait()
}
// ExpandPatterns returns the set of packages matched by patterns,
// which may have the following forms:
//
// golang.org/x/tools/cmd/guru # a single package
// golang.org/x/tools/... # all packages beneath dir
// ... # the entire workspace.
//
// Order is significant: a pattern preceded by '-' removes matching
// packages from the set. For example, these patterns match all encoding
// packages except encoding/xml:
//
// encoding/... -encoding/xml
//
func ExpandPatterns(ctxt *build.Context, patterns []string) map[string]bool {
// TODO(adonovan): support other features of 'go list':
// - "std"/"cmd"/"all" meta-packages
// - "..." not at the end of a pattern
// - relative patterns using "./" or "../" prefix
pkgs := make(map[string]bool)
doPkg := func(pkg string, neg bool) {
if neg {
delete(pkgs, pkg)
} else {
pkgs[pkg] = true
}
}
// Scan entire workspace if wildcards are present.
// TODO(adonovan): opt: scan only the necessary subtrees of the workspace.
var all []string
for _, arg := range patterns {
if strings.HasSuffix(arg, "...") {
all = AllPackages(ctxt)
break
}
}
for _, arg := range patterns {
if arg == "" {
continue
}
neg := arg[0] == '-'
if neg {
arg = arg[1:]
}
if arg == "..." {
// ... matches all packages
for _, pkg := range all {
doPkg(pkg, neg)
}
} else if dir := strings.TrimSuffix(arg, "/..."); dir != arg {
// dir/... matches all packages beneath dir
for _, pkg := range all {
if strings.HasPrefix(pkg, dir) &&
(len(pkg) == len(dir) || pkg[len(dir)] == '/') {
doPkg(pkg, neg)
}
}
} else {
// single package
doPkg(arg, neg)
}
}
return pkgs
}

View File

@@ -1,108 +0,0 @@
package buildutil
import (
"fmt"
"go/build"
"io"
"io/ioutil"
"os"
"path"
"path/filepath"
"sort"
"strings"
"time"
)
// FakeContext returns a build.Context for the fake file tree specified
// by pkgs, which maps package import paths to a mapping from file base
// names to contents.
//
// The fake Context has a GOROOT of "/go" and no GOPATH, and overrides
// the necessary file access methods to read from memory instead of the
// real file system.
//
// Unlike a real file tree, the fake one has only two levels---packages
// and files---so ReadDir("/go/src/") returns all packages under
// /go/src/ including, for instance, "math" and "math/big".
// ReadDir("/go/src/math/big") would return all the files in the
// "math/big" package.
//
func FakeContext(pkgs map[string]map[string]string) *build.Context {
clean := func(filename string) string {
f := path.Clean(filepath.ToSlash(filename))
// Removing "/go/src" while respecting segment
// boundaries has this unfortunate corner case:
if f == "/go/src" {
return ""
}
return strings.TrimPrefix(f, "/go/src/")
}
ctxt := build.Default // copy
ctxt.GOROOT = "/go"
ctxt.GOPATH = ""
ctxt.IsDir = func(dir string) bool {
dir = clean(dir)
if dir == "" {
return true // needed by (*build.Context).SrcDirs
}
return pkgs[dir] != nil
}
ctxt.ReadDir = func(dir string) ([]os.FileInfo, error) {
dir = clean(dir)
var fis []os.FileInfo
if dir == "" {
// enumerate packages
for importPath := range pkgs {
fis = append(fis, fakeDirInfo(importPath))
}
} else {
// enumerate files of package
for basename := range pkgs[dir] {
fis = append(fis, fakeFileInfo(basename))
}
}
sort.Sort(byName(fis))
return fis, nil
}
ctxt.OpenFile = func(filename string) (io.ReadCloser, error) {
filename = clean(filename)
dir, base := path.Split(filename)
content, ok := pkgs[path.Clean(dir)][base]
if !ok {
return nil, fmt.Errorf("file not found: %s", filename)
}
return ioutil.NopCloser(strings.NewReader(content)), nil
}
ctxt.IsAbsPath = func(path string) bool {
path = filepath.ToSlash(path)
// Don't rely on the default (filepath.Path) since on
// Windows, it reports virtual paths as non-absolute.
return strings.HasPrefix(path, "/")
}
return &ctxt
}
type byName []os.FileInfo
func (s byName) Len() int { return len(s) }
func (s byName) Swap(i, j int) { s[i], s[j] = s[j], s[i] }
func (s byName) Less(i, j int) bool { return s[i].Name() < s[j].Name() }
type fakeFileInfo string
func (fi fakeFileInfo) Name() string { return string(fi) }
func (fakeFileInfo) Sys() interface{} { return nil }
func (fakeFileInfo) ModTime() time.Time { return time.Time{} }
func (fakeFileInfo) IsDir() bool { return false }
func (fakeFileInfo) Size() int64 { return 0 }
func (fakeFileInfo) Mode() os.FileMode { return 0644 }
type fakeDirInfo string
func (fd fakeDirInfo) Name() string { return string(fd) }
func (fakeDirInfo) Sys() interface{} { return nil }
func (fakeDirInfo) ModTime() time.Time { return time.Time{} }
func (fakeDirInfo) IsDir() bool { return true }
func (fakeDirInfo) Size() int64 { return 0 }
func (fakeDirInfo) Mode() os.FileMode { return 0755 }

View File

@@ -1,75 +0,0 @@
package buildutil
// This logic was copied from stringsFlag from $GOROOT/src/cmd/go/build.go.
import "fmt"
const TagsFlagDoc = "a list of `build tags` to consider satisfied during the build. " +
"For more information about build tags, see the description of " +
"build constraints in the documentation for the go/build package"
// TagsFlag is an implementation of the flag.Value and flag.Getter interfaces that parses
// a flag value in the same manner as go build's -tags flag and
// populates a []string slice.
//
// See $GOROOT/src/go/build/doc.go for description of build tags.
// See $GOROOT/src/cmd/go/doc.go for description of 'go build -tags' flag.
//
// Example:
// flag.Var((*buildutil.TagsFlag)(&build.Default.BuildTags), "tags", buildutil.TagsFlagDoc)
type TagsFlag []string
func (v *TagsFlag) Set(s string) error {
var err error
*v, err = splitQuotedFields(s)
if *v == nil {
*v = []string{}
}
return err
}
func (v *TagsFlag) Get() interface{} { return *v }
func splitQuotedFields(s string) ([]string, error) {
// Split fields allowing '' or "" around elements.
// Quotes further inside the string do not count.
var f []string
for len(s) > 0 {
for len(s) > 0 && isSpaceByte(s[0]) {
s = s[1:]
}
if len(s) == 0 {
break
}
// Accepted quoted string. No unescaping inside.
if s[0] == '"' || s[0] == '\'' {
quote := s[0]
s = s[1:]
i := 0
for i < len(s) && s[i] != quote {
i++
}
if i >= len(s) {
return nil, fmt.Errorf("unterminated %c string", quote)
}
f = append(f, s[:i])
s = s[i+1:]
continue
}
i := 0
for i < len(s) && !isSpaceByte(s[i]) {
i++
}
f = append(f, s[:i])
s = s[i:]
}
return f, nil
}
func (v *TagsFlag) String() string {
return "<tagsFlag>"
}
func isSpaceByte(c byte) bool {
return c == ' ' || c == '\t' || c == '\n' || c == '\r'
}

View File

@@ -1,167 +0,0 @@
// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package buildutil
import (
"fmt"
"go/ast"
"go/build"
"go/parser"
"go/token"
"io"
"io/ioutil"
"os"
"path"
"path/filepath"
"runtime"
"strings"
)
// ParseFile behaves like parser.ParseFile,
// but uses the build context's file system interface, if any.
//
// If file is not absolute (as defined by IsAbsPath), the (dir, file)
// components are joined using JoinPath; dir must be absolute.
//
// The displayPath function, if provided, is used to transform the
// filename that will be attached to the ASTs.
//
// TODO(adonovan): call this from go/loader.parseFiles when the tree thaws.
//
func ParseFile(fset *token.FileSet, ctxt *build.Context, displayPath func(string) string, dir string, file string, mode parser.Mode) (*ast.File, error) {
if !IsAbsPath(ctxt, file) {
file = JoinPath(ctxt, dir, file)
}
rd, err := OpenFile(ctxt, file)
if err != nil {
return nil, err
}
defer rd.Close() // ignore error
if displayPath != nil {
file = displayPath(file)
}
return parser.ParseFile(fset, file, rd, mode)
}
// ContainingPackage returns the package containing filename.
//
// If filename is not absolute, it is interpreted relative to working directory dir.
// All I/O is via the build context's file system interface, if any.
//
// The '...Files []string' fields of the resulting build.Package are not
// populated (build.FindOnly mode).
//
// TODO(adonovan): call this from oracle when the tree thaws.
//
func ContainingPackage(ctxt *build.Context, dir, filename string) (*build.Package, error) {
if !IsAbsPath(ctxt, filename) {
filename = JoinPath(ctxt, dir, filename)
}
// We must not assume the file tree uses
// "/" always,
// `\` always,
// or os.PathSeparator (which varies by platform),
// but to make any progress, we are forced to assume that
// paths will not use `\` unless the PathSeparator
// is also `\`, thus we can rely on filepath.ToSlash for some sanity.
dirSlash := path.Dir(filepath.ToSlash(filename)) + "/"
// We assume that no source root (GOPATH[i] or GOROOT) contains any other.
for _, srcdir := range ctxt.SrcDirs() {
srcdirSlash := filepath.ToSlash(srcdir) + "/"
if dirHasPrefix(dirSlash, srcdirSlash) {
importPath := dirSlash[len(srcdirSlash) : len(dirSlash)-len("/")]
return ctxt.Import(importPath, dir, build.FindOnly)
}
}
return nil, fmt.Errorf("can't find package containing %s", filename)
}
// dirHasPrefix tests whether the directory dir begins with prefix.
func dirHasPrefix(dir, prefix string) bool {
if runtime.GOOS != "windows" {
return strings.HasPrefix(dir, prefix)
}
return len(dir) >= len(prefix) && strings.EqualFold(dir[:len(prefix)], prefix)
}
// -- Effective methods of file system interface -------------------------
// (go/build.Context defines these as methods, but does not export them.)
// TODO(adonovan): HasSubdir?
// FileExists returns true if the specified file exists,
// using the build context's file system interface.
func FileExists(ctxt *build.Context, path string) bool {
if ctxt.OpenFile != nil {
r, err := ctxt.OpenFile(path)
if err != nil {
return false
}
r.Close() // ignore error
return true
}
_, err := os.Stat(path)
return err == nil
}
// OpenFile behaves like os.Open,
// but uses the build context's file system interface, if any.
func OpenFile(ctxt *build.Context, path string) (io.ReadCloser, error) {
if ctxt.OpenFile != nil {
return ctxt.OpenFile(path)
}
return os.Open(path)
}
// IsAbsPath behaves like filepath.IsAbs,
// but uses the build context's file system interface, if any.
func IsAbsPath(ctxt *build.Context, path string) bool {
if ctxt.IsAbsPath != nil {
return ctxt.IsAbsPath(path)
}
return filepath.IsAbs(path)
}
// JoinPath behaves like filepath.Join,
// but uses the build context's file system interface, if any.
func JoinPath(ctxt *build.Context, path ...string) string {
if ctxt.JoinPath != nil {
return ctxt.JoinPath(path...)
}
return filepath.Join(path...)
}
// IsDir behaves like os.Stat plus IsDir,
// but uses the build context's file system interface, if any.
func IsDir(ctxt *build.Context, path string) bool {
if ctxt.IsDir != nil {
return ctxt.IsDir(path)
}
fi, err := os.Stat(path)
return err == nil && fi.IsDir()
}
// ReadDir behaves like ioutil.ReadDir,
// but uses the build context's file system interface, if any.
func ReadDir(ctxt *build.Context, path string) ([]os.FileInfo, error) {
if ctxt.ReadDir != nil {
return ctxt.ReadDir(path)
}
return ioutil.ReadDir(path)
}
// SplitPathList behaves like filepath.SplitList,
// but uses the build context's file system interface, if any.
func SplitPathList(ctxt *build.Context, s string) []string {
if ctxt.SplitPathList != nil {
return ctxt.SplitPathList(s)
}
return filepath.SplitList(s)
}

View File

@@ -1,209 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.5
package loader
// This file handles cgo preprocessing of files containing `import "C"`.
//
// DESIGN
//
// The approach taken is to run the cgo processor on the package's
// CgoFiles and parse the output, faking the filenames of the
// resulting ASTs so that the synthetic file containing the C types is
// called "C" (e.g. "~/go/src/net/C") and the preprocessed files
// have their original names (e.g. "~/go/src/net/cgo_unix.go"),
// not the names of the actual temporary files.
//
// The advantage of this approach is its fidelity to 'go build'. The
// downside is that the token.Position.Offset for each AST node is
// incorrect, being an offset within the temporary file. Line numbers
// should still be correct because of the //line comments.
//
// The logic of this file is mostly plundered from the 'go build'
// tool, which also invokes the cgo preprocessor.
//
//
// REJECTED ALTERNATIVE
//
// An alternative approach that we explored is to extend go/types'
// Importer mechanism to provide the identity of the importing package
// so that each time `import "C"` appears it resolves to a different
// synthetic package containing just the objects needed in that case.
// The loader would invoke cgo but parse only the cgo_types.go file
// defining the package-level objects, discarding the other files
// resulting from preprocessing.
//
// The benefit of this approach would have been that source-level
// syntax information would correspond exactly to the original cgo
// file, with no preprocessing involved, making source tools like
// godoc, oracle, and eg happy. However, the approach was rejected
// due to the additional complexity it would impose on go/types. (It
// made for a beautiful demo, though.)
//
// cgo files, despite their *.go extension, are not legal Go source
// files per the specification since they may refer to unexported
// members of package "C" such as C.int. Also, a function such as
// C.getpwent has in effect two types, one matching its C type and one
// which additionally returns (errno C.int). The cgo preprocessor
// uses name mangling to distinguish these two functions in the
// processed code, but go/types would need to duplicate this logic in
// its handling of function calls, analogous to the treatment of map
// lookups in which y=m[k] and y,ok=m[k] are both legal.
import (
"fmt"
"go/ast"
"go/build"
"go/parser"
"go/token"
"io/ioutil"
"log"
"os"
"os/exec"
"path/filepath"
"regexp"
"strings"
)
// processCgoFiles invokes the cgo preprocessor on bp.CgoFiles, parses
// the output and returns the resulting ASTs.
//
func processCgoFiles(bp *build.Package, fset *token.FileSet, DisplayPath func(path string) string, mode parser.Mode) ([]*ast.File, error) {
tmpdir, err := ioutil.TempDir("", strings.Replace(bp.ImportPath, "/", "_", -1)+"_C")
if err != nil {
return nil, err
}
defer os.RemoveAll(tmpdir)
pkgdir := bp.Dir
if DisplayPath != nil {
pkgdir = DisplayPath(pkgdir)
}
cgoFiles, cgoDisplayFiles, err := runCgo(bp, pkgdir, tmpdir)
if err != nil {
return nil, err
}
var files []*ast.File
for i := range cgoFiles {
rd, err := os.Open(cgoFiles[i])
if err != nil {
return nil, err
}
display := filepath.Join(bp.Dir, cgoDisplayFiles[i])
f, err := parser.ParseFile(fset, display, rd, mode)
rd.Close()
if err != nil {
return nil, err
}
files = append(files, f)
}
return files, nil
}
var cgoRe = regexp.MustCompile(`[/\\:]`)
// runCgo invokes the cgo preprocessor on bp.CgoFiles and returns two
// lists of files: the resulting processed files (in temporary
// directory tmpdir) and the corresponding names of the unprocessed files.
//
// runCgo is adapted from (*builder).cgo in
// $GOROOT/src/cmd/go/build.go, but these features are unsupported:
// Objective C, CGOPKGPATH, CGO_FLAGS.
//
func runCgo(bp *build.Package, pkgdir, tmpdir string) (files, displayFiles []string, err error) {
cgoCPPFLAGS, _, _, _ := cflags(bp, true)
_, cgoexeCFLAGS, _, _ := cflags(bp, false)
if len(bp.CgoPkgConfig) > 0 {
pcCFLAGS, err := pkgConfigFlags(bp)
if err != nil {
return nil, nil, err
}
cgoCPPFLAGS = append(cgoCPPFLAGS, pcCFLAGS...)
}
// Allows including _cgo_export.h from .[ch] files in the package.
cgoCPPFLAGS = append(cgoCPPFLAGS, "-I", tmpdir)
// _cgo_gotypes.go (displayed "C") contains the type definitions.
files = append(files, filepath.Join(tmpdir, "_cgo_gotypes.go"))
displayFiles = append(displayFiles, "C")
for _, fn := range bp.CgoFiles {
// "foo.cgo1.go" (displayed "foo.go") is the processed Go source.
f := cgoRe.ReplaceAllString(fn[:len(fn)-len("go")], "_")
files = append(files, filepath.Join(tmpdir, f+"cgo1.go"))
displayFiles = append(displayFiles, fn)
}
var cgoflags []string
if bp.Goroot && bp.ImportPath == "runtime/cgo" {
cgoflags = append(cgoflags, "-import_runtime_cgo=false")
}
if bp.Goroot && bp.ImportPath == "runtime/race" || bp.ImportPath == "runtime/cgo" {
cgoflags = append(cgoflags, "-import_syscall=false")
}
args := stringList(
"go", "tool", "cgo", "-objdir", tmpdir, cgoflags, "--",
cgoCPPFLAGS, cgoexeCFLAGS, bp.CgoFiles,
)
if false {
log.Printf("Running cgo for package %q: %s (dir=%s)", bp.ImportPath, args, pkgdir)
}
cmd := exec.Command(args[0], args[1:]...)
cmd.Dir = pkgdir
cmd.Stdout = os.Stderr
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
return nil, nil, fmt.Errorf("cgo failed: %s: %s", args, err)
}
return files, displayFiles, nil
}
// -- unmodified from 'go build' ---------------------------------------
// Return the flags to use when invoking the C or C++ compilers, or cgo.
func cflags(p *build.Package, def bool) (cppflags, cflags, cxxflags, ldflags []string) {
var defaults string
if def {
defaults = "-g -O2"
}
cppflags = stringList(envList("CGO_CPPFLAGS", ""), p.CgoCPPFLAGS)
cflags = stringList(envList("CGO_CFLAGS", defaults), p.CgoCFLAGS)
cxxflags = stringList(envList("CGO_CXXFLAGS", defaults), p.CgoCXXFLAGS)
ldflags = stringList(envList("CGO_LDFLAGS", defaults), p.CgoLDFLAGS)
return
}
// envList returns the value of the given environment variable broken
// into fields, using the default value when the variable is empty.
func envList(key, def string) []string {
v := os.Getenv(key)
if v == "" {
v = def
}
return strings.Fields(v)
}
// stringList's arguments should be a sequence of string or []string values.
// stringList flattens them into a single []string.
func stringList(args ...interface{}) []string {
var x []string
for _, arg := range args {
switch arg := arg.(type) {
case []string:
x = append(x, arg...)
case string:
x = append(x, arg)
default:
panic("stringList: invalid argument")
}
}
return x
}

View File

@@ -1,39 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package loader
import (
"errors"
"fmt"
"go/build"
"os/exec"
"strings"
)
// pkgConfig runs pkg-config with the specified arguments and returns the flags it prints.
func pkgConfig(mode string, pkgs []string) (flags []string, err error) {
cmd := exec.Command("pkg-config", append([]string{mode}, pkgs...)...)
out, err := cmd.CombinedOutput()
if err != nil {
s := fmt.Sprintf("%s failed: %v", strings.Join(cmd.Args, " "), err)
if len(out) > 0 {
s = fmt.Sprintf("%s: %s", s, out)
}
return nil, errors.New(s)
}
if len(out) > 0 {
flags = strings.Fields(string(out))
}
return
}
// pkgConfigFlags calls pkg-config if needed and returns the cflags
// needed to build the package.
func pkgConfigFlags(p *build.Package) (cflags []string, err error) {
if len(p.CgoPkgConfig) == 0 {
return nil, nil
}
return pkgConfig("--cflags", p.CgoPkgConfig)
}

View File

@@ -1,205 +0,0 @@
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package loader loads a complete Go program from source code, parsing
// and type-checking the initial packages plus their transitive closure
// of dependencies. The ASTs and the derived facts are retained for
// later use.
//
// THIS INTERFACE IS EXPERIMENTAL AND IS LIKELY TO CHANGE.
//
// The package defines two primary types: Config, which specifies a
// set of initial packages to load and various other options; and
// Program, which is the result of successfully loading the packages
// specified by a configuration.
//
// The configuration can be set directly, but *Config provides various
// convenience methods to simplify the common cases, each of which can
// be called any number of times. Finally, these are followed by a
// call to Load() to actually load and type-check the program.
//
// var conf loader.Config
//
// // Use the command-line arguments to specify
// // a set of initial packages to load from source.
// // See FromArgsUsage for help.
// rest, err := conf.FromArgs(os.Args[1:], wantTests)
//
// // Parse the specified files and create an ad hoc package with path "foo".
// // All files must have the same 'package' declaration.
// conf.CreateFromFilenames("foo", "foo.go", "bar.go")
//
// // Create an ad hoc package with path "foo" from
// // the specified already-parsed files.
// // All ASTs must have the same 'package' declaration.
// conf.CreateFromFiles("foo", parsedFiles)
//
// // Add "runtime" to the set of packages to be loaded.
// conf.Import("runtime")
//
// // Adds "fmt" and "fmt_test" to the set of packages
// // to be loaded. "fmt" will include *_test.go files.
// conf.ImportWithTests("fmt")
//
// // Finally, load all the packages specified by the configuration.
// prog, err := conf.Load()
//
// See examples_test.go for examples of API usage.
//
//
// CONCEPTS AND TERMINOLOGY
//
// The WORKSPACE is the set of packages accessible to the loader. The
// workspace is defined by Config.Build, a *build.Context. The
// default context treats subdirectories of $GOROOT and $GOPATH as
// packages, but this behavior may be overridden.
//
// An AD HOC package is one specified as a set of source files on the
// command line. In the simplest case, it may consist of a single file
// such as $GOROOT/src/net/http/triv.go.
//
// EXTERNAL TEST packages are those comprised of a set of *_test.go
// files all with the same 'package foo_test' declaration, all in the
// same directory. (go/build.Package calls these files XTestFiles.)
//
// An IMPORTABLE package is one that can be referred to by some import
// spec. Every importable package is uniquely identified by its
// PACKAGE PATH or just PATH, a string such as "fmt", "encoding/json",
// or "cmd/vendor/golang.org/x/arch/x86/x86asm". A package path
// typically denotes a subdirectory of the workspace.
//
// An import declaration uses an IMPORT PATH to refer to a package.
// Most import declarations use the package path as the import path.
//
// Due to VENDORING (https://golang.org/s/go15vendor), the
// interpretation of an import path may depend on the directory in which
// it appears. To resolve an import path to a package path, go/build
// must search the enclosing directories for a subdirectory named
// "vendor".
//
// ad hoc packages and external test packages are NON-IMPORTABLE. The
// path of an ad hoc package is inferred from the package
// declarations of its files and is therefore not a unique package key.
// For example, Config.CreatePkgs may specify two initial ad hoc
// packages, both with path "main".
//
// An AUGMENTED package is an importable package P plus all the
// *_test.go files with same 'package foo' declaration as P.
// (go/build.Package calls these files TestFiles.)
//
// The INITIAL packages are those specified in the configuration. A
// DEPENDENCY is a package loaded to satisfy an import in an initial
// package or another dependency.
//
package loader
// IMPLEMENTATION NOTES
//
// 'go test', in-package test files, and import cycles
// ---------------------------------------------------
//
// An external test package may depend upon members of the augmented
// package that are not in the unaugmented package, such as functions
// that expose internals. (See bufio/export_test.go for an example.)
// So, the loader must ensure that for each external test package
// it loads, it also augments the corresponding non-test package.
//
// The import graph over n unaugmented packages must be acyclic; the
// import graph over n-1 unaugmented packages plus one augmented
// package must also be acyclic. ('go test' relies on this.) But the
// import graph over n augmented packages may contain cycles.
//
// First, all the (unaugmented) non-test packages and their
// dependencies are imported in the usual way; the loader reports an
// error if it detects an import cycle.
//
// Then, each package P for which testing is desired is augmented by
// the list P' of its in-package test files, by calling
// (*types.Checker).Files. This arrangement ensures that P' may
// reference definitions within P, but P may not reference definitions
// within P'. Furthermore, P' may import any other package, including
// ones that depend upon P, without an import cycle error.
//
// Consider two packages A and B, both of which have lists of
// in-package test files we'll call A' and B', and which have the
// following import graph edges:
// B imports A
// B' imports A
// A' imports B
// This last edge would be expected to create an error were it not
// for the special type-checking discipline above.
// Cycles of size greater than two are possible. For example:
// compress/bzip2/bzip2_test.go (package bzip2) imports "io/ioutil"
// io/ioutil/tempfile_test.go (package ioutil) imports "regexp"
// regexp/exec_test.go (package regexp) imports "compress/bzip2"
//
//
// Concurrency
// -----------
//
// Let us define the import dependency graph as follows. Each node is a
// list of files passed to (Checker).Files at once. Many of these lists
// are the production code of an importable Go package, so those nodes
// are labelled by the package's path. The remaining nodes are
// ad hoc packages and lists of in-package *_test.go files that augment
// an importable package; those nodes have no label.
//
// The edges of the graph represent import statements appearing within a
// file. An edge connects a node (a list of files) to the node it
// imports, which is importable and thus always labelled.
//
// Loading is controlled by this dependency graph.
//
// To reduce I/O latency, we start loading a package's dependencies
// asynchronously as soon as we've parsed its files and enumerated its
// imports (scanImports). This performs a preorder traversal of the
// import dependency graph.
//
// To exploit hardware parallelism, we type-check unrelated packages in
// parallel, where "unrelated" means not ordered by the partial order of
// the import dependency graph.
//
// We use a concurrency-safe non-blocking cache (importer.imported) to
// record the results of type-checking, whether success or failure. An
// entry is created in this cache by startLoad the first time the
// package is imported. The first goroutine to request an entry becomes
// responsible for completing the task and broadcasting completion to
// subsequent requestors, which block until then.
//
// Type checking occurs in (parallel) postorder: we cannot type-check a
// set of files until we have loaded and type-checked all of their
// immediate dependencies (and thus all of their transitive
// dependencies). If the input were guaranteed free of import cycles,
// this would be trivial: we could simply wait for completion of the
// dependencies and then invoke the typechecker.
//
// But as we saw in the 'go test' section above, some cycles in the
// import graph over packages are actually legal, so long as the
// cycle-forming edge originates in the in-package test files that
// augment the package. This explains why the nodes of the import
// dependency graph are not packages, but lists of files: the unlabelled
// nodes avoid the cycles. Consider packages A and B where B imports A
// and A's in-package tests AT import B. The naively constructed import
// graph over packages would contain a cycle (A+AT) --> B --> (A+AT) but
// the graph over lists of files is AT --> B --> A, where AT is an
// unlabelled node.
//
// Awaiting completion of the dependencies in a cyclic graph would
// deadlock, so we must materialize the import dependency graph (as
// importer.graph) and check whether each import edge forms a cycle. If
// x imports y, and the graph already contains a path from y to x, then
// there is an import cycle, in which case the processing of x must not
// wait for the completion of processing of y.
//
// When the type-checker makes a callback (doImport) to the loader for a
// given import edge, there are two possible cases. In the normal case,
// the dependency has already been completely type-checked; doImport
// does a cache lookup and returns it. In the cyclic case, the entry in
// the cache is still necessarily incomplete, indicating a cycle. We
// perform the cycle check again to obtain the error message, and return
// the error.
//
// The result of using concurrency is about a 2.5x speedup for stdlib_test.
// TODO(adonovan): overhaul the package documentation.

View File

@@ -1,13 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.6
package loader
import "go/build"
func init() {
ignoreVendor = build.IgnoreVendor
}

View File

@@ -1,124 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package loader
import (
"go/ast"
"go/build"
"go/parser"
"go/token"
"io"
"os"
"strconv"
"sync"
"golang.org/x/tools/go/buildutil"
)
// We use a counting semaphore to limit
// the number of parallel I/O calls per process.
var ioLimit = make(chan bool, 10)
// parseFiles parses the Go source files within directory dir and
// returns the ASTs of the ones that could be at least partially parsed,
// along with a list of I/O and parse errors encountered.
//
// I/O is done via ctxt, which may specify a virtual file system.
// displayPath is used to transform the filenames attached to the ASTs.
//
func parseFiles(fset *token.FileSet, ctxt *build.Context, displayPath func(string) string, dir string, files []string, mode parser.Mode) ([]*ast.File, []error) {
if displayPath == nil {
displayPath = func(path string) string { return path }
}
var wg sync.WaitGroup
n := len(files)
parsed := make([]*ast.File, n)
errors := make([]error, n)
for i, file := range files {
if !buildutil.IsAbsPath(ctxt, file) {
file = buildutil.JoinPath(ctxt, dir, file)
}
wg.Add(1)
go func(i int, file string) {
ioLimit <- true // wait
defer func() {
wg.Done()
<-ioLimit // signal
}()
var rd io.ReadCloser
var err error
if ctxt.OpenFile != nil {
rd, err = ctxt.OpenFile(file)
} else {
rd, err = os.Open(file)
}
if err != nil {
errors[i] = err // open failed
return
}
// ParseFile may return both an AST and an error.
parsed[i], errors[i] = parser.ParseFile(fset, displayPath(file), rd, mode)
rd.Close()
}(i, file)
}
wg.Wait()
// Eliminate nils, preserving order.
var o int
for _, f := range parsed {
if f != nil {
parsed[o] = f
o++
}
}
parsed = parsed[:o]
o = 0
for _, err := range errors {
if err != nil {
errors[o] = err
o++
}
}
errors = errors[:o]
return parsed, errors
}
// scanImports returns the set of all import paths from all
// import specs in the specified files.
func scanImports(files []*ast.File) map[string]bool {
imports := make(map[string]bool)
for _, f := range files {
for _, decl := range f.Decls {
if decl, ok := decl.(*ast.GenDecl); ok && decl.Tok == token.IMPORT {
for _, spec := range decl.Specs {
spec := spec.(*ast.ImportSpec)
// NB: do not assume the program is well-formed!
path, err := strconv.Unquote(spec.Path.Value)
if err != nil {
continue // quietly ignore the error
}
if path == "C" {
continue // skip pseudopackage
}
imports[path] = true
}
}
}
}
return imports
}
// ---------- Internal helpers ----------
// TODO(adonovan): make this a method: func (*token.File) Contains(token.Pos)
func tokenFileContainsPos(f *token.File, pos token.Pos) bool {
p := int(pos)
base := f.Base()
return base <= p && p < base+f.Size()
}

View File

@@ -1,15 +0,0 @@
<div class="toggle" style="display: none">
<div class="collapsed">
<p class="exampleHeading toggleButton"><span class="text">Internal call graph</span></p>
</div>
<div class="expanded">
<p class="exampleHeading toggleButton"><span class="text">Internal call graph</span></p>
<p>
This viewer shows the portion of the internal call
graph of this package that is reachable from this function.
See the <a href='#pkg-callgraph'>package's call
graph</a> for more information.
</p>
<ul style="margin-left: 0.5in" id="callgraph-{{.Index}}" class="treeview"></ul>
</div>
</div>

View File

@@ -1,56 +0,0 @@
<!--
Copyright 2010 The Go Authors. All rights reserved.
Use of this source code is governed by a BSD-style
license that can be found in the LICENSE file.
-->
<style type='text/css'>@import "/doc/codewalk/codewalk.css";</style>
<script type="text/javascript" src="/doc/codewalk/codewalk.js"></script>
<div id="codewalk-main">
<div class="left" id="code-column">
<div id='sizer'></div>
<div id="code-area">
<div id="code-header" align="center">
<a id="code-popout-link" href="" target="_blank">
<img title="View code in new window" alt="Pop Out Code" src="/doc/codewalk/popout.png" style="display: block; float: right;"/>
</a>
<select id="code-selector">
{{range .File}}
<option value="/doc/codewalk/?fileprint=/{{urlquery .}}">{{html .}}</option>
{{end}}
</select>
</div>
<div id="code">
<iframe class="code-display" name="code-display" id="code-display"></iframe>
</div>
</div>
<div id="code-options" class="setting">
<span>code on <a id="set-code-left" class="selected" href="#">left</a> &bull; <a id="set-code-right" href="#">right</a></span>
<span>code width <span id="code-column-width">70%</span></span>
<span>filepaths <a id="show-filepaths" class="selected" href="#">shown</a> &bull; <a id="hide-filepaths" href="#">hidden</a></span>
</div>
</div>
<div class="right" id="comment-column">
<div id="comment-area">
{{range .Step}}
<div class="comment first last">
<a class="comment-link" href="/doc/codewalk/?fileprint=/{{urlquery .File}}&amp;lo={{urlquery .Lo}}&amp;hi={{urlquery .Hi}}#mark" target="code-display"></a>
<div class="comment-title">{{html .Title}}</div>
<div class="comment-text">
{{with .Err}}
ERROR LOADING FILE: {{html .}}<br/><br/>
{{end}}
{{.XML}}
</div>
<div class="comment-text file-name"><span class="path-file">{{html .}}</span></div>
</div>
{{end}}
</div>
<div id="comment-options" class="setting">
<a id="prev-comment" href="#"><span class="hotkey">p</span>revious step</a>
&bull;
<a id="next-comment" href="#"><span class="hotkey">n</span>ext step</a>
</div>
</div>
</div>

View File

@@ -1,16 +0,0 @@
<!--
Copyright 2010 The Go Authors. All rights reserved.
Use of this source code is governed by a BSD-style
license that can be found in the LICENSE file.
-->
<table class="layout">
{{range .}}
<tr>
{{$name_html := html .Name}}
<td><a href="{{$name_html}}">{{$name_html}}</a></td>
<td width="25">&nbsp;</td>
<td>{{html .Title}}</td>
</tr>
{{end}}
</table>

View File

@@ -1,31 +0,0 @@
<!--
Copyright 2009 The Go Authors. All rights reserved.
Use of this source code is governed by a BSD-style
license that can be found in the LICENSE file.
-->
<p>
<table class="layout">
<tr>
<th align="left">File</th>
<td width="25">&nbsp;</td>
<th align="right">Bytes</th>
<td width="25">&nbsp;</td>
<th align="left">Modified</th>
</tr>
<tr>
<td><a href="..">..</a></td>
</tr>
{{range .}}
<tr>
{{$name_html := fileInfoName . | html}}
<td align="left"><a href="{{$name_html}}">{{$name_html}}</a></td>
<td></td>
<td align="right">{{html .Size}}</td>
<td></td>
<td align="left">{{fileInfoTime . | html}}</td>
</tr>
{{end}}
</table>
</p>

View File

@@ -1,9 +0,0 @@
<!--
Copyright 2009 The Go Authors. All rights reserved.
Use of this source code is governed by a BSD-style
license that can be found in the LICENSE file.
-->
<p>
<span class="alert" style="font-size:120%">{{html .}}</span>
</p>

View File

@@ -1,30 +0,0 @@
<div id="example_{{.Name}}" class="toggle">
<div class="collapsed">
<p class="exampleHeading toggleButton"><span class="text">Example{{example_suffix .Name}}</span></p>
</div>
<div class="expanded">
<p class="exampleHeading toggleButton"><span class="text">Example{{example_suffix .Name}}</span></p>
{{with .Doc}}<p>{{html .}}</p>{{end}}
{{$output := .Output}}
{{with .Play}}
<div class="play">
<div class="input"><textarea class="code">{{html .}}</textarea></div>
<div class="output"><pre>{{html $output}}</pre></div>
<div class="buttons">
<a class="run" title="Run this code [shift-enter]">Run</a>
<a class="fmt" title="Format this code">Format</a>
{{if $.Share}}
<a class="share" title="Share this code">Share</a>
{{end}}
</div>
</div>
{{else}}
<p>Code:</p>
<pre class="code">{{.Code}}</pre>
{{with .Output}}
<p>Output:</p>
<pre class="output">{{html .}}</pre>
{{end}}
{{end}}
</div>
</div>

View File

@@ -1,145 +0,0 @@
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#375EAB">
{{with .Tabtitle}}
<title>{{html .}} - Amazon Web Services - Go SDK</title>
{{else}}
<title>Amazon Web Services - Go SDK</title>
{{end}}
<link type="text/css" rel="stylesheet" href="/sdk-for-go/api/lib/godoc/style.css">
<link rel="stylesheet" href="/sdk-for-go/api/lib/godoc/jquery.treeview.css">
<script type="text/javascript" src="/sdk-for-go/api/lib/godoc/jquery.js"></script>
<script type="text/javascript" src="/sdk-for-go/api/lib/godoc/jquery.treeview.js"></script>
<script type="text/javascript" src="/sdk-for-go/api/lib/godoc/jquery.treeview.edit.js"></script>
<script>
((window.gitter = {}).chat = {}).options = {
room: 'aws/aws-sdk-go'
};
</script>
<script src="https://sidecar.gitter.im/dist/sidecar.v1.js" async defer></script>
<script type="text/javascript" src="/sdk-for-go/api/lib/godoc/godocs.js"></script>
<script type="text/javascript">window.initFuncs = [];</script>
<script type="text/javascript">
var navOldStyle = {};
var pageOldStyle = {};
var divOldStyle = {};
var which = true;
function showMobileNav() {
if(which === false) return;
var nav = document.getElementById("pkglist");
var page = document.getElementById("page");
var div = window.frames["pkglist"].document.getElementById("mobile_container");
navOldStyle.display = nav.style.display;
navOldStyle.height = nav.style.height;
navOldStyle.width = nav.style.width;
navOldStyle.textAlign = nav.style.textAlign;
nav.style.display = "inline";
nav.style.height = "100%";
nav.style.width = "100%";
nav.style.textAlign = "center";
pageOldStyle.display = page.style.display;
page.style.display = "none";
divOldStyle.fontSize = div.style.fontSize;
divOldStyle.lineHeight = div.style.lineHeight;
divOldStyle.display = div.style.display;
div.style.fontSize = "x-large";
div.style.lineHeight = "150%";
div.style.display = "inline-block";
which = false;
}
function hideMobileNav() {
if(which == true) return;
var nav = document.getElementById("pkglist");
var page = document.getElementById("page");
var div = window.frames["pkglist"].document.getElementById("mobile_container");
nav.style.display = navOldStyle.display;
nav.style.height = navOldStyle.height;
nav.style.width = navOldStyle.width;
nav.style.textAlign = navOldStyle.textAlign;
page.style.display = pageOldStyle.display;
div.style.fontSize = divOldStyle.fontSize;
div.style.lineHeight = divOldStyle.lineHeight;
div.style.display = divOldStyle.display;
which = true;
}
function toggleMobileNav() {
if (which === true) {
showMobileNav();
} else {
hideMobileNav();
}
}
$(window).resize(function() {
if($('#fixed #topbar #mobile-nav').css('display') == "none") {
hideMobileNav();
}
})
</script>
</head>
<body>
<!-- used by search indexer -->
<div id="product_name" style="display:none;">AWS SDK for Go</div>
<div id="guide_name" style="display:none;">API Reference</div>
<div id='lowframe' style="position: fixed; bottom: 0; left: 0; height: 0; width: 100%; border-top: thin solid grey; background-color: white; overflow: auto;">
...
</div><!-- #lowframe -->
<div id="fixed">
<div id="topbar"{{if .Title}} class="wide"{{end}}>
<div style="display:inline-block; color:white; padding-top:2px;">
<a href="/sdk-for-go/api" id="title" class="title">AWS SDK for Go API Reference</a>
</div>
<div id="mobile-nav" onclick="toggleMobileNav()"></div>
<div class="top_link">
<a class="title" href="https://aws.amazon.com/forms/aws-doc-feedback?hidden_service_name=godoc&hidden_file_name=godoc&hidden_guide_name=godoc&hidden_api_version=HEAD">Feedback</a>
</div>
<div class="top_link">
<a class="title" href="https://aws.amazon.com/blogs/developer/category/go/">Blog</a>
</div>
<div class="top_link">
<a class="title" href="http://aws.amazon.com/sdk-for-go/">Developer Guide</a>
</div>
</div>
<iframe name="pkglist" id="pkglist" src="/sdk-for-go/api/pkglist.html" class="pkg-dir"></iframe>
<div id="page"{{if .Title}} class="wide"{{end}}>
<div class="container">
{{/* The Table of Contents is automatically inserted in this <div>.
Do not delete this <div>. */}}
{{ if not .NoTOC }}
<div id="nav"></div>
{{ end }}
<div id="pkg-nav">
</div>
{{/* Body is HTML-escaped elsewhere */}}
{{printf "%s" .Body}}
<div id="body" class="container-body"> </div>
</div>
</div>
</div><!-- .container -->
</div><!-- #page -->
</body>
</html>

View File

@@ -1,571 +0,0 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
/* A little code to ease navigation of these documents.
*
* On window load we:
* + Bind search box hint placeholder show/hide events (bindSearchEvents)
* + Generate a table of contents (generateTOC)
* + Bind foldable sections (bindToggles)
* + Bind links to foldable sections (bindToggleLinks)
*/
(function() {
'use strict';
// Mobile-friendly topbar menu
$(function() {
var menu = $('#menu');
var menuButton = $('#menu-button');
var menuButtonArrow = $('#menu-button-arrow');
menuButton.click(function(event) {
menu.toggleClass('menu-visible');
menuButtonArrow.toggleClass('vertical-flip');
event.preventDefault();
return false;
});
});
function bindSearchEvents() {
var search = $('#search');
if (search.length === 0) {
return; // no search box
}
function clearInactive() {
if (search.is('.inactive')) {
search.val('');
search.removeClass('inactive');
}
}
function restoreInactive() {
if (search.val() !== '') {
return;
}
search.val(search.attr('placeholder'));
search.addClass('inactive');
}
search.on('focus', clearInactive);
search.on('blur', restoreInactive);
restoreInactive();
}
/* Generates a table of contents: looks for h2 and h3 elements and generates
* links. "Decorates" the element with id=="nav" with this table of contents.
*/
function generateTOC() {
if ($('#manual-nav').length > 0) {
return;
}
var nav = $('#nav');
if (nav.length === 0) {
return;
}
var toc_items = [];
$(nav).nextAll('h2, h3').each(function() {
var node = this;
if (node.id == '')
node.id = 'tmp_' + toc_items.length;
var link = $('<a/>').attr('href', '#' + node.id).text($(node).text());
var item;
if ($(node).is('h2')) {
item = $('<dt/>');
} else { // h3
item = $('<dd class="indent"/>');
}
item.append(link);
toc_items.push(item);
});
if (toc_items.length <= 1) {
return;
}
var dl1 = $('<dl/>');
var dl2 = $('<dl/>');
var split_index = (toc_items.length / 2) + 1;
if (split_index < 8) {
split_index = toc_items.length;
}
for (var i = 0; i < split_index; i++) {
dl1.append(toc_items[i]);
}
for (/* keep using i */; i < toc_items.length; i++) {
dl2.append(toc_items[i]);
}
var tocTable = $('<table class="unruled"/>').appendTo(nav);
var tocBody = $('<tbody/>').appendTo(tocTable);
var tocRow = $('<tr/>').appendTo(tocBody);
// 1st column
$('<td class="first"/>').appendTo(tocRow).append(dl1);
// 2nd column
$('<td/>').appendTo(tocRow).append(dl2);
}
function bindToggle(el) {
$('.toggleButton', el).click(function() {
if ($(el).is('.toggle')) {
$(el).addClass('toggleVisible').removeClass('toggle');
} else {
$(el).addClass('toggle').removeClass('toggleVisible');
}
});
}
function bindToggles(selector) {
$(selector).each(function(i, el) {
bindToggle(el);
});
}
function bindToggleLink(el, prefix) {
$(el).click(function() {
var href = $(el).attr('href');
var i = href.indexOf('#'+prefix);
if (i < 0) {
return;
}
var id = '#' + prefix + href.slice(i+1+prefix.length);
if ($(id).is('.toggle')) {
$(id).find('.toggleButton').first().click();
}
});
}
function bindToggleLinks(selector, prefix) {
$(selector).each(function(i, el) {
bindToggleLink(el, prefix);
});
}
function setupDropdownPlayground() {
if (!$('#page').is('.wide')) {
return; // don't show on front page
}
var button = $('#playgroundButton');
var div = $('#playground');
var setup = false;
button.toggle(function() {
button.addClass('active');
div.show();
if (setup) {
return;
}
setup = true;
playground({
'codeEl': $('.code', div),
'outputEl': $('.output', div),
'runEl': $('.run', div),
'fmtEl': $('.fmt', div),
'shareEl': $('.share', div),
'shareRedirect': '//play.golang.org/p/'
});
},
function() {
button.removeClass('active');
div.hide();
});
button.show();
$('#menu').css('min-width', '+=60');
}
function setupInlinePlayground() {
'use strict';
// Set up playground when each element is toggled.
$('div.play').each(function (i, el) {
// Set up playground for this example.
var setup = function() {
var code = $('.code', el);
playground({
'codeEl': code,
'outputEl': $('.output', el),
'runEl': $('.run', el),
'fmtEl': $('.fmt', el),
'shareEl': $('.share', el),
'shareRedirect': '//play.golang.org/p/'
});
// Make the code textarea resize to fit content.
var resize = function() {
code.height(0);
var h = code[0].scrollHeight;
code.height(h+20); // minimize bouncing.
code.closest('.input').height(h);
};
code.on('keydown', resize);
code.on('keyup', resize);
code.keyup(); // resize now.
};
// If example already visible, set up playground now.
if ($(el).is(':visible')) {
setup();
return;
}
// Otherwise, set up playground when example is expanded.
var built = false;
$(el).closest('.toggle').click(function() {
// Only set up once.
if (!built) {
setup();
built = true;
}
});
});
}
// fixFocus tries to put focus to div#page so that keyboard navigation works.
function fixFocus() {
var page = $('div#page');
var topbar = $('div#topbar');
page.css('outline', 0); // disable outline when focused
page.attr('tabindex', -1); // and set tabindex so that it is focusable
$(window).resize(function (evt) {
// only focus page when the topbar is at fixed position (that is, it's in
// front of page, and keyboard event will go to the former by default.)
// by focusing page, keyboard event will go to page so that up/down arrow,
// space, etc. will work as expected.
if (topbar.css('position') == "fixed")
page.focus();
}).resize();
}
function toggleHash() {
var hash = $(window.location.hash);
if (hash.is('.toggle')) {
hash.find('.toggleButton').first().click();
}
}
function personalizeInstallInstructions() {
var prefix = '?download=';
var s = window.location.search;
if (s.indexOf(prefix) != 0) {
// No 'download' query string; bail.
return;
}
var filename = s.substr(prefix.length);
var filenameRE = /^go1\.\d+(\.\d+)?([a-z0-9]+)?\.([a-z0-9]+)(-[a-z0-9]+)?(-osx10\.[68])?\.([a-z.]+)$/;
$('.downloadFilename').text(filename);
$('.hideFromDownload').hide();
var m = filenameRE.exec(filename);
if (!m) {
// Can't interpret file name; bail.
return;
}
var os = m[3];
var ext = m[6];
if (ext != 'tar.gz') {
$('#tarballInstructions').hide();
}
if (os != 'darwin' || ext != 'pkg') {
$('#darwinPackageInstructions').hide();
}
if (os != 'windows') {
$('#windowsInstructions').hide();
$('.testUnix').show();
$('.testWindows').hide();
} else {
if (ext != 'msi') {
$('#windowsInstallerInstructions').hide();
}
if (ext != 'zip') {
$('#windowsZipInstructions').hide();
}
$('.testUnix').hide();
$('.testWindows').show();
}
var download = "https://storage.googleapis.com/golang/" + filename;
var message = $('<p class="downloading">'+
'Your download should begin shortly. '+
'If it does not, click <a>this link</a>.</p>');
message.find('a').attr('href', download);
message.insertAfter('#nav');
window.location = download;
}
$(document).ready(function() {
bindSearchEvents();
generateTOC();
bindToggles(".toggle");
bindToggles(".toggleVisible");
bindToggleLinks(".exampleLink", "example_");
bindToggleLinks(".overviewLink", "");
bindToggleLinks(".examplesLink", "");
bindToggleLinks(".indexLink", "");
setupDropdownPlayground();
setupInlinePlayground();
fixFocus();
setupTypeInfo();
setupCallgraphs();
toggleHash();
personalizeInstallInstructions();
// godoc.html defines window.initFuncs in the <head> tag, and root.html and
// codewalk.js push their on-page-ready functions to the list.
// We execute those functions here, to avoid loading jQuery until the page
// content is loaded.
for (var i = 0; i < window.initFuncs.length; i++) window.initFuncs[i]();
});
// -- analysis ---------------------------------------------------------
// escapeHTML returns HTML for s, with metacharacters quoted.
// It is safe for use in both elements and attributes
// (unlike the "set innerText, read innerHTML" trick).
function escapeHTML(s) {
return s.replace(/&/g, '&amp;').
replace(/\"/g, '&quot;').
replace(/\'/g, '&#39;').
replace(/</g, '&lt;').
replace(/>/g, '&gt;');
}
// makeAnchor returns HTML for an <a> element, given an anchorJSON object.
function makeAnchor(json) {
var html = escapeHTML(json.Text);
if (json.Href != "") {
html = "<a href='" + escapeHTML(json.Href) + "'>" + html + "</a>";
}
return html;
}
function showLowFrame(html) {
var lowframe = document.getElementById('lowframe');
lowframe.style.height = "200px";
lowframe.innerHTML = "<p style='text-align: left;'>" + html + "</p>\n" +
"<div onclick='hideLowFrame()' style='position: absolute; top: 0; right: 0; cursor: pointer;'>✘</div>"
};
document.hideLowFrame = function() {
var lowframe = document.getElementById('lowframe');
lowframe.style.height = "0px";
}
// onClickCallers is the onclick action for the 'func' tokens of a
// function declaration.
document.onClickCallers = function(index) {
var data = document.ANALYSIS_DATA[index]
if (data.Callers.length == 1 && data.Callers[0].Sites.length == 1) {
document.location = data.Callers[0].Sites[0].Href; // jump to sole caller
return;
}
var html = "Callers of <code>" + escapeHTML(data.Callee) + "</code>:<br/>\n";
for (var i = 0; i < data.Callers.length; i++) {
var caller = data.Callers[i];
html += "<code>" + escapeHTML(caller.Func) + "</code>";
var sites = caller.Sites;
if (sites != null && sites.length > 0) {
html += " at line ";
for (var j = 0; j < sites.length; j++) {
if (j > 0) {
html += ", ";
}
html += "<code>" + makeAnchor(sites[j]) + "</code>";
}
}
html += "<br/>\n";
}
showLowFrame(html);
};
// onClickCallees is the onclick action for the '(' token of a function call.
document.onClickCallees = function(index) {
var data = document.ANALYSIS_DATA[index]
if (data.Callees.length == 1) {
document.location = data.Callees[0].Href; // jump to sole callee
return;
}
var html = "Callees of this " + escapeHTML(data.Descr) + ":<br/>\n";
for (var i = 0; i < data.Callees.length; i++) {
html += "<code>" + makeAnchor(data.Callees[i]) + "</code><br/>\n";
}
showLowFrame(html);
};
// onClickTypeInfo is the onclick action for identifiers declaring a named type.
document.onClickTypeInfo = function(index) {
var data = document.ANALYSIS_DATA[index];
var html = "Type <code>" + data.Name + "</code>: " +
"&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<small>(size=" + data.Size + ", align=" + data.Align + ")</small><br/>\n";
html += implementsHTML(data);
html += methodsetHTML(data);
showLowFrame(html);
};
// implementsHTML returns HTML for the implements relation of the
// specified TypeInfoJSON value.
function implementsHTML(info) {
var html = "";
if (info.ImplGroups != null) {
for (var i = 0; i < info.ImplGroups.length; i++) {
var group = info.ImplGroups[i];
var x = "<code>" + escapeHTML(group.Descr) + "</code> ";
for (var j = 0; j < group.Facts.length; j++) {
var fact = group.Facts[j];
var y = "<code>" + makeAnchor(fact.Other) + "</code>";
if (fact.ByKind != null) {
html += escapeHTML(fact.ByKind) + " type " + y + " implements " + x;
} else {
html += x + " implements " + y;
}
html += "<br/>\n";
}
}
}
return html;
}
// methodsetHTML returns HTML for the methodset of the specified
// TypeInfoJSON value.
function methodsetHTML(info) {
var html = "";
if (info.Methods != null) {
for (var i = 0; i < info.Methods.length; i++) {
html += "<code>" + makeAnchor(info.Methods[i]) + "</code><br/>\n";
}
}
return html;
}
// onClickComm is the onclick action for channel "make" and "<-"
// send/receive tokens.
document.onClickComm = function(index) {
var ops = document.ANALYSIS_DATA[index].Ops
if (ops.length == 1) {
document.location = ops[0].Op.Href; // jump to sole element
return;
}
var html = "Operations on this channel:<br/>\n";
for (var i = 0; i < ops.length; i++) {
html += makeAnchor(ops[i].Op) + " by <code>" + escapeHTML(ops[i].Fn) + "</code><br/>\n";
}
if (ops.length == 0) {
html += "(none)<br/>\n";
}
showLowFrame(html);
};
$(window).load(function() {
// Scroll window so that first selection is visible.
// (This means we don't need to emit id='L%d' spans for each line.)
// TODO(adonovan): ideally, scroll it so that it's under the pointer,
// but I don't know how to get the pointer y coordinate.
var elts = document.getElementsByClassName("selection");
if (elts.length > 0) {
elts[0].scrollIntoView()
}
});
// setupTypeInfo populates the "Implements" and "Method set" toggle for
// each type in the package doc.
function setupTypeInfo() {
for (var i in document.ANALYSIS_DATA) {
var data = document.ANALYSIS_DATA[i];
var el = document.getElementById("implements-" + i);
if (el != null) {
// el != null => data is TypeInfoJSON.
if (data.ImplGroups != null) {
el.innerHTML = implementsHTML(data);
el.parentNode.parentNode.style.display = "block";
}
}
var el = document.getElementById("methodset-" + i);
if (el != null) {
// el != null => data is TypeInfoJSON.
if (data.Methods != null) {
el.innerHTML = methodsetHTML(data);
el.parentNode.parentNode.style.display = "block";
}
}
}
}
function setupCallgraphs() {
if (document.CALLGRAPH == null) {
return
}
document.getElementById("pkg-callgraph").style.display = "block";
var treeviews = document.getElementsByClassName("treeview");
for (var i = 0; i < treeviews.length; i++) {
var tree = treeviews[i];
if (tree.id == null || tree.id.indexOf("callgraph-") != 0) {
continue;
}
var id = tree.id.substring("callgraph-".length);
$(tree).treeview({collapsed: true, animated: "fast"});
document.cgAddChildren(tree, tree, [id]);
tree.parentNode.parentNode.style.display = "block";
}
}
document.cgAddChildren = function(tree, ul, indices) {
if (indices != null) {
for (var i = 0; i < indices.length; i++) {
var li = cgAddChild(tree, ul, document.CALLGRAPH[indices[i]]);
if (i == indices.length - 1) {
$(li).addClass("last");
}
}
}
$(tree).treeview({animated: "fast", add: ul});
}
// cgAddChild adds an <li> element for document.CALLGRAPH node cgn to
// the parent <ul> element ul. tree is the tree's root <ul> element.
function cgAddChild(tree, ul, cgn) {
var li = document.createElement("li");
ul.appendChild(li);
li.className = "closed";
var code = document.createElement("code");
if (cgn.Callees != null) {
$(li).addClass("expandable");
// Event handlers and innerHTML updates don't play nicely together,
// hence all this explicit DOM manipulation.
var hitarea = document.createElement("div");
hitarea.className = "hitarea expandable-hitarea";
li.appendChild(hitarea);
li.appendChild(code);
var childUL = document.createElement("ul");
li.appendChild(childUL);
childUL.setAttribute('style', "display: none;");
var onClick = function() {
document.cgAddChildren(tree, childUL, cgn.Callees);
hitarea.removeEventListener('click', onClick)
};
hitarea.addEventListener('click', onClick);
} else {
li.appendChild(code);
}
code.innerHTML += "&nbsp;" + makeAnchor(cgn.Func);
return li
}
})();

View File

@@ -1,9 +0,0 @@
<div class="toggle" style="display: none">
<div class="collapsed">
<p class="exampleHeading toggleButton"><span class="text">Implements</span></p>
</div>
<div class="expanded">
<p class="exampleHeading toggleButton"><span class="text">Implements</span></p>
<div style="margin-left: 1in" id='implements-{{.Index}}'>...</div>
</div>
</div>

File diff suppressed because one or more lines are too long

View File

@@ -1,76 +0,0 @@
/* https://github.com/jzaefferer/jquery-treeview/blob/master/jquery.treeview.css */
/* License: MIT. */
.treeview, .treeview ul {
padding: 0;
margin: 0;
list-style: none;
}
.treeview ul {
background-color: white;
margin-top: 4px;
}
.treeview .hitarea {
background: url(images/treeview-default.gif) -64px -25px no-repeat;
height: 16px;
width: 16px;
margin-left: -16px;
float: left;
cursor: pointer;
}
/* fix for IE6 */
* html .hitarea {
display: inline;
float:none;
}
.treeview li {
margin: 0;
padding: 3px 0pt 3px 16px;
}
.treeview a.selected {
background-color: #eee;
}
#treecontrol { margin: 1em 0; display: none; }
.treeview .hover { color: red; cursor: pointer; }
.treeview li { background: url(images/treeview-default-line.gif) 0 0 no-repeat; }
.treeview li.collapsable, .treeview li.expandable { background-position: 0 -176px; }
.treeview .expandable-hitarea { background-position: -80px -3px; }
.treeview li.last { background-position: 0 -1766px }
.treeview li.lastCollapsable, .treeview li.lastExpandable { background-image: url(images/treeview-default.gif); }
.treeview li.lastCollapsable { background-position: 0 -111px }
.treeview li.lastExpandable { background-position: -32px -67px }
.treeview div.lastCollapsable-hitarea, .treeview div.lastExpandable-hitarea { background-position: 0; }
.treeview-red li { background-image: url(images/treeview-red-line.gif); }
.treeview-red .hitarea, .treeview-red li.lastCollapsable, .treeview-red li.lastExpandable { background-image: url(images/treeview-red.gif); }
.treeview-black li { background-image: url(images/treeview-black-line.gif); }
.treeview-black .hitarea, .treeview-black li.lastCollapsable, .treeview-black li.lastExpandable { background-image: url(images/treeview-black.gif); }
.treeview-gray li { background-image: url(images/treeview-gray-line.gif); }
.treeview-gray .hitarea, .treeview-gray li.lastCollapsable, .treeview-gray li.lastExpandable { background-image: url(images/treeview-gray.gif); }
.treeview-famfamfam li { background-image: url(images/treeview-famfamfam-line.gif); }
.treeview-famfamfam .hitarea, .treeview-famfamfam li.lastCollapsable, .treeview-famfamfam li.lastExpandable { background-image: url(images/treeview-famfamfam.gif); }
.treeview .placeholder {
background: url(images/ajax-loader.gif) 0 0 no-repeat;
height: 16px;
width: 16px;
display: block;
}
.filetree li { padding: 3px 0 2px 16px; }
.filetree span.folder, .filetree span.file { padding: 1px 0 1px 16px; display: block; }
.filetree span.folder { background: url(images/folder.gif) 0 0 no-repeat; }
.filetree li.expandable span.folder { background: url(images/folder-closed.gif) 0 0 no-repeat; }
.filetree span.file { background: url(images/file.gif) 0 0 no-repeat; }

View File

@@ -1,39 +0,0 @@
/* https://github.com/jzaefferer/jquery-treeview/blob/master/jquery.treeview.edit.js */
/* License: MIT. */
(function($) {
var CLASSES = $.treeview.classes;
var proxied = $.fn.treeview;
$.fn.treeview = function(settings) {
settings = $.extend({}, settings);
if (settings.add) {
return this.trigger("add", [settings.add]);
}
if (settings.remove) {
return this.trigger("remove", [settings.remove]);
}
return proxied.apply(this, arguments).bind("add", function(event, branches) {
$(branches).prev()
.removeClass(CLASSES.last)
.removeClass(CLASSES.lastCollapsable)
.removeClass(CLASSES.lastExpandable)
.find(">.hitarea")
.removeClass(CLASSES.lastCollapsableHitarea)
.removeClass(CLASSES.lastExpandableHitarea);
$(branches).find("li").andSelf().prepareBranches(settings).applyClasses(settings, $(this).data("toggler"));
}).bind("remove", function(event, branches) {
var prev = $(branches).prev();
var parent = $(branches).parent();
$(branches).remove();
prev.filter(":last-child").addClass(CLASSES.last)
.filter("." + CLASSES.expandable).replaceClass(CLASSES.last, CLASSES.lastExpandable).end()
.find(">.hitarea").replaceClass(CLASSES.expandableHitarea, CLASSES.lastExpandableHitarea).end()
.filter("." + CLASSES.collapsable).replaceClass(CLASSES.last, CLASSES.lastCollapsable).end()
.find(">.hitarea").replaceClass(CLASSES.collapsableHitarea, CLASSES.lastCollapsableHitarea);
if (parent.is(":not(:has(>))") && parent[0] != this) {
parent.parent().removeClass(CLASSES.collapsable).removeClass(CLASSES.expandable)
parent.siblings(".hitarea").andSelf().remove();
}
});
};
})(jQuery);

View File

@@ -1,256 +0,0 @@
/*
* Treeview 1.4.1 - jQuery plugin to hide and show branches of a tree
*
* http://bassistance.de/jquery-plugins/jquery-plugin-treeview/
* http://docs.jquery.com/Plugins/Treeview
*
* Copyright (c) 2007 Jörn Zaefferer
*
* Dual licensed under the MIT and GPL licenses:
* http://www.opensource.org/licenses/mit-license.php
* http://www.gnu.org/licenses/gpl.html
*
* Revision: $Id: jquery.treeview.js 5759 2008-07-01 07:50:28Z joern.zaefferer $
*
*/
;(function($) {
// TODO rewrite as a widget, removing all the extra plugins
$.extend($.fn, {
swapClass: function(c1, c2) {
var c1Elements = this.filter('.' + c1);
this.filter('.' + c2).removeClass(c2).addClass(c1);
c1Elements.removeClass(c1).addClass(c2);
return this;
},
replaceClass: function(c1, c2) {
return this.filter('.' + c1).removeClass(c1).addClass(c2).end();
},
hoverClass: function(className) {
className = className || "hover";
return this.hover(function() {
$(this).addClass(className);
}, function() {
$(this).removeClass(className);
});
},
heightToggle: function(animated, callback) {
animated ?
this.animate({ height: "toggle" }, animated, callback) :
this.each(function(){
jQuery(this)[ jQuery(this).is(":hidden") ? "show" : "hide" ]();
if(callback)
callback.apply(this, arguments);
});
},
heightHide: function(animated, callback) {
if (animated) {
this.animate({ height: "hide" }, animated, callback);
} else {
this.hide();
if (callback)
this.each(callback);
}
},
prepareBranches: function(settings) {
if (!settings.prerendered) {
// mark last tree items
this.filter(":last-child:not(ul)").addClass(CLASSES.last);
// collapse whole tree, or only those marked as closed, anyway except those marked as open
this.filter((settings.collapsed ? "" : "." + CLASSES.closed) + ":not(." + CLASSES.open + ")").find(">ul").hide();
}
// return all items with sublists
return this.filter(":has(>ul)");
},
applyClasses: function(settings, toggler) {
// TODO use event delegation
this.filter(":has(>ul):not(:has(>a))").find(">span").unbind("click.treeview").bind("click.treeview", function(event) {
// don't handle click events on children, eg. checkboxes
if ( this == event.target )
toggler.apply($(this).next());
}).add( $("a", this) ).hoverClass();
if (!settings.prerendered) {
// handle closed ones first
this.filter(":has(>ul:hidden)")
.addClass(CLASSES.expandable)
.replaceClass(CLASSES.last, CLASSES.lastExpandable);
// handle open ones
this.not(":has(>ul:hidden)")
.addClass(CLASSES.collapsable)
.replaceClass(CLASSES.last, CLASSES.lastCollapsable);
// create hitarea if not present
var hitarea = this.find("div." + CLASSES.hitarea);
if (!hitarea.length)
hitarea = this.prepend("<div class=\"" + CLASSES.hitarea + "\"/>").find("div." + CLASSES.hitarea);
hitarea.removeClass().addClass(CLASSES.hitarea).each(function() {
var classes = "";
$.each($(this).parent().attr("class").split(" "), function() {
classes += this + "-hitarea ";
});
$(this).addClass( classes );
})
}
// apply event to hitarea
this.find("div." + CLASSES.hitarea).click( toggler );
},
treeview: function(settings) {
settings = $.extend({
cookieId: "treeview"
}, settings);
if ( settings.toggle ) {
var callback = settings.toggle;
settings.toggle = function() {
return callback.apply($(this).parent()[0], arguments);
};
}
// factory for treecontroller
function treeController(tree, control) {
// factory for click handlers
function handler(filter) {
return function() {
// reuse toggle event handler, applying the elements to toggle
// start searching for all hitareas
toggler.apply( $("div." + CLASSES.hitarea, tree).filter(function() {
// for plain toggle, no filter is provided, otherwise we need to check the parent element
return filter ? $(this).parent("." + filter).length : true;
}) );
return false;
};
}
// click on first element to collapse tree
$("a:eq(0)", control).click( handler(CLASSES.collapsable) );
// click on second to expand tree
$("a:eq(1)", control).click( handler(CLASSES.expandable) );
// click on third to toggle tree
$("a:eq(2)", control).click( handler() );
}
// handle toggle event
function toggler() {
$(this)
.parent()
// swap classes for hitarea
.find(">.hitarea")
.swapClass( CLASSES.collapsableHitarea, CLASSES.expandableHitarea )
.swapClass( CLASSES.lastCollapsableHitarea, CLASSES.lastExpandableHitarea )
.end()
// swap classes for parent li
.swapClass( CLASSES.collapsable, CLASSES.expandable )
.swapClass( CLASSES.lastCollapsable, CLASSES.lastExpandable )
// find child lists
.find( ">ul" )
// toggle them
.heightToggle( settings.animated, settings.toggle );
if ( settings.unique ) {
$(this).parent()
.siblings()
// swap classes for hitarea
.find(">.hitarea")
.replaceClass( CLASSES.collapsableHitarea, CLASSES.expandableHitarea )
.replaceClass( CLASSES.lastCollapsableHitarea, CLASSES.lastExpandableHitarea )
.end()
.replaceClass( CLASSES.collapsable, CLASSES.expandable )
.replaceClass( CLASSES.lastCollapsable, CLASSES.lastExpandable )
.find( ">ul" )
.heightHide( settings.animated, settings.toggle );
}
}
this.data("toggler", toggler);
function serialize() {
function binary(arg) {
return arg ? 1 : 0;
}
var data = [];
branches.each(function(i, e) {
data[i] = $(e).is(":has(>ul:visible)") ? 1 : 0;
});
$.cookie(settings.cookieId, data.join(""), settings.cookieOptions );
}
function deserialize() {
var stored = $.cookie(settings.cookieId);
if ( stored ) {
var data = stored.split("");
branches.each(function(i, e) {
$(e).find(">ul")[ parseInt(data[i]) ? "show" : "hide" ]();
});
}
}
// add treeview class to activate styles
this.addClass("treeview");
// prepare branches and find all tree items with child lists
var branches = this.find("li").prepareBranches(settings);
switch(settings.persist) {
case "cookie":
var toggleCallback = settings.toggle;
settings.toggle = function() {
serialize();
if (toggleCallback) {
toggleCallback.apply(this, arguments);
}
};
deserialize();
break;
case "location":
var current = this.find("a").filter(function() {
return this.href.toLowerCase() == location.href.toLowerCase();
});
if ( current.length ) {
// TODO update the open/closed classes
var items = current.addClass("selected").parents("ul, li").add( current.next() ).show();
if (settings.prerendered) {
// if prerendered is on, replicate the basic class swapping
items.filter("li")
.swapClass( CLASSES.collapsable, CLASSES.expandable )
.swapClass( CLASSES.lastCollapsable, CLASSES.lastExpandable )
.find(">.hitarea")
.swapClass( CLASSES.collapsableHitarea, CLASSES.expandableHitarea )
.swapClass( CLASSES.lastCollapsableHitarea, CLASSES.lastExpandableHitarea );
}
}
break;
}
branches.applyClasses(settings, toggler);
// if control option is set, create the treecontroller and show it
if ( settings.control ) {
treeController(this, settings.control);
$(settings.control).show();
}
return this;
}
});
// classes used by the plugin
// need to be styled via external stylesheet, see first example
$.treeview = {};
var CLASSES = ($.treeview.classes = {
open: "open",
closed: "closed",
expandable: "expandable",
expandableHitarea: "expandable-hitarea",
lastExpandableHitarea: "lastExpandable-hitarea",
collapsable: "collapsable",
collapsableHitarea: "collapsable-hitarea",
lastCollapsableHitarea: "lastCollapsable-hitarea",
lastCollapsable: "lastCollapsable",
lastExpandable: "lastExpandable",
last: "last",
hitarea: "hitarea"
});
})(jQuery);

View File

@@ -1,9 +0,0 @@
<div class="toggle" style="display: none">
<div class="collapsed">
<p class="exampleHeading toggleButton"><span class="text">Method set</span></p>
</div>
<div class="expanded">
<p class="exampleHeading toggleButton"><span class="text">Method set</span></p>
<div style="margin-left: 1in" id='methodset-{{.Index}}'>...</div>
</div>
</div>

View File

@@ -1,11 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
<ShortName>godoc</ShortName>
<Description>The Go Programming Language</Description>
<Tags>go golang</Tags>
<Contact />
<Url type="text/html" template="{{.BaseURL}}/search?q={searchTerms}" />
<Image height="15" width="16" type="image/x-icon">/favicon.ico</Image>
<OutputEncoding>UTF-8</OutputEncoding>
<InputEncoding>UTF-8</InputEncoding>
</OpenSearchDescription>

View File

@@ -1,116 +0,0 @@
{{$info := .}}{{$filtered := .IsFiltered}}{{/*
---------------------------------------
*/}}{{if $filtered}}{{range .PAst}}{{range .Decls}}{{node $info .}}
{{end}}{{end}}{{else}}{{with .PAst}}{{range $filename, $ast := .}}{{$filename}}:
{{node $ $ast}}{{end}}{{end}}{{end}}{{/*
---------------------------------------
*/}}{{if and $filtered (not (or .PDoc .PAst))}}No match found.
{{end}}{{with .PDoc}}{{if $.IsMain}}COMMAND DOCUMENTATION
{{comment_text .Doc " " "\t"}}
{{else}}{{if not $filtered}}PACKAGE DOCUMENTATION
package {{.Name}}
import "{{.ImportPath}}"
{{comment_text .Doc " " "\t"}}
{{example_text $ "" " "}}{{end}}{{/*
---------------------------------------
*/}}{{with .Consts}}{{if not $filtered}}CONSTANTS
{{end}}{{range .}}{{node $ .Decl}}
{{comment_text .Doc " " "\t"}}
{{end}}{{end}}{{/*
---------------------------------------
*/}}{{with .Vars}}{{if not $filtered}}VARIABLES
{{end}}{{range .}}{{node $ .Decl}}
{{comment_text .Doc " " "\t"}}
{{end}}{{end}}{{/*
---------------------------------------
*/}}{{with .Funcs}}{{if not $filtered}}FUNCTIONS
{{end}}{{range .}}{{node $ .Decl}}
{{comment_text .Doc " " "\t"}}
{{example_text $ .Name " "}}{{end}}{{end}}{{/*
---------------------------------------
*/}}{{with .Types}}{{if not $filtered}}TYPES
{{end}}{{range .}}{{$tname := .Name}}{{node $ .Decl}}
{{comment_text .Doc " " "\t"}}
{{/*
---------------------------------------
*/}}{{if .Consts}}{{range .Consts}}{{node $ .Decl}}
{{comment_text .Doc " " "\t"}}
{{end}}{{end}}{{/*
---------------------------------------
*/}}{{if .Vars}}{{range .Vars}}{{node $ .Decl}}
{{comment_text .Doc " " "\t"}}
{{range $name := .Names}}{{example_text $ $name " "}}{{end}}{{end}}{{end}}{{/*
---------------------------------------
*/}}{{if .Funcs}}{{range .Funcs}}{{node $ .Decl}}
{{comment_text .Doc " " "\t"}}
{{example_text $ .Name " "}}{{end}}{{end}}{{/*
---------------------------------------
*/}}{{if .Methods}}{{range .Methods}}{{node $ .Decl}}
{{comment_text .Doc " " "\t"}}
{{$name := printf "%s_%s" $tname .Name}}{{example_text $ $name " "}}{{end}}{{end}}{{/*
---------------------------------------
*/}}{{end}}{{end}}{{/*
---------------------------------------
*/}}{{if and $filtered (not (or .Consts (or .Vars (or .Funcs .Types))))}}No match found.
{{end}}{{/*
---------------------------------------
*/}}{{end}}{{/*
---------------------------------------
*/}}{{with $.Notes}}
{{range $marker, $content := .}}
{{$marker}}S
{{range $content}}{{comment_text .Body " " "\t"}}
{{end}}{{end}}{{end}}{{end}}{{/*
---------------------------------------
*/}}{{if not $filtered}}{{with .Dirs}}SUBDIRECTORIES
{{if $.DirFlat}}{{range .List}}{{if .HasPkg}}
{{.Path}}{{end}}{{end}}
{{else}}{{range .List}}
{{repeat `. ` .Depth}}{{.Name}}{{end}}
{{end}}{{end}}{{/*
---------------------------------------
*/}}{{end}}{{/*
Make sure there is no newline at the end of this file.
perl -i -pe 'chomp if eof' package.txt
*/}}

View File

@@ -1,245 +0,0 @@
<!--
Copyright 2009 The Go Authors. All rights reserved.
Use of this source code is governed by a BSD-style
license that can be found in the LICENSE file.
-->
<!--
Note: Static (i.e., not template-generated) href and id
attributes start with "pkg-" to make it impossible for
them to conflict with generated attributes (some of which
correspond to Go identifiers).
-->
{{with .PDoc}}
<script type='text/javascript'>
document.ANALYSIS_DATA = {{$.AnalysisData}};
document.CALLGRAPH = {{$.CallGraph}};
</script>
{{if $.IsMain}}
{{/* command documentation */}}
{{comment_html .Doc}}
{{else}}
{{/* package documentation */}}
<div id="short-nav">
<dl>
<dd><code>import "github.com/aws/aws-sdk-go/{{html .ImportPath}}"</code></dd>
</dl>
<dl>
<dd><a href="#pkg-overview" class="overviewLink">Overview</a></dd>
<dd><a href="#pkg-index" class="indexLink">Index</a></dd>
{{if $.Examples}}
<dd><a href="#pkg-examples" class="examplesLink">Examples</a></dd>
{{end}}
</dl>
</div>
<!-- The package's Name is printed as title by the top-level template -->
<div id="pkg-overview" class="toggleVisible">
<div class="collapsed">
<h2 class="toggleButton" title="Click to show Overview section">Overview ▹</h2>
</div>
<div class="expanded">
<h2 class="toggleButton" title="Click to hide Overview section">Overview ▾</h2>
{{comment_html .Doc}}
</div>
</div>
{{example_html $ ""}}
<div id="pkg-index" class="toggleVisible">
<div class="collapsed">
<h2 class="toggleButton" title="Click to show Index section">Index ▹</h2>
</div>
<div class="expanded">
<h2 class="toggleButton" title="Click to hide Index section">Index ▾</h2>
<!-- Table of contents for API; must be named manual-nav to turn off auto nav. -->
<div id="manual-nav">
<dl>
{{if .Consts}}
<dd><a href="#pkg-constants">Constants</a></dd>
{{end}}
{{if .Vars}}
<dd><a href="#pkg-variables">Variables</a></dd>
{{end}}
{{range .Funcs}}
{{$name_html := html .Name}}
<dd><a href="#{{$name_html}}">{{node_html $ .Decl false | sanitize}}</a></dd>
{{end}}
{{range .Types}}
{{$tname_html := html .Name}}
<dd><a href="#{{$tname_html}}">type {{$tname_html}}</a></dd>
{{range .Funcs}}
{{$name_html := html .Name}}
<dd>&nbsp; &nbsp; <a href="#{{$name_html}}">{{node_html $ .Decl false | sanitize}}</a></dd>
{{end}}
{{range .Methods}}
{{$name_html := html .Name}}
<dd>&nbsp; &nbsp; <a href="#{{$tname_html}}.{{$name_html}}">{{node_html $ .Decl false | sanitize}}</a></dd>
{{end}}
{{end}}
{{if $.Notes}}
{{range $marker, $item := $.Notes}}
<dd><a href="#pkg-note-{{$marker}}">{{noteTitle $marker | html}}s</a></dd>
{{end}}
{{end}}
</dl>
</div><!-- #manual-nav -->
{{if $.Examples}}
<div id="pkg-examples">
<h4>Examples</h4>
<dl>
{{range $.Examples}}
<dd><a class="exampleLink" href="#example_{{.Name}}">{{example_name .Name}}</a></dd>
{{end}}
</dl>
</div>
{{end}}
{{with .Filenames}}
<h4>Package files</h4>
<p>
<span style="font-size:90%">
{{range .}}
<a href="{{.|githubFileLink|html}}">{{.|filename|html}}</a>
{{end}}
</span>
</p>
{{end}}
</div><!-- .expanded -->
</div><!-- #pkg-index -->
<div id="pkg-callgraph" class="toggle" style="display: none">
<div class="collapsed">
<h2 class="toggleButton" title="Click to show Internal Call Graph section">Internal call graph ▹</h2>
</div> <!-- .expanded -->
<div class="expanded">
<h2 class="toggleButton" title="Click to hide Internal Call Graph section">Internal call graph ▾</h2>
<p>
In the call graph viewer below, each node
is a function belonging to this package
and its children are the functions it
calls&mdash;perhaps dynamically.
</p>
<p>
The root nodes are the entry points of the
package: functions that may be called from
outside the package.
There may be non-exported or anonymous
functions among them if they are called
dynamically from another package.
</p>
<p>
Click a node to visit that function's source code.
From there you can visit its callers by
clicking its declaring <code>func</code>
token.
</p>
<p>
Functions may be omitted if they were
determined to be unreachable in the
particular programs or tests that were
analyzed.
</p>
<!-- Zero means show all package entry points. -->
<ul style="margin-left: 0.5in" id="callgraph-0" class="treeview"></ul>
</div>
</div> <!-- #pkg-callgraph -->
{{with .Consts}}
<h2 id="pkg-constants">Constants</h2>
{{range .}}
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{end}}
{{end}}
{{with .Vars}}
<h2 id="pkg-variables">Variables</h2>
{{range .}}
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{end}}
{{end}}
{{range .Funcs}}
{{/* Name is a string - no need for FSet */}}
{{$name_html := html .Name}}
<h2 id="{{$name_html}}">func <a href="{{posLink_url $ .Decl}}">{{$name_html}}</a></h2>
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{example_html $ .Name}}
{{callgraph_html $ "" .Name}}
{{end}}
{{range .Types}}
{{$tname := .Name}}
{{$tname_html := html .Name}}
<h2 id="{{$tname_html}}">type <a href="{{posLink_url $ .Decl}}">{{$tname_html}}</a></h2>
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{range .Consts}}
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{end}}
{{range .Vars}}
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{end}}
{{example_html $ $tname}}
{{implements_html $ $tname}}
{{methodset_html $ $tname}}
{{range .Funcs}}
{{$name_html := html .Name}}
<h3 id="{{$name_html}}">func <a href="{{posLink_url $ .Decl}}">{{$name_html}}</a></h3>
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{example_html $ .Name}}
{{callgraph_html $ "" .Name}}
{{end}}
{{range .Methods}}
{{$name_html := html .Name}}
<h3 id="{{$tname_html}}.{{$name_html}}">func ({{html .Recv}}) <a href="{{posLink_url $ .Decl}}">{{$name_html}}</a></h3>
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{$name := printf "%s_%s" $tname .Name}}
{{example_html $ $name}}
{{callgraph_html $ .Recv .Name}}
{{end}}
{{end}}
{{end}}
{{with $.Notes}}
{{range $marker, $content := .}}
<h2 id="pkg-note-{{$marker}}">{{noteTitle $marker | html}}s</h2>
<ul style="list-style: none; padding: 0;">
{{range .}}
<li><a href="{{posLink_url $ .}}">&#x261e;</a> {{html .Body}}</li>
{{end}}
</ul>
{{end}}
{{end}}
{{end}}
{{with .PAst}}
{{range $filename, $ast := .}}
<a href="{{$filename|srcLink|html}}">{{$filename|filename|html}}</a>:<pre>{{node_html $ $ast false}}</pre>
{{end}}
{{end}}
{{with .Dirs}}
{{if eq $.Dirname "/src"}}
<div id="manual-nav">
<dl>
<dt><a href="#stdlib">Standard library</a></dt>
<dt><a href="#other">Other packages</a></dt>
<dd><a href="#subrepo">Sub-repositories</a></dd>
<dd><a href="#community">Community</a></dd>
</dl>
</div>
<h2 id="stdlib">Standard library</h2>
<img class="gopher" src="/doc/gopher/pkg.png"/>
{{end}}
{{end}}

View File

@@ -1,268 +0,0 @@
<!--
Copyright 2009 The Go Authors. All rights reserved.
Use of this source code is governed by a BSD-style
license that can be found in the LICENSE file.
-->
<!--
Note: Static (i.e., not template-generated) href and id
attributes start with "pkg-" to make it impossible for
them to conflict with generated attributes (some of which
correspond to Go identifiers).
-->
{{with .PDoc}}
<script type='text/javascript'>
document.ANALYSIS_DATA = {{$.AnalysisData}};
document.CALLGRAPH = {{$.CallGraph}};
</script>
{{if $.IsMain}}
{{/* command documentation */}}
{{comment_html .Doc}}
{{else}}
{{/* package documentation */}}
<div id="short-nav">
<dl>
<dd><code>import "github.com/aws/aws-sdk-go/{{html .ImportPath}}"</code></dd>
</dl>
<dl>
<dd><a href="#pkg-overview" class="overviewLink">Overview</a></dd>
{{if .Consts}}
<dd><a href="#pkg-constants" class="examplesLink">Constants</a></dd>
{{end}}
{{if $.Examples}}
<dd><a href="#pkg-examples" class="examplesLink">Examples</a></dd>
{{end}}
</dl>
</div>
<!-- The package's Name is printed as title by the top-level template -->
<div id="pkg-overview" class="toggleVisible">
<div class="collapsed">
<h2 class="toggleButton" title="Click to show Overview section">Overview ▹</h2>
</div>
<div class="expanded">
<h2 class="toggleButton" title="Click to hide Overview section">Overview ▾</h2>
{{comment_html .Doc}}
<p>
{{if ne $.IfaceLink ""}}
The stub package, <a href={{$.IfaceLink}}>{{$.PDoc.Name}}iface</a>, can be used to provide alternative implementations of service clients,
such as mocking the client for testing.
{{end}}
</p?>
</div>
</div>
{{example_html $ ""}}
<div id="pkg-operations" class="toggle">
<div class="collapsed">
<h2 class="toggleButton" title="Click to show Index section">Operations ▹</h2>
</div>
<div class="expanded">
<h2 class="toggleButton" title="Click to hide Index section">Operations ▾</h2>
<!-- Table of contents for API; must be named manual-nav to turn off auto nav. -->
<div id="manual-nav">
<dl>
{{range .Funcs}}
{{$name_html := html .Name}}
<dd><a href="#{{$name_html}}">{{node_html $ .Decl false | sanitize}}</a></dd>
{{end}}
{{range .Types}}
{{$tname_html := html .Name}}
{{range .Funcs}}
{{$name_html := html .Name}}
<dd><a href="#{{$name_html}}">{{node_html $ .Decl false | sanitize}}</a></dd>
{{end}}
{{range .Methods}}
<!-- Hide stringers -->
{{if (and (ne .Name "String") (ne .Name "GoString")) }}
{{if (ne .Name "Validate") }}
{{$name_html := html .Name}}
<dd><a href="#{{$tname_html}}.{{$name_html}}">{{node_html $ .Decl false | sanitize}}</a></dd>
{{end}}
{{end}}
{{end}}
{{end}}
{{if $.Notes}}
{{range $marker, $item := $.Notes}}
<dd><a href="#pkg-note-{{$marker}}">{{noteTitle $marker | html}}s</a></dd>
{{end}}
{{end}}
</dl>
</div><!-- #manual-nav -->
<!--{{if $.Examples}}
<div id="pkg-examples">
<h4>Examples</h4>
<dl>
{{range $.Examples}}
<dd><a class="exampleLink" href="#example_{{.Name}}">{{example_name .Name}}</a></dd>
{{end}}
</dl>
</div>
{{end}}-->
<!--{{with .Filenames}}
<h4>Package files</h4>
<p>
<span style="font-size:90%">
{{range .}}
<a href="{{.|srcLink|html}}">{{.|filename|html}}</a>
{{end}}
</span>
</p>
{{end}}-->
</div><!-- .expanded -->
</div><!-- #pkg-index -->
<div id="pkg-types" class="toggle">
<div class="collapsed">
<h2 class="toggleButton" title="Click to show Index section">Types ▹</h2>
</div>
<div class="expanded">
<h2 class="toggleButton" title="Click to hide Index section">Types ▾</h2>
{{if .Vars}}
<dd><a href="#pkg-variables">Variables</a></dd>
{{end}}
{{range .Types}}
{{$tname_html := html .Name}}
<dd><a href="#{{$tname_html}}">type {{$tname_html}}</a></dd>
{{end}}
</div>
</div>
{{if $.Examples}}
<div id="pkg-examples" class="toggle">
<div class="collapsed">
<h2 class="toggleButton" title="Click to show Index section">Examples ▹</h2>
</div>
<div class="expanded">
<h2 class="toggleButton" title="Click to hide Index section">Examples ▾</h2>
<div id="pkg-examples">
<dl>
{{range $.Examples}}
<dd><a class="exampleLink" href="#example_{{.Name}}">{{example_name .Name}}</a></dd>
{{end}}
</dl>
</div>
</div>
</div>
{{end}}
<div id="pkg-callgraph" class="toggle" style="display: none">
<div class="collapsed">
<h2 class="toggleButton" title="Click to show Internal Call Graph section">Internal call graph ▹</h2>
</div> <!-- .expanded -->
<div class="expanded">
<h2 class="toggleButton" title="Click to hide Internal Call Graph section">Internal call graph ▾</h2>
<p>
In the call graph viewer below, each node
is a function belonging to this package
and its children are the functions it
calls&mdash;perhaps dynamically.
</p>
<p>
The root nodes are the entry points of the
package: functions that may be called from
outside the package.
There may be non-exported or anonymous
functions among them if they are called
dynamically from another package.
</p>
<p>
Click a node to visit that function's source code.
From there you can visit its callers by
clicking its declaring <code>func</code>
token.
</p>
<p>
Functions may be omitted if they were
determined to be unreachable in the
particular programs or tests that were
analyzed.
</p>
<!-- Zero means show all package entry points. -->
<ul style="margin-left: 0.5in" id="callgraph-0" class="treeview"></ul>
</div>
</div> <!-- #pkg-callgraph -->
{{with .Consts}}
<h2 id="pkg-constants">Constants</h2>
{{range .}}
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{end}}
{{end}}
{{with .Vars}}
<h2 id="pkg-variables">Variables</h2>
{{range .}}
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{end}}
{{end}}
{{range .Funcs}}
{{/* Name is a string - no need for FSet */}}
{{$name_html := html .Name}}
<h2 id="{{$name_html}}">func <a href="{{posLink_url $ .Decl}}">{{$name_html}}</a></h2>
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{example_html $ .Name}}
{{callgraph_html $ "" .Name}}
{{end}}
{{range .Types}}
{{$tname := .Name}}
{{$tname_html := html .Name}}
<h2 id="{{$tname_html}}">type <a href="{{posLink_url $ .Decl}}">{{$tname_html}}</a></h2>
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{range .Consts}}
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{end}}
{{range .Vars}}
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{end}}
{{example_html $ $tname}}
{{implements_html $ $tname}}
{{methodset_html $ $tname}}
{{range .Funcs}}
{{$name_html := html .Name}}
<h3 id="{{$name_html}}">func <a href="{{posLink_url $ .Decl}}">{{$name_html}}</a></h3>
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{example_html $ .Name}}
{{callgraph_html $ "" .Name}}
{{end}}
{{range .Methods}}
{{$name_html := html .Name}}
<h3 id="{{$tname_html}}.{{$name_html}}">func ({{html .Recv}}) <a href="{{posLink_url $ .Decl}}">{{$name_html}}</a></h3>
<pre>{{node_html $ .Decl true}}</pre>
{{comment_html .Doc}}
{{$name := printf "%s_%s" $tname .Name}}
{{example_html $ $name}}
{{callgraph_html $ .Recv .Name}}
{{end}}
{{end}}
{{end}}
{{with $.Notes}}
{{range $marker, $content := .}}
<h2 id="pkg-note-{{$marker}}">{{noteTitle $marker | html}}s</h2>
<ul style="list-style: none; padding: 0;">
{{range .}}
<li><a href="{{posLink_url $ .}}">&#x261e;</a> {{html .Body}}</li>
{{end}}
</ul>
{{end}}
{{end}}
{{end}}
{{with .PAst}}
{{range $filename, $ast := .}}
<a href="{{$filename|srcLink|html}}">{{$filename|filename|html}}</a>:<pre>{{node_html $ $ast false}}</pre>
{{end}}
{{end}}

Some files were not shown because too many files have changed in this diff Show More