diff --git a/.jshintrc b/.jshintrc index 99247999e7c..bca48639a70 100644 --- a/.jshintrc +++ b/.jshintrc @@ -28,6 +28,7 @@ "globals": { "System": true, + "Promise": true, "define": true, "require": true, "Chromath": false, diff --git a/CHANGELOG.md b/CHANGELOG.md index 978a531bcf5..0e0721e4182 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,7 @@ * **Metadata**: Settings panel now shows dashboard metadata, closes [#3304](https://github.com/grafana/grafana/issues/3304) * **InfluxDB**: Support for policy selection in query editor, closes [#2018](https://github.com/grafana/grafana/issues/2018) * **Snapshots UI**: Dashboard snapshots list can be managed through UI, closes[#1984](https://github.com/grafana/grafana/issues/1984) +* **Prometheus**: Prometheus annotation support, closes[#2883](https://github.com/grafana/grafana/pull/2883) ### Breaking changes * **Plugin API**: Both datasource and panel plugin api (and plugin.json schema) have been updated, requiring an update to plugins. See [plugin api](https://github.com/grafana/grafana/blob/master/public/app/plugins/plugin_api.md) for more info. @@ -16,6 +17,8 @@ * **mysql**: Grafana now supports ssl for mysql, closes [#3584](https://github.com/grafana/grafana/issues/3584) * **snapshot**: Annotations are now included in snapshots, closes [#3635](https://github.com/grafana/grafana/issues/3635) * **Admin**: Admin can now have global overview of Grafana setup, closes [#3812](https://github.com/grafana/grafana/issues/3812) +* **graph**: Right side legend height is now fixed at row height, closes [#1277](https://github.com/grafana/grafana/issues/1277) +* **Table**: All content in table panel is now html escaped, closes [#3673](https://github.com/grafana/grafana/issues/3673) ### Bug fixes * **Playlist**: Fix for memory leak when running a playlist, closes [#3794](https://github.com/grafana/grafana/pull/3794) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 405be409788..3e7b20ce9c1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,14 +1,19 @@ -If you have any idea for an improvement or found a bug do not hesitate to open an issue. -And if you have time clone this repo and submit a pull request and help me make Grafana the -kickass metrics & devops dashboard we all dream about! +Follow the setup guide in README.md -Prerequisites: - - Nodejs (for jshint & grunt & development server) +### Rebuild frontend assts on source change +``` +grunt && grunt watch +``` -Clone repository: +### Rerun tests on source change +``` +grunt karma:dev +``` + +### Run tests before commit +``` +grunt test +godep go test -v ./pkg/... +``` - npm install - grunt server (starts development web server in src folder) - grunt (runs jshint and less -> css compilation) -Please remember to run grunt before doing pull request to verify that your code passes all the jshint validations. diff --git a/Godeps/Godeps.json b/Godeps/Godeps.json index 2b42526d473..6449d1941c4 100644 --- a/Godeps/Godeps.json +++ b/Godeps/Godeps.json @@ -1,6 +1,6 @@ { "ImportPath": "github.com/grafana/grafana", - "GoVersion": "go1.5", + "GoVersion": "go1.5.1", "Packages": [ "./pkg/..." ], @@ -101,17 +101,18 @@ }, { "ImportPath": "github.com/go-sql-driver/mysql", - "Comment": "v1.2-26-g9543750", - "Rev": "9543750295406ef070f7de8ae9c43ccddd44e15e" + "Comment": "v1.2-171-g267b128", + "Rev": "267b128680c46286b9ca13475c3cca5de8f79bd7" }, { "ImportPath": "github.com/go-xorm/core", - "Rev": "be6e7ac47dc57bd0ada25322fa526944f66ccaa6" + "Comment": "v0.4.4-7-g9e608f7", + "Rev": "9e608f7330b9d16fe2818cfe731128b3f156cb9a" }, { "ImportPath": "github.com/go-xorm/xorm", - "Comment": "v0.4.2-58-ge2889e5", - "Rev": "e2889e5517600b82905f1d2ba8b70deb71823ffe" + "Comment": "v0.4.4-44-gf561133", + "Rev": "f56113384f2c63dfe4cd8e768e349f1c35122b58" }, { "ImportPath": "github.com/gosimple/slug", diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/.travis.yml b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/.travis.yml index dd29a75803f..c48cedd8770 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/.travis.yml +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/.travis.yml @@ -1,8 +1,10 @@ +sudo: false language: go go: - - 1.1 - 1.2 - 1.3 + - 1.4 + - 1.5 - tip before_script: diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/AUTHORS b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/AUTHORS index f0b07024689..986f017cbed 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/AUTHORS +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/AUTHORS @@ -14,19 +14,32 @@ Aaron Hopkins Arne Hormann Carlos Nieto +Chris Moos +Daniel Nichter DisposaBoy Frederick Mayle Gustavo Kristic Hanno Braun Henri Yandell +Hirotaka Yamamoto +INADA Naoki James Harr Jian Zhen +Joshua Prunier +Julien Lefevre Julien Schmidt +Kamil Dziedzic +Kevin Malachowski Leonardo YongUk Kim +Luca Looz Lucas Liu Luke Scott Michael Woolnough Nicola Peduzzi +Runrioter Wung +Soroush Pour +Stan Putrya +Stanley Gunawan Xiaobing Jiang Xiuming Chen @@ -34,3 +47,4 @@ Xiuming Chen Barracuda Networks, Inc. Google Inc. +Stripe Inc. diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/CHANGELOG.md b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/CHANGELOG.md index 9bb92c1cd5f..381d91825d2 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/CHANGELOG.md +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/CHANGELOG.md @@ -2,11 +2,31 @@ Changes: - - Use decimals field from MySQL to format time types + - Go 1.1 is no longer supported + - Use decimals field from MySQL to format time types (#249) + - Buffer optimizations (#269) + - TLS ServerName defaults to the host (#283) Bugfixes: - - Enable microsecond resolution on TIME, DATETIME and TIMESTAMP + - Enable microsecond resolution on TIME, DATETIME and TIMESTAMP (#249) + - Fixed handling of queries without columns and rows (#255) + - Fixed a panic when SetKeepAlive() failed (#298) + - Support receiving ERR packet while reading rows (#321) + - Fixed reading NULL length-encoded integers in MySQL 5.6+ (#349) + - Fixed absolute paths support in LOAD LOCAL DATA INFILE (#356) + - Actually zero out bytes in handshake response (#378) + - Fixed race condition in registering LOAD DATA INFILE handler (#383) + - Fixed tests with MySQL 5.7.9+ (#380) + - QueryUnescape TLS config names (#397) + - Fixed "broken pipe" error by writing to closed socket (#390) + +New Features: + - Support for returning table alias on Columns() (#289, #359, #382) + - Placeholder interpolation, can be actived with the DSN parameter `interpolateParams=true` (#309, #318) + - Support for uint64 parameters with high bit set (#332, #345) + - Cleartext authentication plugin support (#327) + ## Version 1.2 (2014-06-03) diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/README.md b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/README.md index 6a2b61e0e64..3adedc6a566 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/README.md +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/README.md @@ -30,7 +30,7 @@ A MySQL-Driver for Go's [database/sql](http://golang.org/pkg/database/sql) packa ## Features * Lightweight and [fast](https://github.com/go-sql-driver/sql-benchmark "golang MySQL-Driver performance") * Native Go implementation. No C-bindings, just pure Go - * Connections over TCP/IPv4, TCP/IPv6 or Unix domain sockets + * Connections over TCP/IPv4, TCP/IPv6, Unix domain sockets or [custom protocols](http://godoc.org/github.com/go-sql-driver/mysql#DialFunc) * Automatic handling of broken connections * Automatic Connection Pooling *(by database/sql package)* * Supports queries larger than 16MB @@ -38,9 +38,10 @@ A MySQL-Driver for Go's [database/sql](http://golang.org/pkg/database/sql) packa * Intelligent `LONG DATA` handling in prepared statements * Secure `LOAD DATA LOCAL INFILE` support with file Whitelisting and `io.Reader` support * Optional `time.Time` parsing + * Optional placeholder interpolation ## Requirements - * Go 1.1 or higher + * Go 1.2 or higher * MySQL (4.1+), MariaDB, Percona Server, Google CloudSQL or Sphinx (2.2.3+) --------------------------------------- @@ -92,6 +93,8 @@ This has the same effect as an empty DSN string: ``` +Alternatively, [Config.FormatDSN](https://godoc.org/github.com/go-sql-driver/mysql#Config.FormatDSN) can be used to create a DSN string by filling a struct. + #### Password Passwords can consist of any character. Escaping is **not** necessary. @@ -122,6 +125,16 @@ Default: false `allowAllFiles=true` disables the file Whitelist for `LOAD DATA LOCAL INFILE` and allows *all* files. [*Might be insecure!*](http://dev.mysql.com/doc/refman/5.7/en/load-data-local.html) +##### `allowCleartextPasswords` + +``` +Type: bool +Valid Values: true, false +Default: false +``` + +`allowCleartextPasswords=true` allows using the [cleartext client side plugin](http://dev.mysql.com/doc/en/cleartext-authentication-plugin.html) if required by an account, such as one defined with the [PAM authentication plugin](http://dev.mysql.com/doc/en/pam-authentication-plugin.html). Sending passwords in clear text may be a security problem in some configurations. To avoid problems if there is any possibility that the password would be intercepted, clients should connect to MySQL Server using a method that protects the password. Possibilities include [TLS / SSL](#tls), IPsec, or a private network. + ##### `allowOldPasswords` ``` @@ -166,6 +179,33 @@ Default: false `clientFoundRows=true` causes an UPDATE to return the number of matching rows instead of the number of rows changed. +##### `columnsWithAlias` + +``` +Type: bool +Valid Values: true, false +Default: false +``` + +When `columnsWithAlias` is true, calls to `sql.Rows.Columns()` will return the table alias and the column name separated by a dot. For example: + +``` +SELECT u.id FROM users as u +``` + +will return `u.id` instead of just `id` if `columnsWithAlias=true`. + +##### `interpolateParams` + +``` +Type: bool +Valid Values: true, false +Default: false +``` + +If `interpolateParams` is true, placeholders (`?`) in calls to `db.Query()` and `db.Exec()` are interpolated into a single query string with given parameters. This reduces the number of roundtrips, since the driver has to prepare a statement, execute it with given parameters and close the statement again with `interpolateParams=false`. + +*This can not be used together with the multibyte encodings BIG5, CP932, GB2312, GBK or SJIS. These are blacklisted as they may [introduce a SQL injection vulnerability](http://stackoverflow.com/a/12118602/3430118)!* ##### `loc` @@ -177,8 +217,20 @@ Default: UTC Sets the location for time.Time values (when using `parseTime=true`). *"Local"* sets the system's location. See [time.LoadLocation](http://golang.org/pkg/time/#LoadLocation) for details. +Note that this sets the location for time.Time values but does not change MySQL's [time_zone setting](https://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html). For that see the [time_zone system variable](#system-variables), which can also be set as a DSN parameter. + Please keep in mind, that param values must be [url.QueryEscape](http://golang.org/pkg/net/url/#QueryEscape)'ed. Alternatively you can manually replace the `/` with `%2F`. For example `US/Pacific` would be `loc=US%2FPacific`. +##### `multiStatements` + +``` +Type: bool +Valid Values: true, false +Default: false +``` + +Allow multiple statements in one query. While this allows batch queries, it also greatly increases the risk of SQL injections. Only the result of the first query is returned, all other results are silently discarded. + ##### `parseTime` @@ -191,6 +243,16 @@ Default: false `parseTime=true` changes the output type of `DATE` and `DATETIME` values to `time.Time` instead of `[]byte` / `string` +##### `readTimeout` + +``` +Type: decimal number +Default: 0 +``` + +I/O read timeout. The value must be a decimal number with an unit suffix ( *"ms"*, *"s"*, *"m"*, *"h"* ), such as *"30s"*, *"0.5m"* or *"1m30s"*. + + ##### `strict` ``` @@ -211,7 +273,7 @@ Type: decimal number Default: OS default ``` -*Driver* side connection timeout. The value must be a string of decimal numbers, each with optional fraction and a unit suffix ( *"ms"*, *"s"*, *"m"*, *"h"* ), such as *"30s"*, *"0.5m"* or *"1m30s"*. To set a server side timeout, use the parameter [`wait_timeout`](http://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html#sysvar_wait_timeout). +*Driver* side connection timeout. The value must be a decimal number with an unit suffix ( *"ms"*, *"s"*, *"m"*, *"h"* ), such as *"30s"*, *"0.5m"* or *"1m30s"*. To set a server side timeout, use the parameter [`wait_timeout`](http://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html#sysvar_wait_timeout). ##### `tls` @@ -225,11 +287,21 @@ Default: false `tls=true` enables TLS / SSL encrypted connection to the server. Use `skip-verify` if you want to use a self-signed or invalid certificate (server side). Use a custom value registered with [`mysql.RegisterTLSConfig`](http://godoc.org/github.com/go-sql-driver/mysql#RegisterTLSConfig). +##### `writeTimeout` + +``` +Type: decimal number +Default: 0 +``` + +I/O write timeout. The value must be a decimal number with an unit suffix ( *"ms"*, *"s"*, *"m"*, *"h"* ), such as *"30s"*, *"0.5m"* or *"1m30s"*. + + ##### System Variables All other parameters are interpreted as system variables: * `autocommit`: `"SET autocommit="` - * `time_zone`: `"SET time_zone="` + * [`time_zone`](https://dev.mysql.com/doc/refman/5.5/en/time-zone-support.html): `"SET time_zone="` * [`tx_isolation`](https://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_tx_isolation): `"SET tx_isolation="` * `param`: `"SET ="` @@ -291,7 +363,7 @@ import "github.com/go-sql-driver/mysql" Files must be whitelisted by registering them with `mysql.RegisterLocalFile(filepath)` (recommended) or the Whitelist check must be deactivated by using the DSN parameter `allowAllFiles=true` ([*Might be insecure!*](http://dev.mysql.com/doc/refman/5.7/en/load-data-local.html)). -To use a `io.Reader` a handler function must be registered with `mysql.RegisterReaderHandler(name, handler)` which returns a `io.Reader` or `io.ReadCloser`. The Reader is available with the filepath `Reader::` then. +To use a `io.Reader` a handler function must be registered with `mysql.RegisterReaderHandler(name, handler)` which returns a `io.Reader` or `io.ReadCloser`. The Reader is available with the filepath `Reader::` then. Choose different names for different handlers and `DeregisterReaderHandler` when you don't need it anymore. See the [godoc of Go-MySQL-Driver](http://godoc.org/github.com/go-sql-driver/mysql "golang mysql driver documentation") for details. diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/benchmark_test.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/benchmark_test.go index d72a4183fb7..8f721139b99 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/benchmark_test.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/benchmark_test.go @@ -11,10 +11,13 @@ package mysql import ( "bytes" "database/sql" + "database/sql/driver" + "math" "strings" "sync" "sync/atomic" "testing" + "time" ) type TB testing.B @@ -45,7 +48,11 @@ func initDB(b *testing.B, queries ...string) *sql.DB { db := tb.checkDB(sql.Open("mysql", dsn)) for _, query := range queries { if _, err := db.Exec(query); err != nil { - b.Fatalf("Error on %q: %v", query, err) + if w, ok := err.(MySQLWarnings); ok { + b.Logf("warning on %q: %v", query, w) + } else { + b.Fatalf("error on %q: %v", query, err) + } } } return db @@ -206,3 +213,34 @@ func BenchmarkRoundtripBin(b *testing.B) { rows.Close() } } + +func BenchmarkInterpolation(b *testing.B) { + mc := &mysqlConn{ + cfg: &Config{ + InterpolateParams: true, + Loc: time.UTC, + }, + maxPacketAllowed: maxPacketSize, + maxWriteSize: maxPacketSize - 1, + buf: newBuffer(nil), + } + + args := []driver.Value{ + int64(42424242), + float64(math.Pi), + false, + time.Unix(1423411542, 807015000), + []byte("bytes containing special chars ' \" \a \x00"), + "string containing special chars ' \" \a \x00", + } + q := "SELECT ?, ?, ?, ?, ?, ?" + + b.ReportAllocs() + b.ResetTimer() + for i := 0; i < b.N; i++ { + _, err := mc.interpolateParams(q, args) + if err != nil { + b.Fatal(err) + } + } +} diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/buffer.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/buffer.go index 509ce89e468..2001feacd31 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/buffer.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/buffer.go @@ -8,7 +8,11 @@ package mysql -import "io" +import ( + "io" + "net" + "time" +) const defaultBufSize = 4096 @@ -18,17 +22,18 @@ const defaultBufSize = 4096 // The buffer is similar to bufio.Reader / Writer but zero-copy-ish // Also highly optimized for this particular use case. type buffer struct { - buf []byte - rd io.Reader - idx int - length int + buf []byte + nc net.Conn + idx int + length int + timeout time.Duration } -func newBuffer(rd io.Reader) buffer { +func newBuffer(nc net.Conn) buffer { var b [defaultBufSize]byte return buffer{ buf: b[:], - rd: rd, + nc: nc, } } @@ -54,7 +59,13 @@ func (b *buffer) fill(need int) error { b.idx = 0 for { - nn, err := b.rd.Read(b.buf[n:]) + if b.timeout > 0 { + if err := b.nc.SetReadDeadline(time.Now().Add(b.timeout)); err != nil { + return err + } + } + + nn, err := b.nc.Read(b.buf[n:]) n += nn switch err { diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/collations.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/collations.go index aabe0055d7e..82079cfb93d 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/collations.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/collations.go @@ -8,7 +8,7 @@ package mysql -const defaultCollation byte = 33 // utf8_general_ci +const defaultCollation = "utf8_general_ci" // A list of available collations mapped to the internal ID. // To update this map use the following MySQL query: @@ -234,3 +234,17 @@ var collations = map[string]byte{ "utf8mb4_unicode_520_ci": 246, "utf8mb4_vietnamese_ci": 247, } + +// A blacklist of collations which is unsafe to interpolate parameters. +// These multibyte encodings may contains 0x5c (`\`) in their trailing bytes. +var unsafeCollations = map[string]bool{ + "big5_chinese_ci": true, + "sjis_japanese_ci": true, + "gbk_chinese_ci": true, + "big5_bin": true, + "gb2312_bin": true, + "gbk_bin": true, + "sjis_bin": true, + "cp932_japanese_ci": true, + "cp932_bin": true, +} diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/connection.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/connection.go index 67d3dbee8d5..c3899de0eb1 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/connection.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/connection.go @@ -9,10 +9,9 @@ package mysql import ( - "crypto/tls" "database/sql/driver" - "errors" "net" + "strconv" "strings" "time" ) @@ -22,34 +21,20 @@ type mysqlConn struct { netConn net.Conn affectedRows uint64 insertId uint64 - cfg *config + cfg *Config maxPacketAllowed int maxWriteSize int + writeTimeout time.Duration flags clientFlag + status statusFlag sequence uint8 parseTime bool strict bool } -type config struct { - user string - passwd string - net string - addr string - dbname string - params map[string]string - loc *time.Location - tls *tls.Config - timeout time.Duration - collation uint8 - allowAllFiles bool - allowOldPasswords bool - clientFoundRows bool -} - // Handles parameters set in DSN after the connection is established func (mc *mysqlConn) handleParams() (err error) { - for param, val := range mc.cfg.params { + for param, val := range mc.cfg.Params { switch param { // Charset case "charset": @@ -65,27 +50,6 @@ func (mc *mysqlConn) handleParams() (err error) { return } - // time.Time parsing - case "parseTime": - var isBool bool - mc.parseTime, isBool = readBool(val) - if !isBool { - return errors.New("Invalid Bool value: " + val) - } - - // Strict mode - case "strict": - var isBool bool - mc.strict, isBool = readBool(val) - if !isBool { - return errors.New("Invalid Bool value: " + val) - } - - // Compression - case "compress": - err = errors.New("Compression not implemented yet") - return - // System Vars default: err = mc.exec("SET " + param + "=" + val + "") @@ -115,18 +79,27 @@ func (mc *mysqlConn) Close() (err error) { // Makes Close idempotent if mc.netConn != nil { err = mc.writeCommandPacket(comQuit) - if err == nil { - err = mc.netConn.Close() - } else { - mc.netConn.Close() + } + + mc.cleanup() + + return +} + +// Closes the network connection and unsets internal variables. Do not call this +// function after successfully authentication, call Close instead. This function +// is called before auth or on auth failure because MySQL will have already +// closed the network connection. +func (mc *mysqlConn) cleanup() { + // Makes cleanup idempotent + if mc.netConn != nil { + if err := mc.netConn.Close(); err != nil { + errLog.Print(err) } mc.netConn = nil } - mc.cfg = nil - mc.buf.rd = nil - - return + mc.buf.nc = nil } func (mc *mysqlConn) Prepare(query string) (driver.Stmt, error) { @@ -161,28 +134,151 @@ func (mc *mysqlConn) Prepare(query string) (driver.Stmt, error) { return stmt, err } +func (mc *mysqlConn) interpolateParams(query string, args []driver.Value) (string, error) { + buf := mc.buf.takeCompleteBuffer() + if buf == nil { + // can not take the buffer. Something must be wrong with the connection + errLog.Print(ErrBusyBuffer) + return "", driver.ErrBadConn + } + buf = buf[:0] + argPos := 0 + + for i := 0; i < len(query); i++ { + q := strings.IndexByte(query[i:], '?') + if q == -1 { + buf = append(buf, query[i:]...) + break + } + buf = append(buf, query[i:i+q]...) + i += q + + arg := args[argPos] + argPos++ + + if arg == nil { + buf = append(buf, "NULL"...) + continue + } + + switch v := arg.(type) { + case int64: + buf = strconv.AppendInt(buf, v, 10) + case float64: + buf = strconv.AppendFloat(buf, v, 'g', -1, 64) + case bool: + if v { + buf = append(buf, '1') + } else { + buf = append(buf, '0') + } + case time.Time: + if v.IsZero() { + buf = append(buf, "'0000-00-00'"...) + } else { + v := v.In(mc.cfg.Loc) + v = v.Add(time.Nanosecond * 500) // To round under microsecond + year := v.Year() + year100 := year / 100 + year1 := year % 100 + month := v.Month() + day := v.Day() + hour := v.Hour() + minute := v.Minute() + second := v.Second() + micro := v.Nanosecond() / 1000 + + buf = append(buf, []byte{ + '\'', + digits10[year100], digits01[year100], + digits10[year1], digits01[year1], + '-', + digits10[month], digits01[month], + '-', + digits10[day], digits01[day], + ' ', + digits10[hour], digits01[hour], + ':', + digits10[minute], digits01[minute], + ':', + digits10[second], digits01[second], + }...) + + if micro != 0 { + micro10000 := micro / 10000 + micro100 := micro / 100 % 100 + micro1 := micro % 100 + buf = append(buf, []byte{ + '.', + digits10[micro10000], digits01[micro10000], + digits10[micro100], digits01[micro100], + digits10[micro1], digits01[micro1], + }...) + } + buf = append(buf, '\'') + } + case []byte: + if v == nil { + buf = append(buf, "NULL"...) + } else { + buf = append(buf, "_binary'"...) + if mc.status&statusNoBackslashEscapes == 0 { + buf = escapeBytesBackslash(buf, v) + } else { + buf = escapeBytesQuotes(buf, v) + } + buf = append(buf, '\'') + } + case string: + buf = append(buf, '\'') + if mc.status&statusNoBackslashEscapes == 0 { + buf = escapeStringBackslash(buf, v) + } else { + buf = escapeStringQuotes(buf, v) + } + buf = append(buf, '\'') + default: + return "", driver.ErrSkip + } + + if len(buf)+4 > mc.maxPacketAllowed { + return "", driver.ErrSkip + } + } + if argPos != len(args) { + return "", driver.ErrSkip + } + return string(buf), nil +} + func (mc *mysqlConn) Exec(query string, args []driver.Value) (driver.Result, error) { if mc.netConn == nil { errLog.Print(ErrInvalidConn) return nil, driver.ErrBadConn } - if len(args) == 0 { // no args, fastpath - mc.affectedRows = 0 - mc.insertId = 0 - - err := mc.exec(query) - if err == nil { - return &mysqlResult{ - affectedRows: int64(mc.affectedRows), - insertId: int64(mc.insertId), - }, err + if len(args) != 0 { + if !mc.cfg.InterpolateParams { + return nil, driver.ErrSkip } - return nil, err + // try to interpolate the parameters to save extra roundtrips for preparing and closing a statement + prepared, err := mc.interpolateParams(query, args) + if err != nil { + return nil, err + } + query = prepared + args = nil } + mc.affectedRows = 0 + mc.insertId = 0 - // with args, must use prepared stmt - return nil, driver.ErrSkip - + err := mc.exec(query) + if err == nil { + return &mysqlResult{ + affectedRows: int64(mc.affectedRows), + insertId: int64(mc.insertId), + }, err + } + return nil, err } // Internal function to execute commands @@ -211,31 +307,38 @@ func (mc *mysqlConn) Query(query string, args []driver.Value) (driver.Rows, erro errLog.Print(ErrInvalidConn) return nil, driver.ErrBadConn } - if len(args) == 0 { // no args, fastpath - // Send command - err := mc.writeCommandPacketStr(comQuery, query) - if err == nil { - // Read Result - var resLen int - resLen, err = mc.readResultSetHeaderPacket() - if err == nil { - rows := new(textRows) - rows.mc = mc - - if resLen == 0 { - // no columns, no more data - return emptyRows{}, nil - } - // Columns - rows.columns, err = mc.readColumns(resLen) - return rows, err - } + if len(args) != 0 { + if !mc.cfg.InterpolateParams { + return nil, driver.ErrSkip } - return nil, err + // try client-side prepare to reduce roundtrip + prepared, err := mc.interpolateParams(query, args) + if err != nil { + return nil, err + } + query = prepared + args = nil } + // Send command + err := mc.writeCommandPacketStr(comQuery, query) + if err == nil { + // Read Result + var resLen int + resLen, err = mc.readResultSetHeaderPacket() + if err == nil { + rows := new(textRows) + rows.mc = mc - // with args, must use prepared stmt - return nil, driver.ErrSkip + if resLen == 0 { + // no columns, no more data + return emptyRows{}, nil + } + // Columns + rows.columns, err = mc.readColumns(resLen) + return rows, err + } + } + return nil, err } // Gets the value of the given MySQL System Variable @@ -251,6 +354,7 @@ func (mc *mysqlConn) getSystemVar(name string) ([]byte, error) { if err == nil { rows := new(textRows) rows.mc = mc + rows.columns = []mysqlField{{fieldType: fieldTypeVarChar}} if resLen > 0 { // Columns diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/const.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/const.go index 5fcc3e98b12..88cfff3fd8b 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/const.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/const.go @@ -24,6 +24,7 @@ const ( iERR byte = 0xff ) +// https://dev.mysql.com/doc/internals/en/capability-flags.html#packet-Protocol::CapabilityFlags type clientFlag uint32 const ( @@ -45,6 +46,13 @@ const ( clientSecureConn clientMultiStatements clientMultiResults + clientPSMultiResults + clientPluginAuth + clientConnectAttrs + clientPluginAuthLenEncClientData + clientCanHandleExpiredPasswords + clientSessionTrack + clientDeprecateEOF ) const ( @@ -68,7 +76,7 @@ const ( comBinlogDump comTableDump comConnectOut - comRegiserSlave + comRegisterSlave comStmtPrepare comStmtExecute comStmtSendLongData @@ -78,6 +86,7 @@ const ( comStmtFetch ) +// https://dev.mysql.com/doc/internals/en/com-query-response.html#packet-Protocol::ColumnType const ( fieldTypeDecimal byte = iota fieldTypeTiny @@ -98,7 +107,8 @@ const ( fieldTypeBit ) const ( - fieldTypeNewDecimal byte = iota + 0xf6 + fieldTypeJSON byte = iota + 0xf5 + fieldTypeNewDecimal fieldTypeEnum fieldTypeSet fieldTypeTinyBLOB @@ -130,3 +140,24 @@ const ( flagUnknown3 flagUnknown4 ) + +// http://dev.mysql.com/doc/internals/en/status-flags.html +type statusFlag uint16 + +const ( + statusInTrans statusFlag = 1 << iota + statusInAutocommit + statusReserved // Not in documentation + statusMoreResultsExists + statusNoGoodIndexUsed + statusNoIndexUsed + statusCursorExists + statusLastRowSent + statusDbDropped + statusNoBackslashEscapes + statusMetadataChanged + statusQueryWasSlow + statusPsOutParams + statusInTransReadonly + statusSessionStateChanged +) diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/driver.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/driver.go index c0375fe18a3..899f955fbb3 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/driver.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/driver.go @@ -4,7 +4,7 @@ // License, v. 2.0. If a copy of the MPL was not distributed with this file, // You can obtain one at http://mozilla.org/MPL/2.0/. -// Go MySQL Driver - A MySQL-Driver for Go's database/sql package +// Package mysql provides a MySQL driver for Go's database/sql package // // The driver should be used via the database/sql package: // @@ -22,7 +22,7 @@ import ( "net" ) -// This struct is exported to make the driver directly accessible. +// MySQLDriver is exported to make the driver directly accessible. // In general the driver is used via the database/sql package. type MySQLDriver struct{} @@ -53,17 +53,19 @@ func (d MySQLDriver) Open(dsn string) (driver.Conn, error) { maxPacketAllowed: maxPacketSize, maxWriteSize: maxPacketSize - 1, } - mc.cfg, err = parseDSN(dsn) + mc.cfg, err = ParseDSN(dsn) if err != nil { return nil, err } + mc.parseTime = mc.cfg.ParseTime + mc.strict = mc.cfg.Strict // Connect to Server - if dial, ok := dials[mc.cfg.net]; ok { - mc.netConn, err = dial(mc.cfg.addr) + if dial, ok := dials[mc.cfg.Net]; ok { + mc.netConn, err = dial(mc.cfg.Addr) } else { - nd := net.Dialer{Timeout: mc.cfg.timeout} - mc.netConn, err = nd.Dial(mc.cfg.net, mc.cfg.addr) + nd := net.Dialer{Timeout: mc.cfg.Timeout} + mc.netConn, err = nd.Dial(mc.cfg.Net, mc.cfg.Addr) } if err != nil { return nil, err @@ -72,44 +74,39 @@ func (d MySQLDriver) Open(dsn string) (driver.Conn, error) { // Enable TCP Keepalives on TCP connections if tc, ok := mc.netConn.(*net.TCPConn); ok { if err := tc.SetKeepAlive(true); err != nil { - mc.Close() + // Don't send COM_QUIT before handshake. + mc.netConn.Close() + mc.netConn = nil return nil, err } } mc.buf = newBuffer(mc.netConn) + // Set I/O timeouts + mc.buf.timeout = mc.cfg.ReadTimeout + mc.writeTimeout = mc.cfg.WriteTimeout + // Reading Handshake Initialization Packet cipher, err := mc.readInitPacket() if err != nil { - mc.Close() + mc.cleanup() return nil, err } // Send Client Authentication Packet if err = mc.writeAuthPacket(cipher); err != nil { - mc.Close() + mc.cleanup() return nil, err } - // Read Result Packet - err = mc.readResultOK() - if err != nil { - // Retry with old authentication method, if allowed - if mc.cfg != nil && mc.cfg.allowOldPasswords && err == ErrOldPassword { - if err = mc.writeOldAuthPacket(cipher); err != nil { - mc.Close() - return nil, err - } - if err = mc.readResultOK(); err != nil { - mc.Close() - return nil, err - } - } else { - mc.Close() - return nil, err - } - + // Handle response to auth packet, switch methods if possible + if err = handleAuthResult(mc, cipher); err != nil { + // Authentication failed and MySQL has already closed the connection + // (https://dev.mysql.com/doc/internals/en/authentication-fails.html). + // Do not send COM_QUIT, just cleanup and return the error. + mc.cleanup() + return nil, err } // Get max allowed packet size @@ -133,6 +130,38 @@ func (d MySQLDriver) Open(dsn string) (driver.Conn, error) { return mc, nil } +func handleAuthResult(mc *mysqlConn, cipher []byte) error { + // Read Result Packet + err := mc.readResultOK() + if err == nil { + return nil // auth successful + } + + if mc.cfg == nil { + return err // auth failed and retry not possible + } + + // Retry auth if configured to do so. + if mc.cfg.AllowOldPasswords && err == ErrOldPassword { + // Retry with old authentication method. Note: there are edge cases + // where this should work but doesn't; this is currently "wontfix": + // https://github.com/go-sql-driver/mysql/issues/184 + if err = mc.writeOldAuthPacket(cipher); err != nil { + return err + } + err = mc.readResultOK() + } else if mc.cfg.AllowCleartextPasswords && err == ErrCleartextPassword { + // Retry with clear text password for + // http://dev.mysql.com/doc/refman/5.7/en/cleartext-authentication-plugin.html + // http://dev.mysql.com/doc/refman/5.7/en/pam-authentication-plugin.html + if err = mc.writeClearAuthPacket(); err != nil { + return err + } + err = mc.readResultOK() + } + return err +} + func init() { sql.Register("mysql", &MySQLDriver{}) } diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/driver_test.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/driver_test.go index a52cc5cd00e..cd2a9168e92 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/driver_test.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/driver_test.go @@ -9,12 +9,14 @@ package mysql import ( + "bytes" "crypto/tls" "database/sql" "database/sql/driver" "fmt" "io" "io/ioutil" + "log" "net" "net/url" "os" @@ -74,18 +76,20 @@ type DBTest struct { db *sql.DB } -func runTests(t *testing.T, dsn string, tests ...func(dbt *DBTest)) { +func runTestsWithMultiStatement(t *testing.T, dsn string, tests ...func(dbt *DBTest)) { if !available { - t.Skipf("MySQL-Server not running on %s", netAddr) + t.Skipf("MySQL server not running on %s", netAddr) } - db, err := sql.Open("mysql", dsn) - if err != nil { - t.Fatalf("Error connecting: %s", err.Error()) + dsn += "&multiStatements=true" + var db *sql.DB + if _, err := ParseDSN(dsn); err != errInvalidDSNUnsafeCollation { + db, err = sql.Open("mysql", dsn) + if err != nil { + t.Fatalf("error connecting: %s", err.Error()) + } + defer db.Close() } - defer db.Close() - - db.Exec("DROP TABLE IF EXISTS test") dbt := &DBTest{t, db} for _, test := range tests { @@ -94,17 +98,67 @@ func runTests(t *testing.T, dsn string, tests ...func(dbt *DBTest)) { } } +func runTests(t *testing.T, dsn string, tests ...func(dbt *DBTest)) { + if !available { + t.Skipf("MySQL server not running on %s", netAddr) + } + + db, err := sql.Open("mysql", dsn) + if err != nil { + t.Fatalf("error connecting: %s", err.Error()) + } + defer db.Close() + + db.Exec("DROP TABLE IF EXISTS test") + + dsn2 := dsn + "&interpolateParams=true" + var db2 *sql.DB + if _, err := ParseDSN(dsn2); err != errInvalidDSNUnsafeCollation { + db2, err = sql.Open("mysql", dsn2) + if err != nil { + t.Fatalf("error connecting: %s", err.Error()) + } + defer db2.Close() + } + + dsn3 := dsn + "&multiStatements=true" + var db3 *sql.DB + if _, err := ParseDSN(dsn3); err != errInvalidDSNUnsafeCollation { + db3, err = sql.Open("mysql", dsn3) + if err != nil { + t.Fatalf("error connecting: %s", err.Error()) + } + defer db3.Close() + } + + dbt := &DBTest{t, db} + dbt2 := &DBTest{t, db2} + dbt3 := &DBTest{t, db3} + for _, test := range tests { + test(dbt) + dbt.db.Exec("DROP TABLE IF EXISTS test") + if db2 != nil { + test(dbt2) + dbt2.db.Exec("DROP TABLE IF EXISTS test") + } + if db3 != nil { + test(dbt3) + dbt3.db.Exec("DROP TABLE IF EXISTS test") + } + } +} + func (dbt *DBTest) fail(method, query string, err error) { if len(query) > 300 { query = "[query too large to print]" } - dbt.Fatalf("Error on %s %s: %s", method, query, err.Error()) + dbt.Fatalf("error on %s %s: %s", method, query, err.Error()) } func (dbt *DBTest) mustExec(query string, args ...interface{}) (res sql.Result) { res, err := dbt.db.Exec(query, args...) if err != nil { - dbt.fail("Exec", query, err) + dbt.fail("exec", query, err) } return res } @@ -112,7 +166,7 @@ func (dbt *DBTest) mustExec(query string, args ...interface{}) (res sql.Result) func (dbt *DBTest) mustQuery(query string, args ...interface{}) (rows *sql.Rows) { rows, err := dbt.db.Query(query, args...) if err != nil { - dbt.fail("Query", query, err) + dbt.fail("query", query, err) } return rows } @@ -123,7 +177,7 @@ func TestEmptyQuery(t *testing.T) { rows := dbt.mustQuery("--") // will hang before #255 if rows.Next() { - dbt.Errorf("Next on rows must be false") + dbt.Errorf("next on rows must be false") } }) } @@ -147,7 +201,7 @@ func TestCRUD(t *testing.T) { dbt.Fatalf("res.RowsAffected() returned error: %s", err.Error()) } if count != 1 { - dbt.Fatalf("Expected 1 affected row, got %d", count) + dbt.Fatalf("expected 1 affected row, got %d", count) } id, err := res.LastInsertId() @@ -155,7 +209,7 @@ func TestCRUD(t *testing.T) { dbt.Fatalf("res.LastInsertId() returned error: %s", err.Error()) } if id != 0 { - dbt.Fatalf("Expected InsertID 0, got %d", id) + dbt.Fatalf("expected InsertId 0, got %d", id) } // Read @@ -180,7 +234,7 @@ func TestCRUD(t *testing.T) { dbt.Fatalf("res.RowsAffected() returned error: %s", err.Error()) } if count != 1 { - dbt.Fatalf("Expected 1 affected row, got %d", count) + dbt.Fatalf("expected 1 affected row, got %d", count) } // Check Update @@ -205,7 +259,7 @@ func TestCRUD(t *testing.T) { dbt.Fatalf("res.RowsAffected() returned error: %s", err.Error()) } if count != 1 { - dbt.Fatalf("Expected 1 affected row, got %d", count) + dbt.Fatalf("expected 1 affected row, got %d", count) } // Check for unexpected rows @@ -215,11 +269,55 @@ func TestCRUD(t *testing.T) { dbt.Fatalf("res.RowsAffected() returned error: %s", err.Error()) } if count != 0 { - dbt.Fatalf("Expected 0 affected row, got %d", count) + dbt.Fatalf("expected 0 affected row, got %d", count) } }) } +func TestMultiQuery(t *testing.T) { + runTestsWithMultiStatement(t, dsn, func(dbt *DBTest) { + // Create Table + dbt.mustExec("CREATE TABLE `test` (`id` int(11) NOT NULL, `value` int(11) NOT NULL) ") + + // Create Data + res := dbt.mustExec("INSERT INTO test VALUES (1, 1)") + count, err := res.RowsAffected() + if err != nil { + dbt.Fatalf("res.RowsAffected() returned error: %s", err.Error()) + } + if count != 1 { + dbt.Fatalf("expected 1 affected row, got %d", count) + } + + // Update + res = dbt.mustExec("UPDATE test SET value = 3 WHERE id = 1; UPDATE test SET value = 4 WHERE id = 1; UPDATE test SET value = 5 WHERE id = 1;") + count, err = res.RowsAffected() + if err != nil { + dbt.Fatalf("res.RowsAffected() returned error: %s", err.Error()) + } + if count != 1 { + dbt.Fatalf("expected 1 affected row, got %d", count) + } + + // Read + var out int + rows := dbt.mustQuery("SELECT value FROM test WHERE id=1;") + if rows.Next() { + rows.Scan(&out) + if 5 != out { + dbt.Errorf("5 != %t", out) + } + + if rows.Next() { + dbt.Error("unexpected data") + } + } else { + dbt.Error("no data") + } + + }) +} + func TestInt(t *testing.T) { runTests(t, dsn, func(dbt *DBTest) { types := [5]string{"TINYINT", "SMALLINT", "MEDIUMINT", "INT", "BIGINT"} @@ -636,14 +734,14 @@ func TestNULL(t *testing.T) { dbt.Fatal(err) } if nb.Valid { - dbt.Error("Valid NullBool which should be invalid") + dbt.Error("valid NullBool which should be invalid") } // Valid if err = nonNullStmt.QueryRow().Scan(&nb); err != nil { dbt.Fatal(err) } if !nb.Valid { - dbt.Error("Invalid NullBool which should be valid") + dbt.Error("invalid NullBool which should be valid") } else if nb.Bool != true { dbt.Errorf("Unexpected NullBool value: %t (should be true)", nb.Bool) } @@ -655,16 +753,16 @@ func TestNULL(t *testing.T) { dbt.Fatal(err) } if nf.Valid { - dbt.Error("Valid NullFloat64 which should be invalid") + dbt.Error("valid NullFloat64 which should be invalid") } // Valid if err = nonNullStmt.QueryRow().Scan(&nf); err != nil { dbt.Fatal(err) } if !nf.Valid { - dbt.Error("Invalid NullFloat64 which should be valid") + dbt.Error("invalid NullFloat64 which should be valid") } else if nf.Float64 != float64(1) { - dbt.Errorf("Unexpected NullFloat64 value: %f (should be 1.0)", nf.Float64) + dbt.Errorf("unexpected NullFloat64 value: %f (should be 1.0)", nf.Float64) } // NullInt64 @@ -674,16 +772,16 @@ func TestNULL(t *testing.T) { dbt.Fatal(err) } if ni.Valid { - dbt.Error("Valid NullInt64 which should be invalid") + dbt.Error("valid NullInt64 which should be invalid") } // Valid if err = nonNullStmt.QueryRow().Scan(&ni); err != nil { dbt.Fatal(err) } if !ni.Valid { - dbt.Error("Invalid NullInt64 which should be valid") + dbt.Error("invalid NullInt64 which should be valid") } else if ni.Int64 != int64(1) { - dbt.Errorf("Unexpected NullInt64 value: %d (should be 1)", ni.Int64) + dbt.Errorf("unexpected NullInt64 value: %d (should be 1)", ni.Int64) } // NullString @@ -693,16 +791,16 @@ func TestNULL(t *testing.T) { dbt.Fatal(err) } if ns.Valid { - dbt.Error("Valid NullString which should be invalid") + dbt.Error("valid NullString which should be invalid") } // Valid if err = nonNullStmt.QueryRow().Scan(&ns); err != nil { dbt.Fatal(err) } if !ns.Valid { - dbt.Error("Invalid NullString which should be valid") + dbt.Error("invalid NullString which should be valid") } else if ns.String != `1` { - dbt.Error("Unexpected NullString value:" + ns.String + " (should be `1`)") + dbt.Error("unexpected NullString value:" + ns.String + " (should be `1`)") } // nil-bytes @@ -712,14 +810,14 @@ func TestNULL(t *testing.T) { dbt.Fatal(err) } if b != nil { - dbt.Error("Non-nil []byte wich should be nil") + dbt.Error("non-nil []byte wich should be nil") } // Read non-nil if err = nonNullStmt.QueryRow().Scan(&b); err != nil { dbt.Fatal(err) } if b == nil { - dbt.Error("Nil []byte wich should be non-nil") + dbt.Error("nil []byte wich should be non-nil") } // Insert nil b = nil @@ -728,7 +826,7 @@ func TestNULL(t *testing.T) { dbt.Fatal(err) } if !success { - dbt.Error("Inserting []byte(nil) as NULL failed") + dbt.Error("inserting []byte(nil) as NULL failed") } // Check input==output with input==nil b = nil @@ -736,7 +834,7 @@ func TestNULL(t *testing.T) { dbt.Fatal(err) } if b != nil { - dbt.Error("Non-nil echo from nil input") + dbt.Error("non-nil echo from nil input") } // Check input==output with input!=nil b = []byte("") @@ -765,6 +863,49 @@ func TestNULL(t *testing.T) { }) } +func TestUint64(t *testing.T) { + const ( + u0 = uint64(0) + uall = ^u0 + uhigh = uall >> 1 + utop = ^uhigh + s0 = int64(0) + sall = ^s0 + shigh = int64(uhigh) + stop = ^shigh + ) + runTests(t, dsn, func(dbt *DBTest) { + stmt, err := dbt.db.Prepare(`SELECT ?, ?, ? ,?, ?, ?, ?, ?`) + if err != nil { + dbt.Fatal(err) + } + defer stmt.Close() + row := stmt.QueryRow( + u0, uhigh, utop, uall, + s0, shigh, stop, sall, + ) + + var ua, ub, uc, ud uint64 + var sa, sb, sc, sd int64 + + err = row.Scan(&ua, &ub, &uc, &ud, &sa, &sb, &sc, &sd) + if err != nil { + dbt.Fatal(err) + } + switch { + case ua != u0, + ub != uhigh, + uc != utop, + ud != uall, + sa != s0, + sb != shigh, + sc != stop, + sd != sall: + dbt.Fatal("unexpected result value") + } + }) +} + func TestLongData(t *testing.T) { runTests(t, dsn, func(dbt *DBTest) { var maxAllowedPacketSize int @@ -855,7 +996,7 @@ func TestLoadData(t *testing.T) { dbt.Fatalf("%d != %d", i, id) } if values[i-1] != value { - dbt.Fatalf("%s != %s", values[i-1], value) + dbt.Fatalf("%q != %q", values[i-1], value) } } err = rows.Err() @@ -864,7 +1005,7 @@ func TestLoadData(t *testing.T) { } if i != 4 { - dbt.Fatalf("Rows count mismatch. Got %d, want 4", i) + dbt.Fatalf("rows count mismatch. Got %d, want 4", i) } } file, err := ioutil.TempFile("", "gotest") @@ -880,13 +1021,13 @@ func TestLoadData(t *testing.T) { // Local File RegisterLocalFile(file.Name()) - dbt.mustExec(fmt.Sprintf("LOAD DATA LOCAL INFILE '%q' INTO TABLE test", file.Name())) + dbt.mustExec(fmt.Sprintf("LOAD DATA LOCAL INFILE %q INTO TABLE test", file.Name())) verifyLoadDataResult() // negative test _, err = dbt.db.Exec("LOAD DATA LOCAL INFILE 'doesnotexist' INTO TABLE test") if err == nil { - dbt.Fatal("Load non-existent file didn't fail") - } else if err.Error() != "Local File 'doesnotexist' is not registered. Use the DSN parameter 'allowAllFiles=true' to allow all files" { + dbt.Fatal("load non-existent file didn't fail") + } else if err.Error() != "local file 'doesnotexist' is not registered" { dbt.Fatal(err.Error()) } @@ -906,7 +1047,7 @@ func TestLoadData(t *testing.T) { // negative test _, err = dbt.db.Exec("LOAD DATA LOCAL INFILE 'Reader::doesnotexist' INTO TABLE test") if err == nil { - dbt.Fatal("Load non-existent Reader didn't fail") + dbt.Fatal("load non-existent Reader didn't fail") } else if err.Error() != "Reader 'doesnotexist' is not registered" { dbt.Fatal(err.Error()) } @@ -960,7 +1101,7 @@ func TestFoundRows(t *testing.T) { func TestStrict(t *testing.T) { // ALLOW_INVALID_DATES to get rid of stricter modes - we want to test for warnings, not errors - relaxedDsn := dsn + "&sql_mode=ALLOW_INVALID_DATES" + relaxedDsn := dsn + "&sql_mode='ALLOW_INVALID_DATES,NO_AUTO_CREATE_USER'" // make sure the MySQL version is recent enough with a separate connection // before running the test conn, err := MySQLDriver{}.Open(relaxedDsn) @@ -986,7 +1127,7 @@ func TestStrict(t *testing.T) { var checkWarnings = func(err error, mode string, idx int) { if err == nil { - dbt.Errorf("Expected STRICT error on query [%s] %s", mode, queries[idx].in) + dbt.Errorf("expected STRICT error on query [%s] %s", mode, queries[idx].in) } if warnings, ok := err.(MySQLWarnings); ok { @@ -995,18 +1136,18 @@ func TestStrict(t *testing.T) { codes[i] = warnings[i].Code } if len(codes) != len(queries[idx].codes) { - dbt.Errorf("Unexpected STRICT error count on query [%s] %s: Wanted %v, Got %v", mode, queries[idx].in, queries[idx].codes, codes) + dbt.Errorf("unexpected STRICT error count on query [%s] %s: Wanted %v, Got %v", mode, queries[idx].in, queries[idx].codes, codes) } for i := range warnings { if codes[i] != queries[idx].codes[i] { - dbt.Errorf("Unexpected STRICT error codes on query [%s] %s: Wanted %v, Got %v", mode, queries[idx].in, queries[idx].codes, codes) + dbt.Errorf("unexpected STRICT error codes on query [%s] %s: Wanted %v, Got %v", mode, queries[idx].in, queries[idx].codes, codes) return } } } else { - dbt.Errorf("Unexpected error on query [%s] %s: %s", mode, queries[idx].in, err.Error()) + dbt.Errorf("unexpected error on query [%s] %s: %s", mode, queries[idx].in, err.Error()) } } @@ -1022,7 +1163,7 @@ func TestStrict(t *testing.T) { for i := range queries { stmt, err = dbt.db.Prepare(queries[i].in) if err != nil { - dbt.Errorf("Error on preparing query %s: %s", queries[i].in, err.Error()) + dbt.Errorf("error on preparing query %s: %s", queries[i].in, err.Error()) } _, err = stmt.Exec() @@ -1030,7 +1171,7 @@ func TestStrict(t *testing.T) { err = stmt.Close() if err != nil { - dbt.Errorf("Error on closing stmt for query %s: %s", queries[i].in, err.Error()) + dbt.Errorf("error on closing stmt for query %s: %s", queries[i].in, err.Error()) } } }) @@ -1040,9 +1181,9 @@ func TestTLS(t *testing.T) { tlsTest := func(dbt *DBTest) { if err := dbt.db.Ping(); err != nil { if err == ErrNoTLS { - dbt.Skip("Server does not support TLS") + dbt.Skip("server does not support TLS") } else { - dbt.Fatalf("Error on Ping: %s", err.Error()) + dbt.Fatalf("error on Ping: %s", err.Error()) } } @@ -1055,7 +1196,7 @@ func TestTLS(t *testing.T) { } if value == nil { - dbt.Fatal("No Cipher") + dbt.Fatal("no Cipher") } } } @@ -1072,42 +1213,42 @@ func TestTLS(t *testing.T) { func TestReuseClosedConnection(t *testing.T) { // this test does not use sql.database, it uses the driver directly if !available { - t.Skipf("MySQL-Server not running on %s", netAddr) + t.Skipf("MySQL server not running on %s", netAddr) } md := &MySQLDriver{} conn, err := md.Open(dsn) if err != nil { - t.Fatalf("Error connecting: %s", err.Error()) + t.Fatalf("error connecting: %s", err.Error()) } stmt, err := conn.Prepare("DO 1") if err != nil { - t.Fatalf("Error preparing statement: %s", err.Error()) + t.Fatalf("error preparing statement: %s", err.Error()) } _, err = stmt.Exec(nil) if err != nil { - t.Fatalf("Error executing statement: %s", err.Error()) + t.Fatalf("error executing statement: %s", err.Error()) } err = conn.Close() if err != nil { - t.Fatalf("Error closing connection: %s", err.Error()) + t.Fatalf("error closing connection: %s", err.Error()) } defer func() { if err := recover(); err != nil { - t.Errorf("Panic after reusing a closed connection: %v", err) + t.Errorf("panic after reusing a closed connection: %v", err) } }() _, err = stmt.Exec(nil) if err != nil && err != driver.ErrBadConn { - t.Errorf("Unexpected error '%s', expected '%s'", + t.Errorf("unexpected error '%s', expected '%s'", err.Error(), driver.ErrBadConn.Error()) } } func TestCharset(t *testing.T) { if !available { - t.Skipf("MySQL-Server not running on %s", netAddr) + t.Skipf("MySQL server not running on %s", netAddr) } mustSetCharset := func(charsetParam, expected string) { @@ -1116,14 +1257,14 @@ func TestCharset(t *testing.T) { defer rows.Close() if !rows.Next() { - dbt.Fatalf("Error getting connection charset: %s", rows.Err()) + dbt.Fatalf("error getting connection charset: %s", rows.Err()) } var got string rows.Scan(&got) if got != expected { - dbt.Fatalf("Expected connection charset %s but got %s", expected, got) + dbt.Fatalf("expected connection charset %s but got %s", expected, got) } }) } @@ -1145,14 +1286,14 @@ func TestFailingCharset(t *testing.T) { _, err := dbt.db.Exec("SELECT 1") if err == nil { dbt.db.Close() - t.Fatalf("Connection must not succeed without a valid charset") + t.Fatalf("connection must not succeed without a valid charset") } }) } func TestCollation(t *testing.T) { if !available { - t.Skipf("MySQL-Server not running on %s", netAddr) + t.Skipf("MySQL server not running on %s", netAddr) } defaultCollation := "utf8_general_ci" @@ -1182,12 +1323,36 @@ func TestCollation(t *testing.T) { } if got != expected { - dbt.Fatalf("Expected connection collation %s but got %s", expected, got) + dbt.Fatalf("expected connection collation %s but got %s", expected, got) } }) } } +func TestColumnsWithAlias(t *testing.T) { + runTests(t, dsn+"&columnsWithAlias=true", func(dbt *DBTest) { + rows := dbt.mustQuery("SELECT 1 AS A") + defer rows.Close() + cols, _ := rows.Columns() + if len(cols) != 1 { + t.Fatalf("expected 1 column, got %d", len(cols)) + } + if cols[0] != "A" { + t.Fatalf("expected column name \"A\", got \"%s\"", cols[0]) + } + rows.Close() + + rows = dbt.mustQuery("SELECT * FROM (SELECT 1 AS one) AS A") + cols, _ = rows.Columns() + if len(cols) != 1 { + t.Fatalf("expected 1 column, got %d", len(cols)) + } + if cols[0] != "A.one" { + t.Fatalf("expected column name \"A.one\", got \"%s\"", cols[0]) + } + }) +} + func TestRawBytesResultExceedsBuffer(t *testing.T) { runTests(t, dsn, func(dbt *DBTest) { // defaultBufSize from buffer.go @@ -1223,7 +1388,7 @@ func TestTimezoneConversion(t *testing.T) { // Retrieve time from DB rows := dbt.mustQuery("SELECT ts FROM test") if !rows.Next() { - dbt.Fatal("Didn't get any rows out") + dbt.Fatal("did not get any rows out") } var dbTime time.Time @@ -1234,7 +1399,7 @@ func TestTimezoneConversion(t *testing.T) { // Check that dates match if reftime.Unix() != dbTime.Unix() { - dbt.Errorf("Times don't match.\n") + dbt.Errorf("times do not match.\n") dbt.Errorf(" Now(%v)=%v\n", usCentral, reftime) dbt.Errorf(" Now(UTC)=%v\n", dbTime) } @@ -1260,7 +1425,7 @@ func TestRowsClose(t *testing.T) { } if rows.Next() { - dbt.Fatal("Unexpected row after rows.Close()") + dbt.Fatal("unexpected row after rows.Close()") } err = rows.Err() @@ -1292,7 +1457,7 @@ func TestCloseStmtBeforeRows(t *testing.T) { } if !rows.Next() { - dbt.Fatal("Getting row failed") + dbt.Fatal("getting row failed") } else { err = rows.Err() if err != nil { @@ -1302,7 +1467,7 @@ func TestCloseStmtBeforeRows(t *testing.T) { var out bool err = rows.Scan(&out) if err != nil { - dbt.Fatalf("Error on rows.Scan(): %s", err.Error()) + dbt.Fatalf("error on rows.Scan(): %s", err.Error()) } if out != true { dbt.Errorf("true != %t", out) @@ -1338,7 +1503,7 @@ func TestStmtMultiRows(t *testing.T) { // 1 if !rows1.Next() { - dbt.Fatal("1st rows1.Next failed") + dbt.Fatal("first rows1.Next failed") } else { err = rows1.Err() if err != nil { @@ -1347,7 +1512,7 @@ func TestStmtMultiRows(t *testing.T) { err = rows1.Scan(&out) if err != nil { - dbt.Fatalf("Error on rows.Scan(): %s", err.Error()) + dbt.Fatalf("error on rows.Scan(): %s", err.Error()) } if out != true { dbt.Errorf("true != %t", out) @@ -1355,7 +1520,7 @@ func TestStmtMultiRows(t *testing.T) { } if !rows2.Next() { - dbt.Fatal("1st rows2.Next failed") + dbt.Fatal("first rows2.Next failed") } else { err = rows2.Err() if err != nil { @@ -1364,7 +1529,7 @@ func TestStmtMultiRows(t *testing.T) { err = rows2.Scan(&out) if err != nil { - dbt.Fatalf("Error on rows.Scan(): %s", err.Error()) + dbt.Fatalf("error on rows.Scan(): %s", err.Error()) } if out != true { dbt.Errorf("true != %t", out) @@ -1373,7 +1538,7 @@ func TestStmtMultiRows(t *testing.T) { // 2 if !rows1.Next() { - dbt.Fatal("2nd rows1.Next failed") + dbt.Fatal("second rows1.Next failed") } else { err = rows1.Err() if err != nil { @@ -1382,14 +1547,14 @@ func TestStmtMultiRows(t *testing.T) { err = rows1.Scan(&out) if err != nil { - dbt.Fatalf("Error on rows.Scan(): %s", err.Error()) + dbt.Fatalf("error on rows.Scan(): %s", err.Error()) } if out != false { dbt.Errorf("false != %t", out) } if rows1.Next() { - dbt.Fatal("Unexpected row on rows1") + dbt.Fatal("unexpected row on rows1") } err = rows1.Close() if err != nil { @@ -1398,7 +1563,7 @@ func TestStmtMultiRows(t *testing.T) { } if !rows2.Next() { - dbt.Fatal("2nd rows2.Next failed") + dbt.Fatal("second rows2.Next failed") } else { err = rows2.Err() if err != nil { @@ -1407,14 +1572,14 @@ func TestStmtMultiRows(t *testing.T) { err = rows2.Scan(&out) if err != nil { - dbt.Fatalf("Error on rows.Scan(): %s", err.Error()) + dbt.Fatalf("error on rows.Scan(): %s", err.Error()) } if out != false { dbt.Errorf("false != %t", out) } if rows2.Next() { - dbt.Fatal("Unexpected row on rows2") + dbt.Fatal("unexpected row on rows2") } err = rows2.Close() if err != nil { @@ -1459,7 +1624,7 @@ func TestConcurrent(t *testing.T) { if err != nil { dbt.Fatalf("%s", err.Error()) } - dbt.Logf("Testing up to %d concurrent connections \r\n", max) + dbt.Logf("testing up to %d concurrent connections \r\n", max) var remaining, succeeded int32 = int32(max), 0 @@ -1483,7 +1648,7 @@ func TestConcurrent(t *testing.T) { if err != nil { if err.Error() != "Error 1040: Too many connections" { - fatalf("Error on Conn %d: %s", id, err.Error()) + fatalf("error on conn %d: %s", id, err.Error()) } return } @@ -1491,13 +1656,13 @@ func TestConcurrent(t *testing.T) { // keep the connection busy until all connections are open for remaining > 0 { if _, err = tx.Exec("DO 1"); err != nil { - fatalf("Error on Conn %d: %s", id, err.Error()) + fatalf("error on conn %d: %s", id, err.Error()) return } } if err = tx.Commit(); err != nil { - fatalf("Error on Conn %d: %s", id, err.Error()) + fatalf("error on conn %d: %s", id, err.Error()) return } @@ -1513,14 +1678,14 @@ func TestConcurrent(t *testing.T) { dbt.Fatal(fatalError) } - dbt.Logf("Reached %d concurrent connections\r\n", succeeded) + dbt.Logf("reached %d concurrent connections\r\n", succeeded) }) } // Tests custom dial functions func TestCustomDial(t *testing.T) { if !available { - t.Skipf("MySQL-Server not running on %s", netAddr) + t.Skipf("MySQL server not running on %s", netAddr) } // our custom dial function which justs wraps net.Dial here @@ -1530,11 +1695,117 @@ func TestCustomDial(t *testing.T) { db, err := sql.Open("mysql", fmt.Sprintf("%s:%s@mydial(%s)/%s?timeout=30s&strict=true", user, pass, addr, dbname)) if err != nil { - t.Fatalf("Error connecting: %s", err.Error()) + t.Fatalf("error connecting: %s", err.Error()) } defer db.Close() if _, err = db.Exec("DO 1"); err != nil { - t.Fatalf("Connection failed: %s", err.Error()) + t.Fatalf("connection failed: %s", err.Error()) } } + +func TestSQLInjection(t *testing.T) { + createTest := func(arg string) func(dbt *DBTest) { + return func(dbt *DBTest) { + dbt.mustExec("CREATE TABLE test (v INTEGER)") + dbt.mustExec("INSERT INTO test VALUES (?)", 1) + + var v int + // NULL can't be equal to anything, the idea here is to inject query so it returns row + // This test verifies that escapeQuotes and escapeBackslash are working properly + err := dbt.db.QueryRow("SELECT v FROM test WHERE NULL = ?", arg).Scan(&v) + if err == sql.ErrNoRows { + return // success, sql injection failed + } else if err == nil { + dbt.Errorf("sql injection successful with arg: %s", arg) + } else { + dbt.Errorf("error running query with arg: %s; err: %s", arg, err.Error()) + } + } + } + + dsns := []string{ + dsn, + dsn + "&sql_mode='NO_BACKSLASH_ESCAPES,NO_AUTO_CREATE_USER'", + } + for _, testdsn := range dsns { + runTests(t, testdsn, createTest("1 OR 1=1")) + runTests(t, testdsn, createTest("' OR '1'='1")) + } +} + +// Test if inserted data is correctly retrieved after being escaped +func TestInsertRetrieveEscapedData(t *testing.T) { + testData := func(dbt *DBTest) { + dbt.mustExec("CREATE TABLE test (v VARCHAR(255))") + + // All sequences that are escaped by escapeQuotes and escapeBackslash + v := "foo \x00\n\r\x1a\"'\\" + dbt.mustExec("INSERT INTO test VALUES (?)", v) + + var out string + err := dbt.db.QueryRow("SELECT v FROM test").Scan(&out) + if err != nil { + dbt.Fatalf("%s", err.Error()) + } + + if out != v { + dbt.Errorf("%q != %q", out, v) + } + } + + dsns := []string{ + dsn, + dsn + "&sql_mode='NO_BACKSLASH_ESCAPES,NO_AUTO_CREATE_USER'", + } + for _, testdsn := range dsns { + runTests(t, testdsn, testData) + } +} + +func TestUnixSocketAuthFail(t *testing.T) { + runTests(t, dsn, func(dbt *DBTest) { + // Save the current logger so we can restore it. + oldLogger := errLog + + // Set a new logger so we can capture its output. + buffer := bytes.NewBuffer(make([]byte, 0, 64)) + newLogger := log.New(buffer, "prefix: ", 0) + SetLogger(newLogger) + + // Restore the logger. + defer SetLogger(oldLogger) + + // Make a new DSN that uses the MySQL socket file and a bad password, which + // we can make by simply appending any character to the real password. + badPass := pass + "x" + socket := "" + if prot == "unix" { + socket = addr + } else { + // Get socket file from MySQL. + err := dbt.db.QueryRow("SELECT @@socket").Scan(&socket) + if err != nil { + t.Fatalf("error on SELECT @@socket: %s", err.Error()) + } + } + t.Logf("socket: %s", socket) + badDSN := fmt.Sprintf("%s:%s@unix(%s)/%s?timeout=30s&strict=true", user, badPass, socket, dbname) + db, err := sql.Open("mysql", badDSN) + if err != nil { + t.Fatalf("error connecting: %s", err.Error()) + } + defer db.Close() + + // Connect to MySQL for real. This will cause an auth failure. + err = db.Ping() + if err == nil { + t.Error("expected Ping() to return an error") + } + + // The driver should not log anything. + if actual := buffer.String(); actual != "" { + t.Errorf("expected no output, got %q", actual) + } + }) +} diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/dsn.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/dsn.go new file mode 100644 index 00000000000..29cce2d860a --- /dev/null +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/dsn.go @@ -0,0 +1,513 @@ +// Go MySQL Driver - A MySQL-Driver for Go's database/sql package +// +// Copyright 2016 The Go-MySQL-Driver Authors. All rights reserved. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at http://mozilla.org/MPL/2.0/. + +package mysql + +import ( + "bytes" + "crypto/tls" + "errors" + "fmt" + "net" + "net/url" + "strings" + "time" +) + +var ( + errInvalidDSNUnescaped = errors.New("invalid DSN: did you forget to escape a param value?") + errInvalidDSNAddr = errors.New("invalid DSN: network address not terminated (missing closing brace)") + errInvalidDSNNoSlash = errors.New("invalid DSN: missing the slash separating the database name") + errInvalidDSNUnsafeCollation = errors.New("invalid DSN: interpolateParams can not be used with unsafe collations") +) + +// Config is a configuration parsed from a DSN string +type Config struct { + User string // Username + Passwd string // Password + Net string // Network type + Addr string // Network address + DBName string // Database name + Params map[string]string // Connection parameters + Collation string // Connection collation + Loc *time.Location // Location for time.Time values + TLSConfig string // TLS configuration name + tls *tls.Config // TLS configuration + Timeout time.Duration // Dial timeout + ReadTimeout time.Duration // I/O read timeout + WriteTimeout time.Duration // I/O write timeout + + AllowAllFiles bool // Allow all files to be used with LOAD DATA LOCAL INFILE + AllowCleartextPasswords bool // Allows the cleartext client side plugin + AllowOldPasswords bool // Allows the old insecure password method + ClientFoundRows bool // Return number of matching rows instead of rows changed + ColumnsWithAlias bool // Prepend table alias to column names + InterpolateParams bool // Interpolate placeholders into query string + MultiStatements bool // Allow multiple statements in one query + ParseTime bool // Parse time values to time.Time + Strict bool // Return warnings as errors +} + +// FormatDSN formats the given Config into a DSN string which can be passed to +// the driver. +func (cfg *Config) FormatDSN() string { + var buf bytes.Buffer + + // [username[:password]@] + if len(cfg.User) > 0 { + buf.WriteString(cfg.User) + if len(cfg.Passwd) > 0 { + buf.WriteByte(':') + buf.WriteString(cfg.Passwd) + } + buf.WriteByte('@') + } + + // [protocol[(address)]] + if len(cfg.Net) > 0 { + buf.WriteString(cfg.Net) + if len(cfg.Addr) > 0 { + buf.WriteByte('(') + buf.WriteString(cfg.Addr) + buf.WriteByte(')') + } + } + + // /dbname + buf.WriteByte('/') + buf.WriteString(cfg.DBName) + + // [?param1=value1&...¶mN=valueN] + hasParam := false + + if cfg.AllowAllFiles { + hasParam = true + buf.WriteString("?allowAllFiles=true") + } + + if cfg.AllowCleartextPasswords { + if hasParam { + buf.WriteString("&allowCleartextPasswords=true") + } else { + hasParam = true + buf.WriteString("?allowCleartextPasswords=true") + } + } + + if cfg.AllowOldPasswords { + if hasParam { + buf.WriteString("&allowOldPasswords=true") + } else { + hasParam = true + buf.WriteString("?allowOldPasswords=true") + } + } + + if cfg.ClientFoundRows { + if hasParam { + buf.WriteString("&clientFoundRows=true") + } else { + hasParam = true + buf.WriteString("?clientFoundRows=true") + } + } + + if col := cfg.Collation; col != defaultCollation && len(col) > 0 { + if hasParam { + buf.WriteString("&collation=") + } else { + hasParam = true + buf.WriteString("?collation=") + } + buf.WriteString(col) + } + + if cfg.ColumnsWithAlias { + if hasParam { + buf.WriteString("&columnsWithAlias=true") + } else { + hasParam = true + buf.WriteString("?columnsWithAlias=true") + } + } + + if cfg.InterpolateParams { + if hasParam { + buf.WriteString("&interpolateParams=true") + } else { + hasParam = true + buf.WriteString("?interpolateParams=true") + } + } + + if cfg.Loc != time.UTC && cfg.Loc != nil { + if hasParam { + buf.WriteString("&loc=") + } else { + hasParam = true + buf.WriteString("?loc=") + } + buf.WriteString(url.QueryEscape(cfg.Loc.String())) + } + + if cfg.MultiStatements { + if hasParam { + buf.WriteString("&multiStatements=true") + } else { + hasParam = true + buf.WriteString("?multiStatements=true") + } + } + + if cfg.ParseTime { + if hasParam { + buf.WriteString("&parseTime=true") + } else { + hasParam = true + buf.WriteString("?parseTime=true") + } + } + + if cfg.ReadTimeout > 0 { + if hasParam { + buf.WriteString("&readTimeout=") + } else { + hasParam = true + buf.WriteString("?readTimeout=") + } + buf.WriteString(cfg.ReadTimeout.String()) + } + + if cfg.Strict { + if hasParam { + buf.WriteString("&strict=true") + } else { + hasParam = true + buf.WriteString("?strict=true") + } + } + + if cfg.Timeout > 0 { + if hasParam { + buf.WriteString("&timeout=") + } else { + hasParam = true + buf.WriteString("?timeout=") + } + buf.WriteString(cfg.Timeout.String()) + } + + if len(cfg.TLSConfig) > 0 { + if hasParam { + buf.WriteString("&tls=") + } else { + hasParam = true + buf.WriteString("?tls=") + } + buf.WriteString(url.QueryEscape(cfg.TLSConfig)) + } + + if cfg.WriteTimeout > 0 { + if hasParam { + buf.WriteString("&writeTimeout=") + } else { + hasParam = true + buf.WriteString("?writeTimeout=") + } + buf.WriteString(cfg.WriteTimeout.String()) + } + + // other params + if cfg.Params != nil { + for param, value := range cfg.Params { + if hasParam { + buf.WriteByte('&') + } else { + hasParam = true + buf.WriteByte('?') + } + + buf.WriteString(param) + buf.WriteByte('=') + buf.WriteString(url.QueryEscape(value)) + } + } + + return buf.String() +} + +// ParseDSN parses the DSN string to a Config +func ParseDSN(dsn string) (cfg *Config, err error) { + // New config with some default values + cfg = &Config{ + Loc: time.UTC, + Collation: defaultCollation, + } + + // [user[:password]@][net[(addr)]]/dbname[?param1=value1¶mN=valueN] + // Find the last '/' (since the password or the net addr might contain a '/') + foundSlash := false + for i := len(dsn) - 1; i >= 0; i-- { + if dsn[i] == '/' { + foundSlash = true + var j, k int + + // left part is empty if i <= 0 + if i > 0 { + // [username[:password]@][protocol[(address)]] + // Find the last '@' in dsn[:i] + for j = i; j >= 0; j-- { + if dsn[j] == '@' { + // username[:password] + // Find the first ':' in dsn[:j] + for k = 0; k < j; k++ { + if dsn[k] == ':' { + cfg.Passwd = dsn[k+1 : j] + break + } + } + cfg.User = dsn[:k] + + break + } + } + + // [protocol[(address)]] + // Find the first '(' in dsn[j+1:i] + for k = j + 1; k < i; k++ { + if dsn[k] == '(' { + // dsn[i-1] must be == ')' if an address is specified + if dsn[i-1] != ')' { + if strings.ContainsRune(dsn[k+1:i], ')') { + return nil, errInvalidDSNUnescaped + } + return nil, errInvalidDSNAddr + } + cfg.Addr = dsn[k+1 : i-1] + break + } + } + cfg.Net = dsn[j+1 : k] + } + + // dbname[?param1=value1&...¶mN=valueN] + // Find the first '?' in dsn[i+1:] + for j = i + 1; j < len(dsn); j++ { + if dsn[j] == '?' { + if err = parseDSNParams(cfg, dsn[j+1:]); err != nil { + return + } + break + } + } + cfg.DBName = dsn[i+1 : j] + + break + } + } + + if !foundSlash && len(dsn) > 0 { + return nil, errInvalidDSNNoSlash + } + + if cfg.InterpolateParams && unsafeCollations[cfg.Collation] { + return nil, errInvalidDSNUnsafeCollation + } + + // Set default network if empty + if cfg.Net == "" { + cfg.Net = "tcp" + } + + // Set default address if empty + if cfg.Addr == "" { + switch cfg.Net { + case "tcp": + cfg.Addr = "127.0.0.1:3306" + case "unix": + cfg.Addr = "/tmp/mysql.sock" + default: + return nil, errors.New("default addr for network '" + cfg.Net + "' unknown") + } + + } + + return +} + +// parseDSNParams parses the DSN "query string" +// Values must be url.QueryEscape'ed +func parseDSNParams(cfg *Config, params string) (err error) { + for _, v := range strings.Split(params, "&") { + param := strings.SplitN(v, "=", 2) + if len(param) != 2 { + continue + } + + // cfg params + switch value := param[1]; param[0] { + + // Disable INFILE whitelist / enable all files + case "allowAllFiles": + var isBool bool + cfg.AllowAllFiles, isBool = readBool(value) + if !isBool { + return errors.New("invalid bool value: " + value) + } + + // Use cleartext authentication mode (MySQL 5.5.10+) + case "allowCleartextPasswords": + var isBool bool + cfg.AllowCleartextPasswords, isBool = readBool(value) + if !isBool { + return errors.New("invalid bool value: " + value) + } + + // Use old authentication mode (pre MySQL 4.1) + case "allowOldPasswords": + var isBool bool + cfg.AllowOldPasswords, isBool = readBool(value) + if !isBool { + return errors.New("invalid bool value: " + value) + } + + // Switch "rowsAffected" mode + case "clientFoundRows": + var isBool bool + cfg.ClientFoundRows, isBool = readBool(value) + if !isBool { + return errors.New("invalid bool value: " + value) + } + + // Collation + case "collation": + cfg.Collation = value + break + + case "columnsWithAlias": + var isBool bool + cfg.ColumnsWithAlias, isBool = readBool(value) + if !isBool { + return errors.New("invalid bool value: " + value) + } + + // Compression + case "compress": + return errors.New("compression not implemented yet") + + // Enable client side placeholder substitution + case "interpolateParams": + var isBool bool + cfg.InterpolateParams, isBool = readBool(value) + if !isBool { + return errors.New("invalid bool value: " + value) + } + + // Time Location + case "loc": + if value, err = url.QueryUnescape(value); err != nil { + return + } + cfg.Loc, err = time.LoadLocation(value) + if err != nil { + return + } + + // multiple statements in one query + case "multiStatements": + var isBool bool + cfg.MultiStatements, isBool = readBool(value) + if !isBool { + return errors.New("invalid bool value: " + value) + } + + // time.Time parsing + case "parseTime": + var isBool bool + cfg.ParseTime, isBool = readBool(value) + if !isBool { + return errors.New("invalid bool value: " + value) + } + + // I/O read Timeout + case "readTimeout": + cfg.ReadTimeout, err = time.ParseDuration(value) + if err != nil { + return + } + + // Strict mode + case "strict": + var isBool bool + cfg.Strict, isBool = readBool(value) + if !isBool { + return errors.New("invalid bool value: " + value) + } + + // Dial Timeout + case "timeout": + cfg.Timeout, err = time.ParseDuration(value) + if err != nil { + return + } + + // TLS-Encryption + case "tls": + boolValue, isBool := readBool(value) + if isBool { + if boolValue { + cfg.TLSConfig = "true" + cfg.tls = &tls.Config{} + } else { + cfg.TLSConfig = "false" + } + } else if vl := strings.ToLower(value); vl == "skip-verify" { + cfg.TLSConfig = vl + cfg.tls = &tls.Config{InsecureSkipVerify: true} + } else { + name, err := url.QueryUnescape(value) + if err != nil { + return fmt.Errorf("invalid value for TLS config name: %v", err) + } + + if tlsConfig, ok := tlsConfigRegister[name]; ok { + if len(tlsConfig.ServerName) == 0 && !tlsConfig.InsecureSkipVerify { + host, _, err := net.SplitHostPort(cfg.Addr) + if err == nil { + tlsConfig.ServerName = host + } + } + + cfg.TLSConfig = name + cfg.tls = tlsConfig + } else { + return errors.New("invalid value / unknown config name: " + name) + } + } + + // I/O write Timeout + case "writeTimeout": + cfg.WriteTimeout, err = time.ParseDuration(value) + if err != nil { + return + } + + default: + // lazy init + if cfg.Params == nil { + cfg.Params = make(map[string]string) + } + + if cfg.Params[param[0]], err = url.QueryUnescape(value); err != nil { + return + } + } + } + + return +} diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/dsn_test.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/dsn_test.go new file mode 100644 index 00000000000..80949e18ab1 --- /dev/null +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/dsn_test.go @@ -0,0 +1,207 @@ +// Go MySQL Driver - A MySQL-Driver for Go's database/sql package +// +// Copyright 2016 The Go-MySQL-Driver Authors. All rights reserved. +// +// This Source Code Form is subject to the terms of the Mozilla Public +// License, v. 2.0. If a copy of the MPL was not distributed with this file, +// You can obtain one at http://mozilla.org/MPL/2.0/. + +package mysql + +import ( + "crypto/tls" + "fmt" + "net/url" + "testing" +) + +var testDSNs = []struct { + in string + out string +}{ + {"username:password@protocol(address)/dbname?param=value", "&{User:username Passwd:password Net:protocol Addr:address DBName:dbname Params:map[param:value] Collation:utf8_general_ci Loc:UTC TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"username:password@protocol(address)/dbname?param=value&columnsWithAlias=true", "&{User:username Passwd:password Net:protocol Addr:address DBName:dbname Params:map[param:value] Collation:utf8_general_ci Loc:UTC TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:true InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"username:password@protocol(address)/dbname?param=value&columnsWithAlias=true&multiStatements=true", "&{User:username Passwd:password Net:protocol Addr:address DBName:dbname Params:map[param:value] Collation:utf8_general_ci Loc:UTC TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:true InterpolateParams:false MultiStatements:true ParseTime:false Strict:false}"}, + {"user@unix(/path/to/socket)/dbname?charset=utf8", "&{User:user Passwd: Net:unix Addr:/path/to/socket DBName:dbname Params:map[charset:utf8] Collation:utf8_general_ci Loc:UTC TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"user:password@tcp(localhost:5555)/dbname?charset=utf8&tls=true", "&{User:user Passwd:password Net:tcp Addr:localhost:5555 DBName:dbname Params:map[charset:utf8] Collation:utf8_general_ci Loc:UTC TLSConfig:true tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"user:password@tcp(localhost:5555)/dbname?charset=utf8mb4,utf8&tls=skip-verify", "&{User:user Passwd:password Net:tcp Addr:localhost:5555 DBName:dbname Params:map[charset:utf8mb4,utf8] Collation:utf8_general_ci Loc:UTC TLSConfig:skip-verify tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"user:password@/dbname?loc=UTC&timeout=30s&readTimeout=1s&writeTimeout=1s&allowAllFiles=1&clientFoundRows=true&allowOldPasswords=TRUE&collation=utf8mb4_unicode_ci", "&{User:user Passwd:password Net:tcp Addr:127.0.0.1:3306 DBName:dbname Params:map[] Collation:utf8mb4_unicode_ci Loc:UTC TLSConfig: tls: Timeout:30s ReadTimeout:1s WriteTimeout:1s AllowAllFiles:true AllowCleartextPasswords:false AllowOldPasswords:true ClientFoundRows:true ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"user:p@ss(word)@tcp([de:ad:be:ef::ca:fe]:80)/dbname?loc=Local", "&{User:user Passwd:p@ss(word) Net:tcp Addr:[de:ad:be:ef::ca:fe]:80 DBName:dbname Params:map[] Collation:utf8_general_ci Loc:Local TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"/dbname", "&{User: Passwd: Net:tcp Addr:127.0.0.1:3306 DBName:dbname Params:map[] Collation:utf8_general_ci Loc:UTC TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"@/", "&{User: Passwd: Net:tcp Addr:127.0.0.1:3306 DBName: Params:map[] Collation:utf8_general_ci Loc:UTC TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"/", "&{User: Passwd: Net:tcp Addr:127.0.0.1:3306 DBName: Params:map[] Collation:utf8_general_ci Loc:UTC TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"", "&{User: Passwd: Net:tcp Addr:127.0.0.1:3306 DBName: Params:map[] Collation:utf8_general_ci Loc:UTC TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"user:p@/ssword@/", "&{User:user Passwd:p@/ssword Net:tcp Addr:127.0.0.1:3306 DBName: Params:map[] Collation:utf8_general_ci Loc:UTC TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, + {"unix/?arg=%2Fsome%2Fpath.ext", "&{User: Passwd: Net:unix Addr:/tmp/mysql.sock DBName: Params:map[arg:/some/path.ext] Collation:utf8_general_ci Loc:UTC TLSConfig: tls: Timeout:0 ReadTimeout:0 WriteTimeout:0 AllowAllFiles:false AllowCleartextPasswords:false AllowOldPasswords:false ClientFoundRows:false ColumnsWithAlias:false InterpolateParams:false MultiStatements:false ParseTime:false Strict:false}"}, +} + +func TestDSNParser(t *testing.T) { + var cfg *Config + var err error + var res string + + for i, tst := range testDSNs { + cfg, err = ParseDSN(tst.in) + if err != nil { + t.Error(err.Error()) + } + + // pointer not static + cfg.tls = nil + + res = fmt.Sprintf("%+v", cfg) + if res != tst.out { + t.Errorf("%d. ParseDSN(%q) => %q, want %q", i, tst.in, res, tst.out) + } + } +} + +func TestDSNParserInvalid(t *testing.T) { + var invalidDSNs = []string{ + "@net(addr/", // no closing brace + "@tcp(/", // no closing brace + "tcp(/", // no closing brace + "(/", // no closing brace + "net(addr)//", // unescaped + "User:pass@tcp(1.2.3.4:3306)", // no trailing slash + //"/dbname?arg=/some/unescaped/path", + } + + for i, tst := range invalidDSNs { + if _, err := ParseDSN(tst); err == nil { + t.Errorf("invalid DSN #%d. (%s) didn't error!", i, tst) + } + } +} + +func TestDSNReformat(t *testing.T) { + for i, tst := range testDSNs { + dsn1 := tst.in + cfg1, err := ParseDSN(dsn1) + if err != nil { + t.Error(err.Error()) + continue + } + cfg1.tls = nil // pointer not static + res1 := fmt.Sprintf("%+v", cfg1) + + dsn2 := cfg1.FormatDSN() + cfg2, err := ParseDSN(dsn2) + if err != nil { + t.Error(err.Error()) + continue + } + cfg2.tls = nil // pointer not static + res2 := fmt.Sprintf("%+v", cfg2) + + if res1 != res2 { + t.Errorf("%d. %q does not match %q", i, res2, res1) + } + } +} + +func TestDSNWithCustomTLS(t *testing.T) { + baseDSN := "User:password@tcp(localhost:5555)/dbname?tls=" + tlsCfg := tls.Config{} + + RegisterTLSConfig("utils_test", &tlsCfg) + + // Custom TLS is missing + tst := baseDSN + "invalid_tls" + cfg, err := ParseDSN(tst) + if err == nil { + t.Errorf("invalid custom TLS in DSN (%s) but did not error. Got config: %#v", tst, cfg) + } + + tst = baseDSN + "utils_test" + + // Custom TLS with a server name + name := "foohost" + tlsCfg.ServerName = name + cfg, err = ParseDSN(tst) + + if err != nil { + t.Error(err.Error()) + } else if cfg.tls.ServerName != name { + t.Errorf("did not get the correct TLS ServerName (%s) parsing DSN (%s).", name, tst) + } + + // Custom TLS without a server name + name = "localhost" + tlsCfg.ServerName = "" + cfg, err = ParseDSN(tst) + + if err != nil { + t.Error(err.Error()) + } else if cfg.tls.ServerName != name { + t.Errorf("did not get the correct ServerName (%s) parsing DSN (%s).", name, tst) + } + + DeregisterTLSConfig("utils_test") +} + +func TestDSNWithCustomTLSQueryEscape(t *testing.T) { + const configKey = "&%!:" + dsn := "User:password@tcp(localhost:5555)/dbname?tls=" + url.QueryEscape(configKey) + name := "foohost" + tlsCfg := tls.Config{ServerName: name} + + RegisterTLSConfig(configKey, &tlsCfg) + + cfg, err := ParseDSN(dsn) + + if err != nil { + t.Error(err.Error()) + } else if cfg.tls.ServerName != name { + t.Errorf("did not get the correct TLS ServerName (%s) parsing DSN (%s).", name, dsn) + } +} + +func TestDSNUnsafeCollation(t *testing.T) { + _, err := ParseDSN("/dbname?collation=gbk_chinese_ci&interpolateParams=true") + if err != errInvalidDSNUnsafeCollation { + t.Errorf("expected %v, got %v", errInvalidDSNUnsafeCollation, err) + } + + _, err = ParseDSN("/dbname?collation=gbk_chinese_ci&interpolateParams=false") + if err != nil { + t.Errorf("expected %v, got %v", nil, err) + } + + _, err = ParseDSN("/dbname?collation=gbk_chinese_ci") + if err != nil { + t.Errorf("expected %v, got %v", nil, err) + } + + _, err = ParseDSN("/dbname?collation=ascii_bin&interpolateParams=true") + if err != nil { + t.Errorf("expected %v, got %v", nil, err) + } + + _, err = ParseDSN("/dbname?collation=latin1_german1_ci&interpolateParams=true") + if err != nil { + t.Errorf("expected %v, got %v", nil, err) + } + + _, err = ParseDSN("/dbname?collation=utf8_general_ci&interpolateParams=true") + if err != nil { + t.Errorf("expected %v, got %v", nil, err) + } + + _, err = ParseDSN("/dbname?collation=utf8mb4_general_ci&interpolateParams=true") + if err != nil { + t.Errorf("expected %v, got %v", nil, err) + } +} + +func BenchmarkParseDSN(b *testing.B) { + b.ReportAllocs() + + for i := 0; i < b.N; i++ { + for _, tst := range testDSNs { + if _, err := ParseDSN(tst.in); err != nil { + b.Error(err.Error()) + } + } + } +} diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/errors.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/errors.go index 97d7b399620..1543a80546c 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/errors.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/errors.go @@ -19,18 +19,20 @@ import ( // Various errors the driver might return. Can change between driver versions. var ( - ErrInvalidConn = errors.New("Invalid Connection") - ErrMalformPkt = errors.New("Malformed Packet") - ErrNoTLS = errors.New("TLS encryption requested but server does not support TLS") - ErrOldPassword = errors.New("This server only supports the insecure old password authentication. If you still want to use it, please add 'allowOldPasswords=1' to your DSN. See also https://github.com/go-sql-driver/mysql/wiki/old_passwords") - ErrOldProtocol = errors.New("MySQL-Server does not support required Protocol 41+") - ErrPktSync = errors.New("Commands out of sync. You can't run this command now") - ErrPktSyncMul = errors.New("Commands out of sync. Did you run multiple statements at once?") - ErrPktTooLarge = errors.New("Packet for query is too large. You can change this value on the server by adjusting the 'max_allowed_packet' variable.") - ErrBusyBuffer = errors.New("Busy buffer") + ErrInvalidConn = errors.New("invalid connection") + ErrMalformPkt = errors.New("malformed packet") + ErrNoTLS = errors.New("TLS requested but server does not support TLS") + ErrOldPassword = errors.New("this user requires old password authentication. If you still want to use it, please add 'allowOldPasswords=1' to your DSN. See also https://github.com/go-sql-driver/mysql/wiki/old_passwords") + ErrCleartextPassword = errors.New("this user requires clear text authentication. If you still want to use it, please add 'allowCleartextPasswords=1' to your DSN") + ErrUnknownPlugin = errors.New("this authentication plugin is not supported") + ErrOldProtocol = errors.New("MySQL server does not support required protocol 41+") + ErrPktSync = errors.New("commands out of sync. You can't run this command now") + ErrPktSyncMul = errors.New("commands out of sync. Did you run multiple statements at once?") + ErrPktTooLarge = errors.New("packet for query is too large. Try adjusting the 'max_allowed_packet' variable on the server") + ErrBusyBuffer = errors.New("busy buffer") ) -var errLog Logger = log.New(os.Stderr, "[MySQL] ", log.Ldate|log.Ltime|log.Lshortfile) +var errLog = Logger(log.New(os.Stderr, "[mysql] ", log.Ldate|log.Ltime|log.Lshortfile)) // Logger is used to log critical error messages. type Logger interface { diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/infile.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/infile.go index 121a04c712c..e3e5e47d9a4 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/infile.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/infile.go @@ -13,11 +13,14 @@ import ( "io" "os" "strings" + "sync" ) var ( - fileRegister map[string]bool - readerRegister map[string]func() io.Reader + fileRegister map[string]bool + fileRegisterLock sync.RWMutex + readerRegister map[string]func() io.Reader + readerRegisterLock sync.RWMutex ) // RegisterLocalFile adds the given file to the file whitelist, @@ -32,17 +35,21 @@ var ( // ... // func RegisterLocalFile(filePath string) { + fileRegisterLock.Lock() // lazy map init if fileRegister == nil { fileRegister = make(map[string]bool) } fileRegister[strings.Trim(filePath, `"`)] = true + fileRegisterLock.Unlock() } // DeregisterLocalFile removes the given filepath from the whitelist. func DeregisterLocalFile(filePath string) { + fileRegisterLock.Lock() delete(fileRegister, strings.Trim(filePath, `"`)) + fileRegisterLock.Unlock() } // RegisterReaderHandler registers a handler function which is used @@ -61,18 +68,22 @@ func DeregisterLocalFile(filePath string) { // ... // func RegisterReaderHandler(name string, handler func() io.Reader) { + readerRegisterLock.Lock() // lazy map init if readerRegister == nil { readerRegister = make(map[string]func() io.Reader) } readerRegister[name] = handler + readerRegisterLock.Unlock() } // DeregisterReaderHandler removes the ReaderHandler function with // the given name from the registry. func DeregisterReaderHandler(name string) { + readerRegisterLock.Lock() delete(readerRegister, name) + readerRegisterLock.Unlock() } func deferredClose(err *error, closer io.Closer) { @@ -86,9 +97,15 @@ func (mc *mysqlConn) handleInFileRequest(name string) (err error) { var rdr io.Reader var data []byte - if strings.HasPrefix(name, "Reader::") { // io.Reader - name = name[8:] - if handler, inMap := readerRegister[name]; inMap { + if idx := strings.Index(name, "Reader::"); idx == 0 || (idx > 0 && name[idx-1] == '/') { // io.Reader + // The server might return an an absolute path. See issue #355. + name = name[idx+8:] + + readerRegisterLock.RLock() + handler, inMap := readerRegister[name] + readerRegisterLock.RUnlock() + + if inMap { rdr = handler() if rdr != nil { data = make([]byte, 4+mc.maxWriteSize) @@ -104,7 +121,10 @@ func (mc *mysqlConn) handleInFileRequest(name string) (err error) { } } else { // File name = strings.Trim(name, `"`) - if mc.cfg.allowAllFiles || fileRegister[name] { + fileRegisterLock.RLock() + fr := fileRegister[name] + fileRegisterLock.RUnlock() + if mc.cfg.AllowAllFiles || fr { var file *os.File var fi os.FileInfo @@ -119,12 +139,12 @@ func (mc *mysqlConn) handleInFileRequest(name string) (err error) { } else if fileSize <= mc.maxPacketAllowed { data = make([]byte, 4+mc.maxWriteSize) } else { - err = fmt.Errorf("Local File '%s' too large: Size: %d, Max: %d", name, fileSize, mc.maxPacketAllowed) + err = fmt.Errorf("local file '%s' too large: size: %d, max: %d", name, fileSize, mc.maxPacketAllowed) } } } } else { - err = fmt.Errorf("Local File '%s' is not registered. Use the DSN parameter 'allowAllFiles=true' to allow all files", name) + err = fmt.Errorf("local file '%s' is not registered", name) } } @@ -155,8 +175,8 @@ func (mc *mysqlConn) handleInFileRequest(name string) (err error) { // read OK packet if err == nil { return mc.readResultOK() - } else { - mc.readPacket() } + + mc.readPacket() return err } diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/packets.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/packets.go index f2e385bf8c9..d5ea888fd4f 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/packets.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/packets.go @@ -13,6 +13,7 @@ import ( "crypto/tls" "database/sql/driver" "encoding/binary" + "errors" "fmt" "io" "math" @@ -47,9 +48,8 @@ func (mc *mysqlConn) readPacket() ([]byte, error) { if data[3] != mc.sequence { if data[3] > mc.sequence { return nil, ErrPktSyncMul - } else { - return nil, ErrPktSync } + return nil, ErrPktSync } mc.sequence++ @@ -100,6 +100,12 @@ func (mc *mysqlConn) writePacket(data []byte) error { data[3] = mc.sequence // Write packet + if mc.writeTimeout > 0 { + if err := mc.netConn.SetWriteDeadline(time.Now().Add(mc.writeTimeout)); err != nil { + return err + } + } + n, err := mc.netConn.Write(data[:4+size]) if err == nil && n == 4+size { mc.sequence++ @@ -140,7 +146,7 @@ func (mc *mysqlConn) readInitPacket() ([]byte, error) { // protocol version [1 byte] if data[0] < minProtocolVersion { return nil, fmt.Errorf( - "Unsupported MySQL Protocol Version %d. Protocol Version %d or higher is required", + "unsupported protocol version %d. Version %d or higher is required", data[0], minProtocolVersion, ) @@ -196,7 +202,11 @@ func (mc *mysqlConn) readInitPacket() ([]byte, error) { // return //} //return ErrMalformPkt - return cipher, nil + + // make a memory safe copy of the cipher slice + var b [20]byte + copy(b[:], cipher) + return b[:], nil } // make a memory safe copy of the cipher slice @@ -214,9 +224,11 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error { clientLongPassword | clientTransactions | clientLocalFiles | + clientPluginAuth | + clientMultiResults | mc.flags&clientLongFlag - if mc.cfg.clientFoundRows { + if mc.cfg.ClientFoundRows { clientFlags |= clientFoundRows } @@ -225,13 +237,17 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error { clientFlags |= clientSSL } - // User Password - scrambleBuff := scramblePassword(cipher, []byte(mc.cfg.passwd)) + if mc.cfg.MultiStatements { + clientFlags |= clientMultiStatements + } - pktLen := 4 + 4 + 1 + 23 + len(mc.cfg.user) + 1 + 1 + len(scrambleBuff) + // User Password + scrambleBuff := scramblePassword(cipher, []byte(mc.cfg.Passwd)) + + pktLen := 4 + 4 + 1 + 23 + len(mc.cfg.User) + 1 + 1 + len(scrambleBuff) + 21 + 1 // To specify a db name - if n := len(mc.cfg.dbname); n > 0 { + if n := len(mc.cfg.DBName); n > 0 { clientFlags |= clientConnectWithDB pktLen += n + 1 } @@ -257,7 +273,14 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error { data[11] = 0x00 // Charset [1 byte] - data[12] = mc.cfg.collation + var found bool + data[12], found = collations[mc.cfg.Collation] + if !found { + // Note possibility for false negatives: + // could be triggered although the collation is valid if the + // collations map does not contain entries the server supports. + return errors.New("unknown collation") + } // SSL Connection Request Packet // http://dev.mysql.com/doc/internals/en/connection-phase-packets.html#packet-Protocol::SSLRequest @@ -273,15 +296,18 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error { return err } mc.netConn = tlsConn - mc.buf.rd = tlsConn + mc.buf.nc = tlsConn } // Filler [23 bytes] (all 0x00) - pos := 13 + 23 + pos := 13 + for ; pos < 13+23; pos++ { + data[pos] = 0 + } // User [null terminated string] - if len(mc.cfg.user) > 0 { - pos += copy(data[pos:], mc.cfg.user) + if len(mc.cfg.User) > 0 { + pos += copy(data[pos:], mc.cfg.User) } data[pos] = 0x00 pos++ @@ -291,11 +317,16 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error { pos += 1 + copy(data[pos+1:], scrambleBuff) // Databasename [null terminated string] - if len(mc.cfg.dbname) > 0 { - pos += copy(data[pos:], mc.cfg.dbname) + if len(mc.cfg.DBName) > 0 { + pos += copy(data[pos:], mc.cfg.DBName) data[pos] = 0x00 + pos++ } + // Assume native client during response + pos += copy(data[pos:], "mysql_native_password") + data[pos] = 0x00 + // Send Auth packet return mc.writePacket(data) } @@ -304,9 +335,9 @@ func (mc *mysqlConn) writeAuthPacket(cipher []byte) error { // http://dev.mysql.com/doc/internals/en/connection-phase-packets.html#packet-Protocol::AuthSwitchResponse func (mc *mysqlConn) writeOldAuthPacket(cipher []byte) error { // User password - scrambleBuff := scrambleOldPassword(cipher, []byte(mc.cfg.passwd)) + scrambleBuff := scrambleOldPassword(cipher, []byte(mc.cfg.Passwd)) - // Calculate the packet lenght and add a tailing 0 + // Calculate the packet length and add a tailing 0 pktLen := len(scrambleBuff) + 1 data := mc.buf.takeSmallBuffer(4 + pktLen) if data == nil { @@ -322,6 +353,25 @@ func (mc *mysqlConn) writeOldAuthPacket(cipher []byte) error { return mc.writePacket(data) } +// Client clear text authentication packet +// http://dev.mysql.com/doc/internals/en/connection-phase-packets.html#packet-Protocol::AuthSwitchResponse +func (mc *mysqlConn) writeClearAuthPacket() error { + // Calculate the packet length and add a tailing 0 + pktLen := len(mc.cfg.Passwd) + 1 + data := mc.buf.takeSmallBuffer(4 + pktLen) + if data == nil { + // can not take the buffer. Something must be wrong with the connection + errLog.Print(ErrBusyBuffer) + return driver.ErrBadConn + } + + // Add the clear password [null terminated string] + copy(data[4:], mc.cfg.Passwd) + data[4+pktLen-1] = 0x00 + + return mc.writePacket(data) +} + /****************************************************************************** * Command Packets * ******************************************************************************/ @@ -405,8 +455,20 @@ func (mc *mysqlConn) readResultOK() error { return mc.handleOkPacket(data) case iEOF: - // someone is using old_passwords - return ErrOldPassword + if len(data) > 1 { + plugin := string(data[1:bytes.IndexByte(data, 0x00)]) + if plugin == "mysql_old_password" { + // using old_passwords + return ErrOldPassword + } else if plugin == "mysql_clear_password" { + // using clear text password + return ErrCleartextPassword + } else { + return ErrUnknownPlugin + } + } else { + return ErrOldPassword + } default: // Error otherwise return mc.handleErrorPacket(data) @@ -470,6 +532,10 @@ func (mc *mysqlConn) handleErrorPacket(data []byte) error { } } +func readStatus(b []byte) statusFlag { + return statusFlag(b[0]) | statusFlag(b[1])<<8 +} + // Ok Packet // http://dev.mysql.com/doc/internals/en/generic-response-packets.html#packet-OK_Packet func (mc *mysqlConn) handleOkPacket(data []byte) error { @@ -484,17 +550,21 @@ func (mc *mysqlConn) handleOkPacket(data []byte) error { mc.insertId, _, m = readLengthEncodedInteger(data[1+n:]) // server_status [2 bytes] + mc.status = readStatus(data[1+n+m : 1+n+m+2]) + if err := mc.discardResults(); err != nil { + return err + } // warning count [2 bytes] if !mc.strict { return nil - } else { - pos := 1 + n + m + 2 - if binary.LittleEndian.Uint16(data[pos:pos+2]) > 0 { - return mc.getWarnings() - } - return nil } + + pos := 1 + n + m + 2 + if binary.LittleEndian.Uint16(data[pos:pos+2]) > 0 { + return mc.getWarnings() + } + return nil } // Read Packets as Field Packets until EOF-Packet or an Error appears @@ -513,7 +583,7 @@ func (mc *mysqlConn) readColumns(count int) ([]mysqlField, error) { if i == count { return columns, nil } - return nil, fmt.Errorf("ColumnsCount mismatch n:%d len:%d", count, len(columns)) + return nil, fmt.Errorf("column count mismatch n:%d len:%d", count, len(columns)) } // Catalog @@ -530,11 +600,20 @@ func (mc *mysqlConn) readColumns(count int) ([]mysqlField, error) { pos += n // Table [len coded string] - n, err = skipLengthEncodedString(data[pos:]) - if err != nil { - return nil, err + if mc.cfg.ColumnsWithAlias { + tableName, _, n, err := readLengthEncodedString(data[pos:]) + if err != nil { + return nil, err + } + pos += n + columns[i].tableName = string(tableName) + } else { + n, err = skipLengthEncodedString(data[pos:]) + if err != nil { + return nil, err + } + pos += n } - pos += n // Original table [len coded string] n, err = skipLengthEncodedString(data[pos:]) @@ -593,8 +672,18 @@ func (rows *textRows) readRow(dest []driver.Value) error { // EOF Packet if data[0] == iEOF && len(data) == 5 { + // server_status [2 bytes] + rows.mc.status = readStatus(data[3:]) + if err := rows.mc.discardResults(); err != nil { + return err + } + rows.mc = nil return io.EOF } + if data[0] == iERR { + rows.mc = nil + return mc.handleErrorPacket(data) + } // RowSet Packet var n int @@ -615,7 +704,7 @@ func (rows *textRows) readRow(dest []driver.Value) error { fieldTypeDate, fieldTypeNewDate: dest[i], err = parseDateTime( string(dest[i].([]byte)), - mc.cfg.loc, + mc.cfg.Loc, ) if err == nil { continue @@ -645,6 +734,10 @@ func (mc *mysqlConn) readUntilEOF() error { if err == nil && data[0] != iEOF { continue } + if err == nil && data[0] == iEOF && len(data) == 5 { + mc.status = readStatus(data[3:]) + } + return err // Err or EOF } } @@ -677,13 +770,13 @@ func (stmt *mysqlStmt) readPrepareResultPacket() (uint16, error) { // Warning count [16 bit uint] if !stmt.mc.strict { return columnCount, nil - } else { - // Check for warnings count > 0, only available in MySQL > 4.1 - if len(data) >= 12 && binary.LittleEndian.Uint16(data[10:12]) > 0 { - return columnCount, stmt.mc.getWarnings() - } - return columnCount, nil } + + // Check for warnings count > 0, only available in MySQL > 4.1 + if len(data) >= 12 && binary.LittleEndian.Uint16(data[10:12]) > 0 { + return columnCount, stmt.mc.getWarnings() + } + return columnCount, nil } return 0, err } @@ -745,7 +838,7 @@ func (stmt *mysqlStmt) writeCommandLongData(paramID int, arg []byte) error { func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error { if len(args) != stmt.paramCount { return fmt.Errorf( - "Arguments count mismatch (Got: %d Has: %d)", + "argument count mismatch (got: %d; has: %d)", len(args), stmt.paramCount, ) @@ -922,7 +1015,7 @@ func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error { if v.IsZero() { val = []byte("0000-00-00") } else { - val = []byte(v.In(mc.cfg.loc).Format(timeFormat)) + val = []byte(v.In(mc.cfg.Loc).Format(timeFormat)) } paramValues = appendLengthEncodedInteger(paramValues, @@ -931,7 +1024,7 @@ func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error { paramValues = append(paramValues, val...) default: - return fmt.Errorf("Can't convert type: %T", arg) + return fmt.Errorf("can not convert type: %T", arg) } } @@ -949,6 +1042,28 @@ func (stmt *mysqlStmt) writeExecutePacket(args []driver.Value) error { return mc.writePacket(data) } +func (mc *mysqlConn) discardResults() error { + for mc.status&statusMoreResultsExists != 0 { + resLen, err := mc.readResultSetHeaderPacket() + if err != nil { + return err + } + if resLen > 0 { + // columns + if err := mc.readUntilEOF(); err != nil { + return err + } + // rows + if err := mc.readUntilEOF(); err != nil { + return err + } + } else { + mc.status &^= statusMoreResultsExists + } + } + return nil +} + // http://dev.mysql.com/doc/internals/en/binary-protocol-resultset-row.html func (rows *binaryRows) readRow(dest []driver.Value) error { data, err := rows.mc.readPacket() @@ -960,8 +1075,14 @@ func (rows *binaryRows) readRow(dest []driver.Value) error { if data[0] != iOK { // EOF Packet if data[0] == iEOF && len(data) == 5 { + rows.mc.status = readStatus(data[3:]) + if err := rows.mc.discardResults(); err != nil { + return err + } + rows.mc = nil return io.EOF } + rows.mc = nil // Error otherwise return rows.mc.handleErrorPacket(data) @@ -1041,7 +1162,7 @@ func (rows *binaryRows) readRow(dest []driver.Value) error { case fieldTypeDecimal, fieldTypeNewDecimal, fieldTypeVarChar, fieldTypeBit, fieldTypeEnum, fieldTypeSet, fieldTypeTinyBLOB, fieldTypeMediumBLOB, fieldTypeLongBLOB, fieldTypeBLOB, - fieldTypeVarString, fieldTypeString, fieldTypeGeometry: + fieldTypeVarString, fieldTypeString, fieldTypeGeometry, fieldTypeJSON: var isNull bool var n int dest[i], isNull, n, err = readLengthEncodedString(data[pos:]) @@ -1078,13 +1199,13 @@ func (rows *binaryRows) readRow(dest []driver.Value) error { dstlen = 8 + 1 + decimals default: return fmt.Errorf( - "MySQL protocol error, illegal decimals value %d", + "protocol error, illegal decimals value %d", rows.columns[i].decimals, ) } dest[i], err = formatBinaryDateTime(data[pos:pos+int(num)], dstlen, true) case rows.mc.parseTime: - dest[i], err = parseBinaryDateTime(num, data[pos:], rows.mc.cfg.loc) + dest[i], err = parseBinaryDateTime(num, data[pos:], rows.mc.cfg.Loc) default: var dstlen uint8 if rows.columns[i].fieldType == fieldTypeDate { @@ -1097,7 +1218,7 @@ func (rows *binaryRows) readRow(dest []driver.Value) error { dstlen = 19 + 1 + decimals default: return fmt.Errorf( - "MySQL protocol error, illegal decimals value %d", + "protocol error, illegal decimals value %d", rows.columns[i].decimals, ) } @@ -1114,7 +1235,7 @@ func (rows *binaryRows) readRow(dest []driver.Value) error { // Please report if this happens! default: - return fmt.Errorf("Unknown FieldType %d", rows.columns[i].fieldType) + return fmt.Errorf("unknown field type %d", rows.columns[i].fieldType) } } diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/rows.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/rows.go index 5ecaa94931b..c08255eeec8 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/rows.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/rows.go @@ -14,6 +14,7 @@ import ( ) type mysqlField struct { + tableName string name string flags fieldFlag fieldType byte @@ -37,8 +38,18 @@ type emptyRows struct{} func (rows *mysqlRows) Columns() []string { columns := make([]string, len(rows.columns)) - for i := range columns { - columns[i] = rows.columns[i].name + if rows.mc != nil && rows.mc.cfg.ColumnsWithAlias { + for i := range columns { + if tableName := rows.columns[i].tableName; len(tableName) > 0 { + columns[i] = tableName + "." + rows.columns[i].name + } else { + columns[i] = rows.columns[i].name + } + } + } else { + for i := range columns { + columns[i] = rows.columns[i].name + } } return columns } @@ -54,6 +65,12 @@ func (rows *mysqlRows) Close() error { // Remove unread packets from stream err := mc.readUntilEOF() + if err == nil { + if err = mc.discardResults(); err != nil { + return err + } + } + rows.mc = nil return err } @@ -65,10 +82,7 @@ func (rows *binaryRows) Next(dest []driver.Value) error { } // Fetch next row from stream - if err := rows.readRow(dest); err != io.EOF { - return err - } - rows.mc = nil + return rows.readRow(dest) } return io.EOF } @@ -80,10 +94,7 @@ func (rows *textRows) Next(dest []driver.Value) error { } // Fetch next row from stream - if err := rows.readRow(dest); err != io.EOF { - return err - } - rows.mc = nil + return rows.readRow(dest) } return io.EOF } diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/statement.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/statement.go index 142ef5416ae..ead9a6bf471 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/statement.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/statement.go @@ -10,6 +10,9 @@ package mysql import ( "database/sql/driver" + "fmt" + "reflect" + "strconv" ) type mysqlStmt struct { @@ -34,6 +37,10 @@ func (stmt *mysqlStmt) NumInput() int { return stmt.paramCount } +func (stmt *mysqlStmt) ColumnConverter(idx int) driver.ValueConverter { + return converter{} +} + func (stmt *mysqlStmt) Exec(args []driver.Value) (driver.Result, error) { if stmt.mc.netConn == nil { errLog.Print(ErrInvalidConn) @@ -94,9 +101,9 @@ func (stmt *mysqlStmt) Query(args []driver.Value) (driver.Rows, error) { } rows := new(binaryRows) - rows.mc = mc if resLen > 0 { + rows.mc = mc // Columns // If not cached, read them and cache them if stmt.columns == nil { @@ -110,3 +117,34 @@ func (stmt *mysqlStmt) Query(args []driver.Value) (driver.Rows, error) { return rows, err } + +type converter struct{} + +func (c converter) ConvertValue(v interface{}) (driver.Value, error) { + if driver.IsValue(v) { + return v, nil + } + + rv := reflect.ValueOf(v) + switch rv.Kind() { + case reflect.Ptr: + // indirect pointers + if rv.IsNil() { + return nil, nil + } + return c.ConvertValue(rv.Elem().Interface()) + case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64: + return rv.Int(), nil + case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32: + return int64(rv.Uint()), nil + case reflect.Uint64: + u64 := rv.Uint() + if u64 >= 1<<63 { + return strconv.FormatUint(u64, 10), nil + } + return int64(u64), nil + case reflect.Float32, reflect.Float64: + return rv.Float(), nil + } + return nil, fmt.Errorf("unsupported type %T, a %s", v, rv.Kind()) +} diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/utils.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/utils.go index 56f1b082ef0..d523b7ffdff 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/utils.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/utils.go @@ -13,26 +13,16 @@ import ( "crypto/tls" "database/sql/driver" "encoding/binary" - "errors" "fmt" "io" - "net/url" "strings" "time" ) var ( tlsConfigRegister map[string]*tls.Config // Register for custom tls.Configs - - errInvalidDSNUnescaped = errors.New("Invalid DSN: Did you forget to escape a param value?") - errInvalidDSNAddr = errors.New("Invalid DSN: Network Address not terminated (missing closing brace)") - errInvalidDSNNoSlash = errors.New("Invalid DSN: Missing the slash separating the database name") ) -func init() { - tlsConfigRegister = make(map[string]*tls.Config) -} - // RegisterTLSConfig registers a custom tls.Config to be used with sql.Open. // Use the key as a value in the DSN where tls=value. // @@ -58,7 +48,11 @@ func init() { // func RegisterTLSConfig(key string, config *tls.Config) error { if _, isBool := readBool(key); isBool || strings.ToLower(key) == "skip-verify" { - return fmt.Errorf("Key '%s' is reserved", key) + return fmt.Errorf("key '%s' is reserved", key) + } + + if tlsConfigRegister == nil { + tlsConfigRegister = make(map[string]*tls.Config) } tlsConfigRegister[key] = config @@ -67,202 +61,9 @@ func RegisterTLSConfig(key string, config *tls.Config) error { // DeregisterTLSConfig removes the tls.Config associated with key. func DeregisterTLSConfig(key string) { - delete(tlsConfigRegister, key) -} - -// parseDSN parses the DSN string to a config -func parseDSN(dsn string) (cfg *config, err error) { - // New config with some default values - cfg = &config{ - loc: time.UTC, - collation: defaultCollation, + if tlsConfigRegister != nil { + delete(tlsConfigRegister, key) } - - // TODO: use strings.IndexByte when we can depend on Go 1.2 - - // [user[:password]@][net[(addr)]]/dbname[?param1=value1¶mN=valueN] - // Find the last '/' (since the password or the net addr might contain a '/') - foundSlash := false - for i := len(dsn) - 1; i >= 0; i-- { - if dsn[i] == '/' { - foundSlash = true - var j, k int - - // left part is empty if i <= 0 - if i > 0 { - // [username[:password]@][protocol[(address)]] - // Find the last '@' in dsn[:i] - for j = i; j >= 0; j-- { - if dsn[j] == '@' { - // username[:password] - // Find the first ':' in dsn[:j] - for k = 0; k < j; k++ { - if dsn[k] == ':' { - cfg.passwd = dsn[k+1 : j] - break - } - } - cfg.user = dsn[:k] - - break - } - } - - // [protocol[(address)]] - // Find the first '(' in dsn[j+1:i] - for k = j + 1; k < i; k++ { - if dsn[k] == '(' { - // dsn[i-1] must be == ')' if an address is specified - if dsn[i-1] != ')' { - if strings.ContainsRune(dsn[k+1:i], ')') { - return nil, errInvalidDSNUnescaped - } - return nil, errInvalidDSNAddr - } - cfg.addr = dsn[k+1 : i-1] - break - } - } - cfg.net = dsn[j+1 : k] - } - - // dbname[?param1=value1&...¶mN=valueN] - // Find the first '?' in dsn[i+1:] - for j = i + 1; j < len(dsn); j++ { - if dsn[j] == '?' { - if err = parseDSNParams(cfg, dsn[j+1:]); err != nil { - return - } - break - } - } - cfg.dbname = dsn[i+1 : j] - - break - } - } - - if !foundSlash && len(dsn) > 0 { - return nil, errInvalidDSNNoSlash - } - - // Set default network if empty - if cfg.net == "" { - cfg.net = "tcp" - } - - // Set default address if empty - if cfg.addr == "" { - switch cfg.net { - case "tcp": - cfg.addr = "127.0.0.1:3306" - case "unix": - cfg.addr = "/tmp/mysql.sock" - default: - return nil, errors.New("Default addr for network '" + cfg.net + "' unknown") - } - - } - - return -} - -// parseDSNParams parses the DSN "query string" -// Values must be url.QueryEscape'ed -func parseDSNParams(cfg *config, params string) (err error) { - for _, v := range strings.Split(params, "&") { - param := strings.SplitN(v, "=", 2) - if len(param) != 2 { - continue - } - - // cfg params - switch value := param[1]; param[0] { - - // Disable INFILE whitelist / enable all files - case "allowAllFiles": - var isBool bool - cfg.allowAllFiles, isBool = readBool(value) - if !isBool { - return fmt.Errorf("Invalid Bool value: %s", value) - } - - // Use old authentication mode (pre MySQL 4.1) - case "allowOldPasswords": - var isBool bool - cfg.allowOldPasswords, isBool = readBool(value) - if !isBool { - return fmt.Errorf("Invalid Bool value: %s", value) - } - - // Switch "rowsAffected" mode - case "clientFoundRows": - var isBool bool - cfg.clientFoundRows, isBool = readBool(value) - if !isBool { - return fmt.Errorf("Invalid Bool value: %s", value) - } - - // Collation - case "collation": - collation, ok := collations[value] - if !ok { - // Note possibility for false negatives: - // could be triggered although the collation is valid if the - // collations map does not contain entries the server supports. - err = errors.New("unknown collation") - return - } - cfg.collation = collation - break - - // Time Location - case "loc": - if value, err = url.QueryUnescape(value); err != nil { - return - } - cfg.loc, err = time.LoadLocation(value) - if err != nil { - return - } - - // Dial Timeout - case "timeout": - cfg.timeout, err = time.ParseDuration(value) - if err != nil { - return - } - - // TLS-Encryption - case "tls": - boolValue, isBool := readBool(value) - if isBool { - if boolValue { - cfg.tls = &tls.Config{} - } - } else { - if strings.ToLower(value) == "skip-verify" { - cfg.tls = &tls.Config{InsecureSkipVerify: true} - } else if tlsConfig, ok := tlsConfigRegister[value]; ok { - cfg.tls = tlsConfig - } else { - return fmt.Errorf("Invalid value / unknown config name: %s", value) - } - } - - default: - // lazy init - if cfg.params == nil { - cfg.params = make(map[string]string) - } - - if cfg.params[param[0]], err = url.QueryUnescape(value); err != nil { - return - } - } - } - - return } // Returns the bool value of the input. @@ -459,7 +260,7 @@ func parseDateTime(str string, loc *time.Location) (t time.Time, err error) { } t, err = time.Parse(timeFormat[:len(str)], str) default: - err = fmt.Errorf("Invalid Time-String: %s", str) + err = fmt.Errorf("invalid time string: %s", str) return } @@ -508,7 +309,7 @@ func parseBinaryDateTime(num uint64, data []byte, loc *time.Location) (driver.Va loc, ), nil } - return nil, fmt.Errorf("Invalid DATETIME-packet length %d", num) + return nil, fmt.Errorf("invalid DATETIME packet length %d", num) } // zeroDateTime is used in formatBinaryDateTime to avoid an allocation @@ -517,11 +318,12 @@ func parseBinaryDateTime(num uint64, data []byte, loc *time.Location) (driver.Va // The current behavior depends on database/sql copying the result. var zeroDateTime = []byte("0000-00-00 00:00:00.000000") +const digits01 = "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" +const digits10 = "0000000000111111111122222222223333333333444444444455555555556666666666777777777788888888889999999999" + func formatBinaryDateTime(src []byte, length uint8, justTime bool) (driver.Value, error) { // length expects the deterministic length of the zero value, // negative time and 100+ hours are automatically added if needed - const digits01 = "0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789" - const digits10 = "0000000000111111111122222222223333333333444444444455555555556666666666777777777788888888889999999999" if len(src) == 0 { if justTime { return zeroDateTime[11 : 11+length], nil @@ -542,7 +344,7 @@ func formatBinaryDateTime(src []byte, length uint8, justTime bool) (driver.Value switch len(src) { case 8, 12: default: - return nil, fmt.Errorf("Invalid TIME-packet length %d", len(src)) + return nil, fmt.Errorf("invalid TIME packet length %d", len(src)) } // +2 to enable negative time and 100+ hours dst = make([]byte, 0, length+2) @@ -576,7 +378,7 @@ func formatBinaryDateTime(src []byte, length uint8, justTime bool) (driver.Value if length > 10 { t += "TIME" } - return nil, fmt.Errorf("illegal %s-packet length %d", t, len(src)) + return nil, fmt.Errorf("illegal %s packet length %d", t, len(src)) } dst = make([]byte, 0, length) // start with the date @@ -742,6 +544,10 @@ func skipLengthEncodedString(b []byte) (int, error) { // returns the number read, whether the value is NULL and the number of bytes read func readLengthEncodedInteger(b []byte) (uint64, bool, int) { + // See issue #349 + if len(b) == 0 { + return 0, true, 1 + } switch b[0] { // 251: NULL @@ -783,3 +589,152 @@ func appendLengthEncodedInteger(b []byte, n uint64) []byte { return append(b, 0xfe, byte(n), byte(n>>8), byte(n>>16), byte(n>>24), byte(n>>32), byte(n>>40), byte(n>>48), byte(n>>56)) } + +// reserveBuffer checks cap(buf) and expand buffer to len(buf) + appendSize. +// If cap(buf) is not enough, reallocate new buffer. +func reserveBuffer(buf []byte, appendSize int) []byte { + newSize := len(buf) + appendSize + if cap(buf) < newSize { + // Grow buffer exponentially + newBuf := make([]byte, len(buf)*2+appendSize) + copy(newBuf, buf) + buf = newBuf + } + return buf[:newSize] +} + +// escapeBytesBackslash escapes []byte with backslashes (\) +// This escapes the contents of a string (provided as []byte) by adding backslashes before special +// characters, and turning others into specific escape sequences, such as +// turning newlines into \n and null bytes into \0. +// https://github.com/mysql/mysql-server/blob/mysql-5.7.5/mysys/charset.c#L823-L932 +func escapeBytesBackslash(buf, v []byte) []byte { + pos := len(buf) + buf = reserveBuffer(buf, len(v)*2) + + for _, c := range v { + switch c { + case '\x00': + buf[pos] = '\\' + buf[pos+1] = '0' + pos += 2 + case '\n': + buf[pos] = '\\' + buf[pos+1] = 'n' + pos += 2 + case '\r': + buf[pos] = '\\' + buf[pos+1] = 'r' + pos += 2 + case '\x1a': + buf[pos] = '\\' + buf[pos+1] = 'Z' + pos += 2 + case '\'': + buf[pos] = '\\' + buf[pos+1] = '\'' + pos += 2 + case '"': + buf[pos] = '\\' + buf[pos+1] = '"' + pos += 2 + case '\\': + buf[pos] = '\\' + buf[pos+1] = '\\' + pos += 2 + default: + buf[pos] = c + pos++ + } + } + + return buf[:pos] +} + +// escapeStringBackslash is similar to escapeBytesBackslash but for string. +func escapeStringBackslash(buf []byte, v string) []byte { + pos := len(buf) + buf = reserveBuffer(buf, len(v)*2) + + for i := 0; i < len(v); i++ { + c := v[i] + switch c { + case '\x00': + buf[pos] = '\\' + buf[pos+1] = '0' + pos += 2 + case '\n': + buf[pos] = '\\' + buf[pos+1] = 'n' + pos += 2 + case '\r': + buf[pos] = '\\' + buf[pos+1] = 'r' + pos += 2 + case '\x1a': + buf[pos] = '\\' + buf[pos+1] = 'Z' + pos += 2 + case '\'': + buf[pos] = '\\' + buf[pos+1] = '\'' + pos += 2 + case '"': + buf[pos] = '\\' + buf[pos+1] = '"' + pos += 2 + case '\\': + buf[pos] = '\\' + buf[pos+1] = '\\' + pos += 2 + default: + buf[pos] = c + pos++ + } + } + + return buf[:pos] +} + +// escapeBytesQuotes escapes apostrophes in []byte by doubling them up. +// This escapes the contents of a string by doubling up any apostrophes that +// it contains. This is used when the NO_BACKSLASH_ESCAPES SQL_MODE is in +// effect on the server. +// https://github.com/mysql/mysql-server/blob/mysql-5.7.5/mysys/charset.c#L963-L1038 +func escapeBytesQuotes(buf, v []byte) []byte { + pos := len(buf) + buf = reserveBuffer(buf, len(v)*2) + + for _, c := range v { + if c == '\'' { + buf[pos] = '\'' + buf[pos+1] = '\'' + pos += 2 + } else { + buf[pos] = c + pos++ + } + } + + return buf[:pos] +} + +// escapeStringQuotes is similar to escapeBytesQuotes but for string. +func escapeStringQuotes(buf []byte, v string) []byte { + pos := len(buf) + buf = reserveBuffer(buf, len(v)*2) + + for i := 0; i < len(v); i++ { + c := v[i] + if c == '\'' { + buf[pos] = '\'' + buf[pos+1] = '\'' + pos += 2 + } else { + buf[pos] = c + pos++ + } + } + + return buf[:pos] +} diff --git a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/utils_test.go b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/utils_test.go index 6e50b09b9cb..0d6c6684f28 100644 --- a/Godeps/_workspace/src/github.com/go-sql-driver/mysql/utils_test.go +++ b/Godeps/_workspace/src/github.com/go-sql-driver/mysql/utils_test.go @@ -16,76 +16,6 @@ import ( "time" ) -var testDSNs = []struct { - in string - out string - loc *time.Location -}{ - {"username:password@protocol(address)/dbname?param=value", "&{user:username passwd:password net:protocol addr:address dbname:dbname params:map[param:value] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.UTC}, - {"user@unix(/path/to/socket)/dbname?charset=utf8", "&{user:user passwd: net:unix addr:/path/to/socket dbname:dbname params:map[charset:utf8] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.UTC}, - {"user:password@tcp(localhost:5555)/dbname?charset=utf8&tls=true", "&{user:user passwd:password net:tcp addr:localhost:5555 dbname:dbname params:map[charset:utf8] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.UTC}, - {"user:password@tcp(localhost:5555)/dbname?charset=utf8mb4,utf8&tls=skip-verify", "&{user:user passwd:password net:tcp addr:localhost:5555 dbname:dbname params:map[charset:utf8mb4,utf8] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.UTC}, - {"user:password@/dbname?loc=UTC&timeout=30s&allowAllFiles=1&clientFoundRows=true&allowOldPasswords=TRUE&collation=utf8mb4_unicode_ci", "&{user:user passwd:password net:tcp addr:127.0.0.1:3306 dbname:dbname params:map[] loc:%p tls: timeout:30000000000 collation:224 allowAllFiles:true allowOldPasswords:true clientFoundRows:true}", time.UTC}, - {"user:p@ss(word)@tcp([de:ad:be:ef::ca:fe]:80)/dbname?loc=Local", "&{user:user passwd:p@ss(word) net:tcp addr:[de:ad:be:ef::ca:fe]:80 dbname:dbname params:map[] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.Local}, - {"/dbname", "&{user: passwd: net:tcp addr:127.0.0.1:3306 dbname:dbname params:map[] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.UTC}, - {"@/", "&{user: passwd: net:tcp addr:127.0.0.1:3306 dbname: params:map[] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.UTC}, - {"/", "&{user: passwd: net:tcp addr:127.0.0.1:3306 dbname: params:map[] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.UTC}, - {"", "&{user: passwd: net:tcp addr:127.0.0.1:3306 dbname: params:map[] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.UTC}, - {"user:p@/ssword@/", "&{user:user passwd:p@/ssword net:tcp addr:127.0.0.1:3306 dbname: params:map[] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.UTC}, - {"unix/?arg=%2Fsome%2Fpath.ext", "&{user: passwd: net:unix addr:/tmp/mysql.sock dbname: params:map[arg:/some/path.ext] loc:%p tls: timeout:0 collation:33 allowAllFiles:false allowOldPasswords:false clientFoundRows:false}", time.UTC}, -} - -func TestDSNParser(t *testing.T) { - var cfg *config - var err error - var res string - - for i, tst := range testDSNs { - cfg, err = parseDSN(tst.in) - if err != nil { - t.Error(err.Error()) - } - - // pointer not static - cfg.tls = nil - - res = fmt.Sprintf("%+v", cfg) - if res != fmt.Sprintf(tst.out, tst.loc) { - t.Errorf("%d. parseDSN(%q) => %q, want %q", i, tst.in, res, fmt.Sprintf(tst.out, tst.loc)) - } - } -} - -func TestDSNParserInvalid(t *testing.T) { - var invalidDSNs = []string{ - "@net(addr/", // no closing brace - "@tcp(/", // no closing brace - "tcp(/", // no closing brace - "(/", // no closing brace - "net(addr)//", // unescaped - "user:pass@tcp(1.2.3.4:3306)", // no trailing slash - //"/dbname?arg=/some/unescaped/path", - } - - for i, tst := range invalidDSNs { - if _, err := parseDSN(tst); err == nil { - t.Errorf("invalid DSN #%d. (%s) didn't error!", i, tst) - } - } -} - -func BenchmarkParseDSN(b *testing.B) { - b.ReportAllocs() - - for i := 0; i < b.N; i++ { - for _, tst := range testDSNs { - if _, err := parseDSN(tst.in); err != nil { - b.Error(err.Error()) - } - } - } -} - func TestScanNullTime(t *testing.T) { var scanTests = []struct { in interface{} @@ -210,3 +140,58 @@ func TestFormatBinaryDateTime(t *testing.T) { expect("1978-12-30 15:46:23", 7, 19) expect("1978-12-30 15:46:23.987654", 11, 26) } + +func TestEscapeBackslash(t *testing.T) { + expect := func(expected, value string) { + actual := string(escapeBytesBackslash([]byte{}, []byte(value))) + if actual != expected { + t.Errorf( + "expected %s, got %s", + expected, actual, + ) + } + + actual = string(escapeStringBackslash([]byte{}, value)) + if actual != expected { + t.Errorf( + "expected %s, got %s", + expected, actual, + ) + } + } + + expect("foo\\0bar", "foo\x00bar") + expect("foo\\nbar", "foo\nbar") + expect("foo\\rbar", "foo\rbar") + expect("foo\\Zbar", "foo\x1abar") + expect("foo\\\"bar", "foo\"bar") + expect("foo\\\\bar", "foo\\bar") + expect("foo\\'bar", "foo'bar") +} + +func TestEscapeQuotes(t *testing.T) { + expect := func(expected, value string) { + actual := string(escapeBytesQuotes([]byte{}, []byte(value))) + if actual != expected { + t.Errorf( + "expected %s, got %s", + expected, actual, + ) + } + + actual = string(escapeStringQuotes([]byte{}, value)) + if actual != expected { + t.Errorf( + "expected %s, got %s", + expected, actual, + ) + } + } + + expect("foo\x00bar", "foo\x00bar") // not affected + expect("foo\nbar", "foo\nbar") // not affected + expect("foo\rbar", "foo\rbar") // not affected + expect("foo\x1abar", "foo\x1abar") // not affected + expect("foo''bar", "foo'bar") // affected + expect("foo\"bar", "foo\"bar") // not affected +} diff --git a/Godeps/_workspace/src/github.com/go-xorm/core/LICENSE b/Godeps/_workspace/src/github.com/go-xorm/core/LICENSE new file mode 100644 index 00000000000..1130797806c --- /dev/null +++ b/Godeps/_workspace/src/github.com/go-xorm/core/LICENSE @@ -0,0 +1,27 @@ +Copyright (c) 2013 - 2015 Lunny Xiao +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +* Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +* Neither the name of the {organization} nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/Godeps/_workspace/src/github.com/go-xorm/core/cache.go b/Godeps/_workspace/src/github.com/go-xorm/core/cache.go index 2a61ef7521d..bf81bd52ba4 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/core/cache.go +++ b/Godeps/_workspace/src/github.com/go-xorm/core/cache.go @@ -1,10 +1,11 @@ package core import ( - "encoding/json" "errors" "fmt" "time" + "bytes" + "encoding/gob" ) const ( @@ -47,16 +48,20 @@ type Cacher interface { } func encodeIds(ids []PK) (string, error) { - b, err := json.Marshal(ids) - if err != nil { - return "", err - } - return string(b), nil + buf := new(bytes.Buffer) + enc := gob.NewEncoder(buf) + err := enc.Encode(ids) + + return buf.String(), err } + func decodeIds(s string) ([]PK, error) { pks := make([]PK, 0) - err := json.Unmarshal([]byte(s), &pks) + + dec := gob.NewDecoder(bytes.NewBufferString(s)) + err := dec.Decode(&pks) + return pks, err } diff --git a/Godeps/_workspace/src/github.com/go-xorm/core/column.go b/Godeps/_workspace/src/github.com/go-xorm/core/column.go index 52468aa20e6..0e0def08c2f 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/core/column.go +++ b/Godeps/_workspace/src/github.com/go-xorm/core/column.go @@ -1,10 +1,10 @@ package core import ( - "errors" "fmt" "reflect" "strings" + "time" ) const ( @@ -35,6 +35,8 @@ type Column struct { DefaultIsEmpty bool EnumOptions map[string]int SetOptions map[string]int + DisableTimeZone bool + TimeZone *time.Location // column specified time zone } func NewColumn(name, fieldName string, sqlType SQLType, len1, len2 int, nullable bool) *Column { @@ -122,50 +124,34 @@ func (col *Column) ValueOfV(dataStruct *reflect.Value) (*reflect.Value, error) { } if dataStruct.Type().Kind() == reflect.Map { - var keyValue reflect.Value - - if len(col.fieldPath) == 1 { - keyValue = reflect.ValueOf(col.FieldName) - } else if len(col.fieldPath) == 2 { - keyValue = reflect.ValueOf(col.fieldPath[1]) - } else { - return nil, fmt.Errorf("Unsupported mutliderive %v", col.FieldName) - } - + keyValue := reflect.ValueOf(col.fieldPath[len(col.fieldPath)-1]) fieldValue = dataStruct.MapIndex(keyValue) return &fieldValue, nil + } else if dataStruct.Type().Kind() == reflect.Interface { + structValue := reflect.ValueOf(dataStruct.Interface()) + dataStruct = &structValue } - if len(col.fieldPath) == 1 { - fieldValue = dataStruct.FieldByName(col.FieldName) - } else if len(col.fieldPath) == 2 { - parentField := dataStruct.FieldByName(col.fieldPath[0]) - if parentField.IsValid() { - if parentField.Kind() == reflect.Struct { - fieldValue = parentField.FieldByName(col.fieldPath[1]) - } else if parentField.Kind() == reflect.Ptr { - if parentField.IsNil() { - parentField.Set(reflect.New(parentField.Type().Elem())) - fieldValue = parentField.Elem().FieldByName(col.fieldPath[1]) - } else { - parentField = parentField.Elem() - if parentField.IsValid() { - fieldValue = parentField.FieldByName(col.fieldPath[1]) - } else { - return nil, fmt.Errorf("field %v is not valid", col.FieldName) - } - } - } - } else { - // so we can use a different struct as conditions - fieldValue = dataStruct.FieldByName(col.fieldPath[1]) + level := len(col.fieldPath) + fieldValue = dataStruct.FieldByName(col.fieldPath[0]) + for i := 0; i < level-1; i++ { + if !fieldValue.IsValid() { + break + } + if fieldValue.Kind() == reflect.Struct { + fieldValue = fieldValue.FieldByName(col.fieldPath[i+1]) + } else if fieldValue.Kind() == reflect.Ptr { + if fieldValue.IsNil() { + fieldValue.Set(reflect.New(fieldValue.Type().Elem())) + } + fieldValue = fieldValue.Elem().FieldByName(col.fieldPath[i+1]) + } else { + return nil, fmt.Errorf("field %v is not valid", col.FieldName) } - } else { - return nil, fmt.Errorf("Unsupported mutliderive %v", col.FieldName) } if !fieldValue.IsValid() { - return nil, errors.New("no find field matched") + return nil, fmt.Errorf("field %v is not valid", col.FieldName) } return &fieldValue, nil diff --git a/Godeps/_workspace/src/github.com/go-xorm/core/db.go b/Godeps/_workspace/src/github.com/go-xorm/core/db.go index 1f70a926116..169d855315e 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/core/db.go +++ b/Godeps/_workspace/src/github.com/go-xorm/core/db.go @@ -2,6 +2,7 @@ package core import ( "database/sql" + "database/sql/driver" "errors" "reflect" "regexp" @@ -29,10 +30,24 @@ func StructToSlice(query string, st interface{}) (string, []interface{}, error) } args := make([]interface{}, 0) + var err error query = re.ReplaceAllStringFunc(query, func(src string) string { - args = append(args, vv.Elem().FieldByName(src[1:]).Interface()) + fv := vv.Elem().FieldByName(src[1:]).Interface() + if v, ok := fv.(driver.Valuer); ok { + var value driver.Value + value, err = v.Value() + if err != nil { + return "?" + } + args = append(args, value) + } else { + args = append(args, fv) + } return "?" }) + if err != nil { + return "", []interface{}{}, err + } return query, args, nil } @@ -43,12 +58,25 @@ type DB struct { func Open(driverName, dataSourceName string) (*DB, error) { db, err := sql.Open(driverName, dataSourceName) - return &DB{db, NewCacheMapper(&SnakeMapper{})}, err + if err != nil { + return nil, err + } + return &DB{db, NewCacheMapper(&SnakeMapper{})}, nil +} + +func FromDB(db *sql.DB) *DB { + return &DB{db, NewCacheMapper(&SnakeMapper{})} } func (db *DB) Query(query string, args ...interface{}) (*Rows, error) { rows, err := db.DB.Query(query, args...) - return &Rows{rows, db.Mapper}, err + if err != nil { + if rows != nil { + rows.Close() + } + return nil, err + } + return &Rows{rows, db.Mapper}, nil } func (db *DB) QueryMap(query string, mp interface{}) (*Rows, error) { @@ -68,28 +96,87 @@ func (db *DB) QueryStruct(query string, st interface{}) (*Rows, error) { } type Row struct { - *sql.Row + rows *Rows // One of these two will be non-nil: - err error // deferred error for easy chaining - Mapper IMapper + err error // deferred error for easy chaining +} + +func (row *Row) Columns() ([]string, error) { + if row.err != nil { + return nil, row.err + } + return row.rows.Columns() } func (row *Row) Scan(dest ...interface{}) error { if row.err != nil { return row.err } - return row.Row.Scan(dest...) + defer row.rows.Close() + + for _, dp := range dest { + if _, ok := dp.(*sql.RawBytes); ok { + return errors.New("sql: RawBytes isn't allowed on Row.Scan") + } + } + + if !row.rows.Next() { + if err := row.rows.Err(); err != nil { + return err + } + return sql.ErrNoRows + } + err := row.rows.Scan(dest...) + if err != nil { + return err + } + // Make sure the query can be processed to completion with no errors. + if err := row.rows.Close(); err != nil { + return err + } + + return nil +} + +func (row *Row) ScanStructByName(dest interface{}) error { + if row.err != nil { + return row.err + } + return row.rows.ScanStructByName(dest) +} + +func (row *Row) ScanStructByIndex(dest interface{}) error { + if row.err != nil { + return row.err + } + return row.rows.ScanStructByIndex(dest) +} + +// scan data to a slice's pointer, slice's length should equal to columns' number +func (row *Row) ScanSlice(dest interface{}) error { + if row.err != nil { + return row.err + } + return row.rows.ScanSlice(dest) +} + +// scan data to a map's pointer +func (row *Row) ScanMap(dest interface{}) error { + if row.err != nil { + return row.err + } + return row.rows.ScanMap(dest) } func (db *DB) QueryRow(query string, args ...interface{}) *Row { - row := db.DB.QueryRow(query, args...) - return &Row{row, nil, db.Mapper} + rows, err := db.Query(query, args...) + return &Row{rows, err} } func (db *DB) QueryRowMap(query string, mp interface{}) *Row { query, args, err := MapToSlice(query, mp) if err != nil { - return &Row{nil, err, db.Mapper} + return &Row{nil, err} } return db.QueryRow(query, args...) } @@ -97,7 +184,7 @@ func (db *DB) QueryRowMap(query string, mp interface{}) *Row { func (db *DB) QueryRowStruct(query string, st interface{}) *Row { query, args, err := StructToSlice(query, st) if err != nil { - return &Row{nil, err, db.Mapper} + return &Row{nil, err} } return db.QueryRow(query, args...) } @@ -187,14 +274,14 @@ func (s *Stmt) QueryStruct(st interface{}) (*Rows, error) { } func (s *Stmt) QueryRow(args ...interface{}) *Row { - row := s.Stmt.QueryRow(args...) - return &Row{row, nil, s.Mapper} + rows, err := s.Query(args...) + return &Row{rows, err} } func (s *Stmt) QueryRowMap(mp interface{}) *Row { vv := reflect.ValueOf(mp) if vv.Kind() != reflect.Ptr || vv.Elem().Kind() != reflect.Map { - return &Row{nil, errors.New("mp should be a map's pointer"), s.Mapper} + return &Row{nil, errors.New("mp should be a map's pointer")} } args := make([]interface{}, len(s.names)) @@ -208,7 +295,7 @@ func (s *Stmt) QueryRowMap(mp interface{}) *Row { func (s *Stmt) QueryRowStruct(st interface{}) *Row { vv := reflect.ValueOf(st) if vv.Kind() != reflect.Ptr || vv.Elem().Kind() != reflect.Struct { - return &Row{nil, errors.New("st should be a struct's pointer"), s.Mapper} + return &Row{nil, errors.New("st should be a struct's pointer")} } args := make([]interface{}, len(s.names)) @@ -540,14 +627,14 @@ func (tx *Tx) QueryStruct(query string, st interface{}) (*Rows, error) { } func (tx *Tx) QueryRow(query string, args ...interface{}) *Row { - row := tx.Tx.QueryRow(query, args...) - return &Row{row, nil, tx.Mapper} + rows, err := tx.Query(query, args...) + return &Row{rows, err} } func (tx *Tx) QueryRowMap(query string, mp interface{}) *Row { query, args, err := MapToSlice(query, mp) if err != nil { - return &Row{nil, err, tx.Mapper} + return &Row{nil, err} } return tx.QueryRow(query, args...) } @@ -555,7 +642,7 @@ func (tx *Tx) QueryRowMap(query string, mp interface{}) *Row { func (tx *Tx) QueryRowStruct(query string, st interface{}) *Row { query, args, err := StructToSlice(query, st) if err != nil { - return &Row{nil, err, tx.Mapper} + return &Row{nil, err} } return tx.QueryRow(query, args...) } diff --git a/Godeps/_workspace/src/github.com/go-xorm/core/db_test.go b/Godeps/_workspace/src/github.com/go-xorm/core/db_test.go index 65bf241446d..94c4ea4f9f6 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/core/db_test.go +++ b/Godeps/_workspace/src/github.com/go-xorm/core/db_test.go @@ -24,7 +24,7 @@ type User struct { Age float32 Alias string NickName string - Created time.Time + Created NullTime } func init() { @@ -85,7 +85,7 @@ func BenchmarkOriQuery(b *testing.B) { var Id int64 var Name, Title, Alias, NickName string var Age float32 - var Created time.Time + var Created NullTime err = rows.Scan(&Id, &Name, &Title, &Age, &Alias, &NickName, &Created) if err != nil { b.Error(err) @@ -600,7 +600,7 @@ func TestExecStruct(t *testing.T) { Age: 1.2, Alias: "lunny", NickName: "lunny xiao", - Created: time.Now(), + Created: NullTime(time.Now()), } _, err = db.ExecStruct("insert into user (`name`, title, age, alias, nick_name,created) "+ @@ -645,7 +645,7 @@ func BenchmarkExecStruct(b *testing.B) { Age: 1.2, Alias: "lunny", NickName: "lunny xiao", - Created: time.Now(), + Created: NullTime(time.Now()), } for i := 0; i < b.N; i++ { diff --git a/Godeps/_workspace/src/github.com/go-xorm/core/dialect.go b/Godeps/_workspace/src/github.com/go-xorm/core/dialect.go index 43a22670913..45bc5c20883 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/core/dialect.go +++ b/Godeps/_workspace/src/github.com/go-xorm/core/dialect.go @@ -54,7 +54,7 @@ type Dialect interface { IndexCheckSql(tableName, idxName string) (string, []interface{}) TableCheckSql(tableName string) (string, []interface{}) - IsColumnExist(tableName string, col *Column) (bool, error) + IsColumnExist(tableName string, colName string) (bool, error) CreateTableSql(table *Table, tableName, storeEngine, charset string) string DropTableSql(tableName string) string @@ -63,6 +63,8 @@ type Dialect interface { ModifyColumnSql(tableName string, col *Column) string + ForUpdateSql(query string) string + //CreateTableIfNotExists(table *Table, tableName, storeEngine, charset string) error //MustDropTable(tableName string) error @@ -164,10 +166,10 @@ func (db *Base) HasRecords(query string, args ...interface{}) (bool, error) { return false, nil } -func (db *Base) IsColumnExist(tableName string, col *Column) (bool, error) { +func (db *Base) IsColumnExist(tableName, colName string) (bool, error) { query := "SELECT `COLUMN_NAME` FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_SCHEMA` = ? AND `TABLE_NAME` = ? AND `COLUMN_NAME` = ?" query = strings.Replace(query, "`", db.dialect.QuoteStr(), -1) - return db.HasRecords(query, db.DbName, tableName, col.Name) + return db.HasRecords(query, db.DbName, tableName, colName) } /* @@ -229,28 +231,33 @@ func (b *Base) CreateTableSql(table *Table, tableName, storeEngine, charset stri tableName = table.Name } - sql += b.dialect.Quote(tableName) + " (" + sql += b.dialect.Quote(tableName) + sql += " (" - pkList := table.PrimaryKeys + if len(table.ColumnsSeq()) > 0 { + pkList := table.PrimaryKeys - for _, colName := range table.ColumnsSeq() { - col := table.GetColumn(colName) - if col.IsPrimaryKey && len(pkList) == 1 { - sql += col.String(b.dialect) - } else { - sql += col.StringNoPk(b.dialect) + for _, colName := range table.ColumnsSeq() { + col := table.GetColumn(colName) + if col.IsPrimaryKey && len(pkList) == 1 { + sql += col.String(b.dialect) + } else { + sql += col.StringNoPk(b.dialect) + } + sql = strings.TrimSpace(sql) + sql += ", " } - sql = strings.TrimSpace(sql) - sql += ", " - } - if len(pkList) > 1 { - sql += "PRIMARY KEY ( " - sql += b.dialect.Quote(strings.Join(pkList, b.dialect.Quote(","))) - sql += " ), " - } + if len(pkList) > 1 { + sql += "PRIMARY KEY ( " + sql += b.dialect.Quote(strings.Join(pkList, b.dialect.Quote(","))) + sql += " ), " + } + + sql = sql[:len(sql)-2] + } + sql += ")" - sql = sql[:len(sql)-2] + ")" if b.dialect.SupportEngine() && storeEngine != "" { sql += " ENGINE=" + storeEngine } @@ -262,21 +269,25 @@ func (b *Base) CreateTableSql(table *Table, tableName, storeEngine, charset stri sql += " DEFAULT CHARSET " + charset } } - sql += ";" + return sql } +func (b *Base) ForUpdateSql(query string) string { + return query + " FOR UPDATE" +} + var ( - dialects = map[DbType]Dialect{} + dialects = map[DbType]func() Dialect{} ) -func RegisterDialect(dbName DbType, dialect Dialect) { - if dialect == nil { +func RegisterDialect(dbName DbType, dialectFunc func() Dialect) { + if dialectFunc == nil { panic("core: Register dialect is nil") } - dialects[dbName] = dialect // !nashtsai! allow override dialect + dialects[dbName] = dialectFunc // !nashtsai! allow override dialect } func QueryDialect(dbName DbType) Dialect { - return dialects[dbName] + return dialects[dbName]() } diff --git a/Godeps/_workspace/src/github.com/go-xorm/core/error.go b/Godeps/_workspace/src/github.com/go-xorm/core/error.go index 414ba037d1e..13c179251d9 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/core/error.go +++ b/Godeps/_workspace/src/github.com/go-xorm/core/error.go @@ -4,7 +4,7 @@ import "errors" var ( ErrNoMapPointer = errors.New("mp should be a map's pointer") - ErrNoStructPointer = errors.New("mp should be a map's pointer") + ErrNoStructPointer = errors.New("mp should be a struct's pointer") //ErrNotExist = errors.New("Not exist") //ErrIgnore = errors.New("Ignore") ) diff --git a/Godeps/_workspace/src/github.com/go-xorm/core/scan.go b/Godeps/_workspace/src/github.com/go-xorm/core/scan.go new file mode 100644 index 00000000000..7da338d8645 --- /dev/null +++ b/Godeps/_workspace/src/github.com/go-xorm/core/scan.go @@ -0,0 +1,52 @@ +package core + +import ( + "database/sql/driver" + "fmt" + "time" +) + +type NullTime time.Time + +var ( + _ driver.Valuer = NullTime{} +) + +func (ns *NullTime) Scan(value interface{}) error { + if value == nil { + return nil + } + return convertTime(ns, value) +} + +// Value implements the driver Valuer interface. +func (ns NullTime) Value() (driver.Value, error) { + if (time.Time)(ns).IsZero() { + return nil, nil + } + return (time.Time)(ns).Format("2006-01-02 15:04:05"), nil +} + +func convertTime(dest *NullTime, src interface{}) error { + // Common cases, without reflect. + switch s := src.(type) { + case string: + t, err := time.Parse("2006-01-02 15:04:05", s) + if err != nil { + return err + } + *dest = NullTime(t) + return nil + case []uint8: + t, err := time.Parse("2006-01-02 15:04:05", string(s)) + if err != nil { + return err + } + *dest = NullTime(t) + return nil + case nil: + default: + return fmt.Errorf("unsupported driver -> Scan pair: %T -> %T", src, dest) + } + return nil +} diff --git a/Godeps/_workspace/src/github.com/go-xorm/core/type.go b/Godeps/_workspace/src/github.com/go-xorm/core/type.go index 73b9921ee63..c7c4d7bd08f 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/core/type.go +++ b/Godeps/_workspace/src/github.com/go-xorm/core/type.go @@ -53,6 +53,10 @@ func (s *SQLType) IsNumeric() bool { return s.IsType(NUMERIC_TYPE) } +func (s *SQLType) IsJson() bool { + return s.Name == Json +} + var ( Bit = "BIT" TinyInt = "TINYINT" @@ -101,6 +105,8 @@ var ( Serial = "SERIAL" BigSerial = "BIGSERIAL" + Json = "JSON" + SqlTypes = map[string]int{ Bit: NUMERIC_TYPE, TinyInt: NUMERIC_TYPE, @@ -112,6 +118,7 @@ var ( Enum: TEXT_TYPE, Set: TEXT_TYPE, + Json: TEXT_TYPE, Char: TEXT_TYPE, Varchar: TEXT_TYPE, @@ -229,7 +236,6 @@ var ( ) func Type2SQLType(t reflect.Type) (st SQLType) { - switch k := t.Kind(); k { case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32: st = SQLType{Int, 0, 0} @@ -252,7 +258,7 @@ func Type2SQLType(t reflect.Type) (st SQLType) { case reflect.String: st = SQLType{Varchar, 255, 0} case reflect.Struct: - if t.ConvertibleTo(reflect.TypeOf(c_TIME_DEFAULT)) { + if t.ConvertibleTo(TimeType) { st = SQLType{DateTime, 0, 0} } else { // TODO need to handle association struct diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/LICENSE b/Godeps/_workspace/src/github.com/go-xorm/xorm/LICENSE index 9ac0c261f59..84d2ae5386d 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/LICENSE +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/LICENSE @@ -1,4 +1,4 @@ -Copyright (c) 2013 - 2014 +Copyright (c) 2013 - 2015 The Xorm Authors All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/README.md b/Godeps/_workspace/src/github.com/go-xorm/xorm/README.md index fe8aca3c374..4b4685ea72b 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/README.md +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/README.md @@ -2,6 +2,8 @@ Xorm is a simple and powerful ORM for Go. +[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/go-xorm/xorm?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) + [![Build Status](https://drone.io/github.com/go-xorm/tests/status.png)](https://drone.io/github.com/go-xorm/tests/latest) [![Go Walker](http://gowalker.org/api/v1/badge)](http://gowalker.org/github.com/go-xorm/xorm) [![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/lunny/xorm/trend.png)](https://bitdeli.com/free "Bitdeli Badge") # Features @@ -9,7 +11,7 @@ Xorm is a simple and powerful ORM for Go. * Struct <-> Table Mapping Support * Chainable APIs - + * Transaction Support * Both ORM and raw SQL operation Support @@ -33,38 +35,44 @@ Drivers for Go's sql package which currently support database/sql includes: * MyMysql: [github.com/ziutek/mymysql/godrv](https://github.com/ziutek/mymysql/godrv) -* SQLite: [github.com/mattn/go-sqlite3](https://github.com/mattn/go-sqlite3) - * Postgres: [github.com/lib/pq](https://github.com/lib/pq) +* Tidb: [github.com/pingcap/tidb](https://github.com/pingcap/tidb) + +* SQLite: [github.com/mattn/go-sqlite3](https://github.com/mattn/go-sqlite3) + * MsSql: [github.com/denisenkom/go-mssqldb](https://github.com/denisenkom/go-mssqldb) * MsSql: [github.com/lunny/godbc](https://github.com/lunny/godbc) +* Oracle: [github.com/mattn/go-oci8](https://github.com/mattn/go-oci8) (experiment) +* ql: [github.com/cznic/ql](https://github.com/cznic/ql) (experiment) # Changelog -* **v0.4.1** - Features: - * Add deleted xorm tag for soft delete and add unscoped +* **v0.4.4** + * ql database expriment support + * tidb database expriment support + * sql.NullString and etc. field support + * select ForUpdate support + * many bugs fixed -* **v0.4.0 RC1** - Changes: - * moved xorm cmd to [github.com/go-xorm/cmd](github.com/go-xorm/cmd) - * refactored general DB operation a core lib at [github.com/go-xorm/core](https://github.com/go-xorm/core) - * moved tests to github.com/go-xorm/tests [github.com/go-xorm/tests](github.com/go-xorm/tests) +* **v0.4.3** + * Json column type support + * oracle expirement support + * bug fixed - Improvements: - * Prepared statement cache - * Add Incr API - * Specify Timezone Location +* **v0.4.2** + * Transaction will auto rollback if not Rollback or Commit be called. + * Gonic Mapper support + * bug fixed -[More changelogs ...](https://github.com/go-xorm/manual-en-US/tree/master/chapter-15) +[More changelogs ...](https://github.com/go-xorm/manual-en-US/tree/master/chapter-16) # Installation -If you have [gopm](https://github.com/gpmgo/gopm) installed, +If you have [gopm](https://github.com/gpmgo/gopm) installed, gopm get github.com/go-xorm/xorm @@ -80,8 +88,152 @@ Or * [GoWalker](http://gowalker.org/github.com/go-xorm/xorm) +# Quick Start + +* Create Engine + +```Go +engine, err := xorm.NewEngine(driverName, dataSourceName) +``` + +* Define a struct and Sync2 table struct to database + +```Go +type User struct { + Id int64 + Name string + Salt string + Age int + Passwd string `xorm:"varchar(200)"` + Created time.Time `xorm:"created"` + Updated time.Time `xorm:"updated"` +} + +err := engine.Sync2(new(User)) +``` + +* Query a SQL string, the returned results is []map[string][]byte + +```Go +results, err := engine.Query("select * from user") +``` + +* Execute a SQL string, the returned results + +```Go +affected, err := engine.Exec("update user set age = ? where name = ?", age, name) +``` + +* Insert one or multipe records to database + +```Go +affected, err := engine.Insert(&user) +// INSERT INTO struct () values () +affected, err := engine.Insert(&user1, &user2) +// INSERT INTO struct1 () values () +// INSERT INTO struct2 () values () +affected, err := engine.Insert(&users) +// INSERT INTO struct () values (),(),() +affected, err := engine.Insert(&user1, &users) +// INSERT INTO struct1 () values () +// INSERT INTO struct2 () values (),(),() +``` + +* Query one record from database + +```Go +has, err := engine.Get(&user) +// SELECT * FROM user LIMIT 1 +has, err := engine.Where("name = ?", name).Desc("id").Get(&user) +// SELECT * FROM user WHERE name = ? ORDER BY id DESC LIMIT 1 +``` + +* Query multiple records from database, also you can use join and extends + +```Go +var users []User +err := engine.Where("name = ?", name).And("age > 10").Limit(10, 0).Find(&users) +// SELECT * FROM user WHERE name = ? AND age > 10 limit 0 offset 10 + +type Detail struct { + Id int64 + UserId int64 `xorm:"index"` +} + +type UserDetail struct { + User `xorm:"extends"` + Detail `xorm:"extends"` +} + +var users []UserDetail +err := engine.Table("user").Select("user.*, detail.*") + Join("INNER", "detail", "detail.user_id = user.id"). + Where("user.name = ?", name).Limit(10, 0). + Find(&users) +// SELECT user.*, detail.* FROM user INNER JOIN detail WHERE user.name = ? limit 0 offset 10 +``` + +* Query multiple records and record by record handle, there two methods Iterate and Rows + +```Go +err := engine.Iterate(&User{Name:name}, func(idx int, bean interface{}) error { + user := bean.(*User) + return nil +}) +// SELECT * FROM user + +rows, err := engine.Rows(&User{Name:name}) +// SELECT * FROM user +defer rows.Close() +bean := new(Struct) +for rows.Next() { + err = rows.Scan(bean) +} +``` + +* Update one or more records, default will update non-empty and non-zero fields except to use Cols, AllCols and etc. + +```Go +affected, err := engine.Id(1).Update(&user) +// UPDATE user SET ... Where id = ? + +affected, err := engine.Update(&user, &User{Name:name}) +// UPDATE user SET ... Where name = ? + +var ids = []int64{1, 2, 3} +affected, err := engine.In(ids).Update(&user) +// UPDATE user SET ... Where id IN (?, ?, ?) + +// force update indicated columns by Cols +affected, err := engine.Id(1).Cols("age").Update(&User{Name:name, Age: 12}) +// UPDATE user SET age = ?, updated=? Where id = ? + +// force NOT update indicated columns by Omit +affected, err := engine.Id(1).Omit("name").Update(&User{Name:name, Age: 12}) +// UPDATE user SET age = ?, updated=? Where id = ? + +affected, err := engine.Id(1).AllCols().Update(&user) +// UPDATE user SET name=?,age=?,salt=?,passwd=?,updated=? Where id = ? +``` + +* Delete one or more records, Delete MUST has conditon + +```Go +affected, err := engine.Where(...).Delete(&user) +// DELETE FROM user Where ... +``` + +* Count records + +```Go +counts, err := engine.Count(&user) +// SELECT count(*) AS total FROM user +``` + # Cases +* [github.com/m3ng9i/qreader](https://github.com/m3ng9i/qreader) + * [Wego](http://github.com/go-tango/wego) * [Docker.cn](https://docker.cn/) diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/README_CN.md b/Godeps/_workspace/src/github.com/go-xorm/xorm/README_CN.md index 5a167f9b148..4466bc707cd 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/README_CN.md +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/README_CN.md @@ -4,6 +4,8 @@ xorm是一个简单而强大的Go语言ORM库. 通过它可以使数据库操作非常简便。 +[![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/go-xorm/xorm?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) + [![Build Status](https://drone.io/github.com/go-xorm/tests/status.png)](https://drone.io/github.com/go-xorm/tests/latest) [![Go Walker](http://gowalker.org/api/v1/badge)](http://gowalker.org/github.com/go-xorm/xorm) ## 特性 @@ -18,7 +20,7 @@ xorm是一个简单而强大的Go语言ORM库. 通过它可以使数据库操作 * 支持使用Id, In, Where, Limit, Join, Having, Table, Sql, Cols等函数和结构体等方式作为条件 -* 支持级联加载Struct +* 支持级联加载Struct * 支持缓存 @@ -34,29 +36,42 @@ xorm是一个简单而强大的Go语言ORM库. 通过它可以使数据库操作 * MyMysql: [github.com/ziutek/mymysql/godrv](https://github.com/ziutek/mymysql/godrv) -* SQLite: [github.com/mattn/go-sqlite3](https://github.com/mattn/go-sqlite3) - * Postgres: [github.com/lib/pq](https://github.com/lib/pq) +* Tidb: [github.com/pingcap/tidb](https://github.com/pingcap/tidb) + +* SQLite: [github.com/mattn/go-sqlite3](https://github.com/mattn/go-sqlite3) + * MsSql: [github.com/denisenkom/go-mssqldb](https://github.com/denisenkom/go-mssqldb) * MsSql: [github.com/lunny/godbc](https://github.com/lunny/godbc) +* Oracle: [github.com/mattn/go-oci8](https://github.com/mattn/go-oci8) (试验性支持) + +* ql: [github.com/cznic/ql](https://github.com/cznic/ql) (试验性支持) + ## 更新日志 -* **v0.4.2** - 新特性： - * deleted标记 - * bug fixed +* **v0.4.4** + * Tidb 数据库支持 + * QL 试验性支持 + * sql.NullString支持 + * ForUpdate 支持 + * bug修正 + +* **v0.4.3** + * Json 字段类型支持 + * oracle实验性支持 + * bug修正 [更多更新日志...](https://github.com/go-xorm/manual-zh-CN/tree/master/chapter-16) ## 安装 -推荐使用 [gopm](https://github.com/gpmgo/gopm) 进行安装： +推荐使用 [gopm](https://github.com/gpmgo/gopm) 进行安装： gopm get github.com/go-xorm/xorm - + 或者您也可以使用go工具进行安装： go get github.com/go-xorm/xorm @@ -69,8 +84,151 @@ xorm是一个简单而强大的Go语言ORM库. 通过它可以使数据库操作 * [Godoc代码文档](http://godoc.org/github.com/go-xorm/xorm) +# 快速开始 -## 案例 +* 第一步创建引擎，driverName, dataSourceName和database/sql接口相同 + +```Go +engine, err := xorm.NewEngine(driverName, dataSourceName) +``` + +* 定义一个和表同步的结构体，并且自动同步结构体到数据库 + +```Go +type User struct { + Id int64 + Name string + Salt string + Age int + Passwd string `xorm:"varchar(200)"` + Created time.Time `xorm:"created"` + Updated time.Time `xorm:"updated"` +} + +err := engine.Sync2(new(User)) +``` + +* 最原始的也支持SQL语句查询，返回的结果类型为 []map[string][]byte + +```Go +results, err := engine.Query("select * from user") +``` + +* 执行一个SQL语句 + +```Go +affected, err := engine.Exec("update user set age = ? where name = ?", age, name) +``` + +* 插入一条或者多条记录 + +```Go +affected, err := engine.Insert(&user) +// INSERT INTO struct () values () +affected, err := engine.Insert(&user1, &user2) +// INSERT INTO struct1 () values () +// INSERT INTO struct2 () values () +affected, err := engine.Insert(&users) +// INSERT INTO struct () values (),(),() +affected, err := engine.Insert(&user1, &users) +// INSERT INTO struct1 () values () +// INSERT INTO struct2 () values (),(),() +``` + +* 查询单条记录 + +```Go +has, err := engine.Get(&user) +// SELECT * FROM user LIMIT 1 +has, err := engine.Where("name = ?", name).Desc("id").Get(&user) +// SELECT * FROM user WHERE name = ? ORDER BY id DESC LIMIT 1 +``` + +* 查询多条记录，当然可以使用Join和extends来组合使用 + +```Go +var users []User +err := engine.Where("name = ?", name).And("age > 10").Limit(10, 0).Find(&users) +// SELECT * FROM user WHERE name = ? AND age > 10 limit 0 offset 10 + +type Detail struct { + Id int64 + UserId int64 `xorm:"index"` +} + +type UserDetail struct { + User `xorm:"extends"` + Detail `xorm:"extends"` +} + +var users []UserDetail +err := engine.Table("user").Select("user.*, detail.*") + Join("INNER", "detail", "detail.user_id = user.id"). + Where("user.name = ?", name).Limit(10, 0). + Find(&users) +// SELECT user.*, detail.* FROM user INNER JOIN detail WHERE user.name = ? limit 0 offset 10 +``` + +* 根据条件遍历数据库，可以有两种方式: Iterate and Rows + +```Go +err := engine.Iterate(&User{Name:name}, func(idx int, bean interface{}) error { + user := bean.(*User) + return nil +}) +// SELECT * FROM user + +rows, err := engine.Rows(&User{Name:name}) +// SELECT * FROM user +defer rows.Close() +bean := new(Struct) +for rows.Next() { + err = rows.Scan(bean) +} +``` + +* 更新数据，除非使用Cols,AllCols函数指明，默认只更新非空和非0的字段 + +```Go +affected, err := engine.Id(1).Update(&user) +// UPDATE user SET ... Where id = ? + +affected, err := engine.Update(&user, &User{Name:name}) +// UPDATE user SET ... Where name = ? + +var ids = []int64{1, 2, 3} +affected, err := engine.In(ids).Update(&user) +// UPDATE user SET ... Where id IN (?, ?, ?) + +// force update indicated columns by Cols +affected, err := engine.Id(1).Cols("age").Update(&User{Name:name, Age: 12}) +// UPDATE user SET age = ?, updated=? Where id = ? + +// force NOT update indicated columns by Omit +affected, err := engine.Id(1).Omit("name").Update(&User{Name:name, Age: 12}) +// UPDATE user SET age = ?, updated=? Where id = ? + +affected, err := engine.Id(1).AllCols().Update(&user) +// UPDATE user SET name=?,age=?,salt=?,passwd=?,updated=? Where id = ? +``` + +* 删除记录，需要注意，删除必须至少有一个条件，否则会报错。要清空数据库可以用EmptyTable + +```Go +affected, err := engine.Where(...).Delete(&user) +// DELETE FROM user Where ... +``` + +* 获取记录条数 + +```Go +counts, err := engine.Count(&user) +// SELECT count(*) AS total FROM user +``` + +# 案例 + +* [github.com/m3ng9i/qreader](https://github.com/m3ng9i/qreader) * [Wego](http://github.com/go-tango/wego) diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/VERSION b/Godeps/_workspace/src/github.com/go-xorm/xorm/VERSION index 4e64e0e9e47..d1a98fc7b2e 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/VERSION +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/VERSION @@ -1 +1 @@ -xorm v0.4.2.0225 +xorm v0.4.5.0204 diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/doc.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/doc.go index 722088ca775..54ce2b985b7 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/doc.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/doc.go @@ -1,4 +1,4 @@ -// Copyright 2013 - 2014 The XORM Authors. All rights reserved. +// Copyright 2013 - 2015 The Xorm Authors. All rights reserved. // Use of this source code is governed by a BSD // license that can be found in the LICENSE file. diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/engine.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/engine.go index 8f0c805dc34..afb38766db3 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/engine.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/engine.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( @@ -316,6 +320,12 @@ func (engine *Engine) NoAutoTime() *Session { return session.NoAutoTime() } +func (engine *Engine) NoAutoCondition(no ...bool) *Session { + session := engine.NewSession() + session.IsAutoClose = true + return session.NoAutoCondition(no...) +} + // Retrieve all tables, columns, indexes' informations from database. func (engine *Engine) DBMetas() ([]*core.Table, error) { tables, err := engine.dialect.GetTables() @@ -373,13 +383,25 @@ func (engine *Engine) DumpAll(w io.Writer) error { return err } - for _, table := range tables { - _, err = io.WriteString(w, engine.dialect.CreateTableSql(table, "", table.StoreEngine, "")+"\n\n") + _, err = io.WriteString(w, fmt.Sprintf("/*Generated by xorm v%s %s*/\n\n", + Version, time.Now().In(engine.TZLocation).Format("2006-01-02 15:04:05"))) + if err != nil { + return err + } + + for i, table := range tables { + if i > 0 { + _, err = io.WriteString(w, "\n") + if err != nil { + return err + } + } + _, err = io.WriteString(w, engine.dialect.CreateTableSql(table, "", table.StoreEngine, "")+";\n") if err != nil { return err } for _, index := range table.Indexes { - _, err = io.WriteString(w, engine.dialect.CreateIndexSql(table.Name, index)+"\n\n") + _, err = io.WriteString(w, engine.dialect.CreateIndexSql(table.Name, index)+";\n") if err != nil { return err } @@ -439,7 +461,7 @@ func (engine *Engine) DumpAll(w io.Writer) error { } } } - _, err = io.WriteString(w, temp[2:]+");\n\n") + _, err = io.WriteString(w, temp[2:]+");\n") if err != nil { return err } @@ -506,6 +528,12 @@ func (engine *Engine) Distinct(columns ...string) *Session { return session.Distinct(columns...) } +func (engine *Engine) Select(str string) *Session { + session := engine.NewSession() + session.IsAutoClose = true + return session.Select(str) +} + // only use the paramters as select or update columns func (engine *Engine) Cols(columns ...string) *Session { session := engine.NewSession() @@ -543,6 +571,13 @@ func (engine *Engine) Omit(columns ...string) *Session { return session.Omit(columns...) } +// Set null when column is zero-value and nullable for update +func (engine *Engine) Nullable(columns ...string) *Session { + session := engine.NewSession() + session.IsAutoClose = true + return session.Nullable(columns...) +} + // This method will generate "column IN (?, ?)" func (engine *Engine) In(column string, args ...interface{}) *Session { session := engine.NewSession() @@ -642,20 +677,20 @@ func (engine *Engine) Having(conditions string) *Session { func (engine *Engine) autoMapType(v reflect.Value) *core.Table { t := v.Type() - engine.mutex.RLock() + engine.mutex.Lock() table, ok := engine.Tables[t] - engine.mutex.RUnlock() if !ok { table = engine.mapType(v) - engine.mutex.Lock() engine.Tables[t] = table - if v.CanAddr() { - engine.GobRegister(v.Addr().Interface()) - } else { - engine.GobRegister(v.Interface()) + if engine.Cacher != nil { + if v.CanAddr() { + engine.GobRegister(v.Addr().Interface()) + } else { + engine.GobRegister(v.Interface()) + } } - engine.mutex.Unlock() } + engine.mutex.Unlock() return table } @@ -691,33 +726,24 @@ func (engine *Engine) newTable() *core.Table { return table } +type TableName interface { + TableName() string +} + func (engine *Engine) mapType(v reflect.Value) *core.Table { t := v.Type() table := engine.newTable() - method := v.MethodByName("TableName") - if !method.IsValid() { - if v.CanAddr() { - method = v.Addr().MethodByName("TableName") - } - } - if method.IsValid() { - params := []reflect.Value{} - results := method.Call(params) - if len(results) == 1 { - table.Name = results[0].Interface().(string) - } - } - - if table.Name == "" { + if tb, ok := v.Interface().(TableName); ok { + table.Name = tb.TableName() + } else { table.Name = engine.TableMapper.Obj2Table(t.Name()) } + table.Type = t var idFieldColName string var err error - - hasCacheTag := false - hasNoCacheTag := false + var hasCacheTag, hasNoCacheTag bool for i := 0; i < t.NumField(); i++ { tag := t.Field(i).Tag @@ -730,7 +756,7 @@ func (engine *Engine) mapType(v reflect.Value) *core.Table { if ormTagStr != "" { col = &core.Column{FieldName: t.Field(i).Name, Nullable: true, IsPrimaryKey: false, IsAutoIncrement: false, MapType: core.TWOSIDES, Indexes: make(map[string]bool)} - tags := strings.Split(ormTagStr, " ") + tags := splitTag(ormTagStr) if len(tags) > 0 { if tags[0] == "-" { @@ -804,6 +830,16 @@ func (engine *Engine) mapType(v reflect.Value) *core.Table { case k == "VERSION": col.IsVersion = true col.Default = "1" + case k == "UTC": + col.TimeZone = time.UTC + case k == "LOCAL": + col.TimeZone = time.Local + case strings.HasPrefix(k, "LOCALE(") && strings.HasSuffix(k, ")"): + location := k[len("INDEX")+1 : len(k)-1] + col.TimeZone, err = time.LoadLocation(location) + if err != nil { + engine.LogError(err) + } case k == "UPDATED": col.IsUpdated = true case k == "DELETED": @@ -1112,7 +1148,7 @@ func (engine *Engine) Sync(beans ...interface{}) error { session := engine.NewSession() session.Statement.RefTable = table defer session.Close() - isExist, err := session.Engine.dialect.IsColumnExist(table.Name, col) + isExist, err := session.Engine.dialect.IsColumnExist(table.Name, col.Name) if err != nil { return err } @@ -1387,7 +1423,6 @@ var ( ) func (engine *Engine) TZTime(t time.Time) time.Time { - if NULL_TIME != t { // if time is not initialized it's not suitable for Time.In() return t.In(engine.TZLocation) } @@ -1405,35 +1440,51 @@ func (engine *Engine) NowTime2(sqlTypeName string) (interface{}, time.Time) { } func (engine *Engine) FormatTime(sqlTypeName string, t time.Time) (v interface{}) { + return engine.formatTime(engine.TZLocation, sqlTypeName, t) +} + +func (engine *Engine) formatColTime(col *core.Column, t time.Time) (v interface{}) { + if col.DisableTimeZone { + return engine.formatTime(nil, col.SQLType.Name, t) + } else if col.TimeZone != nil { + return engine.formatTime(col.TimeZone, col.SQLType.Name, t) + } + return engine.formatTime(engine.TZLocation, col.SQLType.Name, t) +} + +func (engine *Engine) formatTime(tz *time.Location, sqlTypeName string, t time.Time) (v interface{}) { if engine.dialect.DBType() == core.ORACLE { return t } + if tz != nil { + t = engine.TZTime(t) + } switch sqlTypeName { case core.Time: - s := engine.TZTime(t).Format("2006-01-02 15:04:05") //time.RFC3339 + s := t.Format("2006-01-02 15:04:05") //time.RFC3339 v = s[11:19] case core.Date: - v = engine.TZTime(t).Format("2006-01-02") + v = t.Format("2006-01-02") case core.DateTime, core.TimeStamp: if engine.dialect.DBType() == "ql" { - v = engine.TZTime(t) + v = t } else if engine.dialect.DBType() == "sqlite3" { - v = engine.TZTime(t).UTC().Format("2006-01-02 15:04:05") + v = t.UTC().Format("2006-01-02 15:04:05") } else { - v = engine.TZTime(t).Format("2006-01-02 15:04:05") + v = t.Format("2006-01-02 15:04:05") } case core.TimeStampz: if engine.dialect.DBType() == core.MSSQL { - v = engine.TZTime(t).Format("2006-01-02T15:04:05.9999999Z07:00") + v = t.Format("2006-01-02T15:04:05.9999999Z07:00") } else if engine.DriverName() == "mssql" { - v = engine.TZTime(t) + v = t } else { - v = engine.TZTime(t).Format(time.RFC3339Nano) + v = t.Format(time.RFC3339Nano) } case core.BigInt, core.Int: - v = engine.TZTime(t).Unix() + v = t.Unix() default: - v = engine.TZTime(t) + v = t } return } diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/error.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/error.go index c868173f708..61537a34626 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/error.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/error.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/examples/goroutine.db-journal b/Godeps/_workspace/src/github.com/go-xorm/xorm/examples/goroutine.db-journal deleted file mode 100644 index 95fc238e294..00000000000 Binary files a/Godeps/_workspace/src/github.com/go-xorm/xorm/examples/goroutine.db-journal and /dev/null differ diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/examples/goroutine.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/examples/goroutine.go index b18fe4f87de..f99c9fbe472 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/examples/goroutine.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/examples/goroutine.go @@ -33,7 +33,7 @@ func test(engine *xorm.Engine) { return } - size := 500 + size := 100 queue := make(chan int, size) for i := 0; i < size; i++ { @@ -83,7 +83,7 @@ func test(engine *xorm.Engine) { } func main() { - runtime.GOMAXPROCS(1) + runtime.GOMAXPROCS(2) fmt.Println("-----start sqlite go routines-----") engine, err := sqliteEngine() if err != nil { diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/goracle_driver.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/goracle_driver.go index f24fe339c1a..9fcde48f059 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/goracle_driver.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/goracle_driver.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/helpers.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/helpers.go index 7eaa1dd1c21..7fa85d11664 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/helpers.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/helpers.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( @@ -11,6 +15,30 @@ import ( "github.com/go-xorm/core" ) +func splitTag(tag string) (tags []string) { + tag = strings.TrimSpace(tag) + var hasQuote = false + var lastIdx = 0 + for i, t := range tag { + if t == '\'' { + hasQuote = !hasQuote + } else if t == ' ' { + if lastIdx < i && !hasQuote { + tags = append(tags, strings.TrimSpace(tag[lastIdx:i])) + lastIdx = i + 1 + } + } + } + if lastIdx < len(tag) { + tags = append(tags, strings.TrimSpace(tag[lastIdx:len(tag)])) + } + return +} + +type zeroable interface { + IsZero() bool +} + func isZero(k interface{}) bool { switch k.(type) { case int: @@ -33,12 +61,41 @@ func isZero(k interface{}) bool { return k.(uint32) == 0 case uint64: return k.(uint64) == 0 + case float32: + return k.(float32) == 0 + case float64: + return k.(float64) == 0 + case bool: + return k.(bool) == false case string: return k.(string) == "" + case zeroable: + return k.(zeroable).IsZero() } return false } +func int64ToInt(id int64, k reflect.Kind) interface{} { + var v interface{} = id + switch k { + case reflect.Int16: + v = int16(id) + case reflect.Int32: + v = int32(id) + case reflect.Int: + v = int(id) + case reflect.Uint16: + v = uint16(id) + case reflect.Uint32: + v = uint32(id) + case reflect.Uint64: + v = uint64(id) + case reflect.Uint: + v = uint(id) + } + return v +} + func isPKZero(pk core.PK) bool { for _, k := range pk { if isZero(k) { @@ -48,6 +105,10 @@ func isPKZero(pk core.PK) bool { return false } +func equalNoCase(s1, s2 string) bool { + return strings.ToLower(s1) == strings.ToLower(s2) +} + func indexNoCase(s, sep string) int { return strings.Index(strings.ToLower(s), strings.ToLower(sep)) } @@ -129,8 +190,8 @@ func reflect2value(rawValue *reflect.Value) (str string, err error) { } //时间类型 case reflect.Struct: - if aa == core.TimeType { - str = rawValue.Interface().(time.Time).Format(time.RFC3339Nano) + if aa.ConvertibleTo(core.TimeType) { + str = vv.Convert(core.TimeType).Interface().(time.Time).Format(time.RFC3339Nano) } else { err = fmt.Errorf("Unsupported struct type %v", vv.Type().Name()) } @@ -352,6 +413,14 @@ func genCols(table *core.Table, session *Session, bean interface{}, useCol bool, } } + // !evalphobia! set fieldValue as nil when column is nullable and zero-value + if _, ok := session.Statement.nullableMap[lColName]; ok { + if col.Nullable && isZero(fieldValue.Interface()) { + var nilValue *int + fieldValue = reflect.ValueOf(nilValue) + } + } + if (col.IsCreated || col.IsUpdated) && session.Statement.UseAutoTime { val, t := session.Engine.NowTime2(col.SQLType.Name) args = append(args, val) diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/helpers_test.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/helpers_test.go new file mode 100644 index 00000000000..7d17383d337 --- /dev/null +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/helpers_test.go @@ -0,0 +1,22 @@ +package xorm + +import "testing" + +func TestSplitTag(t *testing.T) { + var cases = []struct { + tag string + tags []string + }{ + {"not null default '2000-01-01 00:00:00' TIMESTAMP", []string{"not", "null", "default", "'2000-01-01 00:00:00'", "TIMESTAMP"}}, + {"TEXT", []string{"TEXT"}}, + {"default('2000-01-01 00:00:00')", []string{"default('2000-01-01 00:00:00')"}}, + {"json binary", []string{"json", "binary"}}, + } + + for _, kase := range cases { + tags := splitTag(kase.tag) + if !sliceEq(tags, kase.tags) { + t.Fatalf("[%d]%v is not equal [%d]%v", len(tags), tags, len(kase.tags), kase.tags) + } + } +} diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/logger.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/logger.go index da3ba019b86..31787ba0e83 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/logger.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/logger.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/lru_cacher.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/lru_cacher.go index 8c912879ccc..855cce2ee40 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/lru_cacher.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/lru_cacher.go @@ -1,4 +1,7 @@ -//LRUCacher implements Cacher according to LRU algorithm +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/memroy_store.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/memory_store.go similarity index 82% rename from Godeps/_workspace/src/github.com/go-xorm/xorm/memroy_store.go rename to Godeps/_workspace/src/github.com/go-xorm/xorm/memory_store.go index c5800e46d1e..60330995707 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/memroy_store.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/memory_store.go @@ -1,4 +1,7 @@ -// MemoryStore implements CacheStore provide local machine +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/mssql_dialect.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/mssql_dialect.go index 6fe50fc8c0d..41989b91807 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/mssql_dialect.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/mssql_dialect.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( @@ -216,6 +220,11 @@ func (db *mssql) SqlType(c *core.Column) string { switch t := c.SQLType.Name; t { case core.Bool: res = core.TinyInt + if c.Default == "true" { + c.Default = "1" + } else if c.Default == "false" { + c.Default = "0" + } case core.Serial: c.IsAutoIncrement = true c.IsPrimaryKey = true @@ -238,7 +247,7 @@ func (db *mssql) SqlType(c *core.Column) string { c.Length = 7 case core.MediumInt: res = core.Int - case core.MediumText, core.TinyText, core.LongText: + case core.MediumText, core.TinyText, core.LongText, core.Json: res = core.Text case core.Double: res = core.Real @@ -311,10 +320,10 @@ func (db *mssql) IndexCheckSql(tableName, idxName string) (string, []interface{} return sql, args }*/ -func (db *mssql) IsColumnExist(tableName string, col *core.Column) (bool, error) { +func (db *mssql) IsColumnExist(tableName, colName string) (bool, error) { query := `SELECT "COLUMN_NAME" FROM "INFORMATION_SCHEMA"."COLUMNS" WHERE "TABLE_NAME" = ? AND "COLUMN_NAME" = ?` - return db.HasRecords(query, tableName, col.Name) + return db.HasRecords(query, tableName, colName) } func (db *mssql) TableCheckSql(tableName string) (string, []interface{}) { @@ -500,6 +509,10 @@ func (db *mssql) CreateTableSql(table *core.Table, tableName, storeEngine, chars return sql } +func (db *mssql) ForUpdateSql(query string) string { + return query +} + func (db *mssql) Filters() []core.Filter { return []core.Filter{&core.IdFilter{}, &core.QuoteFilter{}} } diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/mymysql_driver.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/mymysql_driver.go index a4e726a445f..53094de9a7c 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/mymysql_driver.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/mymysql_driver.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/mysql_dialect.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/mysql_dialect.go index 602cd0ecff7..76a84a7fa70 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/mysql_dialect.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/mysql_dialect.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( @@ -221,6 +225,8 @@ func (db *mysql) SqlType(c *core.Column) string { case core.Uuid: res = core.Varchar c.Length = 40 + case core.Json: + res = core.Text default: res = t } diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/mysql_driver.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/mysql_driver.go index c36fa8b60b8..872d854d14b 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/mysql_driver.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/mysql_driver.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/oci8_driver.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/oci8_driver.go index 5bb1d16c44b..f16a2a37d7c 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/oci8_driver.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/oci8_driver.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/odbc_driver.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/odbc_driver.go index 0f6d4e183a9..81b87fde8a8 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/odbc_driver.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/odbc_driver.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/oracle_dialect.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/oracle_dialect.go index be71c288d0c..dc64c00e5e9 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/oracle_dialect.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/oracle_dialect.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( @@ -518,7 +522,7 @@ func (db *oracle) SqlType(c *core.Column) string { res = "TIMESTAMP WITH TIME ZONE" case core.Float, core.Double, core.Numeric, core.Decimal: res = "NUMBER" - case core.Text, core.MediumText, core.LongText: + case core.Text, core.MediumText, core.LongText, core.Json: res = "CLOB" case core.Char, core.Varchar, core.TinyText: res = "VARCHAR2" @@ -661,8 +665,8 @@ func (db *oracle) MustDropTable(tableName string) error { " AND column_name = ?", args }*/ -func (db *oracle) IsColumnExist(tableName string, col *core.Column) (bool, error) { - args := []interface{}{tableName, col.Name} +func (db *oracle) IsColumnExist(tableName, colName string) (bool, error) { + args := []interface{}{tableName, colName} query := "SELECT column_name FROM USER_TAB_COLUMNS WHERE table_name = :1" + " AND column_name = :2" rows, err := db.DB().Query(query, args...) @@ -740,6 +744,8 @@ func (db *oracle) GetColumns(tableName string) ([]string, map[string]*core.Colum switch dt { case "VARCHAR2": col.SQLType = core.SQLType{core.Varchar, len1, len2} + case "NVARCHAR2": + col.SQLType = core.SQLType{core.NVarchar, len1, len2} case "TIMESTAMP WITH TIME ZONE": col.SQLType = core.SQLType{core.TimeStampz, 0, 0} case "NUMBER": diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/postgres_dialect.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/postgres_dialect.go index abe51c3e55a..972a214f6fd 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/postgres_dialect.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/postgres_dialect.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( @@ -892,8 +896,8 @@ func (db *postgres) DropIndexSql(tableName string, index *core.Index) string { return fmt.Sprintf("DROP INDEX %v", quote(idxName)) } -func (db *postgres) IsColumnExist(tableName string, col *core.Column) (bool, error) { - args := []interface{}{tableName, col.Name} +func (db *postgres) IsColumnExist(tableName, colName string) (bool, error) { + args := []interface{}{tableName, colName} query := "SELECT column_name FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name = $1" + " AND column_name = $2" rows, err := db.DB().Query(query, args...) @@ -909,7 +913,8 @@ func (db *postgres) IsColumnExist(tableName string, col *core.Column) (bool, err } func (db *postgres) GetColumns(tableName string) ([]string, map[string]*core.Column, error) { - args := []interface{}{tableName} + pgSchema := "public" + args := []interface{}{tableName,pgSchema} s := `SELECT column_name, column_default, is_nullable, data_type, character_maximum_length, numeric_precision, numeric_precision_radix , CASE WHEN p.contype = 'p' THEN true ELSE false END AS primarykey, CASE WHEN p.contype = 'u' THEN true ELSE false END AS uniquekey @@ -920,7 +925,7 @@ FROM pg_attribute f LEFT JOIN pg_constraint p ON p.conrelid = c.oid AND f.attnum = ANY (p.conkey) LEFT JOIN pg_class AS g ON p.confrelid = g.oid LEFT JOIN INFORMATION_SCHEMA.COLUMNS s ON s.column_name=f.attname AND c.relname=s.table_name -WHERE c.relkind = 'r'::char AND c.relname = $1 AND f.attnum > 0 ORDER BY f.attnum;` +WHERE c.relkind = 'r'::char AND c.relname = $1 AND s.table_schema = $2 AND f.attnum > 0 ORDER BY f.attnum;` rows, err := db.DB().Query(s, args...) if db.Logger != nil { diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/pq_driver.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/pq_driver.go index c8dd5aa009c..a4e269756b6 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/pq_driver.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/pq_driver.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( @@ -37,7 +41,7 @@ func parseURL(connstr string) (string, error) { return "", err } - if u.Scheme != "postgres" { + if u.Scheme != "postgresql" && u.Scheme != "postgres" { return "", fmt.Errorf("invalid connection protocol: %s", u.Scheme) } @@ -99,7 +103,7 @@ func (p *pqDriver) Parse(driverName, dataSourceName string) (*core.Uri, error) { db := &core.Uri{DbType: core.POSTGRES} o := make(values) var err error - if strings.HasPrefix(dataSourceName, "postgres://") { + if strings.HasPrefix(dataSourceName, "postgresql://") || strings.HasPrefix(dataSourceName, "postgres://") { dataSourceName, err = parseURL(dataSourceName) if err != nil { return nil, err diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/processors.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/processors.go index 03ae8e0f80c..8f95ae3be73 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/processors.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/processors.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm // Executed before an object is initially persisted to the database @@ -19,6 +23,10 @@ type BeforeSetProcessor interface { BeforeSet(string, Cell) } +type AfterSetProcessor interface { + AfterSet(string, Cell) +} + // !nashtsai! TODO enable BeforeValidateProcessor when xorm start to support validations //// Executed before an object is validated //type BeforeValidateProcessor interface { diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/rows.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/rows.go index 0def55757c8..fb18454d1dc 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/rows.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/rows.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( @@ -41,7 +45,7 @@ func newRows(session *Session, bean interface{}) (*Rows, error) { sqlStr = filter.Do(sqlStr, session.Engine.dialect, rows.session.Statement.RefTable) } - rows.session.Engine.logSQL(sqlStr, args) + rows.session.saveLastSQL(sqlStr, args) var err error rows.stmt, err = rows.session.DB().Prepare(sqlStr) if err != nil { diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/session.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/session.go index 0d11d99fd0f..b7ce7ff6498 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/session.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/session.go @@ -1,7 +1,12 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( "database/sql" + "database/sql/driver" "encoding/json" "errors" "fmt" @@ -39,18 +44,25 @@ type Session struct { beforeClosures []func(interface{}) afterClosures []func(interface{}) + prepareStmt bool stmtCache map[uint32]*core.Stmt //key: hash.Hash32 of (queryStr, len(queryStr)) cascadeDeep int + + // !evalphobia! stored the last executed query on this session + //beforeSQLExec func(string, ...interface{}) + lastSQL string + lastSQLArgs []interface{} } // Method Init reset the session as the init status. func (session *Session) Init() { - session.Statement = Statement{Engine: session.Engine} session.Statement.Init() + session.Statement.Engine = session.Engine session.IsAutoCommit = true session.IsCommitedOrRollbacked = false session.IsAutoClose = false session.AutoResetStatement = true + session.prepareStmt = false // !nashtsai! is lazy init better? session.afterInsertBeans = make(map[interface{}]*[]func(interface{}), 0) @@ -58,6 +70,9 @@ func (session *Session) Init() { session.afterDeleteBeans = make(map[interface{}]*[]func(interface{}), 0) session.beforeClosures = make([]func(interface{}), 0) session.afterClosures = make([]func(interface{}), 0) + + session.lastSQL = "" + session.lastSQLArgs = []interface{}{} } // Method Close release the connection from pool @@ -67,11 +82,15 @@ func (session *Session) Close() { } if session.db != nil { - //session.Engine.Pool.ReleaseDB(session.Engine, session.Db) - session.db = nil + // When Close be called, if session is a transaction and do not call + // Commit or Rollback, then call Rollback. + if session.Tx != nil && !session.IsCommitedOrRollbacked { + session.Rollback() + } session.Tx = nil session.stmtCache = nil session.Init() + session.db = nil } } @@ -81,6 +100,12 @@ func (session *Session) resetStatement() { } } +// Prepare +func (session *Session) Prepare() *Session { + session.prepareStmt = true + return session +} + // Method Sql provides raw sql input parameter. When you have a complex SQL statement // and cannot use Where, Id, In and etc. Methods to describe, you can use Sql. func (session *Session) Sql(querystring string, args ...interface{}) *Session { @@ -164,6 +189,12 @@ func (session *Session) SetExpr(column string, expression string) *Session { return session } +// Method Cols provides some columns to special +func (session *Session) Select(str string) *Session { + session.Statement.Select(str) + return session +} + // Method Cols provides some columns to special func (session *Session) Cols(columns ...string) *Session { session.Statement.Cols(columns...) @@ -203,12 +234,24 @@ func (session *Session) Distinct(columns ...string) *Session { return session } +// Set Read/Write locking for UPDATE +func (session *Session) ForUpdate() *Session { + session.Statement.IsForUpdate = true + return session +} + // Only not use the paramters as select or update columns func (session *Session) Omit(columns ...string) *Session { session.Statement.Omit(columns...) return session } +// Set null when column is zero-value and nullable for update +func (session *Session) Nullable(columns ...string) *Session { + session.Statement.Nullable(columns...) + return session +} + // Method NoAutoTime means do not automatically give created field and updated field // the current time on the current session temporarily func (session *Session) NoAutoTime() *Session { @@ -216,6 +259,11 @@ func (session *Session) NoAutoTime() *Session { return session } +func (session *Session) NoAutoCondition(no ...bool) *Session { + session.Statement.NoAutoCondition(no...) + return session +} + // Method Limit provide limit and offset query condition func (session *Session) Limit(limit int, start ...int) *Session { session.Statement.Limit(limit, start...) @@ -304,8 +352,7 @@ func (session *Session) Begin() error { session.IsAutoCommit = false session.IsCommitedOrRollbacked = false session.Tx = tx - - session.Engine.logSQL("BEGIN TRANSACTION") + session.saveLastSQL("BEGIN TRANSACTION") } return nil } @@ -313,7 +360,7 @@ func (session *Session) Begin() error { // When using transaction, you can rollback if any error func (session *Session) Rollback() error { if !session.IsAutoCommit && !session.IsCommitedOrRollbacked { - session.Engine.logSQL(session.Engine.dialect.RollBackStr()) + session.saveLastSQL(session.Engine.dialect.RollBackStr()) session.IsCommitedOrRollbacked = true return session.Tx.Rollback() } @@ -323,7 +370,7 @@ func (session *Session) Rollback() error { // When using transaction, Commit will commit all operations. func (session *Session) Commit() error { if !session.IsAutoCommit && !session.IsCommitedOrRollbacked { - session.Engine.logSQL("COMMIT") + session.saveLastSQL("COMMIT") session.IsCommitedOrRollbacked = true var err error if err = session.Tx.Commit(); err == nil { @@ -426,17 +473,20 @@ func (session *Session) scanMapIntoStruct(obj interface{}, objMap map[string][]b //Execute sql func (session *Session) innerExec(sqlStr string, args ...interface{}) (sql.Result, error) { - stmt, err := session.doPrepare(sqlStr) - if err != nil { - return nil, err - } - //defer stmt.Close() + if session.prepareStmt { + stmt, err := session.doPrepare(sqlStr) + if err != nil { + return nil, err + } - res, err := stmt.Exec(args...) - if err != nil { - return nil, err + res, err := stmt.Exec(args...) + if err != nil { + return nil, err + } + return res, nil } - return res, nil + + return session.DB().Exec(sqlStr, args...) } func (session *Session) exec(sqlStr string, args ...interface{}) (sql.Result, error) { @@ -444,7 +494,7 @@ func (session *Session) exec(sqlStr string, args ...interface{}) (sql.Result, er sqlStr = filter.Do(sqlStr, session.Engine.dialect, session.Statement.RefTable) } - session.Engine.logSQL(sqlStr, args...) + session.saveLastSQL(sqlStr, args...) return session.Engine.LogSQLExecutionTime(sqlStr, args, func() (sql.Result, error) { if session.IsAutoCommit { @@ -587,11 +637,15 @@ func (session *Session) DropTable(beanOrTableName interface{}) error { return nil } -func (statement *Statement) JoinColumns(cols []*core.Column) string { +func (statement *Statement) JoinColumns(cols []*core.Column, includeTableName bool) string { var colnames = make([]string, len(cols)) for i, col := range cols { - colnames[i] = statement.Engine.Quote(statement.TableName()) + - "." + statement.Engine.Quote(col.Name) + if includeTableName { + colnames[i] = statement.Engine.Quote(statement.TableName()) + + "." + statement.Engine.Quote(col.Name) + } else { + colnames[i] = statement.Engine.Quote(col.Name) + } } return strings.Join(colnames, ", ") } @@ -603,21 +657,33 @@ func (statement *Statement) convertIdSql(sqlStr string) string { return "" } - colstrs := statement.JoinColumns(cols) - sqls := splitNNoCase(sqlStr, "from", 2) + colstrs := statement.JoinColumns(cols, false) + sqls := splitNNoCase(sqlStr, " from ", 2) if len(sqls) != 2 { return "" } + if statement.Engine.dialect.DBType() == "ql" { + return fmt.Sprintf("SELECT id() FROM %v", sqls[1]) + } return fmt.Sprintf("SELECT %s FROM %v", colstrs, sqls[1]) } return "" } -func (session *Session) cacheGet(bean interface{}, sqlStr string, args ...interface{}) (has bool, err error) { - // if has no reftable, then don't use cache currently +func (session *Session) canCache() bool { if session.Statement.RefTable == nil || session.Statement.JoinStr != "" || - session.Statement.RawSQL != "" { + session.Statement.RawSQL != "" || + session.Tx != nil || + len(session.Statement.selectStr) > 0 { + return false + } + return true +} + +func (session *Session) cacheGet(bean interface{}, sqlStr string, args ...interface{}) (has bool, err error) { + // if has no reftable, then don't use cache currently + if !session.canCache() { return false, ErrCacheFailed } @@ -715,7 +781,7 @@ func (session *Session) cacheGet(bean interface{}, sqlStr string, args ...interf } func (session *Session) cacheFind(t reflect.Type, sqlStr string, rowsSlicePtr interface{}, args ...interface{}) (err error) { - if session.Statement.RefTable == nil || + if !session.canCache() || indexNoCase(sqlStr, "having") != -1 || indexNoCase(sqlStr, "group by") != -1 { return ErrCacheFailed @@ -859,7 +925,7 @@ func (session *Session) cacheFind(t reflect.Type, sqlStr string, rowsSlicePtr in } temps[ididxes[sid]] = bean - session.Engine.LogDebug("[cacheFind] cache bean:", tableName, id, bean) + session.Engine.LogDebug("[cacheFind] cache bean:", tableName, id, bean, temps) cacher.PutBean(tableName, sid, bean) } } @@ -867,7 +933,7 @@ func (session *Session) cacheFind(t reflect.Type, sqlStr string, rowsSlicePtr in for j := 0; j < len(temps); j++ { bean := temps[j] if bean == nil { - session.Engine.LogWarn("[cacheFind] cache no hit:", tableName, ides[j]) + session.Engine.LogWarn("[cacheFind] cache no hit:", tableName, ids[j], temps) // return errors.New("cache error") // !nashtsai! no need to return error, but continue instead continue } @@ -992,12 +1058,16 @@ func (session *Session) Get(bean interface{}) (bool, error) { var err error session.queryPreprocess(&sqlStr, args...) if session.IsAutoCommit { - stmt, err := session.doPrepare(sqlStr) - if err != nil { - return false, err + if session.prepareStmt { + stmt, errPrepare := session.doPrepare(sqlStr) + if errPrepare != nil { + return false, errPrepare + } + // defer stmt.Close() // !nashtsai! don't close due to stmt is cached and bounded to this session + rawRows, err = stmt.Query(args...) + } else { + rawRows, err = session.DB().Query(sqlStr, args...) } - // defer stmt.Close() // !nashtsai! don't close due to stmt is cached and bounded to this session - rawRows, err = stmt.Query(args...) } else { rawRows, err = session.Tx.Query(sqlStr, args...) } @@ -1154,18 +1224,25 @@ func (session *Session) Find(rowsSlicePtr interface{}, condiBean ...interface{}) table = session.Statement.RefTable } - if len(condiBean) > 0 { - colNames, args := buildConditions(session.Engine, table, condiBean[0], true, true, - false, true, session.Statement.allUseBool, session.Statement.useAllCols, - session.Statement.unscoped, session.Statement.mustColumnMap) + var addedTableName = (len(session.Statement.JoinStr) > 0) + if !session.Statement.noAutoCondition && len(condiBean) > 0 { + colNames, args := session.Statement.buildConditions(table, condiBean[0], true, true, false, true, addedTableName) session.Statement.ConditionStr = strings.Join(colNames, " AND ") session.Statement.BeanArgs = args } else { // !oinume! Add " IS NULL" to WHERE whatever condiBean is given. // See https://github.com/go-xorm/xorm/issues/179 if col := table.DeletedColumn(); col != nil && !session.Statement.unscoped { // tag "deleted" is enabled - session.Statement.ConditionStr = fmt.Sprintf("(%v IS NULL or %v = '0001-01-01 00:00:00') ", - session.Engine.Quote(col.Name), session.Engine.Quote(col.Name)) + var colName string = session.Engine.Quote(col.Name) + if addedTableName { + var nm = session.Statement.TableName() + if len(session.Statement.TableAlias) > 0 { + nm = session.Statement.TableAlias + } + colName = session.Engine.Quote(nm) + "." + colName + } + session.Statement.ConditionStr = fmt.Sprintf("%v IS NULL OR %v = '0001-01-01 00:00:00'", + colName, colName) } } @@ -1173,20 +1250,24 @@ func (session *Session) Find(rowsSlicePtr interface{}, condiBean ...interface{}) var args []interface{} if session.Statement.RawSQL == "" { var columnStr string = session.Statement.ColumnStr - if session.Statement.JoinStr == "" { - if columnStr == "" { - if session.Statement.GroupByStr != "" { - columnStr = session.Statement.Engine.Quote(strings.Replace(session.Statement.GroupByStr, ",", session.Engine.Quote(","), -1)) - } else { - columnStr = session.Statement.genColumnStr() - } - } + if len(session.Statement.selectStr) > 0 { + columnStr = session.Statement.selectStr } else { - if columnStr == "" { - if session.Statement.GroupByStr != "" { - columnStr = session.Statement.Engine.Quote(strings.Replace(session.Statement.GroupByStr, ",", session.Engine.Quote(","), -1)) - } else { - columnStr = "*" + if session.Statement.JoinStr == "" { + if columnStr == "" { + if session.Statement.GroupByStr != "" { + columnStr = session.Statement.Engine.Quote(strings.Replace(session.Statement.GroupByStr, ",", session.Engine.Quote(","), -1)) + } else { + columnStr = session.Statement.genColumnStr() + } + } + } else { + if columnStr == "" { + if session.Statement.GroupByStr != "" { + columnStr = session.Statement.Engine.Quote(strings.Replace(session.Statement.GroupByStr, ",", session.Engine.Quote(","), -1)) + } else { + columnStr = "*" + } } } } @@ -1227,11 +1308,15 @@ func (session *Session) Find(rowsSlicePtr interface{}, condiBean ...interface{}) session.queryPreprocess(&sqlStr, args...) if session.IsAutoCommit { - stmt, err = session.doPrepare(sqlStr) - if err != nil { - return err + if session.prepareStmt { + stmt, err = session.doPrepare(sqlStr) + if err != nil { + return err + } + rawRows, err = stmt.Query(args...) + } else { + rawRows, err = session.DB().Query(sqlStr, args...) } - rawRows, err = stmt.Query(args...) } else { rawRows, err = session.Tx.Query(sqlStr, args...) } @@ -1279,7 +1364,6 @@ func (session *Session) Find(rowsSlicePtr interface{}, condiBean ...interface{}) } table := session.Engine.autoMapType(dataStruct) - return session.rows2Beans(rawRows, fields, fieldsCount, table, newElemFunc, sliceValueSetFunc) } else { resultsSlice, err := session.query(sqlStr, args...) @@ -1333,20 +1417,6 @@ func (session *Session) Find(rowsSlicePtr interface{}, condiBean ...interface{}) return nil } -// func (session *Session) queryRows(rawStmt **sql.Stmt, rawRows **sql.Rows, sqlStr string, args ...interface{}) error { -// var err error -// if session.IsAutoCommit { -// *rawStmt, err = session.doPrepare(sqlStr) -// if err != nil { -// return err -// } -// *rawRows, err = (*rawStmt).Query(args...) -// } else { -// *rawRows, err = session.Tx.Query(sqlStr, args...) -// } -// return err -// } - // Test if database is ok func (session *Session) Ping() error { defer session.resetStatement() @@ -1422,7 +1492,7 @@ func (session *Session) isTableEmpty(tableName string) (bool, error) { var total int64 sql := fmt.Sprintf("select count(*) from %s", session.Engine.Quote(tableName)) err := session.DB().QueryRow(sql).Scan(&total) - session.Engine.logSQL(sql) + session.saveLastSQL(sql) if err != nil { return true, err } @@ -1550,7 +1620,6 @@ type Cell *interface{} func (session *Session) rows2Beans(rows *core.Rows, fields []string, fieldsCount int, table *core.Table, newElemFunc func() reflect.Value, sliceValueSetFunc func(*reflect.Value)) error { - for rows.Next() { var newValue reflect.Value = newElemFunc() bean := newValue.Interface() @@ -1560,7 +1629,6 @@ func (session *Session) rows2Beans(rows *core.Rows, fields []string, fieldsCount return err } sliceValueSetFunc(&newValue) - } return nil } @@ -1591,6 +1659,14 @@ func (session *Session) _row2Bean(rows *core.Rows, fields []string, fieldsCount } } + defer func() { + if b, hasAfterSet := bean.(AfterSetProcessor); hasAfterSet { + for ii, key := range fields { + b.AfterSet(key, Cell(scanResults[ii].(*interface{}))) + } + } + }() + var tempMap = make(map[string]int) for ii, key := range fields { var idx int @@ -1640,7 +1716,6 @@ func (session *Session) _row2Bean(rows *core.Rows, fields []string, fieldsCount hasAssigned := false switch fieldType.Kind() { - case reflect.Complex64, reflect.Complex128: if rawValueType.Kind() == reflect.String { hasAssigned = true @@ -1651,6 +1726,15 @@ func (session *Session) _row2Bean(rows *core.Rows, fields []string, fieldsCount return err } fieldValue.Set(x.Elem()) + } else if rawValueType.Kind() == reflect.Slice { + hasAssigned = true + x := reflect.New(fieldType) + err := json.Unmarshal(vv.Bytes(), x.Interface()) + if err != nil { + session.Engine.LogError(err) + return err + } + fieldValue.Set(x.Elem()) } case reflect.Slice, reflect.Array: switch rawValueType.Kind() { @@ -1695,6 +1779,7 @@ func (session *Session) _row2Bean(rows *core.Rows, fields []string, fieldsCount fieldValue.SetUint(uint64(vv.Int())) } case reflect.Struct: + col := table.GetColumn(key) if fieldType.ConvertibleTo(core.TimeType) { if rawValueType == core.TimeType { hasAssigned = true @@ -1702,12 +1787,16 @@ func (session *Session) _row2Bean(rows *core.Rows, fields []string, fieldsCount t := vv.Convert(core.TimeType).Interface().(time.Time) z, _ := t.Zone() if len(z) == 0 || t.Year() == 0 { // !nashtsai! HACK tmp work around for lib/pq doesn't properly time with location - session.Engine.LogDebug("empty zone key[%v] : %v | zone: %v | location: %+v\n", key, t, z, *t.Location()) + session.Engine.LogDebugf("empty zone key[%v] : %v | zone: %v | location: %+v\n", key, t, z, *t.Location()) t = time.Date(t.Year(), t.Month(), t.Day(), t.Hour(), t.Minute(), t.Second(), t.Nanosecond(), time.Local) } // !nashtsai! convert to engine location - t = t.In(session.Engine.TZLocation) + if col.TimeZone == nil { + t = t.In(session.Engine.TZLocation) + } else { + t = t.In(col.TimeZone) + } fieldValue.Set(reflect.ValueOf(t).Convert(fieldType)) // t = fieldValue.Interface().(time.Time) @@ -1716,17 +1805,66 @@ func (session *Session) _row2Bean(rows *core.Rows, fields []string, fieldsCount } else if rawValueType == core.IntType || rawValueType == core.Int64Type || rawValueType == core.Int32Type { hasAssigned = true - t := time.Unix(vv.Int(), 0).In(session.Engine.TZLocation) - vv = reflect.ValueOf(t) - fieldValue.Set(vv) + var tz *time.Location + if col.TimeZone == nil { + tz = session.Engine.TZLocation + } else { + tz = col.TimeZone + } + t := time.Unix(vv.Int(), 0).In(tz) + //vv = reflect.ValueOf(t) + fieldValue.Set(reflect.ValueOf(t).Convert(fieldType)) + } else { + if d, ok := vv.Interface().([]uint8); ok { + hasAssigned = true + t, err := session.byte2Time(col, d) + //fmt.Println(string(d), t, err) + if err != nil { + session.Engine.LogError("byte2Time error:", err.Error()) + hasAssigned = false + } else { + fieldValue.Set(reflect.ValueOf(t).Convert(fieldType)) + } + } else { + panic(fmt.Sprintf("rawValueType is %v, value is %v", rawValueType, vv.Interface())) + } + } + } else if nulVal, ok := fieldValue.Addr().Interface().(sql.Scanner); ok { + // !! 增加支持sql.Scanner接口的结构，如sql.NullString + hasAssigned = true + if err := nulVal.Scan(vv.Interface()); err != nil { + //fmt.Println("sql.Sanner error:", err.Error()) + session.Engine.LogError("sql.Sanner error:", err.Error()) + hasAssigned = false + } + } else if col.SQLType.IsJson() { + if rawValueType.Kind() == reflect.String { + hasAssigned = true + x := reflect.New(fieldType) + err := json.Unmarshal([]byte(vv.String()), x.Interface()) + if err != nil { + session.Engine.LogError(err) + return err + } + fieldValue.Set(x.Elem()) + } else if rawValueType.Kind() == reflect.Slice { + hasAssigned = true + x := reflect.New(fieldType) + err := json.Unmarshal(vv.Bytes(), x.Interface()) + if err != nil { + session.Engine.LogError(err) + return err + } + fieldValue.Set(x.Elem()) } } else if session.Statement.UseCascade { table := session.Engine.autoMapType(*fieldValue) if table != nil { - if len(table.PrimaryKeys) > 1 { - panic("unsupported composited primary key cascade") + if len(table.PrimaryKeys) != 1 { + panic("unsupported non or composited primary key cascade") } var pk = make(core.PK, len(table.PrimaryKeys)) + switch rawValueType.Kind() { case reflect.Int64: pk[0] = vv.Int() @@ -1750,8 +1888,10 @@ func (session *Session) _row2Bean(rows *core.Rows, fields []string, fieldsCount pk[0] = uint8(vv.Uint()) case reflect.String: pk[0] = vv.String() + case reflect.Slice: + pk[0], _ = strconv.ParseInt(string(rawValue.Interface().([]byte)), 10, 64) default: - panic("unsupported primary key type cascade") + panic(fmt.Sprintf("unsupported primary key type: %v, %v", rawValueType, fieldValue)) } if !isPKZero(pk) { @@ -1914,7 +2054,7 @@ func (session *Session) queryPreprocess(sqlStr *string, paramStr ...interface{}) *sqlStr = filter.Do(*sqlStr, session.Engine.dialect, session.Statement.RefTable) } - session.Engine.logSQL(*sqlStr, paramStr...) + session.saveLastSQL(*sqlStr, paramStr...) } func (session *Session) query(sqlStr string, paramStr ...interface{}) (resultsSlice []map[string][]byte, err error) { @@ -1922,7 +2062,7 @@ func (session *Session) query(sqlStr string, paramStr ...interface{}) (resultsSl session.queryPreprocess(&sqlStr, paramStr...) if session.IsAutoCommit { - return session.innerQuery(session.DB(), sqlStr, paramStr...) + return session.innerQuery(sqlStr, paramStr...) } return session.txQuery(session.Tx, sqlStr, paramStr...) } @@ -1937,22 +2077,33 @@ func (session *Session) txQuery(tx *core.Tx, sqlStr string, params ...interface{ return rows2maps(rows) } -func (session *Session) innerQuery(db *core.DB, sqlStr string, params ...interface{}) (resultsSlice []map[string][]byte, err error) { - stmt, rows, err := session.Engine.LogSQLQueryTime(sqlStr, params, func() (*core.Stmt, *core.Rows, error) { - stmt, err := db.Prepare(sqlStr) - if err != nil { - return stmt, nil, err +func (session *Session) innerQuery(sqlStr string, params ...interface{}) ([]map[string][]byte, error) { + var callback func() (*core.Stmt, *core.Rows, error) + if session.prepareStmt { + callback = func() (*core.Stmt, *core.Rows, error) { + stmt, err := session.doPrepare(sqlStr) + if err != nil { + return nil, nil, err + } + rows, err := stmt.Query(params...) + if err != nil { + return nil, nil, err + } + return stmt, rows, nil } - rows, err := stmt.Query(params...) - - return stmt, rows, err - }) + } else { + callback = func() (*core.Stmt, *core.Rows, error) { + rows, err := session.DB().Query(sqlStr, params...) + if err != nil { + return nil, nil, err + } + return nil, rows, err + } + } + _, rows, err := session.Engine.LogSQLQueryTime(sqlStr, params, callback) if rows != nil { defer rows.Close() } - if stmt != nil { - defer stmt.Close() - } if err != nil { return nil, err } @@ -2044,7 +2195,9 @@ func (session *Session) innerInsertMulti(rowsSlicePtr interface{}) (int64, error cols := make([]*core.Column, 0) for i := 0; i < size; i++ { - elemValue := sliceValue.Index(i).Interface() + v := sliceValue.Index(i) + vv := reflect.Indirect(v) + elemValue := v.Interface() colPlaces := make([]string, 0) // handle BeforeInsertProcessor @@ -2060,8 +2213,12 @@ func (session *Session) innerInsertMulti(rowsSlicePtr interface{}) (int64, error if i == 0 { for _, col := range table.Columns() { - fieldValue := reflect.Indirect(reflect.ValueOf(elemValue)).FieldByName(col.FieldName) - if col.IsAutoIncrement && fieldValue.Int() == 0 { + ptrFieldValue, err := col.ValueOfV(&vv) + if err != nil { + return 0, err + } + fieldValue := *ptrFieldValue + if col.IsAutoIncrement && isZero(fieldValue.Interface()) { continue } if col.MapType == core.ONLYFROMDB { @@ -2103,8 +2260,13 @@ func (session *Session) innerInsertMulti(rowsSlicePtr interface{}) (int64, error } } else { for _, col := range cols { - fieldValue := reflect.Indirect(reflect.ValueOf(elemValue)).FieldByName(col.FieldName) - if col.IsAutoIncrement && fieldValue.Int() == 0 { + ptrFieldValue, err := col.ValueOfV(&vv) + if err != nil { + return 0, err + } + fieldValue := *ptrFieldValue + + if col.IsAutoIncrement && isZero(fieldValue.Interface()) { continue } if col.MapType == core.ONLYFROMDB { @@ -2167,7 +2329,8 @@ func (session *Session) innerInsertMulti(rowsSlicePtr interface{}) (int64, error lenAfterClosures := len(session.afterClosures) for i := 0; i < size; i++ { - elemValue := sliceValue.Index(i).Interface() + elemValue := reflect.Indirect(sliceValue.Index(i)).Addr().Interface() + // handle AfterInsertProcessor if session.IsAutoCommit { // !nashtsai! does user expect it's same slice to passed closure when using Before()/After() when insert multi?? @@ -2226,17 +2389,20 @@ func (session *Session) byte2Time(col *core.Column, data []byte) (outTime time.T // time stamp sd, err := strconv.ParseInt(sdata, 10, 64) if err == nil { - x = time.Unix(0, sd) + x = time.Unix(sd, 0) // !nashtsai! HACK mymysql driver is casuing Local location being change to CHAT and cause wrong time conversion - x = x.In(time.UTC) - x = time.Date(x.Year(), x.Month(), x.Day(), x.Hour(), - x.Minute(), x.Second(), x.Nanosecond(), session.Engine.TZLocation) + //fmt.Println(x.In(session.Engine.TZLocation), "===") + if col.TimeZone == nil { + x = x.In(session.Engine.TZLocation) + } else { + x = x.In(col.TimeZone) + } + //fmt.Println(x, "=====") session.Engine.LogDebugf("time(0) key[%v]: %+v | sdata: [%v]\n", col.FieldName, x, sdata) } else { session.Engine.LogDebugf("time(0) err key[%v]: %+v | sdata: [%v]\n", col.FieldName, x, sdata) } - } else if len(sdata) > 19 { - + } else if len(sdata) > 19 && strings.Contains(sdata, "-") { x, err = time.ParseInLocation(time.RFC3339Nano, sdata, session.Engine.TZLocation) session.Engine.LogDebugf("time(1) key[%v]: %+v | sdata: [%v]\n", col.FieldName, x, sdata) if err != nil { @@ -2248,7 +2414,7 @@ func (session *Session) byte2Time(col *core.Column, data []byte) (outTime time.T session.Engine.LogDebugf("time(3) key[%v]: %+v | sdata: [%v]\n", col.FieldName, x, sdata) } - } else if len(sdata) == 19 { + } else if len(sdata) == 19 && strings.Contains(sdata, "-") { x, err = time.ParseInLocation("2006-01-02 15:04:05", sdata, session.Engine.TZLocation) session.Engine.LogDebugf("time(4) key[%v]: %+v | sdata: [%v]\n", col.FieldName, x, sdata) } else if len(sdata) == 10 && sdata[4] == '-' && sdata[7] == '-' { @@ -2352,7 +2518,6 @@ func (session *Session) bytes2Value(col *core.Column, fieldValue *reflect.Value, } else { x = 0 } - //fmt.Println("######", x, data) } else if strings.HasPrefix(sdata, "0x") { x, err = strconv.ParseInt(sdata, 16, 64) } else if strings.HasPrefix(sdata, "0") { @@ -2382,108 +2547,115 @@ func (session *Session) bytes2Value(col *core.Column, fieldValue *reflect.Value, fieldValue.SetUint(x) //Currently only support Time type case reflect.Struct: - if fieldType.ConvertibleTo(core.TimeType) { - x, err := session.byte2Time(col, data) - if err != nil { - return err + // !! 增加支持sql.Scanner接口的结构，如sql.NullString + if nulVal, ok := fieldValue.Addr().Interface().(sql.Scanner); ok { + if err := nulVal.Scan(data); err != nil { + return fmt.Errorf("sql.Scan(%v) failed: %s ", data, err.Error()) } - v = x - fieldValue.Set(reflect.ValueOf(v).Convert(fieldType)) - } else if session.Statement.UseCascade { - table := session.Engine.autoMapType(*fieldValue) - if table != nil { - if len(table.PrimaryKeys) > 1 { - panic("unsupported composited primary key cascade") + } else { + if fieldType.ConvertibleTo(core.TimeType) { + x, err := session.byte2Time(col, data) + if err != nil { + return err } - var pk = make(core.PK, len(table.PrimaryKeys)) - rawValueType := table.ColumnType(table.PKColumns()[0].FieldName) - switch rawValueType.Kind() { - case reflect.Int64: - x, err := strconv.ParseInt(string(data), 10, 64) - if err != nil { - return fmt.Errorf("arg %v as int: %s", key, err.Error()) + v = x + fieldValue.Set(reflect.ValueOf(v).Convert(fieldType)) + } else if session.Statement.UseCascade { + table := session.Engine.autoMapType(*fieldValue) + if table != nil { + if len(table.PrimaryKeys) > 1 { + panic("unsupported composited primary key cascade") } - pk[0] = x - case reflect.Int: - x, err := strconv.ParseInt(string(data), 10, 64) - if err != nil { - return fmt.Errorf("arg %v as int: %s", key, err.Error()) + var pk = make(core.PK, len(table.PrimaryKeys)) + rawValueType := table.ColumnType(table.PKColumns()[0].FieldName) + switch rawValueType.Kind() { + case reflect.Int64: + x, err := strconv.ParseInt(string(data), 10, 64) + if err != nil { + return fmt.Errorf("arg %v as int: %s", key, err.Error()) + } + pk[0] = x + case reflect.Int: + x, err := strconv.ParseInt(string(data), 10, 64) + if err != nil { + return fmt.Errorf("arg %v as int: %s", key, err.Error()) + } + pk[0] = int(x) + case reflect.Int32: + x, err := strconv.ParseInt(string(data), 10, 64) + if err != nil { + return fmt.Errorf("arg %v as int: %s", key, err.Error()) + } + pk[0] = int32(x) + case reflect.Int16: + x, err := strconv.ParseInt(string(data), 10, 64) + if err != nil { + return fmt.Errorf("arg %v as int: %s", key, err.Error()) + } + pk[0] = int16(x) + case reflect.Int8: + x, err := strconv.ParseInt(string(data), 10, 64) + if err != nil { + return fmt.Errorf("arg %v as int: %s", key, err.Error()) + } + pk[0] = int8(x) + case reflect.Uint64: + x, err := strconv.ParseUint(string(data), 10, 64) + if err != nil { + return fmt.Errorf("arg %v as int: %s", key, err.Error()) + } + pk[0] = x + case reflect.Uint: + x, err := strconv.ParseUint(string(data), 10, 64) + if err != nil { + return fmt.Errorf("arg %v as int: %s", key, err.Error()) + } + pk[0] = uint(x) + case reflect.Uint32: + x, err := strconv.ParseUint(string(data), 10, 64) + if err != nil { + return fmt.Errorf("arg %v as int: %s", key, err.Error()) + } + pk[0] = uint32(x) + case reflect.Uint16: + x, err := strconv.ParseUint(string(data), 10, 64) + if err != nil { + return fmt.Errorf("arg %v as int: %s", key, err.Error()) + } + pk[0] = uint16(x) + case reflect.Uint8: + x, err := strconv.ParseUint(string(data), 10, 64) + if err != nil { + return fmt.Errorf("arg %v as int: %s", key, err.Error()) + } + pk[0] = uint8(x) + case reflect.String: + pk[0] = string(data) + default: + panic("unsupported primary key type cascade") } - pk[0] = int(x) - case reflect.Int32: - x, err := strconv.ParseInt(string(data), 10, 64) - if err != nil { - return fmt.Errorf("arg %v as int: %s", key, err.Error()) - } - pk[0] = int32(x) - case reflect.Int16: - x, err := strconv.ParseInt(string(data), 10, 64) - if err != nil { - return fmt.Errorf("arg %v as int: %s", key, err.Error()) - } - pk[0] = int16(x) - case reflect.Int8: - x, err := strconv.ParseInt(string(data), 10, 64) - if err != nil { - return fmt.Errorf("arg %v as int: %s", key, err.Error()) - } - pk[0] = int8(x) - case reflect.Uint64: - x, err := strconv.ParseUint(string(data), 10, 64) - if err != nil { - return fmt.Errorf("arg %v as int: %s", key, err.Error()) - } - pk[0] = x - case reflect.Uint: - x, err := strconv.ParseUint(string(data), 10, 64) - if err != nil { - return fmt.Errorf("arg %v as int: %s", key, err.Error()) - } - pk[0] = uint(x) - case reflect.Uint32: - x, err := strconv.ParseUint(string(data), 10, 64) - if err != nil { - return fmt.Errorf("arg %v as int: %s", key, err.Error()) - } - pk[0] = uint32(x) - case reflect.Uint16: - x, err := strconv.ParseUint(string(data), 10, 64) - if err != nil { - return fmt.Errorf("arg %v as int: %s", key, err.Error()) - } - pk[0] = uint16(x) - case reflect.Uint8: - x, err := strconv.ParseUint(string(data), 10, 64) - if err != nil { - return fmt.Errorf("arg %v as int: %s", key, err.Error()) - } - pk[0] = uint8(x) - case reflect.String: - pk[0] = string(data) - default: - panic("unsupported primary key type cascade") - } - if !isPKZero(pk) { - // !nashtsai! TODO for hasOne relationship, it's preferred to use join query for eager fetch - // however, also need to consider adding a 'lazy' attribute to xorm tag which allow hasOne - // property to be fetched lazily - structInter := reflect.New(fieldValue.Type()) - newsession := session.Engine.NewSession() - defer newsession.Close() - has, err := newsession.Id(pk).NoCascade().Get(structInter.Interface()) - if err != nil { - return err - } - if has { - v = structInter.Elem().Interface() - fieldValue.Set(reflect.ValueOf(v)) - } else { - return errors.New("cascade obj is not exist!") + if !isPKZero(pk) { + // !nashtsai! TODO for hasOne relationship, it's preferred to use join query for eager fetch + // however, also need to consider adding a 'lazy' attribute to xorm tag which allow hasOne + // property to be fetched lazily + structInter := reflect.New(fieldValue.Type()) + newsession := session.Engine.NewSession() + defer newsession.Close() + has, err := newsession.Id(pk).NoCascade().Get(structInter.Interface()) + if err != nil { + return err + } + if has { + v = structInter.Elem().Interface() + fieldValue.Set(reflect.ValueOf(v)) + } else { + return errors.New("cascade obj is not exist!") + } } + } else { + return fmt.Errorf("unsupported struct type in Scan: %s", fieldValue.Type().String()) } - } else { - return fmt.Errorf("unsupported struct type in Scan: %s", fieldValue.Type().String()) } } case reflect.Ptr: @@ -2601,7 +2773,6 @@ func (session *Session) bytes2Value(col *core.Column, fieldValue *reflect.Value, } else { x = 0 } - //fmt.Println("######", x, data) } else if strings.HasPrefix(sdata, "0x") { x, err = strconv.ParseInt(sdata, 16, 64) } else if strings.HasPrefix(sdata, "0") { @@ -2627,7 +2798,6 @@ func (session *Session) bytes2Value(col *core.Column, fieldValue *reflect.Value, } else { x = 0 } - //fmt.Println("######", x, data) } else if strings.HasPrefix(sdata, "0x") { x1, err = strconv.ParseInt(sdata, 16, 64) x = int(x1) @@ -2656,7 +2826,6 @@ func (session *Session) bytes2Value(col *core.Column, fieldValue *reflect.Value, } else { x = 0 } - //fmt.Println("######", x, data) } else if strings.HasPrefix(sdata, "0x") { x1, err = strconv.ParseInt(sdata, 16, 64) x = int32(x1) @@ -2685,7 +2854,6 @@ func (session *Session) bytes2Value(col *core.Column, fieldValue *reflect.Value, } else { x = 0 } - //fmt.Println("######", x, data) } else if strings.HasPrefix(sdata, "0x") { x1, err = strconv.ParseInt(sdata, 16, 64) x = int8(x1) @@ -2714,7 +2882,6 @@ func (session *Session) bytes2Value(col *core.Column, fieldValue *reflect.Value, } else { x = 0 } - //fmt.Println("######", x, data) } else if strings.HasPrefix(sdata, "0x") { x1, err = strconv.ParseInt(sdata, 16, 64) x = int16(x1) @@ -2892,31 +3059,47 @@ func (session *Session) value2Interface(col *core.Column, fieldValue reflect.Val case reflect.String: return fieldValue.String(), nil case reflect.Struct: - if fieldType == core.TimeType { - switch fieldValue.Interface().(type) { - case time.Time: - t := fieldValue.Interface().(time.Time) - if session.Engine.dialect.DBType() == core.MSSQL { - if t.IsZero() { - return nil, nil - } + if fieldType.ConvertibleTo(core.TimeType) { + t := fieldValue.Convert(core.TimeType).Interface().(time.Time) + if session.Engine.dialect.DBType() == core.MSSQL { + if t.IsZero() { + return nil, nil } - tf := session.Engine.FormatTime(col.SQLType.Name, t) - return tf, nil - default: - return fieldValue.Interface(), nil } + tf := session.Engine.FormatTime(col.SQLType.Name, t) + return tf, nil } - if fieldTable, ok := session.Engine.Tables[fieldValue.Type()]; ok { + + if !col.SQLType.IsJson() { + // !! 增加支持driver.Valuer接口的结构，如sql.NullString + if v, ok := fieldValue.Interface().(driver.Valuer); ok { + return v.Value() + } + + fieldTable := session.Engine.autoMapType(fieldValue) if len(fieldTable.PrimaryKeys) == 1 { pkField := reflect.Indirect(fieldValue).FieldByName(fieldTable.PKColumns()[0].FieldName) return pkField.Interface(), nil - } else { - return 0, fmt.Errorf("no primary key for col %v", col.Name) } - } else { - return 0, fmt.Errorf("Unsupported type %v\n", fieldValue.Type()) + return 0, fmt.Errorf("no primary key for col %v", col.Name) } + + if col.SQLType.IsText() { + bytes, err := json.Marshal(fieldValue.Interface()) + if err != nil { + session.Engine.LogError(err) + return 0, err + } + return string(bytes), nil + } else if col.SQLType.IsBlob() { + bytes, err := json.Marshal(fieldValue.Interface()) + if err != nil { + session.Engine.LogError(err) + return 0, err + } + return bytes, nil + } + return nil, fmt.Errorf("Unsupported type %v", fieldValue.Type()) case reflect.Complex64, reflect.Complex128: bytes, err := json.Marshal(fieldValue.Interface()) if err != nil { @@ -2950,9 +3133,8 @@ func (session *Session) value2Interface(col *core.Column, fieldValue reflect.Val } } return bytes, nil - } else { - return nil, ErrUnSupportedType } + return nil, ErrUnSupportedType case reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint: return int64(fieldValue.Uint()), nil default: @@ -2974,12 +3156,10 @@ func (session *Session) innerInsert(bean interface{}) (int64, error) { processor.BeforeInsert() } // -- - colNames, args, err := genCols(table, session, bean, false, false) if err != nil { return 0, err } - // insert expr columns, override if exists exprColumns := session.Statement.getExpr() exprColVals := make([]string, 0, len(exprColumns)) @@ -3044,9 +3224,10 @@ func (session *Session) innerInsert(bean interface{}) (int64, error) { // for postgres, many of them didn't implement lastInsertId, so we should // implemented it ourself. + if session.Engine.dialect.DBType() == core.ORACLE && len(table.AutoIncrement) > 0 { + //assert table.AutoIncrement != "" + res, err := session.query("select seq_atable.currval from dual", args...) - if session.Engine.DriverName() != core.POSTGRES || table.AutoIncrement == "" { - res, err := session.exec(sqlStr, args...) if err != nil { return 0, err } else { @@ -3066,14 +3247,14 @@ func (session *Session) innerInsert(bean interface{}) (int64, error) { } } - if table.AutoIncrement == "" { - return res.RowsAffected() + if len(res) < 1 { + return 0, errors.New("insert no error but not returned id") } - var id int64 = 0 - id, err = res.LastInsertId() - if err != nil || id <= 0 { - return res.RowsAffected() + idByte := res[0][table.AutoIncrement] + id, err := strconv.ParseInt(string(idByte), 10, 64) + if err != nil { + return 1, err } aiValue, err := table.AutoIncrColumn().ValueOf(bean) @@ -3081,27 +3262,15 @@ func (session *Session) innerInsert(bean interface{}) (int64, error) { session.Engine.LogError(err) } - if aiValue == nil || !aiValue.IsValid() /*|| aiValue.Int() != 0*/ || !aiValue.CanSet() { - return res.RowsAffected() + if aiValue == nil || !aiValue.IsValid() /*|| aiValue. != 0*/ || !aiValue.CanSet() { + return 1, nil } - var v interface{} = id - switch aiValue.Type().Kind() { - case reflect.Int32: - v = int32(id) - case reflect.Int: - v = int(id) - case reflect.Uint32: - v = uint32(id) - case reflect.Uint64: - v = uint64(id) - case reflect.Uint: - v = uint(id) - } + v := int64ToInt(id, aiValue.Type().Kind()) aiValue.Set(reflect.ValueOf(v)) - return res.RowsAffected() - } else { + return 1, nil + } else if session.Engine.dialect.DBType() == core.POSTGRES && len(table.AutoIncrement) > 0 { //assert table.AutoIncrement != "" sqlStr = sqlStr + " RETURNING " + session.Engine.Quote(table.AutoIncrement) res, err := session.query(sqlStr, args...) @@ -3144,22 +3313,54 @@ func (session *Session) innerInsert(bean interface{}) (int64, error) { return 1, nil } - var v interface{} = id - switch aiValue.Type().Kind() { - case reflect.Int32: - v = int32(id) - case reflect.Int: - v = int(id) - case reflect.Uint32: - v = uint32(id) - case reflect.Uint64: - v = uint64(id) - case reflect.Uint: - v = uint(id) - } + v := int64ToInt(id, aiValue.Type().Kind()) aiValue.Set(reflect.ValueOf(v)) return 1, nil + } else { + res, err := session.exec(sqlStr, args...) + if err != nil { + return 0, err + } else { + handleAfterInsertProcessorFunc(bean) + } + + if cacher := session.Engine.getCacher2(table); cacher != nil && session.Statement.UseCache { + session.cacheInsert(session.Statement.TableName()) + } + + if table.Version != "" && session.Statement.checkVersion { + verValue, err := table.VersionColumn().ValueOf(bean) + if err != nil { + session.Engine.LogError(err) + } else if verValue.IsValid() && verValue.CanSet() { + verValue.SetInt(1) + } + } + + if table.AutoIncrement == "" { + return res.RowsAffected() + } + + var id int64 = 0 + id, err = res.LastInsertId() + if err != nil || id <= 0 { + return res.RowsAffected() + } + + aiValue, err := table.AutoIncrColumn().ValueOf(bean) + if err != nil { + session.Engine.LogError(err) + } + + if aiValue == nil || !aiValue.IsValid() /*|| aiValue.Int() != 0*/ || !aiValue.CanSet() { + return res.RowsAffected() + } + + v := int64ToInt(id, aiValue.Type().Kind()) + aiValue.Set(reflect.ValueOf(v)) + + return res.RowsAffected() } } @@ -3180,7 +3381,7 @@ func (statement *Statement) convertUpdateSql(sqlStr string) (string, string) { return "", "" } - colstrs := statement.JoinColumns(statement.RefTable.PKColumns()) + colstrs := statement.JoinColumns(statement.RefTable.PKColumns(), true) sqls := splitNNoCase(sqlStr, "where", 2) if len(sqls) != 2 { if len(sqls) == 1 { @@ -3233,7 +3434,8 @@ func (session *Session) cacheInsert(tables ...string) error { } func (session *Session) cacheUpdate(sqlStr string, args ...interface{}) error { - if session.Statement.RefTable == nil || len(session.Statement.RefTable.PrimaryKeys) != 1 { + if session.Statement.RefTable == nil || + session.Tx != nil { return ErrCacheFailed } @@ -3382,21 +3584,24 @@ func (session *Session) Update(bean interface{}, condiBean ...interface{}) (int6 // -- var err error - if t.Kind() == reflect.Struct { + var isMap = t.Kind() == reflect.Map + var isStruct = t.Kind() == reflect.Struct + if isStruct { table = session.Engine.TableInfo(bean) session.Statement.RefTable = table if session.Statement.ColumnStr == "" { colNames, args = buildUpdates(session.Engine, table, bean, false, false, false, false, session.Statement.allUseBool, session.Statement.useAllCols, - session.Statement.mustColumnMap, session.Statement.columnMap, true) + session.Statement.mustColumnMap, session.Statement.nullableMap, + session.Statement.columnMap, true, session.Statement.unscoped) } else { colNames, args, err = genCols(table, session, bean, true, true) if err != nil { return 0, err } } - } else if t.Kind() == reflect.Map { + } else if isMap { if session.Statement.RefTable == nil { return 0, ErrTableNotFound } @@ -3420,10 +3625,12 @@ func (session *Session) Update(bean interface{}, condiBean ...interface{}) (int6 args = append(args, val) var colName = col.Name - session.afterClosures = append(session.afterClosures, func(bean interface{}) { - col := table.GetColumn(colName) - setColumnTime(bean, col, t) - }) + if isStruct { + session.afterClosures = append(session.afterClosures, func(bean interface{}) { + col := table.GetColumn(colName) + setColumnTime(bean, col, t) + }) + } } //for update action to like "column = column + ?" @@ -3447,10 +3654,8 @@ func (session *Session) Update(bean interface{}, condiBean ...interface{}) (int6 var condiColNames []string var condiArgs []interface{} - if len(condiBean) > 0 { - condiColNames, condiArgs = buildConditions(session.Engine, session.Statement.RefTable, condiBean[0], true, true, - false, true, session.Statement.allUseBool, session.Statement.useAllCols, - session.Statement.unscoped, session.Statement.mustColumnMap) + if !session.Statement.noAutoCondition && len(condiBean) > 0 { + condiColNames, condiArgs = session.Statement.buildConditions(session.Statement.RefTable, condiBean[0], true, true, false, true, false) } var condition = "" @@ -3567,6 +3772,7 @@ func (session *Session) Update(bean interface{}, condiBean ...interface{}) (int6 } else { afterClosures := make([]func(interface{}), lenAfterClosures) copy(afterClosures, session.afterClosures) + // FIXME: if bean is a map type, it will panic because map cannot be as map key session.afterUpdateBeans[bean] = &afterClosures } @@ -3583,7 +3789,8 @@ func (session *Session) Update(bean interface{}, condiBean ...interface{}) (int6 } func (session *Session) cacheDelete(sqlStr string, args ...interface{}) error { - if session.Statement.RefTable == nil || len(session.Statement.RefTable.PrimaryKeys) != 1 { + if session.Statement.RefTable == nil || + session.Tx != nil { return ErrCacheFailed } @@ -3608,15 +3815,25 @@ func (session *Session) cacheDelete(sqlStr string, args ...interface{}) error { if len(resultsSlice) > 0 { for _, data := range resultsSlice { var id int64 - if v, ok := data[session.Statement.RefTable.PrimaryKeys[0]]; !ok { - return errors.New("no id") - } else { - id, err = strconv.ParseInt(string(v), 10, 64) - if err != nil { - return err + var pk core.PK = make([]interface{}, 0) + for _, col := range session.Statement.RefTable.PKColumns() { + if v, ok := data[col.Name]; !ok { + return errors.New("no id") + } else { + if col.SQLType.IsText() { + pk = append(pk, string(v)) + } else if col.SQLType.IsNumeric() { + id, err = strconv.ParseInt(string(v), 10, 64) + if err != nil { + return err + } + pk = append(pk, id) + } else { + return errors.New("not supported primary key type") + } } } - ids = append(ids, core.PK{id}) + ids = append(ids, pk) } } } /*else { @@ -3657,10 +3874,12 @@ func (session *Session) Delete(bean interface{}) (int64, error) { table := session.Engine.TableInfo(bean) session.Statement.RefTable = table - colNames, args := buildConditions(session.Engine, table, bean, true, true, - false, true, session.Statement.allUseBool, session.Statement.useAllCols, - session.Statement.unscoped, session.Statement.mustColumnMap) + var colNames []string + var args []interface{} + if !session.Statement.noAutoCondition { + colNames, args = session.Statement.buildConditions(table, bean, true, true, false, true, false) + } var condition = "" var andStr = session.Engine.dialect.AndStr() @@ -3681,32 +3900,91 @@ func (session *Session) Delete(bean interface{}) (int64, error) { condition += inSql args = append(args, inArgs...) } - if len(condition) == 0 { + if len(condition) == 0 && session.Statement.LimitN == 0 { return 0, ErrNeedDeletedCond } - sqlStr, sqlStrForCache := "", "" + var deleteSql, realSql string + var tableName = session.Engine.Quote(session.Statement.TableName()) + + if len(condition) > 0 { + deleteSql = fmt.Sprintf("DELETE FROM %v WHERE %v", tableName, condition) + } else { + deleteSql = fmt.Sprintf("DELETE FROM %v", tableName) + } + + var orderSql string + if len(session.Statement.OrderStr) > 0 { + orderSql += fmt.Sprintf(" ORDER BY %s", session.Statement.OrderStr) + } + if session.Statement.LimitN > 0 { + orderSql += fmt.Sprintf(" LIMIT %d", session.Statement.LimitN) + } + + if len(orderSql) > 0 { + switch session.Engine.dialect.DBType() { + case core.POSTGRES: + inSql := fmt.Sprintf("ctid IN (SELECT ctid FROM %s%s)", tableName, orderSql) + if len(condition) > 0 { + deleteSql += " AND " + inSql + } else { + deleteSql += " WHERE " + inSql + } + case core.SQLITE: + inSql := fmt.Sprintf("rowid IN (SELECT rowid FROM %s%s)", tableName, orderSql) + if len(condition) > 0 { + deleteSql += " AND " + inSql + } else { + deleteSql += " WHERE " + inSql + } + // TODO: how to handle delete limit on mssql? + case core.MSSQL: + return 0, ErrNotImplemented + default: + deleteSql += orderSql + } + } + argsForCache := make([]interface{}, 0, len(args)*2) if session.Statement.unscoped || table.DeletedColumn() == nil { // tag "deleted" is disabled - sqlStr = fmt.Sprintf("DELETE FROM %v WHERE %v", - session.Engine.Quote(session.Statement.TableName()), condition) - - sqlStrForCache = sqlStr + realSql = deleteSql copy(argsForCache, args) argsForCache = append(session.Statement.Params, argsForCache...) } else { // !oinume! sqlStrForCache and argsForCache is needed to behave as executing "DELETE FROM ..." for cache. - sqlStrForCache = fmt.Sprintf("DELETE FROM %v WHERE %v", - session.Engine.Quote(session.Statement.TableName()), condition) copy(argsForCache, args) argsForCache = append(session.Statement.Params, argsForCache...) deletedColumn := table.DeletedColumn() - sqlStr = fmt.Sprintf("UPDATE %v SET %v = ? WHERE %v", + realSql = fmt.Sprintf("UPDATE %v SET %v = ? WHERE %v", session.Engine.Quote(session.Statement.TableName()), session.Engine.Quote(deletedColumn.Name), condition) + if len(orderSql) > 0 { + switch session.Engine.dialect.DBType() { + case core.POSTGRES: + inSql := fmt.Sprintf("ctid IN (SELECT ctid FROM %s%s)", tableName, orderSql) + if len(condition) > 0 { + realSql += " AND " + inSql + } else { + realSql += " WHERE " + inSql + } + case core.SQLITE: + inSql := fmt.Sprintf("rowid IN (SELECT rowid FROM %s%s)", tableName, orderSql) + if len(condition) > 0 { + realSql += " AND " + inSql + } else { + realSql += " WHERE " + inSql + } + // TODO: how to handle delete limit on mssql? + case core.MSSQL: + return 0, ErrNotImplemented + default: + realSql += orderSql + } + } + // !oinume! Insert NowTime to the head of session.Statement.Params session.Statement.Params = append(session.Statement.Params, "") paramsLen := len(session.Statement.Params) @@ -3725,10 +4003,10 @@ func (session *Session) Delete(bean interface{}) (int64, error) { args = append(session.Statement.Params, args...) if cacher := session.Engine.getCacher2(session.Statement.RefTable); cacher != nil && session.Statement.UseCache { - session.cacheDelete(sqlStrForCache, argsForCache...) + session.cacheDelete(deleteSql, argsForCache...) } - res, err := session.exec(sqlStr, args...) + res, err := session.exec(realSql, args...) if err != nil { return 0, err } @@ -3763,6 +4041,18 @@ func (session *Session) Delete(bean interface{}) (int64, error) { return res.RowsAffected() } +// saveLastSQL stores executed query information +func (session *Session) saveLastSQL(sql string, args ...interface{}) { + session.lastSQL = sql + session.lastSQLArgs = args + session.Engine.logSQL(sql, args...) +} + +// LastSQL returns last query information +func (session *Session) LastSQL() (string, []interface{}) { + return session.lastSQL, session.lastSQLArgs +} + func (s *Session) Sync2(beans ...interface{}) error { engine := s.Engine @@ -3779,7 +4069,7 @@ func (s *Session) Sync2(beans ...interface{}) error { var oriTable *core.Table for _, tb := range tables { - if tb.Name == table.Name { + if equalNoCase(tb.Name, table.Name) { oriTable = tb break } @@ -3804,7 +4094,7 @@ func (s *Session) Sync2(beans ...interface{}) error { for _, col := range table.Columns() { var oriCol *core.Column for _, col2 := range oriTable.Columns() { - if col.Name == col2.Name { + if equalNoCase(col.Name, col2.Name) { oriCol = col2 break } @@ -3826,10 +4116,26 @@ func (s *Session) Sync2(beans ...interface{}) error { engine.LogWarnf("Table %s column %s db type is %s, struct type is %s\n", table.Name, col.Name, curType, expectedType) } + } else if strings.HasPrefix(curType, core.Varchar) && strings.HasPrefix(expectedType, core.Varchar) { + if engine.dialect.DBType() == core.MYSQL { + if oriCol.Length < col.Length { + engine.LogInfof("Table %s column %s change type from varchar(%d) to varchar(%d)\n", + table.Name, col.Name, oriCol.Length, col.Length) + _, err = engine.Exec(engine.dialect.ModifyColumnSql(table.Name, col)) + } + } } else { engine.LogWarnf("Table %s column %s db type is %s, struct type is %s", table.Name, col.Name, curType, expectedType) } + } else if expectedType == core.Varchar { + if engine.dialect.DBType() == core.MYSQL { + if oriCol.Length < col.Length { + engine.LogInfof("Table %s column %s change type from varchar(%d) to varchar(%d)\n", + table.Name, col.Name, oriCol.Length, col.Length) + _, err = engine.Exec(engine.dialect.ModifyColumnSql(table.Name, col)) + } + } } if col.Default != oriCol.Default { engine.LogWarnf("Table %s Column %s db default is %s, struct default is %s", @@ -3851,6 +4157,7 @@ func (s *Session) Sync2(beans ...interface{}) error { } var foundIndexNames = make(map[string]bool) + var addedNames = make(map[string]*core.Index) for name, index := range table.Indexes { var oriIndex *core.Index @@ -3874,20 +4181,7 @@ func (s *Session) Sync2(beans ...interface{}) error { } if oriIndex == nil { - if index.Type == core.UniqueType { - session := engine.NewSession() - session.Statement.RefTable = table - defer session.Close() - err = session.addUnique(table.Name, name) - } else if index.Type == core.IndexType { - session := engine.NewSession() - session.Statement.RefTable = table - defer session.Close() - err = session.addIndex(table.Name, name) - } - if err != nil { - return err - } + addedNames[name] = index } } @@ -3900,13 +4194,30 @@ func (s *Session) Sync2(beans ...interface{}) error { } } } + + for name, index := range addedNames { + if index.Type == core.UniqueType { + session := engine.NewSession() + session.Statement.RefTable = table + defer session.Close() + err = session.addUnique(table.Name, name) + } else if index.Type == core.IndexType { + session := engine.NewSession() + session.Statement.RefTable = table + defer session.Close() + err = session.addIndex(table.Name, name) + } + if err != nil { + return err + } + } } } for _, table := range tables { var oriTable *core.Table for _, structTable := range structTables { - if table.Name == structTable.Name { + if equalNoCase(table.Name, structTable.Name) { oriTable = structTable break } diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/sqlite3_dialect.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/sqlite3_dialect.go index cb9e7f54a78..9f29d587cdb 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/sqlite3_dialect.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/sqlite3_dialect.go @@ -1,9 +1,14 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( "database/sql" "errors" "fmt" + "regexp" "strings" "github.com/go-xorm/core" @@ -152,13 +157,21 @@ func (db *sqlite3) Init(d *core.DB, uri *core.Uri, drivername, dataSourceName st func (db *sqlite3) SqlType(c *core.Column) string { switch t := c.SQLType.Name; t { + case core.Bool: + if c.Default == "true" { + c.Default = "1" + } else if c.Default == "false" { + c.Default = "0" + } + return core.Integer case core.Date, core.DateTime, core.TimeStamp, core.Time: return core.DateTime case core.TimeStampz: return core.Text - case core.Char, core.Varchar, core.NVarchar, core.TinyText, core.Text, core.MediumText, core.LongText: + case core.Char, core.Varchar, core.NVarchar, core.TinyText, + core.Text, core.MediumText, core.LongText, core.Json: return core.Text - case core.Bit, core.TinyInt, core.SmallInt, core.MediumInt, core.Int, core.Integer, core.BigInt, core.Bool: + case core.Bit, core.TinyInt, core.SmallInt, core.MediumInt, core.Int, core.Integer, core.BigInt: return core.Integer case core.Float, core.Double, core.Real: return core.Real @@ -238,15 +251,19 @@ func (db *sqlite3) DropIndexSql(tableName string, index *core.Index) string { return fmt.Sprintf("DROP INDEX %v", quote(idxName)) } +func (db *sqlite3) ForUpdateSql(query string) string { + return query +} + /*func (db *sqlite3) ColumnCheckSql(tableName, colName string) (string, []interface{}) { args := []interface{}{tableName} sql := "SELECT name FROM sqlite_master WHERE type='table' and name = ? and ((sql like '%`" + colName + "`%') or (sql like '%[" + colName + "]%'))" return sql, args }*/ -func (db *sqlite3) IsColumnExist(tableName string, col *core.Column) (bool, error) { +func (db *sqlite3) IsColumnExist(tableName, colName string) (bool, error) { args := []interface{}{tableName} - query := "SELECT name FROM sqlite_master WHERE type='table' and name = ? and ((sql like '%`" + col.Name + "`%') or (sql like '%[" + col.Name + "]%'))" + query := "SELECT name FROM sqlite_master WHERE type='table' and name = ? and ((sql like '%`" + colName + "`%') or (sql like '%[" + colName + "]%'))" rows, err := db.DB().Query(query, args...) if db.Logger != nil { db.Logger.Info("[sql]", query, args) @@ -290,10 +307,13 @@ func (db *sqlite3) GetColumns(tableName string) ([]string, map[string]*core.Colu nStart := strings.Index(name, "(") nEnd := strings.LastIndex(name, ")") - colCreates := strings.Split(name[nStart+1:nEnd], ",") + reg := regexp.MustCompile(`[^\(,\)]*(\([^\(]*\))?`) + colCreates := reg.FindAllString(name[nStart+1:nEnd], -1) cols := make(map[string]*core.Column) colSeq := make([]string, 0) for _, colStr := range colCreates { + reg = regexp.MustCompile(`,\s`) + colStr = reg.ReplaceAllString(colStr, ",") fields := strings.Fields(strings.TrimSpace(colStr)) col := new(core.Column) col.Indexes = make(map[string]bool) diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/sqlite3_driver.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/sqlite3_driver.go index 2ecd9edf5d5..6ae19569ef8 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/sqlite3_driver.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/sqlite3_driver.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/statement.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/statement.go index b8f859714ea..9b9042fa9ad 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/statement.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/statement.go @@ -1,6 +1,12 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( + "bytes" + "database/sql/driver" "encoding/json" "errors" "fmt" @@ -33,42 +39,46 @@ type exprParam struct { // statement save all the sql info for executing SQL type Statement struct { - RefTable *core.Table - Engine *Engine - Start int - LimitN int - WhereStr string - IdParam *core.PK - Params []interface{} - OrderStr string - JoinStr string - GroupByStr string - HavingStr string - ColumnStr string - columnMap map[string]bool - useAllCols bool - OmitStr string - ConditionStr string - AltTableName string - RawSQL string - RawParams []interface{} - UseCascade bool - UseAutoJoin bool - StoreEngine string - Charset string - BeanArgs []interface{} - UseCache bool - UseAutoTime bool - IsDistinct bool - TableAlias string - allUseBool bool - checkVersion bool - unscoped bool - mustColumnMap map[string]bool - inColumns map[string]*inParam - incrColumns map[string]incrParam - decrColumns map[string]decrParam - exprColumns map[string]exprParam + RefTable *core.Table + Engine *Engine + Start int + LimitN int + WhereStr string + IdParam *core.PK + Params []interface{} + OrderStr string + JoinStr string + GroupByStr string + HavingStr string + ColumnStr string + selectStr string + columnMap map[string]bool + useAllCols bool + OmitStr string + ConditionStr string + AltTableName string + RawSQL string + RawParams []interface{} + UseCascade bool + UseAutoJoin bool + StoreEngine string + Charset string + BeanArgs []interface{} + UseCache bool + UseAutoTime bool + noAutoCondition bool + IsDistinct bool + IsForUpdate bool + TableAlias string + allUseBool bool + checkVersion bool + unscoped bool + mustColumnMap map[string]bool + nullableMap map[string]bool + inColumns map[string]*inParam + incrColumns map[string]incrParam + decrColumns map[string]decrParam + exprColumns map[string]exprParam } // init @@ -94,11 +104,15 @@ func (statement *Statement) Init() { statement.BeanArgs = make([]interface{}, 0) statement.UseCache = true statement.UseAutoTime = true + statement.noAutoCondition = false statement.IsDistinct = false + statement.IsForUpdate = false statement.TableAlias = "" + statement.selectStr = "" statement.allUseBool = false statement.useAllCols = false statement.mustColumnMap = make(map[string]bool) + statement.nullableMap = make(map[string]bool) statement.checkVersion = true statement.unscoped = false statement.inColumns = make(map[string]*inParam) @@ -107,20 +121,29 @@ func (statement *Statement) Init() { statement.exprColumns = make(map[string]exprParam) } -// add the raw sql statement +// NoAutoCondition if you do not want convert bean's field as query condition, then use this function +func (statement *Statement) NoAutoCondition(no ...bool) *Statement { + statement.noAutoCondition = true + if len(no) > 0 { + statement.noAutoCondition = no[0] + } + return statement +} + +// Sql add the raw sql statement func (statement *Statement) Sql(querystring string, args ...interface{}) *Statement { statement.RawSQL = querystring statement.RawParams = args return statement } -// set the table alias +// Alias set the table alias func (statement *Statement) Alias(alias string) *Statement { statement.TableAlias = alias return statement } -// add Where statment +// Where add Where statment func (statement *Statement) Where(querystring string, args ...interface{}) *Statement { if !strings.Contains(querystring, statement.Engine.dialect.EqStr()) { querystring = strings.Replace(querystring, "=", statement.Engine.dialect.EqStr(), -1) @@ -130,11 +153,13 @@ func (statement *Statement) Where(querystring string, args ...interface{}) *Stat return statement } -// add Where & and statment +// And add Where & and statment func (statement *Statement) And(querystring string, args ...interface{}) *Statement { - if statement.WhereStr != "" { - statement.WhereStr = fmt.Sprintf("(%v) %s (%v)", statement.WhereStr, + if len(statement.WhereStr) > 0 { + var buf bytes.Buffer + fmt.Fprintf(&buf, "(%v) %s (%v)", statement.WhereStr, statement.Engine.dialect.AndStr(), querystring) + statement.WhereStr = buf.String() } else { statement.WhereStr = querystring } @@ -142,11 +167,13 @@ func (statement *Statement) And(querystring string, args ...interface{}) *Statem return statement } -// add Where & Or statment +// Or add Where & Or statment func (statement *Statement) Or(querystring string, args ...interface{}) *Statement { - if statement.WhereStr != "" { - statement.WhereStr = fmt.Sprintf("(%v) %s (%v)", statement.WhereStr, + if len(statement.WhereStr) > 0 { + var buf bytes.Buffer + fmt.Fprintf(&buf, "(%v) %s (%v)", statement.WhereStr, statement.Engine.dialect.OrStr(), querystring) + statement.WhereStr = buf.String() } else { statement.WhereStr = querystring } @@ -154,7 +181,7 @@ func (statement *Statement) Or(querystring string, args ...interface{}) *Stateme return statement } -// tempororily set table name +// Table tempororily set table name, the parameter could be a string or a pointer of struct func (statement *Statement) Table(tableNameOrBean interface{}) *Statement { v := rValue(tableNameOrBean) t := v.Type() @@ -166,127 +193,12 @@ func (statement *Statement) Table(tableNameOrBean interface{}) *Statement { return statement } -/*func (statement *Statement) genFields(bean interface{}) map[string]interface{} { - results := make(map[string]interface{}) - table := statement.Engine.TableInfo(bean) - for _, col := range table.Columns { - fieldValue := col.ValueOf(bean) - fieldType := reflect.TypeOf(fieldValue.Interface()) - var val interface{} - switch fieldType.Kind() { - case reflect.Bool: - if allUseBool { - val = fieldValue.Interface() - } else if _, ok := boolColumnMap[col.Name]; ok { - val = fieldValue.Interface() - } else { - // if a bool in a struct, it will not be as a condition because it default is false, - // please use Where() instead - continue - } - case reflect.String: - if fieldValue.String() == "" { - continue - } - // for MyString, should convert to string or panic - if fieldType.String() != reflect.String.String() { - val = fieldValue.String() - } else { - val = fieldValue.Interface() - } - case reflect.Int8, reflect.Int16, reflect.Int, reflect.Int32, reflect.Int64: - if fieldValue.Int() == 0 { - continue - } - val = fieldValue.Interface() - case reflect.Float32, reflect.Float64: - if fieldValue.Float() == 0.0 { - continue - } - val = fieldValue.Interface() - case reflect.Uint8, reflect.Uint16, reflect.Uint, reflect.Uint32, reflect.Uint64: - if fieldValue.Uint() == 0 { - continue - } - val = fieldValue.Interface() - case reflect.Struct: - if fieldType == reflect.TypeOf(time.Now()) { - t := fieldValue.Interface().(time.Time) - if t.IsZero() || !fieldValue.IsValid() { - continue - } - var str string - if col.SQLType.Name == Time { - s := t.UTC().Format("2006-01-02 15:04:05") - val = s[11:19] - } else if col.SQLType.Name == Date { - str = t.Format("2006-01-02") - val = str - } else { - val = t - } - } else { - engine.autoMapType(fieldValue.Type()) - if table, ok := engine.Tables[fieldValue.Type()]; ok { - pkField := reflect.Indirect(fieldValue).FieldByName(table.PKColumn().FieldName) - if pkField.Int() != 0 { - val = pkField.Interface() - } else { - continue - } - } else { - val = fieldValue.Interface() - } - } - case reflect.Array, reflect.Slice, reflect.Map: - if fieldValue == reflect.Zero(fieldType) { - continue - } - if fieldValue.IsNil() || !fieldValue.IsValid() { - continue - } - - if col.SQLType.IsText() { - bytes, err := json.Marshal(fieldValue.Interface()) - if err != nil { - engine.LogError(err) - continue - } - val = string(bytes) - } else if col.SQLType.IsBlob() { - var bytes []byte - var err error - if (fieldType.Kind() == reflect.Array || fieldType.Kind() == reflect.Slice) && - fieldType.Elem().Kind() == reflect.Uint8 { - if fieldValue.Len() > 0 { - val = fieldValue.Bytes() - } else { - continue - } - } else { - bytes, err = json.Marshal(fieldValue.Interface()) - if err != nil { - engine.LogError(err) - continue - } - val = bytes - } - } else { - continue - } - default: - val = fieldValue.Interface() - } - results[col.Name] = val - } - return results -}*/ - -// Auto generating conditions according a struct +// Auto generating update columnes and values according a struct func buildUpdates(engine *Engine, table *core.Table, bean interface{}, includeVersion bool, includeUpdated bool, includeNil bool, includeAutoIncr bool, allUseBool bool, useAllCols bool, - mustColumnMap map[string]bool, columnMap map[string]bool, update bool) ([]string, []interface{}) { + mustColumnMap map[string]bool, nullableMap map[string]bool, + columnMap map[string]bool, update, unscoped bool) ([]string, []interface{}) { colNames := make([]string, 0) var args = make([]interface{}, 0) @@ -303,17 +215,13 @@ func buildUpdates(engine *Engine, table *core.Table, bean interface{}, if !includeAutoIncr && col.IsAutoIncrement { continue } - if col.IsDeleted { + if col.IsDeleted && !unscoped { continue } if use, ok := columnMap[col.Name]; ok && !use { continue } - if engine.dialect.DBType() == core.MSSQL && col.SQLType.Name == core.Text { - continue - } - fieldValuePtr, err := col.ValueOf(bean) if err != nil { engine.LogError(err) @@ -325,7 +233,9 @@ func buildUpdates(engine *Engine, table *core.Table, bean interface{}, requiredField := useAllCols includeNil := useAllCols - if b, ok := mustColumnMap[strings.ToLower(col.Name)]; ok { + lColName := strings.ToLower(col.Name) + + if b, ok := mustColumnMap[lColName]; ok { if b { requiredField = true } else { @@ -333,6 +243,16 @@ func buildUpdates(engine *Engine, table *core.Table, bean interface{}, } } + // !evalphobia! set fieldValue as nil when column is nullable and zero-value + if b, ok := nullableMap[lColName]; ok { + if b && col.Nullable && isZero(fieldValue.Interface()) { + var nilValue *int + fieldValue = reflect.ValueOf(nilValue) + fieldType = reflect.TypeOf(fieldValue.Interface()) + includeNil = true + } + } + var val interface{} if fieldValue.CanAddr() { @@ -410,38 +330,53 @@ func buildUpdates(engine *Engine, table *core.Table, bean interface{}, t := int64(fieldValue.Uint()) val = reflect.ValueOf(&t).Interface() case reflect.Struct: - if fieldType == reflect.TypeOf(time.Now()) { - t := fieldValue.Interface().(time.Time) + if fieldType.ConvertibleTo(core.TimeType) { + t := fieldValue.Convert(core.TimeType).Interface().(time.Time) if !requiredField && (t.IsZero() || !fieldValue.IsValid()) { continue } val = engine.FormatTime(col.SQLType.Name, t) + } else if nulType, ok := fieldValue.Interface().(driver.Valuer); ok { + val, _ = nulType.Value() } else { - engine.autoMapType(fieldValue) - if table, ok := engine.Tables[fieldValue.Type()]; ok { - if len(table.PrimaryKeys) == 1 { - pkField := reflect.Indirect(fieldValue).FieldByName(table.PKColumns()[0].FieldName) - // fix non-int pk issues - //if pkField.Int() != 0 { - if pkField.IsValid() && !isZero(pkField.Interface()) { - val = pkField.Interface() + if !col.SQLType.IsJson() { + engine.autoMapType(fieldValue) + if table, ok := engine.Tables[fieldValue.Type()]; ok { + if len(table.PrimaryKeys) == 1 { + pkField := reflect.Indirect(fieldValue).FieldByName(table.PKColumns()[0].FieldName) + // fix non-int pk issues + if pkField.IsValid() && !isZero(pkField.Interface()) { + val = pkField.Interface() + } else { + continue + } } else { - continue + //TODO: how to handler? + panic("not supported") } } else { - //TODO: how to handler? - panic("not supported") + val = fieldValue.Interface() } } else { - val = fieldValue.Interface() + bytes, err := json.Marshal(fieldValue.Interface()) + if err != nil { + panic(fmt.Sprintf("mashal %v failed", fieldValue.Interface())) + } + if col.SQLType.IsText() { + val = string(bytes) + } else if col.SQLType.IsBlob() { + val = bytes + } } } case reflect.Array, reflect.Slice, reflect.Map: - if fieldValue == reflect.Zero(fieldType) { - continue - } - if fieldValue.IsNil() || !fieldValue.IsValid() || fieldValue.Len() == 0 { - continue + if !requiredField { + if fieldValue == reflect.Zero(fieldType) { + continue + } + if fieldValue.IsNil() || !fieldValue.IsValid() || fieldValue.Len() == 0 { + continue + } } if col.SQLType.IsText() { @@ -492,8 +427,7 @@ func buildUpdates(engine *Engine, table *core.Table, bean interface{}, func buildConditions(engine *Engine, table *core.Table, bean interface{}, includeVersion bool, includeUpdated bool, includeNil bool, includeAutoIncr bool, allUseBool bool, useAllCols bool, unscoped bool, - mustColumnMap map[string]bool) ([]string, []interface{}) { - + mustColumnMap map[string]bool, tableName, aliasName string, addedTableName bool) ([]string, []interface{}) { colNames := make([]string, 0) var args = make([]interface{}, 0) for _, col := range table.Columns() { @@ -510,6 +444,21 @@ func buildConditions(engine *Engine, table *core.Table, bean interface{}, if engine.dialect.DBType() == core.MSSQL && col.SQLType.Name == core.Text { continue } + if col.SQLType.IsJson() { + continue + } + + var colName string + if addedTableName { + var nm = tableName + if len(aliasName) > 0 { + nm = aliasName + } + colName = engine.Quote(nm) + "." + engine.Quote(col.Name) + } else { + colName = engine.Quote(col.Name) + } + fieldValuePtr, err := col.ValueOf(bean) if err != nil { engine.LogError(err) @@ -517,7 +466,8 @@ func buildConditions(engine *Engine, table *core.Table, bean interface{}, } if col.IsDeleted && !unscoped { // tag "deleted" is enabled - colNames = append(colNames, fmt.Sprintf("(%v IS NULL or %v = '0001-01-01 00:00:00')", engine.Quote(col.Name), engine.Quote(col.Name))) + colNames = append(colNames, fmt.Sprintf("%v IS NULL or %v = '0001-01-01 00:00:00'", + colName, colName)) } fieldValue := *fieldValuePtr @@ -539,7 +489,7 @@ func buildConditions(engine *Engine, table *core.Table, bean interface{}, if fieldValue.IsNil() { if includeNil { args = append(args, nil) - colNames = append(colNames, fmt.Sprintf("%v %s ?", engine.Quote(col.Name), engine.dialect.EqStr())) + colNames = append(colNames, fmt.Sprintf("%v %s ?", colName, engine.dialect.EqStr())) } continue } else if !fieldValue.IsValid() { @@ -597,24 +547,49 @@ func buildConditions(engine *Engine, table *core.Table, bean interface{}, val = engine.FormatTime(col.SQLType.Name, t) } else if _, ok := reflect.New(fieldType).Interface().(core.Conversion); ok { continue + } else if valNul, ok := fieldValue.Interface().(driver.Valuer); ok { + val, _ = valNul.Value() + if val == nil { + continue + } } else { - engine.autoMapType(fieldValue) - if table, ok := engine.Tables[fieldValue.Type()]; ok { - if len(table.PrimaryKeys) == 1 { - pkField := reflect.Indirect(fieldValue).FieldByName(table.PKColumns()[0].FieldName) - // fix non-int pk issues - //if pkField.Int() != 0 { - if pkField.IsValid() && !isZero(pkField.Interface()) { - val = pkField.Interface() - } else { + if col.SQLType.IsJson() { + if col.SQLType.IsText() { + bytes, err := json.Marshal(fieldValue.Interface()) + if err != nil { + engine.LogError(err) continue } - } else { - //TODO: how to handler? - panic("not supported") + val = string(bytes) + } else if col.SQLType.IsBlob() { + var bytes []byte + var err error + bytes, err = json.Marshal(fieldValue.Interface()) + if err != nil { + engine.LogError(err) + continue + } + val = bytes } } else { - val = fieldValue.Interface() + engine.autoMapType(fieldValue) + if table, ok := engine.Tables[fieldValue.Type()]; ok { + if len(table.PrimaryKeys) == 1 { + pkField := reflect.Indirect(fieldValue).FieldByName(table.PKColumns()[0].FieldName) + // fix non-int pk issues + //if pkField.Int() != 0 { + if pkField.IsValid() && !isZero(pkField.Interface()) { + val = pkField.Interface() + } else { + continue + } + } else { + //TODO: how to handler? + panic(fmt.Sprintln("not supported", fieldValue.Interface(), "as", table.PrimaryKeys)) + } + } else { + val = fieldValue.Interface() + } } } case reflect.Array, reflect.Slice, reflect.Map: @@ -662,7 +637,7 @@ func buildConditions(engine *Engine, table *core.Table, bean interface{}, if col.IsPrimaryKey && engine.dialect.DBType() == "ql" { condi = "id() == ?" } else { - condi = fmt.Sprintf("%v %s ?", engine.Quote(col.Name), engine.dialect.EqStr()) + condi = fmt.Sprintf("%v %s ?", colName, engine.dialect.EqStr()) } colNames = append(colNames, condi) } @@ -709,7 +684,7 @@ func (statement *Statement) Id(id interface{}) *Statement { return statement } -// Generate "Update ... Set column = column + arg" statment +// Incr Generate "Update ... Set column = column + arg" statment func (statement *Statement) Incr(column string, arg ...interface{}) *Statement { k := strings.ToLower(column) if len(arg) > 0 { @@ -720,7 +695,7 @@ func (statement *Statement) Incr(column string, arg ...interface{}) *Statement { return statement } -// Generate "Update ... Set column = column - arg" statment +// Decr Generate "Update ... Set column = column - arg" statment func (statement *Statement) Decr(column string, arg ...interface{}) *Statement { k := strings.ToLower(column) if len(arg) > 0 { @@ -731,7 +706,7 @@ func (statement *Statement) Decr(column string, arg ...interface{}) *Statement { return statement } -// Generate "Update ... Set column = {expression}" statment +// SetExpr Generate "Update ... Set column = {expression}" statment func (statement *Statement) SetExpr(column string, expression string) *Statement { k := strings.ToLower(column) statement.exprColumns[k] = exprParam{column, expression} @@ -755,9 +730,14 @@ func (statement *Statement) getExpr() map[string]exprParam { // Generate "Where column IN (?) " statment func (statement *Statement) In(column string, args ...interface{}) *Statement { + length := len(args) + if length == 0 { + return statement + } + k := strings.ToLower(column) var newargs []interface{} - if len(args) == 1 && + if length == 1 && reflect.TypeOf(args[0]).Kind() == reflect.Slice { newargs = make([]interface{}, 0) v := reflect.ValueOf(args[0]) @@ -781,12 +761,17 @@ func (statement *Statement) genInSql() (string, []interface{}) { return "", []interface{}{} } - inStrs := make([]string, 0, len(statement.inColumns)) + inStrs := make([]string, len(statement.inColumns), len(statement.inColumns)) args := make([]interface{}, 0) + var buf bytes.Buffer + var i int for _, params := range statement.inColumns { - inStrs = append(inStrs, fmt.Sprintf("(%v IN (%v))", + buf.Reset() + fmt.Fprintf(&buf, "(%v IN (%v))", statement.Engine.autoQuote(params.colName), - strings.Join(makeArray("?", len(params.args)), ","))) + strings.Join(makeArray("?", len(params.args)), ",")) + inStrs[i] = buf.String() + i++ args = append(args, params.args...) } @@ -799,7 +784,7 @@ func (statement *Statement) genInSql() (string, []interface{}) { func (statement *Statement) attachInSql() { inSql, inArgs := statement.genInSql() if len(inSql) > 0 { - if statement.ConditionStr != "" { + if len(statement.ConditionStr) > 0 { statement.ConditionStr += " " + statement.Engine.dialect.AndStr() + " " } statement.ConditionStr += inSql @@ -858,6 +843,18 @@ func (statement *Statement) Distinct(columns ...string) *Statement { return statement } +// Generate "SELECT ... FOR UPDATE" statment +func (statement *Statement) ForUpdate() *Statement { + statement.IsForUpdate = true + return statement +} + +// replace select +func (s *Statement) Select(str string) *Statement { + s.selectStr = str + return s +} + // Generate "col1, col2" statement func (statement *Statement) Cols(columns ...string) *Statement { newColumns := col2NewCols(columns...) @@ -868,6 +865,7 @@ func (statement *Statement) Cols(columns ...string) *Statement { if strings.Contains(statement.ColumnStr, ".") { statement.ColumnStr = strings.Replace(statement.ColumnStr, ".", statement.Engine.Quote("."), -1) } + statement.ColumnStr = strings.Replace(statement.ColumnStr, statement.Engine.Quote("*"), "*", -1) return statement } @@ -886,15 +884,6 @@ func (statement *Statement) MustCols(columns ...string) *Statement { return statement } -// Update use only: not update columns -/*func (statement *Statement) NotCols(columns ...string) *Statement { - newColumns := col2NewCols(columns...) - for _, nc := range newColumns { - statement.mustColumnMap[strings.ToLower(nc)] = false - } - return statement -}*/ - // indicates that use bool fields as update contents and query contiditions func (statement *Statement) UseBool(columns ...string) *Statement { if len(columns) > 0 { @@ -914,6 +903,14 @@ func (statement *Statement) Omit(columns ...string) { statement.OmitStr = statement.Engine.Quote(strings.Join(newColumns, statement.Engine.Quote(", "))) } +// Update use only: update columns to null when value is nullable and zero-value +func (statement *Statement) Nullable(columns ...string) { + newColumns := col2NewCols(columns...) + for _, nc := range newColumns { + statement.nullableMap[strings.ToLower(nc)] = true + } +} + // Generate LIMIT limit statement func (statement *Statement) Top(limit int) *Statement { statement.Limit(limit) @@ -931,7 +928,7 @@ func (statement *Statement) Limit(limit int, start ...int) *Statement { // Generate "Order By order" statement func (statement *Statement) OrderBy(order string) *Statement { - if statement.OrderStr != "" { + if len(statement.OrderStr) > 0 { statement.OrderStr += ", " } statement.OrderStr += order @@ -939,44 +936,51 @@ func (statement *Statement) OrderBy(order string) *Statement { } func (statement *Statement) Desc(colNames ...string) *Statement { - if statement.OrderStr != "" { - statement.OrderStr += ", " + var buf bytes.Buffer + fmt.Fprintf(&buf, statement.OrderStr) + if len(statement.OrderStr) > 0 { + fmt.Fprint(&buf, ", ") } newColNames := statement.col2NewColsWithQuote(colNames...) - sqlStr := strings.Join(newColNames, " DESC, ") - statement.OrderStr += sqlStr + " DESC" + fmt.Fprintf(&buf, "%v DESC", strings.Join(newColNames, " DESC, ")) + statement.OrderStr = buf.String() return statement } // Method Asc provide asc order by query condition, the input parameters are columns. func (statement *Statement) Asc(colNames ...string) *Statement { - if statement.OrderStr != "" { - statement.OrderStr += ", " + var buf bytes.Buffer + fmt.Fprintf(&buf, statement.OrderStr) + if len(statement.OrderStr) > 0 { + fmt.Fprint(&buf, ", ") } newColNames := statement.col2NewColsWithQuote(colNames...) - sqlStr := strings.Join(newColNames, " ASC, ") - statement.OrderStr += sqlStr + " ASC" + fmt.Fprintf(&buf, "%v ASC", strings.Join(newColNames, " ASC, ")) + statement.OrderStr = buf.String() return statement } //The join_operator should be one of INNER, LEFT OUTER, CROSS etc - this will be prepended to JOIN func (statement *Statement) Join(join_operator string, tablename interface{}, condition string) *Statement { - var joinTable string + var buf bytes.Buffer + if len(statement.JoinStr) > 0 { + fmt.Fprintf(&buf, "%v %v JOIN ", statement.JoinStr, join_operator) + } else { + fmt.Fprintf(&buf, "%v JOIN ", join_operator) + } + switch tablename.(type) { case []string: t := tablename.([]string) - l := len(t) - if l > 1 { - table := t[0] - joinTable = statement.Engine.Quote(table) + " AS " + statement.Engine.Quote(t[1]) - } else if l == 1 { - table := t[0] - joinTable = statement.Engine.Quote(table) + if len(t) > 1 { + fmt.Fprintf(&buf, "%v AS %v", statement.Engine.Quote(t[0]), statement.Engine.Quote(t[1])) + } else if len(t) == 1 { + fmt.Fprintf(&buf, statement.Engine.Quote(t[0])) } case []interface{}: t := tablename.([]interface{}) l := len(t) - table := "" + var table string if l > 0 { f := t[0] v := rValue(f) @@ -989,21 +993,17 @@ func (statement *Statement) Join(join_operator string, tablename interface{}, co } } if l > 1 { - joinTable = statement.Engine.Quote(table) + " AS " + statement.Engine.Quote(fmt.Sprintf("%v", t[1])) + fmt.Fprintf(&buf, "%v AS %v", statement.Engine.Quote(table), + statement.Engine.Quote(fmt.Sprintf("%v", t[1]))) } else if l == 1 { - joinTable = statement.Engine.Quote(table) + fmt.Fprintf(&buf, statement.Engine.Quote(table)) } default: - t := fmt.Sprintf("%v", tablename) - joinTable = statement.Engine.Quote(t) - } - if statement.JoinStr != "" { - statement.JoinStr = statement.JoinStr + fmt.Sprintf(" %v JOIN %v ON %v", join_operator, - joinTable, condition) - } else { - statement.JoinStr = fmt.Sprintf("%v JOIN %v ON %v", join_operator, - joinTable, condition) + fmt.Fprintf(&buf, statement.Engine.Quote(fmt.Sprintf("%v", tablename))) } + + fmt.Fprintf(&buf, " ON %v", condition) + statement.JoinStr = buf.String() return statement } @@ -1120,11 +1120,6 @@ func (s *Statement) genDelIndexSQL() []string { return sqls } -/* -func (s *Statement) genDropSQL() string { - return s.Engine.dialect.MustDropTa(s.TableName()) + ";" -}*/ - func (statement *Statement) genGetSql(bean interface{}) (string, []interface{}) { var table *core.Table if statement.RefTable == nil { @@ -1134,28 +1129,34 @@ func (statement *Statement) genGetSql(bean interface{}) (string, []interface{}) table = statement.RefTable } - colNames, args := buildConditions(statement.Engine, table, bean, true, true, - false, true, statement.allUseBool, statement.useAllCols, - statement.unscoped, statement.mustColumnMap) + var addedTableName = (len(statement.JoinStr) > 0) - statement.ConditionStr = strings.Join(colNames, " "+statement.Engine.dialect.AndStr()+" ") - statement.BeanArgs = args + if !statement.noAutoCondition { + colNames, args := statement.buildConditions(table, bean, true, true, false, true, addedTableName) + + statement.ConditionStr = strings.Join(colNames, " "+statement.Engine.dialect.AndStr()+" ") + statement.BeanArgs = args + } var columnStr string = statement.ColumnStr - if len(statement.JoinStr) == 0 { - if len(columnStr) == 0 { - if statement.GroupByStr != "" { - columnStr = statement.Engine.Quote(strings.Replace(statement.GroupByStr, ",", statement.Engine.Quote(","), -1)) - } else { - columnStr = statement.genColumnStr() - } - } + if len(statement.selectStr) > 0 { + columnStr = statement.selectStr } else { - if len(columnStr) == 0 { - if statement.GroupByStr != "" { - columnStr = statement.Engine.Quote(strings.Replace(statement.GroupByStr, ",", statement.Engine.Quote(","), -1)) - } else { - columnStr = "*" + if len(statement.JoinStr) == 0 { + if len(columnStr) == 0 { + if statement.GroupByStr != "" { + columnStr = statement.Engine.Quote(strings.Replace(statement.GroupByStr, ",", statement.Engine.Quote(","), -1)) + } else { + columnStr = statement.genColumnStr() + } + } + } else { + if len(columnStr) == 0 { + if statement.GroupByStr != "" { + columnStr = statement.Engine.Quote(strings.Replace(statement.GroupByStr, ",", statement.Engine.Quote(","), -1)) + } else { + columnStr = "*" + } } } } @@ -1185,16 +1186,23 @@ func (s *Statement) genAddUniqueStr(uqeName string, cols []string) (string, []in return sql, []interface{}{} }*/ +func (statement *Statement) buildConditions(table *core.Table, bean interface{}, includeVersion bool, includeUpdated bool, includeNil bool, includeAutoIncr bool, addedTableName bool) ([]string, []interface{}) { + return buildConditions(statement.Engine, table, bean, includeVersion, includeUpdated, includeNil, includeAutoIncr, statement.allUseBool, statement.useAllCols, + statement.unscoped, statement.mustColumnMap, statement.TableName(), statement.TableAlias, addedTableName) +} + func (statement *Statement) genCountSql(bean interface{}) (string, []interface{}) { table := statement.Engine.TableInfo(bean) statement.RefTable = table - colNames, args := buildConditions(statement.Engine, table, bean, true, true, false, - true, statement.allUseBool, statement.useAllCols, - statement.unscoped, statement.mustColumnMap) + var addedTableName = (len(statement.JoinStr) > 0) - statement.ConditionStr = strings.Join(colNames, " "+statement.Engine.Dialect().AndStr()+" ") - statement.BeanArgs = args + if !statement.noAutoCondition { + colNames, args := statement.buildConditions(table, bean, true, true, false, true, addedTableName) + + statement.ConditionStr = strings.Join(colNames, " "+statement.Engine.Dialect().AndStr()+" ") + statement.BeanArgs = args + } // count(index fieldname) > count(0) > count(*) var id string = "*" @@ -1206,47 +1214,46 @@ func (statement *Statement) genCountSql(bean interface{}) (string, []interface{} } func (statement *Statement) genSelectSql(columnStr string) (a string) { - /*if statement.GroupByStr != "" { - if columnStr == "" { - columnStr = statement.Engine.Quote(strings.Replace(statement.GroupByStr, ",", statement.Engine.Quote(","), -1)) - } - //statement.GroupByStr = columnStr - }*/ var distinct string if statement.IsDistinct { distinct = "DISTINCT " } + var dialect = statement.Engine.Dialect() + var quote = statement.Engine.Quote var top string var mssqlCondi string - /*var orderBy string - if statement.OrderStr != "" { - orderBy = fmt.Sprintf(" ORDER BY %v", statement.OrderStr) - }*/ + statement.processIdParam() - var whereStr string - if statement.WhereStr != "" { - whereStr = fmt.Sprintf(" WHERE %v", statement.WhereStr) - if statement.ConditionStr != "" { - whereStr = fmt.Sprintf("%v %s %v", whereStr, statement.Engine.Dialect().AndStr(), - statement.ConditionStr) - } - } else if statement.ConditionStr != "" { - whereStr = fmt.Sprintf(" WHERE %v", statement.ConditionStr) - } - var fromStr string = " FROM " + statement.Engine.Quote(statement.TableName()) - if statement.TableAlias != "" { - if statement.Engine.dialect.DBType() == core.ORACLE { - fromStr += " " + statement.Engine.Quote(statement.TableAlias) + + var buf bytes.Buffer + if len(statement.WhereStr) > 0 { + if len(statement.ConditionStr) > 0 { + fmt.Fprintf(&buf, " WHERE (%v)", statement.WhereStr) } else { - fromStr += " AS " + statement.Engine.Quote(statement.TableAlias) + fmt.Fprintf(&buf, " WHERE %v", statement.WhereStr) + } + if statement.ConditionStr != "" { + fmt.Fprintf(&buf, " %s (%v)", dialect.AndStr(), statement.ConditionStr) + } + } else if len(statement.ConditionStr) > 0 { + fmt.Fprintf(&buf, " WHERE %v", statement.ConditionStr) + } + var whereStr = buf.String() + + var fromStr string = " FROM " + quote(statement.TableName()) + if statement.TableAlias != "" { + if dialect.DBType() == core.ORACLE { + fromStr += " " + quote(statement.TableAlias) + } else { + fromStr += " AS " + quote(statement.TableAlias) } } if statement.JoinStr != "" { fromStr = fmt.Sprintf("%v %v", fromStr, statement.JoinStr) } - if statement.Engine.dialect.DBType() == core.MSSQL { + if dialect.DBType() == core.MSSQL { if statement.LimitN > 0 { top = fmt.Sprintf(" TOP %d ", statement.LimitN) } @@ -1277,10 +1284,9 @@ func (statement *Statement) genSelectSql(columnStr string) (a string) { } // !nashtsai! REVIEW Sprintf is considered slowest mean of string concatnation, better to work with builder pattern - a = fmt.Sprintf("SELECT %v%v%v%v%v", top, distinct, columnStr, - fromStr, whereStr) - if mssqlCondi != "" { - if whereStr != "" { + a = fmt.Sprintf("SELECT %v%v%v%v%v", top, distinct, columnStr, fromStr, whereStr) + if len(mssqlCondi) > 0 { + if len(whereStr) > 0 { a += " AND " + mssqlCondi } else { a += " WHERE " + mssqlCondi @@ -1296,17 +1302,20 @@ func (statement *Statement) genSelectSql(columnStr string) (a string) { if statement.OrderStr != "" { a = fmt.Sprintf("%v ORDER BY %v", a, statement.OrderStr) } - if statement.Engine.dialect.DBType() != core.MSSQL && statement.Engine.dialect.DBType() != core.ORACLE { + if dialect.DBType() != core.MSSQL && dialect.DBType() != core.ORACLE { if statement.Start > 0 { a = fmt.Sprintf("%v LIMIT %v OFFSET %v", a, statement.LimitN, statement.Start) } else if statement.LimitN > 0 { a = fmt.Sprintf("%v LIMIT %v", a, statement.LimitN) } - } else if statement.Engine.dialect.DBType() == core.ORACLE { + } else if dialect.DBType() == core.ORACLE { if statement.Start != 0 || statement.LimitN != 0 { a = fmt.Sprintf("SELECT %v FROM (SELECT %v,ROWNUM RN FROM (%v) at WHERE ROWNUM <= %d) aat WHERE RN > %d", columnStr, columnStr, a, statement.Start+statement.LimitN, statement.Start) } } + if statement.IsForUpdate { + a = dialect.ForUpdateSql(a) + } return } diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/syslogger.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/syslogger.go index 5c78fecb0d2..eff69d11194 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/syslogger.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/syslogger.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + // +build !windows,!nacl,!plan9 package xorm diff --git a/Godeps/_workspace/src/github.com/go-xorm/xorm/xorm.go b/Godeps/_workspace/src/github.com/go-xorm/xorm/xorm.go index 71644e6c039..2da9949ebc2 100644 --- a/Godeps/_workspace/src/github.com/go-xorm/xorm/xorm.go +++ b/Godeps/_workspace/src/github.com/go-xorm/xorm/xorm.go @@ -1,3 +1,7 @@ +// Copyright 2015 The Xorm Authors. All rights reserved. +// Use of this source code is governed by a BSD-style +// license that can be found in the LICENSE file. + package xorm import ( @@ -13,7 +17,7 @@ import ( ) const ( - Version string = "0.4.2.0225" + Version string = "0.4.5.0204" ) func regDrvsNDialects() bool { @@ -35,7 +39,7 @@ func regDrvsNDialects() bool { for driverName, v := range providedDrvsNDialects { if driver := core.QueryDriver(driverName); driver == nil { core.RegisterDriver(driverName, v.getDriver()) - core.RegisterDialect(v.dbType, v.getDialect()) + core.RegisterDialect(v.dbType, v.getDialect) } } return true diff --git a/README.md b/README.md index 3886c32a971..a3336e0373b 100644 --- a/README.md +++ b/README.md @@ -80,13 +80,13 @@ the latest master builds [here](http://grafana.org/download/builds) ### Get Code -``` +```bash go get github.com/grafana/grafana ``` ### Building the backend Replace X.Y.Z by actual version number. -``` +```bash cd $GOPATH/src/github.com/grafana/grafana go run build.go setup (only needed once to install godep) godep restore (will pull down all golang lib dependencies in your current GOPATH) @@ -98,7 +98,7 @@ go run build.go build To build less to css for the frontend you will need a recent version of of node (v0.12.0), npm (v2.5.0) and grunt (v0.4.5). Run the following: -``` +```bash npm install npm install -g grunt-cli grunt @@ -106,13 +106,13 @@ grunt ### Recompile backend on source change To rebuild on source change (requires that you executed godep restore) -``` +```bash go get github.com/Unknwon/bra bra run ``` ### Running -``` +```bash ./bin/grafana-server ``` diff --git a/bower.json b/bower.json index 2e45c456274..9e0b307c80c 100644 --- a/bower.json +++ b/bower.json @@ -14,10 +14,10 @@ ], "dependencies": { "jquery": "~2.1.4", - "angular": "~1.5.0", - "angular-route": "~1.5.0", - "angular-mocks": "~1.5.0", - "angular-sanitize": "~1.5.0", + "angular": "~1.5.1", + "angular-route": "~1.5.1", + "angular-mocks": "~1.5.1", + "angular-sanitize": "~1.5.1", "angular-bindonce": "~0.3.3" } } diff --git a/docs/sources/datasources/prometheus.md b/docs/sources/datasources/prometheus.md index a7959a41c51..9ad435270c4 100644 --- a/docs/sources/datasources/prometheus.md +++ b/docs/sources/datasources/prometheus.md @@ -48,6 +48,7 @@ Name | Description `label_values(label)` | Returns a list of label values for the `label` in every metric. `label_values(metric, label)` | Returns a list of label values for the `label` in the specified metric. `metrics(metric)` | Returns a list of metrics matching the specified `metric` regex. +`query_result(query)` | Returns a list of Prometheus query result for the `query`. For details of `metric names` & `label names`, and `label values`, please refer to the [Prometheus documentation](http://prometheus.io/docs/concepts/data_model/#metric-names-and-labels). diff --git a/docs/sources/installation/configuration.md b/docs/sources/installation/configuration.md index 247cfbb0d19..b652abcabb6 100644 --- a/docs/sources/installation/configuration.md +++ b/docs/sources/installation/configuration.md @@ -268,6 +268,7 @@ example: scopes = user:email auth_url = https://github.com/login/oauth/authorize token_url = https://github.com/login/oauth/access_token + api_url = https://api.github.com allow_sign_up = false team_ids = diff --git a/public/app/core/plugins/directive.ts b/examples/nginx-app/css/dark.css similarity index 100% rename from public/app/core/plugins/directive.ts rename to examples/nginx-app/css/dark.css diff --git a/examples/nginx-app/css/light.css b/examples/nginx-app/css/light.css new file mode 100644 index 00000000000..e69de29bb2d diff --git a/examples/nginx-app/img/logo_large.png b/examples/nginx-app/img/logo_large.png new file mode 100644 index 00000000000..c28955960e4 Binary files /dev/null and b/examples/nginx-app/img/logo_large.png differ diff --git a/examples/nginx-app/img/logo_small.png b/examples/nginx-app/img/logo_small.png new file mode 100644 index 00000000000..a6040f66f3d Binary files /dev/null and b/examples/nginx-app/img/logo_small.png differ diff --git a/examples/nginx-app/module.js b/examples/nginx-app/module.js new file mode 100644 index 00000000000..9c1cccb2e52 --- /dev/null +++ b/examples/nginx-app/module.js @@ -0,0 +1,20 @@ +define([ +], function() { + 'use strict'; + + function StreamPageCtrl() {} + StreamPageCtrl.templateUrl = 'partials/stream.html'; + + function LogsPageCtrl() {} + LogsPageCtrl.templateUrl = 'partials/logs.html'; + + function NginxConfigCtrl() {} + NginxConfigCtrl.templateUrl = 'partials/config.html'; + + return { + ConfigCtrl: NginxConfigCtrl, + StreamPageCtrl: StreamPageCtrl, + LogsPageCtrl: LogsPageCtrl, + }; + +}); diff --git a/examples/nginx-app/panel/module.js b/examples/nginx-app/panel/module.js new file mode 100644 index 00000000000..9ccc5b6c962 --- /dev/null +++ b/examples/nginx-app/panel/module.js @@ -0,0 +1,21 @@ +define([ + 'app/plugins/sdk' +], function(sdk) { + 'use strict'; + + var NginxPanel = (function(_super) { + function NginxPanel($scope, $injector) { + _super.call(this, $scope, $injector); + } + + NginxPanel.template = '

nginx!

'; + NginxPanel.prototype = Object.create(_super.prototype); + NginxPanel.prototype.constructor = NginxPanel; + + return NginxPanel; + })(sdk.PanelCtrl); + + return { + PanelCtrl: NginxPanel + }; +}); diff --git a/examples/nginx-app/panel/plugin.json b/examples/nginx-app/panel/plugin.json new file mode 100644 index 00000000000..f3548c987f3 --- /dev/null +++ b/examples/nginx-app/panel/plugin.json @@ -0,0 +1,5 @@ +{ + "type": "panel", + "name": "Nginx Panel", + "id": "nginx-panel" +} diff --git a/examples/nginx-app/partials/config.html b/examples/nginx-app/partials/config.html new file mode 100644 index 00000000000..cdd6b8d9b60 --- /dev/null +++ b/examples/nginx-app/partials/config.html @@ -0,0 +1 @@ +

nginx config

diff --git a/examples/nginx-app/partials/logs.html b/examples/nginx-app/partials/logs.html new file mode 100644 index 00000000000..f6bef036395 --- /dev/null +++ b/examples/nginx-app/partials/logs.html @@ -0,0 +1,2 @@ + +Logs! diff --git a/examples/nginx-app/partials/stream.html b/examples/nginx-app/partials/stream.html new file mode 100644 index 00000000000..092c36cc24c --- /dev/null +++ b/examples/nginx-app/partials/stream.html @@ -0,0 +1 @@ +streams! diff --git a/examples/nginx-app/plugin.json b/examples/nginx-app/plugin.json new file mode 100644 index 00000000000..7e3cdd7ad08 --- /dev/null +++ b/examples/nginx-app/plugin.json @@ -0,0 +1,44 @@ +{ + "type": "app", + "name": "Nginx", + "id": "nginx-app", + + "staticRoot": ".", + + "pages": [ + { "name": "Live stream", "component": "StreamPageCtrl", "role": "Editor"}, + { "name": "Log view", "component": "LogsPageCtrl", "role": "Viewer"} + ], + + "css": { + "dark": "css/dark.css", + "light": "css/light.css" + }, + + "info": { + "description": "Official Grafana Nginx App & Dashboard bundle", + "author": { + "name": "Nginx Inc.", + "url": "http://nginx.com" + }, + "keywords": ["nginx"], + "logos": { + "small": "img/logo_small.png", + "large": "img/logo_large.png" + }, + "links": [ + {"name": "Project site", "url": "http://project.com"}, + {"name": "License & Terms", "url": "http://license.com"} + ], + "version": "1.0.0", + "updated": "2015-02-10" + }, + + "dependencies": { + "grafanaVersion": "2.6.x", + "plugins": [ + {"type": "datasource", "id": "graphite", "name": "Graphite", "version": "1.0.0"}, + {"type": "panel", "id": "graph", "name": "Graph", "version": "1.0.0"} + ] + } +} diff --git a/examples/panel-boilerplate-es5/module.js b/examples/panel-boilerplate-es5/module.js new file mode 100644 index 00000000000..b991ad72f76 --- /dev/null +++ b/examples/panel-boilerplate-es5/module.js @@ -0,0 +1,28 @@ +define([ + 'app/plugins/sdk' +], function(sdk) { + + var BoilerPlatePanel = (function(_super) { + + function BoilerPlatePanel($scope, $injector) { + _super.call(this, $scope, $injector); + } + + // you do not need a templateUrl, you can use a inline template here + // BoilerPlatePanel.template = '

boilerplate

'; + + // all panel static assets can be accessed via 'public/plugins// + BoilerPlatePanel.templateUrl = 'panel.html'; + + BoilerPlatePanel.prototype = Object.create(_super.prototype); + BoilerPlatePanel.prototype.constructor = BoilerPlatePanel; + + return BoilerPlatePanel; + + })(sdk.PanelCtrl); + + + return { + PanelCtrl: BoilerPlatePanel + }; +}); diff --git a/examples/panel-boilerplate-es5/panel.html b/examples/panel-boilerplate-es5/panel.html new file mode 100644 index 00000000000..5d3e44c756a --- /dev/null +++ b/examples/panel-boilerplate-es5/panel.html @@ -0,0 +1,4 @@ +

+ Boilerplate panel +

+ diff --git a/examples/panel-boilerplate-es5/plugin.json b/examples/panel-boilerplate-es5/plugin.json new file mode 100644 index 00000000000..b585698c20a --- /dev/null +++ b/examples/panel-boilerplate-es5/plugin.json @@ -0,0 +1,6 @@ +{ + "type": "panel", + "name": "Panel Boilerplate", + "id": "panel-boilerplate-es5", + "staticRoot": "." +} diff --git a/pkg/api/dtos/apps.go b/pkg/api/dtos/apps.go index d10883a26d7..ba26e3eb617 100644 --- a/pkg/api/dtos/apps.go +++ b/pkg/api/dtos/apps.go @@ -6,15 +6,16 @@ import ( ) type AppSettings struct { - Name string `json:"name"` - AppId string `json:"appId"` - Enabled bool `json:"enabled"` - Pinned bool `json:"pinned"` - Module string `json:"module"` - Info *plugins.PluginInfo `json:"info"` - Pages []plugins.AppPluginPage `json:"pages"` - Includes []plugins.AppIncludeInfo `json:"includes"` - JsonData map[string]interface{} `json:"jsonData"` + Name string `json:"name"` + AppId string `json:"appId"` + Enabled bool `json:"enabled"` + Pinned bool `json:"pinned"` + Module string `json:"module"` + BaseUrl string `json:"baseUrl"` + Info *plugins.PluginInfo `json:"info"` + Pages []*plugins.AppPluginPage `json:"pages"` + Includes []*plugins.AppIncludeInfo `json:"includes"` + JsonData map[string]interface{} `json:"jsonData"` } func NewAppSettingsDto(def *plugins.AppPlugin, data *models.AppSettings) *AppSettings { @@ -23,6 +24,7 @@ func NewAppSettingsDto(def *plugins.AppPlugin, data *models.AppSettings) *AppSet Name: def.Name, Info: &def.Info, Module: def.Module, + BaseUrl: def.BaseUrl, Pages: def.Pages, Includes: def.Includes, } diff --git a/pkg/api/dtos/index.go b/pkg/api/dtos/index.go index c5b81ee0e98..180ec767281 100644 --- a/pkg/api/dtos/index.go +++ b/pkg/api/dtos/index.go @@ -19,8 +19,9 @@ type PluginCss struct { } type NavLink struct { - Text string `json:"text"` - Icon string `json:"icon"` - Img string `json:"img"` - Url string `json:"url"` + Text string `json:"text"` + Icon string `json:"icon"` + Img string `json:"img"` + Url string `json:"url"` + Children []*NavLink `json:"children"` } diff --git a/pkg/api/frontendsettings.go b/pkg/api/frontendsettings.go index 0759f1e83b8..8c86f98220a 100644 --- a/pkg/api/frontendsettings.go +++ b/pkg/api/frontendsettings.go @@ -121,10 +121,11 @@ func getFrontendSettingsMap(c *middleware.Context) (map[string]interface{}, erro panels := map[string]interface{}{} for _, panel := range enabledPlugins.Panels { panels[panel.Id] = map[string]interface{}{ - "module": panel.Module, - "name": panel.Name, - "id": panel.Id, - "info": panel.Info, + "module": panel.Module, + "baseUrl": panel.BaseUrl, + "name": panel.Name, + "id": panel.Id, + "info": panel.Info, } } diff --git a/pkg/api/index.go b/pkg/api/index.go index 342de21c670..094511df1ea 100644 --- a/pkg/api/index.go +++ b/pkg/api/index.go @@ -36,7 +36,7 @@ func setIndexViewData(c *middleware.Context) (*dtos.IndexViewData, error) { } if setting.DisableGravatar { - data.User.GravatarUrl = setting.AppSubUrl + "/img/user_profile.png" + data.User.GravatarUrl = setting.AppSubUrl + "/public/img/user_profile.png" } if len(data.User.Name) == 0 { @@ -51,32 +51,27 @@ func setIndexViewData(c *middleware.Context) (*dtos.IndexViewData, error) { data.MainNavLinks = append(data.MainNavLinks, &dtos.NavLink{ Text: "Dashboards", Icon: "fa fa-fw fa-th-large", - Url: "/", + Url: setting.AppSubUrl + "/", + // Children: []*dtos.NavLink{ + // {Text: "Playlists", Icon: "fa fa-fw fa-list", Url: setting.AppSubUrl + "/playlists"}, + // {Text: "Snapshots", Icon: "fa-fw icon-gf icon-gf-snapshot", Url: setting.AppSubUrl + "/dashboard/snapshots"}, + // }, }) - data.MainNavLinks = append(data.MainNavLinks, &dtos.NavLink{ - Text: "Playlists", - Icon: "fa fa-fw fa-list", - Url: "/playlists", - }) - - data.MainNavLinks = append(data.MainNavLinks, &dtos.NavLink{ - Text: "Snapshots", - Icon: "fa-fw icon-gf icon-gf-snapshot", - Url: "/dashboard/snapshots", - }) + data.MainNavLinks = append(data.MainNavLinks, &dtos.NavLink{Text: "Playlists", Icon: "fa fa-fw fa-list", Url: setting.AppSubUrl + "/playlists"}) + data.MainNavLinks = append(data.MainNavLinks, &dtos.NavLink{Text: "Snapshots", Icon: "fa-fw icon-gf icon-gf-snapshot", Url: setting.AppSubUrl + "/dashboard/snapshots"}) if c.OrgRole == m.ROLE_ADMIN { data.MainNavLinks = append(data.MainNavLinks, &dtos.NavLink{ Text: "Data Sources", Icon: "fa fa-fw fa-database", - Url: "/datasources", + Url: setting.AppSubUrl + "/datasources", }) data.MainNavLinks = append(data.MainNavLinks, &dtos.NavLink{ Text: "Apps", Icon: "fa fa-fw fa-cubes", - Url: "/apps", + Url: setting.AppSubUrl + "/apps", }) } @@ -86,20 +81,25 @@ func setIndexViewData(c *middleware.Context) (*dtos.IndexViewData, error) { } for _, plugin := range enabledPlugins.Apps { - if plugin.Module != "" { - data.PluginModules = append(data.PluginModules, plugin.Module) - } - if plugin.Css != nil { data.PluginCss = append(data.PluginCss, &dtos.PluginCss{Light: plugin.Css.Light, Dark: plugin.Css.Dark}) } if plugin.Pinned { - data.MainNavLinks = append(data.MainNavLinks, &dtos.NavLink{ + pageLink := &dtos.NavLink{ Text: plugin.Name, - Url: "/apps/edit/" + plugin.Id, + Url: setting.AppSubUrl + "/apps/" + plugin.Id + "/edit", Img: plugin.Info.Logos.Small, - }) + } + + for _, page := range plugin.Pages { + pageLink.Children = append(pageLink.Children, &dtos.NavLink{ + Url: setting.AppSubUrl + "/apps/" + plugin.Id + "/page/" + page.Slug, + Text: page.Name, + }) + } + + data.MainNavLinks = append(data.MainNavLinks, pageLink) } } diff --git a/pkg/cmd/web.go b/pkg/cmd/web.go index 897cfc4f3af..61534e7d346 100644 --- a/pkg/cmd/web.go +++ b/pkg/cmd/web.go @@ -31,14 +31,11 @@ func newMacaron() *macaron.Macaron { for _, route := range plugins.StaticRoutes { pluginRoute := path.Join("/public/plugins/", route.PluginId) - log.Info("Plugin: Adding static route %s -> %s", pluginRoute, route.Directory) + log.Info("Plugins: Adding route %s -> %s", pluginRoute, route.Directory) mapStatic(m, route.Directory, "", pluginRoute) } mapStatic(m, setting.StaticRootPath, "", "public") - mapStatic(m, setting.StaticRootPath, "css", "css") - mapStatic(m, setting.StaticRootPath, "img", "img") - mapStatic(m, setting.StaticRootPath, "fonts", "fonts") mapStatic(m, setting.StaticRootPath, "robots.txt", "robots.txt") m.Use(macaron.Renderer(macaron.RenderOptions{ diff --git a/pkg/plugins/app_plugin.go b/pkg/plugins/app_plugin.go index 547cd91283c..474b7354762 100644 --- a/pkg/plugins/app_plugin.go +++ b/pkg/plugins/app_plugin.go @@ -4,13 +4,15 @@ import ( "encoding/json" "strings" + "github.com/gosimple/slug" "github.com/grafana/grafana/pkg/models" ) type AppPluginPage struct { - Name string `json:"name"` - Url string `json:"url"` - ReqRole models.RoleType `json:"reqRole"` + Name string `json:"name"` + Slug string `json:"slug"` + Component string `json:"component"` + Role models.RoleType `json:"role"` } type AppPluginCss struct { @@ -27,9 +29,9 @@ type AppIncludeInfo struct { type AppPlugin struct { FrontendPluginBase Css *AppPluginCss `json:"css"` - Pages []AppPluginPage `json:"pages"` + Pages []*AppPluginPage `json:"pages"` Routes []*AppPluginRoute `json:"routes"` - Includes []AppIncludeInfo `json:"-"` + Includes []*AppIncludeInfo `json:"-"` Pinned bool `json:"-"` Enabled bool `json:"-"` @@ -55,19 +57,27 @@ func (app *AppPlugin) Load(decoder *json.Decoder, pluginDir string) error { return err } + if err := app.registerPlugin(pluginDir); err != nil { + return err + } + + Apps[app.Id] = app + return nil +} + +func (app *AppPlugin) initApp() { + app.initFrontendPlugin() + if app.Css != nil { app.Css.Dark = evalRelativePluginUrlPath(app.Css.Dark, app.Id) app.Css.Light = evalRelativePluginUrlPath(app.Css.Light, app.Id) } - app.PluginDir = pluginDir - app.initFrontendPlugin() - // check if we have child panels for _, panel := range Panels { if strings.HasPrefix(panel.PluginDir, app.PluginDir) { - panel.IncludedInAppId = app.Id - app.Includes = append(app.Includes, AppIncludeInfo{ + panel.setPathsBasedOnApp(app) + app.Includes = append(app.Includes, &AppIncludeInfo{ Name: panel.Name, Id: panel.Id, Type: panel.Type, @@ -75,6 +85,22 @@ func (app *AppPlugin) Load(decoder *json.Decoder, pluginDir string) error { } } - Apps[app.Id] = app - return nil + // check if we have child datasources + for _, ds := range DataSources { + if strings.HasPrefix(ds.PluginDir, app.PluginDir) { + ds.setPathsBasedOnApp(app) + app.Includes = append(app.Includes, &AppIncludeInfo{ + Name: ds.Name, + Id: ds.Id, + Type: ds.Type, + }) + } + } + + // slugify pages + for _, page := range app.Pages { + if page.Slug == "" { + page.Slug = slug.Make(page.Name) + } + } } diff --git a/pkg/plugins/datasource_plugin.go b/pkg/plugins/datasource_plugin.go index e1bb9047297..ecffb87b8db 100644 --- a/pkg/plugins/datasource_plugin.go +++ b/pkg/plugins/datasource_plugin.go @@ -17,9 +17,10 @@ func (p *DataSourcePlugin) Load(decoder *json.Decoder, pluginDir string) error { return err } - p.PluginDir = pluginDir - p.initFrontendPlugin() - DataSources[p.Id] = p + if err := p.registerPlugin(pluginDir); err != nil { + return err + } + DataSources[p.Id] = p return nil } diff --git a/pkg/plugins/frontend_plugin.go b/pkg/plugins/frontend_plugin.go index 3dd26a78ea5..a1bcd0c68b5 100644 --- a/pkg/plugins/frontend_plugin.go +++ b/pkg/plugins/frontend_plugin.go @@ -4,11 +4,15 @@ import ( "net/url" "path" "path/filepath" + "strings" + + "github.com/grafana/grafana/pkg/util" ) type FrontendPluginBase struct { PluginBase Module string `json:"module"` + BaseUrl string `json:"baseUrl"` StaticRoot string `json:"staticRoot"` StaticRootAbs string `json:"-"` } @@ -22,22 +26,33 @@ func (fp *FrontendPluginBase) initFrontendPlugin() { }) } + fp.handleModuleDefaults() + fp.Info.Logos.Small = evalRelativePluginUrlPath(fp.Info.Logos.Small, fp.Id) fp.Info.Logos.Large = evalRelativePluginUrlPath(fp.Info.Logos.Large, fp.Id) - fp.handleModuleDefaults() + + for i := 0; i < len(fp.Info.Screenshots); i++ { + fp.Info.Screenshots[i].Path = evalRelativePluginUrlPath(fp.Info.Screenshots[i].Path, fp.Id) + } +} + +func (fp *FrontendPluginBase) setPathsBasedOnApp(app *AppPlugin) { + appSubPath := strings.Replace(fp.PluginDir, app.StaticRootAbs, "", 1) + fp.IncludedInAppId = app.Id + fp.BaseUrl = app.BaseUrl + fp.Module = util.JoinUrlFragments("plugins/"+app.Id, appSubPath) + "/module" } func (fp *FrontendPluginBase) handleModuleDefaults() { - if fp.Module != "" { - return - } if fp.StaticRoot != "" { fp.Module = path.Join("plugins", fp.Id, "module") + fp.BaseUrl = path.Join("public/plugins", fp.Id) return } fp.Module = path.Join("app/plugins", fp.Type, fp.Id, "module") + fp.BaseUrl = path.Join("public/app/plugins", fp.Type, fp.Id) } func evalRelativePluginUrlPath(pathStr string, pluginId string) string { diff --git a/pkg/plugins/models.go b/pkg/plugins/models.go index 55ba8f97767..83f43c1f544 100644 --- a/pkg/plugins/models.go +++ b/pkg/plugins/models.go @@ -2,6 +2,11 @@ package plugins import ( "encoding/json" + "errors" + "strings" + + "github.com/grafana/grafana/pkg/log" + "github.com/grafana/grafana/pkg/setting" ) type PluginLoader interface { @@ -18,13 +23,28 @@ type PluginBase struct { PluginDir string `json:"-"` } +func (pb *PluginBase) registerPlugin(pluginDir string) error { + if _, exists := Plugins[pb.Id]; exists { + return errors.New("Plugin with same id already exists") + } + + if !strings.HasPrefix(pluginDir, setting.StaticRootPath) { + log.Info("Plugins: Registering plugin %v", pb.Name) + } + + pb.PluginDir = pluginDir + Plugins[pb.Id] = pb + return nil +} + type PluginInfo struct { - Author PluginInfoLink `json:"author"` - Description string `json:"description"` - Links []PluginInfoLink `json:"links"` - Logos PluginLogos `json:"logos"` - Version string `json:"version"` - Updated string `json:"updated"` + Author PluginInfoLink `json:"author"` + Description string `json:"description"` + Links []PluginInfoLink `json:"links"` + Logos PluginLogos `json:"logos"` + Screenshots []PluginScreenshots `json:"screenshots"` + Version string `json:"version"` + Updated string `json:"updated"` } type PluginInfoLink struct { @@ -37,6 +57,11 @@ type PluginLogos struct { Large string `json:"large"` } +type PluginScreenshots struct { + Path string `json:"path"` + Name string `json:"name"` +} + type PluginStaticRoute struct { Directory string PluginId string diff --git a/pkg/plugins/panel_plugin.go b/pkg/plugins/panel_plugin.go index 5b99ac52344..49cbfb19fc6 100644 --- a/pkg/plugins/panel_plugin.go +++ b/pkg/plugins/panel_plugin.go @@ -11,9 +11,10 @@ func (p *PanelPlugin) Load(decoder *json.Decoder, pluginDir string) error { return err } - p.PluginDir = pluginDir - p.initFrontendPlugin() - Panels[p.Id] = p + if err := p.registerPlugin(pluginDir); err != nil { + return err + } + Panels[p.Id] = p return nil } diff --git a/pkg/plugins/plugins.go b/pkg/plugins/plugins.go index 5878a25a2da..e4a8f52b259 100644 --- a/pkg/plugins/plugins.go +++ b/pkg/plugins/plugins.go @@ -19,6 +19,7 @@ var ( Panels map[string]*PanelPlugin StaticRoutes []*PluginStaticRoute Apps map[string]*AppPlugin + Plugins map[string]*PluginBase PluginTypes map[string]interface{} ) @@ -32,45 +33,50 @@ func Init() error { StaticRoutes = make([]*PluginStaticRoute, 0) Panels = make(map[string]*PanelPlugin) Apps = make(map[string]*AppPlugin) + Plugins = make(map[string]*PluginBase) PluginTypes = map[string]interface{}{ "panel": PanelPlugin{}, "datasource": DataSourcePlugin{}, "app": AppPlugin{}, } + log.Info("Plugins: Scan starting") scan(path.Join(setting.StaticRootPath, "app/plugins")) - scan(setting.PluginsPath) + + // check if plugins dir exists + if _, err := os.Stat(setting.PluginsPath); os.IsNotExist(err) { + log.Warn("Plugins: Plugin dir %v does not exist", setting.PluginsPath) + if err = os.MkdirAll(setting.PluginsPath, os.ModePerm); err != nil { + log.Warn("Plugins: Failed to create plugin dir: %v, error: %v", setting.PluginsPath, err) + } else { + log.Info("Plugins: Plugin dir %v created", setting.PluginsPath) + scan(setting.PluginsPath) + } + } else { + scan(setting.PluginsPath) + } + + // check plugin paths defined in config checkPluginPaths() - // checkDependencies() + + for _, panel := range Panels { + panel.initFrontendPlugin() + } + for _, panel := range DataSources { + panel.initFrontendPlugin() + } + for _, app := range Apps { + app.initApp() + } + return nil } -// func checkDependencies() { -// for appType, app := range Apps { -// for _, reqPanel := range app.PanelPlugins { -// if _, ok := Panels[reqPanel]; !ok { -// log.Fatal(4, "App %s requires Panel type %s, but it is not present.", appType, reqPanel) -// } -// } -// for _, reqDataSource := range app.DatasourcePlugins { -// if _, ok := DataSources[reqDataSource]; !ok { -// log.Fatal(4, "App %s requires DataSource type %s, but it is not present.", appType, reqDataSource) -// } -// } -// for _, reqApiPlugin := range app.ApiPlugins { -// if _, ok := ApiPlugins[reqApiPlugin]; !ok { -// log.Fatal(4, "App %s requires ApiPlugin type %s, but it is not present.", appType, reqApiPlugin) -// } -// } -// } -// } - func checkPluginPaths() error { for _, section := range setting.Cfg.Sections() { if strings.HasPrefix(section.Name(), "plugin.") { path := section.Key("path").String() if path != "" { - log.Info("Plugin: Scaning dir %s", path) scan(path) } } @@ -83,6 +89,8 @@ func scan(pluginDir string) error { pluginPath: pluginDir, } + log.Info("Plugins: Scaning dir %s", pluginDir) + if err := util.Walk(pluginDir, true, true, scanner.walker); err != nil { if pluginDir != "data/plugins" { log.Warn("Could not scan dir \"%v\" error: %s", pluginDir, err) @@ -102,6 +110,10 @@ func (scanner *PluginScanner) walker(currentPath string, f os.FileInfo, err erro return err } + if f.Name() == "node_modules" { + return util.WalkSkipDir + } + if f.IsDir() { return nil } @@ -109,7 +121,7 @@ func (scanner *PluginScanner) walker(currentPath string, f os.FileInfo, err erro if f.Name() == "plugin.json" { err := scanner.loadPluginJson(currentPath) if err != nil { - log.Error(3, "Failed to load plugin json file: %v, err: %v", currentPath, err) + log.Error(3, "Plugins: Failed to load plugin json file: %v, err: %v", currentPath, err) scanner.errors = append(scanner.errors, err) } } diff --git a/pkg/plugins/plugins_test.go b/pkg/plugins/plugins_test.go index 1808f72ec0d..08ff3cbdfdd 100644 --- a/pkg/plugins/plugins_test.go +++ b/pkg/plugins/plugins_test.go @@ -34,6 +34,7 @@ func TestPluginScans(t *testing.T) { So(err, ShouldBeNil) So(len(Apps), ShouldBeGreaterThan, 0) So(Apps["app-example"].Info.Logos.Large, ShouldEqual, "public/plugins/app-example/img/logo_large.png") + So(Apps["app-example"].Info.Screenshots[1].Path, ShouldEqual, "public/plugins/app-example/img/screenshot2.png") }) } diff --git a/public/app/app.js b/public/app/app.js deleted file mode 100644 index 35f195dd093..00000000000 --- a/public/app/app.js +++ /dev/null @@ -1,7 +0,0 @@ -define([ - './grafana' -], function(app) { - 'use strict'; - // backward compatability hack; - return app.default; -}); diff --git a/public/app/grafana.ts b/public/app/app.ts similarity index 92% rename from public/app/grafana.ts rename to public/app/app.ts index d9ad3a449d3..a4e1bebca6e 100644 --- a/public/app/grafana.ts +++ b/public/app/app.ts @@ -72,12 +72,6 @@ export class GrafanaApp { this.useModule(coreModule); var preBootRequires = [System.import('app/features/all')]; - var pluginModules = config.bootData.pluginModules || []; - - // add plugin modules - for (var i = 0; i < pluginModules.length; i++) { - preBootRequires.push(System.import(pluginModules[i])); - } Promise.all(preBootRequires).then(() => { // disable tool tip animation diff --git a/public/app/boot.js b/public/app/boot.js index 4ca0646f158..26922e9d612 100644 --- a/public/app/boot.js +++ b/public/app/boot.js @@ -1,8 +1,17 @@ (function bootGrafana() { 'use strict'; + var systemLocate = System.locate; + System.locate = function(load) { + var System = this; + return Promise.resolve(systemLocate.call(this, load)).then(function(address) { + return address + System.cacheBust; + }); + }; + System.cacheBust = '?bust=' + Date.now(); + System.import('app/app').then(function(app) { - app.init(); + app.default.init(); }).catch(function(err) { console.log('Loading app module failed: ', err); }); diff --git a/public/app/core/components/grafana_app.ts b/public/app/core/components/grafana_app.ts index 9b627e930a0..1dd1661aff0 100644 --- a/public/app/core/components/grafana_app.ts +++ b/public/app/core/components/grafana_app.ts @@ -187,13 +187,17 @@ export function grafanaAppDirective(playlistSrv) { // hide search if (elem.find('.search-container').length > 0) { if (target.parents('.search-container').length === 0) { - scope.appEvent('hide-dash-search'); + scope.$apply(function() { + scope.appEvent('hide-dash-search'); + }); } } // hide sidemenu if (!ignoreSideMenuHide && !scope.contextSrv.pinned && elem.find('.sidemenu').length > 0) { if (target.parents('.sidemenu').length === 0) { - scope.$apply(() => scope.contextSrv.toggleSideMenu()); + scope.$apply(function() { + scope.contextSrv.toggleSideMenu(); + }); } } diff --git a/public/app/core/components/navbar/navbar.html b/public/app/core/components/navbar/navbar.html index 3ebac53e8d1..2ba53897696 100644 --- a/public/app/core/components/navbar/navbar.html +++ b/public/app/core/components/navbar/navbar.html @@ -3,7 +3,7 @@
- + diff --git a/public/app/core/components/search/search.html b/public/app/core/components/search/search.html index 7a78f1a4a3e..35c4431d80c 100644 --- a/public/app/core/components/search/search.html +++ b/public/app/core/components/search/search.html @@ -1,3 +1,4 @@ +
@@ -44,7 +45,7 @@
No dashboards matching your query were found.
 + ng-class="{'selected': $index == ctrl.selectedIndex}" ng-href="{{row.url}}"> @@ -71,4 +72,4 @@
- +
diff --git a/public/app/core/components/search/search.ts b/public/app/core/components/search/search.ts index 3f8cb495e71..e296acf56e1 100644 --- a/public/app/core/components/search/search.ts +++ b/public/app/core/components/search/search.ts @@ -7,6 +7,7 @@ import $ from 'jquery'; import coreModule from '../../core_module'; export class SearchCtrl { + isOpen: boolean; query: any; giveSearchFocus: number; selectedIndex: number; @@ -15,16 +16,34 @@ export class SearchCtrl { tagsMode: boolean; showImport: boolean; dismiss: any; + ignoreClose: any; /** @ngInject */ - constructor(private $scope, private $location, private $timeout, private backendSrv, private contextSrv) { + constructor(private $scope, private $location, private $timeout, private backendSrv, private contextSrv, private $rootScope) { + $rootScope.onAppEvent('show-dash-search', this.openSearch.bind(this), $scope); + $rootScope.onAppEvent('hide-dash-search', this.closeSearch.bind(this), $scope); + } + + closeSearch() { + this.isOpen = this.ignoreClose; + } + + openSearch() { + if (this.isOpen) { + this.isOpen = false; + return; + } + + this.isOpen = true; this.giveSearchFocus = 0; this.selectedIndex = -1; this.results = []; this.query = { query: '', tag: [], starred: false }; this.currentSearchId = 0; + this.ignoreClose = true; - $timeout(() => { + this.$timeout(() => { + this.ignoreClose = false; this.giveSearchFocus = this.giveSearchFocus + 1; this.query.query = ''; this.search(); @@ -33,7 +52,7 @@ export class SearchCtrl { keyDown(evt) { if (evt.keyCode === 27) { - this.dismiss(); + this.closeSearch(); } if (evt.keyCode === 40) { this.moveSelection(1); @@ -141,10 +160,7 @@ export function searchDirective() { controller: SearchCtrl, bindToController: true, controllerAs: 'ctrl', - scope: { - dismiss: '&' - }, }; } -coreModule.directive('search', searchDirective); +coreModule.directive('dashboardSearch', searchDirective); diff --git a/public/app/core/components/sidemenu/sidemenu.html b/public/app/core/components/sidemenu/sidemenu.html index a651c87aa7c..7fcd41a2824 100644 --- a/public/app/core/components/sidemenu/sidemenu.html +++ b/public/app/core/components/sidemenu/sidemenu.html @@ -1,6 +1,6 @@
    -
  • +
  • @@ -44,6 +44,13 @@ {{item.text}} +
      • diff --git a/public/app/core/components/sidemenu/sidemenu.ts b/public/app/core/components/sidemenu/sidemenu.ts index 833bba1913f..0394d143ab9 100644 --- a/public/app/core/components/sidemenu/sidemenu.ts +++ b/public/app/core/components/sidemenu/sidemenu.ts @@ -16,7 +16,7 @@ export class SideMenuCtrl { appSubUrl: string; /** @ngInject */ - constructor(private $scope, private $location, private contextSrv, private backendSrv) { + constructor(private $scope, private $location, private contextSrv, private backendSrv, private $element) { this.isSignedIn = contextSrv.isSignedIn; this.user = contextSrv.user; this.appSubUrl = config.appSubUrl; @@ -29,6 +29,7 @@ export class SideMenuCtrl { this.contextSrv.sidemenu = false; } }); + } getUrl(url) { @@ -36,9 +37,7 @@ export class SideMenuCtrl { } setupMainNav() { - this.mainLinks = config.bootData.mainNavLinks.map(item => { - return {text: item.text, icon: item.icon, img: item.img, url: this.getUrl(item.url)}; - }); + this.mainLinks = config.bootData.mainNavLinks; } openUserDropdown() { diff --git a/public/app/core/controllers/login_ctrl.js b/public/app/core/controllers/login_ctrl.js index a2f39a1bd56..89757c3d141 100644 --- a/public/app/core/controllers/login_ctrl.js +++ b/public/app/core/controllers/login_ctrl.js @@ -17,6 +17,7 @@ function (angular, coreModule, config) { $scope.googleAuthEnabled = config.googleAuthEnabled; $scope.githubAuthEnabled = config.githubAuthEnabled; + $scope.oauthEnabled = config.githubAuthEnabled || config.googleAuthEnabled; $scope.disableUserSignUp = config.disableUserSignUp; $scope.loginHint = config.loginHint; @@ -84,7 +85,5 @@ function (angular, coreModule, config) { }; $scope.init(); - }); - }); diff --git a/public/app/core/core.ts b/public/app/core/core.ts index 55c0f4ec049..361dea334ef 100644 --- a/public/app/core/core.ts +++ b/public/app/core/core.ts @@ -28,7 +28,7 @@ import {navbarDirective} from './components/navbar/navbar'; import {arrayJoin} from './directives/array_join'; import 'app/core/controllers/all'; import 'app/core/services/all'; -import 'app/core/routes/all'; +import 'app/core/routes/routes'; import './filters/filters'; import coreModule from './core_module'; diff --git a/public/app/core/directives/body_class.js b/public/app/core/directives/body_class.js index 73426c60e00..dcf04a0bfd6 100644 --- a/public/app/core/directives/body_class.js +++ b/public/app/core/directives/body_class.js @@ -12,6 +12,14 @@ function (_, $, coreModule) { var lastHideControlsVal; + $scope.onAppEvent('panel-fullscreen-enter', function() { + elem.toggleClass('panel-in-fullscreen', true); + }); + + $scope.onAppEvent('panel-fullscreen-exit', function() { + elem.toggleClass('panel-in-fullscreen', false); + }); + $scope.$watch('dashboard.hideControls', function() { if (!$scope.dashboard) { return; diff --git a/public/app/core/directives/plugin_component.ts b/public/app/core/directives/plugin_component.ts index 94e50bf5c86..5c3baf3d223 100644 --- a/public/app/core/directives/plugin_component.ts +++ b/public/app/core/directives/plugin_component.ts @@ -7,7 +7,7 @@ import config from 'app/core/config'; import coreModule from 'app/core/core_module'; import {UnknownPanelCtrl} from 'app/plugins/panel/unknown/module'; -/** @ngInject */ +/** @ngInject **/ function pluginDirectiveLoader($compile, datasourceSrv, $rootScope, $q, $http, $templateCache) { function getTemplate(component) { @@ -23,7 +23,16 @@ function pluginDirectiveLoader($compile, datasourceSrv, $rootScope, $q, $http, $ }); } + function relativeTemplateUrlToAbs(templateUrl, baseUrl) { + if (!templateUrl) { return undefined; } + if (templateUrl.indexOf('public') === 0) { return templateUrl; } + return baseUrl + '/' + templateUrl; + } + function getPluginComponentDirective(options) { + // handle relative template urls for plugin templates + options.Component.templateUrl = relativeTemplateUrlToAbs(options.Component.templateUrl, options.baseUrl); + return function() { return { templateUrl: options.Component.templateUrl, @@ -74,6 +83,10 @@ function pluginDirectiveLoader($compile, datasourceSrv, $rootScope, $q, $http, $ }); } + if (panelInfo) { + PanelCtrl.templateUrl = relativeTemplateUrlToAbs(PanelCtrl.templateUrl, panelInfo.baseUrl); + } + PanelCtrl.templatePromise = getTemplate(PanelCtrl).then(template => { PanelCtrl.templateUrl = null; PanelCtrl.template = `${template}`; @@ -94,6 +107,7 @@ function pluginDirectiveLoader($compile, datasourceSrv, $rootScope, $q, $http, $ return System.import(ds.meta.module).then(dsModule => { return { + baseUrl: ds.meta.baseUrl, name: 'query-ctrl-' + ds.meta.id, bindings: {target: "=", panelCtrl: "=", datasource: "="}, attrs: {"target": "target", "panel-ctrl": "ctrl", datasource: "datasource"}, @@ -111,6 +125,7 @@ function pluginDirectiveLoader($compile, datasourceSrv, $rootScope, $q, $http, $ } return { + baseUrl: ds.meta.baseUrl, name: 'query-options-ctrl-' + ds.meta.id, bindings: {panelCtrl: "="}, attrs: {"panel-ctrl": "ctrl"}, @@ -123,6 +138,7 @@ function pluginDirectiveLoader($compile, datasourceSrv, $rootScope, $q, $http, $ case "annotations-query-ctrl": { return System.import(scope.currentDatasource.meta.module).then(function(dsModule) { return { + baseUrl: scope.currentDatasource.meta.baseUrl, name: 'annotations-query-ctrl-' + scope.currentDatasource.meta.id, bindings: {annotation: "=", datasource: "="}, attrs: {"annotation": "currentAnnotation", datasource: "currentDatasource"}, @@ -134,6 +150,7 @@ function pluginDirectiveLoader($compile, datasourceSrv, $rootScope, $q, $http, $ case 'datasource-config-ctrl': { return System.import(scope.datasourceMeta.module).then(function(dsModule) { return { + baseUrl: scope.datasourceMeta.baseUrl, name: 'ds-config-' + scope.datasourceMeta.id, bindings: {meta: "=", current: "="}, attrs: {meta: "datasourceMeta", current: "current"}, @@ -141,6 +158,32 @@ function pluginDirectiveLoader($compile, datasourceSrv, $rootScope, $q, $http, $ }; }); } + // AppConfigCtrl + case 'app-config-ctrl': { + let appModel = scope.ctrl.appModel; + return System.import(appModel.module).then(function(appModule) { + return { + baseUrl: appModel.baseUrl, + name: 'app-config-' + appModel.appId, + bindings: {appModel: "=", appEditCtrl: "="}, + attrs: {"app-model": "ctrl.appModel", "app-edit-ctrl": "ctrl"}, + Component: appModule.ConfigCtrl, + }; + }); + } + // App Page + case 'app-page': { + let appModel = scope.ctrl.appModel; + return System.import(appModel.module).then(function(appModule) { + return { + baseUrl: appModel.baseUrl, + name: 'app-page-' + appModel.appId + '-' + scope.ctrl.page.slug, + bindings: {appModel: "="}, + attrs: {"app-model": "ctrl.appModel"}, + Component: appModule[scope.ctrl.page.component], + }; + }); + } // Panel case 'panel': { return loadPanelComponentInfo(scope, attrs); diff --git a/public/app/core/routes/all.js b/public/app/core/routes/all.js deleted file mode 100644 index 2d313d7322f..00000000000 --- a/public/app/core/routes/all.js +++ /dev/null @@ -1,166 +0,0 @@ -define([ - 'angular', - '../core_module', - './bundle_loader', - './dashboard_loaders', -], function(angular, coreModule, BundleLoader) { - "use strict"; - - coreModule.default.config(function($routeProvider, $locationProvider) { - $locationProvider.html5Mode(true); - - var loadOrgBundle = new BundleLoader.BundleLoader('app/features/org/all'); - var loadAppsBundle = new BundleLoader.BundleLoader('app/features/apps/all'); - - $routeProvider - .when('/', { - templateUrl: 'public/app/partials/dashboard.html', - controller : 'LoadDashboardCtrl', - reloadOnSearch: false, - }) - .when('/dashboard/:type/:slug', { - templateUrl: 'public/app/partials/dashboard.html', - controller : 'LoadDashboardCtrl', - reloadOnSearch: false, - }) - .when('/dashboard-solo/:type/:slug', { - templateUrl: 'public/app/features/panel/partials/soloPanel.html', - controller : 'SoloPanelCtrl', - }) - .when('/dashboard-import/:file', { - templateUrl: 'public/app/partials/dashboard.html', - controller : 'DashFromImportCtrl', - reloadOnSearch: false, - }) - .when('/dashboard/new', { - templateUrl: 'public/app/partials/dashboard.html', - controller : 'NewDashboardCtrl', - reloadOnSearch: false, - }) - .when('/import/dashboard', { - templateUrl: 'public/app/features/dashboard/partials/import.html', - controller : 'DashboardImportCtrl', - }) - .when('/datasources', { - templateUrl: 'public/app/features/datasources/partials/list.html', - controller : 'DataSourcesCtrl', - resolve: loadOrgBundle, - }) - .when('/datasources/edit/:id', { - templateUrl: 'public/app/features/datasources/partials/edit.html', - controller : 'DataSourceEditCtrl', - resolve: loadOrgBundle, - }) - .when('/datasources/new', { - templateUrl: 'public/app/features/datasources/partials/edit.html', - controller : 'DataSourceEditCtrl', - resolve: loadOrgBundle, - }) - .when('/org', { - templateUrl: 'public/app/features/org/partials/orgDetails.html', - controller : 'OrgDetailsCtrl', - resolve: loadOrgBundle, - }) - .when('/org/new', { - templateUrl: 'public/app/features/org/partials/newOrg.html', - controller : 'NewOrgCtrl', - resolve: loadOrgBundle, - }) - .when('/org/users', { - templateUrl: 'public/app/features/org/partials/orgUsers.html', - controller : 'OrgUsersCtrl', - resolve: loadOrgBundle, - }) - .when('/org/apikeys', { - templateUrl: 'public/app/features/org/partials/orgApiKeys.html', - controller : 'OrgApiKeysCtrl', - resolve: loadOrgBundle, - }) - .when('/profile', { - templateUrl: 'public/app/features/profile/partials/profile.html', - controller : 'ProfileCtrl', - }) - .when('/profile/password', { - templateUrl: 'public/app/features/profile/partials/password.html', - controller : 'ChangePasswordCtrl', - }) - .when('/profile/select-org', { - templateUrl: 'public/app/features/profile/partials/select_org.html', - controller : 'SelectOrgCtrl', - }) - .when('/admin/settings', { - templateUrl: 'public/app/features/admin/partials/settings.html', - controller : 'AdminSettingsCtrl', - }) - .when('/admin/users', { - templateUrl: 'public/app/features/admin/partials/users.html', - controller : 'AdminListUsersCtrl', - }) - .when('/admin/users/create', { - templateUrl: 'public/app/features/admin/partials/new_user.html', - controller : 'AdminEditUserCtrl', - }) - .when('/admin/users/edit/:id', { - templateUrl: 'public/app/features/admin/partials/edit_user.html', - controller : 'AdminEditUserCtrl', - }) - .when('/admin/orgs', { - templateUrl: 'public/app/features/admin/partials/orgs.html', - controller : 'AdminListOrgsCtrl', - }) - .when('/admin/orgs/edit/:id', { - templateUrl: 'public/app/features/admin/partials/edit_org.html', - controller : 'AdminEditOrgCtrl', - }) - .when('/admin/stats', { - templateUrl: 'public/app/features/admin/partials/stats.html', - controller : 'AdminStatsCtrl', - controllerAs: 'ctrl', - }) - .when('/login', { - templateUrl: 'public/app/partials/login.html', - controller : 'LoginCtrl', - }) - .when('/invite/:code', { - templateUrl: 'public/app/partials/signup_invited.html', - controller : 'InvitedCtrl', - }) - .when('/signup', { - templateUrl: 'public/app/partials/signup_step2.html', - controller : 'SignUpCtrl', - }) - .when('/user/password/send-reset-email', { - templateUrl: 'public/app/partials/reset_password.html', - controller : 'ResetPasswordCtrl', - }) - .when('/user/password/reset', { - templateUrl: 'public/app/partials/reset_password.html', - controller : 'ResetPasswordCtrl', - }) - .when('/dashboard/snapshots', { - templateUrl: 'public/app/features/snapshot/partials/snapshots.html', - controller : 'SnapshotsCtrl', - controllerAs: 'ctrl', - }) - .when('/apps', { - templateUrl: 'public/app/features/apps/partials/list.html', - controller: 'AppListCtrl', - controllerAs: 'ctrl', - resolve: loadAppsBundle, - }) - .when('/apps/edit/:appId', { - templateUrl: 'public/app/features/apps/partials/edit.html', - controller: 'AppEditCtrl', - controllerAs: 'ctrl', - resolve: loadAppsBundle, - }) - .when('/global-alerts', { - templateUrl: 'public/app/features/dashboard/partials/globalAlerts.html', - }) - .otherwise({ - templateUrl: 'public/app/partials/error.html', - controller: 'ErrorCtrl' - }); - }); - -}); diff --git a/public/app/core/routes/routes.ts b/public/app/core/routes/routes.ts new file mode 100644 index 00000000000..8963228210b --- /dev/null +++ b/public/app/core/routes/routes.ts @@ -0,0 +1,173 @@ +/// + +import './dashboard_loaders'; + +import angular from 'angular'; +import coreModule from 'app/core/core_module'; +import {BundleLoader} from './bundle_loader'; + +/** @ngInject **/ +function setupAngularRoutes($routeProvider, $locationProvider) { + $locationProvider.html5Mode(true); + + var loadOrgBundle = new BundleLoader('app/features/org/all'); + var loadAppsBundle = new BundleLoader('app/features/apps/all'); + + $routeProvider + .when('/', { + templateUrl: 'public/app/partials/dashboard.html', + controller : 'LoadDashboardCtrl', + reloadOnSearch: false, + }) + .when('/dashboard/:type/:slug', { + templateUrl: 'public/app/partials/dashboard.html', + controller : 'LoadDashboardCtrl', + reloadOnSearch: false, + }) + .when('/dashboard-solo/:type/:slug', { + templateUrl: 'public/app/features/panel/partials/soloPanel.html', + controller : 'SoloPanelCtrl', + }) + .when('/dashboard-import/:file', { + templateUrl: 'public/app/partials/dashboard.html', + controller : 'DashFromImportCtrl', + reloadOnSearch: false, + }) + .when('/dashboard/new', { + templateUrl: 'public/app/partials/dashboard.html', + controller : 'NewDashboardCtrl', + reloadOnSearch: false, + }) + .when('/import/dashboard', { + templateUrl: 'public/app/features/dashboard/partials/import.html', + controller : 'DashboardImportCtrl', + }) + .when('/datasources', { + templateUrl: 'public/app/features/datasources/partials/list.html', + controller : 'DataSourcesCtrl', + resolve: loadOrgBundle, + }) + .when('/datasources/edit/:id', { + templateUrl: 'public/app/features/datasources/partials/edit.html', + controller : 'DataSourceEditCtrl', + resolve: loadOrgBundle, + }) + .when('/datasources/new', { + templateUrl: 'public/app/features/datasources/partials/edit.html', + controller : 'DataSourceEditCtrl', + resolve: loadOrgBundle, + }) + .when('/org', { + templateUrl: 'public/app/features/org/partials/orgDetails.html', + controller : 'OrgDetailsCtrl', + resolve: loadOrgBundle, + }) + .when('/org/new', { + templateUrl: 'public/app/features/org/partials/newOrg.html', + controller : 'NewOrgCtrl', + resolve: loadOrgBundle, + }) + .when('/org/users', { + templateUrl: 'public/app/features/org/partials/orgUsers.html', + controller : 'OrgUsersCtrl', + resolve: loadOrgBundle, + }) + .when('/org/apikeys', { + templateUrl: 'public/app/features/org/partials/orgApiKeys.html', + controller : 'OrgApiKeysCtrl', + resolve: loadOrgBundle, + }) + .when('/profile', { + templateUrl: 'public/app/features/profile/partials/profile.html', + controller : 'ProfileCtrl', + }) + .when('/profile/password', { + templateUrl: 'public/app/features/profile/partials/password.html', + controller : 'ChangePasswordCtrl', + }) + .when('/profile/select-org', { + templateUrl: 'public/app/features/profile/partials/select_org.html', + controller : 'SelectOrgCtrl', + }) + .when('/admin/settings', { + templateUrl: 'public/app/features/admin/partials/settings.html', + controller : 'AdminSettingsCtrl', + }) + .when('/admin/users', { + templateUrl: 'public/app/features/admin/partials/users.html', + controller : 'AdminListUsersCtrl', + }) + .when('/admin/users/create', { + templateUrl: 'public/app/features/admin/partials/new_user.html', + controller : 'AdminEditUserCtrl', + }) + .when('/admin/users/edit/:id', { + templateUrl: 'public/app/features/admin/partials/edit_user.html', + controller : 'AdminEditUserCtrl', + }) + .when('/admin/orgs', { + templateUrl: 'public/app/features/admin/partials/orgs.html', + controller : 'AdminListOrgsCtrl', + }) + .when('/admin/orgs/edit/:id', { + templateUrl: 'public/app/features/admin/partials/edit_org.html', + controller : 'AdminEditOrgCtrl', + }) + .when('/admin/stats', { + templateUrl: 'public/app/features/admin/partials/stats.html', + controller : 'AdminStatsCtrl', + controllerAs: 'ctrl', + }) + .when('/login', { + templateUrl: 'public/app/partials/login.html', + controller : 'LoginCtrl', + }) + .when('/invite/:code', { + templateUrl: 'public/app/partials/signup_invited.html', + controller : 'InvitedCtrl', + }) + .when('/signup', { + templateUrl: 'public/app/partials/signup_step2.html', + controller : 'SignUpCtrl', + }) + .when('/user/password/send-reset-email', { + templateUrl: 'public/app/partials/reset_password.html', + controller : 'ResetPasswordCtrl', + }) + .when('/user/password/reset', { + templateUrl: 'public/app/partials/reset_password.html', + controller : 'ResetPasswordCtrl', + }) + .when('/dashboard/snapshots', { + templateUrl: 'public/app/features/snapshot/partials/snapshots.html', + controller : 'SnapshotsCtrl', + controllerAs: 'ctrl', + }) + .when('/apps', { + templateUrl: 'public/app/features/apps/partials/list.html', + controller: 'AppListCtrl', + controllerAs: 'ctrl', + resolve: loadAppsBundle, + }) + .when('/apps/:appId/edit', { + templateUrl: 'public/app/features/apps/partials/edit.html', + controller: 'AppEditCtrl', + controllerAs: 'ctrl', + resolve: loadAppsBundle, + }) + .when('/apps/:appId/page/:slug', { + templateUrl: 'public/app/features/apps/partials/page.html', + controller: 'AppPageCtrl', + controllerAs: 'ctrl', + resolve: loadAppsBundle, + }) + .when('/global-alerts', { + templateUrl: 'public/app/features/dashboard/partials/globalAlerts.html', + }) + .otherwise({ + templateUrl: 'public/app/partials/error.html', + controller: 'ErrorCtrl' + }); +} + +coreModule.config(setupAngularRoutes); diff --git a/public/app/features/apps/all.ts b/public/app/features/apps/all.ts index 6f15e8b1bf6..9d54165e56b 100644 --- a/public/app/features/apps/all.ts +++ b/public/app/features/apps/all.ts @@ -1,3 +1,3 @@ import './edit_ctrl'; +import './page_ctrl'; import './list_ctrl'; -import './config_view'; diff --git a/public/app/features/apps/config_view.ts b/public/app/features/apps/config_view.ts deleted file mode 100644 index 31e85967cb2..00000000000 --- a/public/app/features/apps/config_view.ts +++ /dev/null @@ -1,23 +0,0 @@ -/// - -import angular from 'angular'; - -/** @ngInject */ -function appConfigView(dynamicDirectiveSrv) { - return dynamicDirectiveSrv.create({ - scope: { - appModel: "=" - }, - directive: scope => { - return System.import(scope.appModel.module).then(function(appModule) { - return { - name: 'app-config-' + scope.appModel.appId, - fn: appModule.configView, - }; - }); - }, - }); -} - - -angular.module('grafana.directives').directive('appConfigView', appConfigView); diff --git a/public/app/features/apps/edit_ctrl.ts b/public/app/features/apps/edit_ctrl.ts index ccdaf529b5e..1cb4dd269ef 100644 --- a/public/app/features/apps/edit_ctrl.ts +++ b/public/app/features/apps/edit_ctrl.ts @@ -5,19 +5,23 @@ import _ from 'lodash'; export class AppEditCtrl { appModel: any; + appId: any; includedPanels: any; + includedDatasources: any; /** @ngInject */ constructor(private backendSrv: any, private $routeParams: any) { this.appModel = {}; + this.appId = $routeParams.appId; - this.backendSrv.get(`/api/org/apps/${this.$routeParams.appId}/settings`).then(result => { + this.backendSrv.get(`/api/org/apps/${this.appId}/settings`).then(result => { this.appModel = result; this.includedPanels = _.where(result.includes, {type: 'panel'}); + this.includedDatasources = _.where(result.includes, {type: 'datasource'}); }); } - update(options) { + update() { var updateCmd = _.extend({ appId: this.appModel.appId, orgId: this.appModel.orgId, @@ -25,19 +29,19 @@ export class AppEditCtrl { pinned: this.appModel.pinned, jsonData: this.appModel.jsonData, secureJsonData: this.appModel.secureJsonData, - }, options); + }, {}); - this.backendSrv.post(`/api/org/apps/${this.$routeParams.appId}/settings`, updateCmd).then(function() { + this.backendSrv.post(`/api/org/apps/${this.appId}/settings`, updateCmd).then(function() { window.location.href = window.location.href; }); } toggleEnabled() { - this.update({enabled: this.appModel.enabled}); + this.update(); } togglePinned() { - this.update({pinned: this.appModel.pinned}); + this.update(); } } diff --git a/public/app/features/apps/list_ctrl.ts b/public/app/features/apps/list_ctrl.ts index 142f0e12b0a..8777f206d88 100644 --- a/public/app/features/apps/list_ctrl.ts +++ b/public/app/features/apps/list_ctrl.ts @@ -6,9 +6,8 @@ export class AppListCtrl { apps: any[]; /** @ngInject */ - constructor(private backendSrv: any) {} + constructor(private backendSrv: any) { - init() { this.backendSrv.get('api/org/apps').then(apps => { this.apps = apps; }); diff --git a/public/app/features/apps/page_ctrl.ts b/public/app/features/apps/page_ctrl.ts new file mode 100644 index 00000000000..15b08ca1cfe --- /dev/null +++ b/public/app/features/apps/page_ctrl.ts @@ -0,0 +1,26 @@ +/// + +import angular from 'angular'; +import _ from 'lodash'; + +export class AppPageCtrl { + page: any; + appId: any; + appModel: any; + + /** @ngInject */ + constructor(private backendSrv, private $routeParams: any, private $rootScope) { + this.appId = $routeParams.appId; + + this.backendSrv.get(`/api/org/apps/${this.appId}/settings`).then(app => { + this.appModel = app; + this.page = _.findWhere(app.pages, {slug: this.$routeParams.slug}); + if (!this.page) { + $rootScope.appEvent('alert-error', ['App Page Not Found', '']); + } + }); + } +} + +angular.module('grafana.controllers').controller('AppPageCtrl', AppPageCtrl); + diff --git a/public/app/features/apps/partials/edit.html b/public/app/features/apps/partials/edit.html index 20df40bd247..4a06d99f8ec 100644 --- a/public/app/features/apps/partials/edit.html +++ b/public/app/features/apps/partials/edit.html @@ -68,7 +68,10 @@ Datasources
    -
  • None
    • +
  • None
    • +
  • + {{ds.name}} +
@@ -78,7 +81,7 @@
@@ -97,7 +100,7 @@

Configuration:

- +
diff --git a/public/app/features/apps/partials/list.html b/public/app/features/apps/partials/list.html index 6d2d8cc24df..98f2cfea5c8 100644 --- a/public/app/features/apps/partials/list.html +++ b/public/app/features/apps/partials/list.html @@ -2,7 +2,7 @@
-
+

Apps

@@ -18,13 +18,13 @@
  • - +
    - + {{app.name}}     diff --git a/public/app/features/apps/partials/page.html b/public/app/features/apps/partials/page.html new file mode 100644 index 00000000000..506708fa654 --- /dev/null +++ b/public/app/features/apps/partials/page.html @@ -0,0 +1,17 @@ + + + + +
    +
    +

    {{ctrl.page.name}}

    + +
    + + +
    + +
    +
    diff --git a/public/app/features/dashboard/all.js b/public/app/features/dashboard/all.js index 9d7cd06019b..073ca2ae1d9 100644 --- a/public/app/features/dashboard/all.js +++ b/public/app/features/dashboard/all.js @@ -12,7 +12,6 @@ define([ './viewStateSrv', './timeSrv', './unsavedChangesSrv', - './directives/dashSearchView', './timepicker/timepicker', './graphiteImportCtrl', './dynamicDashboardSrv', diff --git a/public/app/features/dashboard/dashnav/dashnav.html b/public/app/features/dashboard/dashnav/dashnav.html index 89f9f301729..67c98520bd3 100644 --- a/public/app/features/dashboard/dashnav/dashnav.html +++ b/public/app/features/dashboard/dashnav/dashnav.html @@ -1,6 +1,6 @@ -
    +
    {{dashboard.title}} @@ -8,8 +8,8 @@
    -
    - +
    + {{dashboard.title}} @@ -18,7 +18,7 @@
    -
      +
        -
      • +
      • Back to dashboard diff --git a/public/app/features/dashboard/directives/dashSearchView.js b/public/app/features/dashboard/directives/dashSearchView.js deleted file mode 100644 index cee97fee758..00000000000 --- a/public/app/features/dashboard/directives/dashSearchView.js +++ /dev/null @@ -1,54 +0,0 @@ -define([ - 'angular', - 'jquery' -], -function (angular, $) { - 'use strict'; - - angular - .module('grafana.directives') - .directive('dashSearchView', function($compile) { - return { - restrict: 'A', - link: function(scope, elem) { - var editorScope; - var ignoreHide; - - function showSearch() { - if (editorScope) { - editorScope.dismiss(); - return; - } - - ignoreHide = true; - editorScope = scope.$new(); - editorScope.dismiss = function() { - editorScope.$destroy(); - elem.empty(); - elem.unbind(); - editorScope = null; - }; - - var view = $(''); - - elem.append(view); - $compile(elem.contents())(editorScope); - - setTimeout(function() { - ignoreHide = false; - }, 300); - } - - function hideSearch() { - if (editorScope && !ignoreHide) { - editorScope.dismiss(); - } - } - - scope.onAppEvent('show-dash-search', showSearch); - scope.onAppEvent('hide-dash-search', hideSearch); - } - }; - }); - -}); diff --git a/public/app/features/dashboard/dynamicDashboardSrv.js b/public/app/features/dashboard/dynamicDashboardSrv.js index 5c0101be10f..9e369733f45 100644 --- a/public/app/features/dashboard/dynamicDashboardSrv.js +++ b/public/app/features/dashboard/dynamicDashboardSrv.js @@ -31,7 +31,6 @@ function (angular, _) { var i, j, row, panel; for (i = 0; i < this.dashboard.rows.length; i++) { row = this.dashboard.rows[i]; - // handle row repeats if (row.repeat) { this.repeatRow(row, i); @@ -40,6 +39,7 @@ function (angular, _) { else if (row.repeatRowId && row.repeatIteration !== this.iteration) { this.dashboard.rows.splice(i, 1); i = i - 1; + continue; } // repeat panels @@ -52,6 +52,8 @@ function (angular, _) { else if (panel.repeatPanelId && panel.repeatIteration !== this.iteration) { row.panels = _.without(row.panels, panel); j = j - 1; + } else if (!_.isEmpty(panel.scopedVars) && panel.repeatIteration !== this.iteration) { + panel.scopedVars = {}; } } } @@ -116,8 +118,9 @@ function (angular, _) { panel = copy.panels[i]; panel.scopedVars = {}; panel.scopedVars[variable.name] = option; + panel.repeatIteration = this.iteration; } - }); + }, this); }; this.getPanelClone = function(sourcePanel, row, index) { diff --git a/public/app/features/dashboard/timepicker/timepicker.html b/public/app/features/dashboard/timepicker/timepicker.html index 66093110700..10a1d6820a3 100644 --- a/public/app/features/dashboard/timepicker/timepicker.html +++ b/public/app/features/dashboard/timepicker/timepicker.html @@ -1,13 +1,13 @@
          -
        • +
        • Zoom Out
          • -
        • - +
        • + @@ -21,7 +21,7 @@
          • -
        • +
        • diff --git a/public/app/features/panel/metrics_panel_ctrl.ts b/public/app/features/panel/metrics_panel_ctrl.ts index 1e6f34775f5..537bdbbcbdb 100644 --- a/public/app/features/panel/metrics_panel_ctrl.ts +++ b/public/app/features/panel/metrics_panel_ctrl.ts @@ -6,8 +6,8 @@ import _ from 'lodash'; import kbn from 'app/core/utils/kbn'; import {PanelCtrl} from './panel_ctrl'; -import * as rangeUtil from '../../core/utils/rangeutil'; -import * as dateMath from '../../core/utils/datemath'; +import * as rangeUtil from 'app/core/utils/rangeutil'; +import * as dateMath from 'app/core/utils/datemath'; class MetricsPanelCtrl extends PanelCtrl { error: boolean; @@ -111,49 +111,49 @@ class MetricsPanelCtrl extends PanelCtrl { var panelInterval = this.panel.interval; var datasourceInterval = (this.datasource || {}).interval; - this.interval = kbn.calculateInterval(this.range, this.resolution, panelInterval || datasourceInterval); - }; + this.interval = kbn.calculateInterval(this.range, this.resolution, panelInterval || datasourceInterval); + }; - applyPanelTimeOverrides() { + applyPanelTimeOverrides() { + this.timeInfo = ''; + + // check panel time overrrides + if (this.panel.timeFrom) { + var timeFromInfo = rangeUtil.describeTextRange(this.panel.timeFrom); + if (timeFromInfo.invalid) { + this.timeInfo = 'invalid time override'; + return; + } + + if (_.isString(this.rangeRaw.from)) { + var timeFromDate = dateMath.parse(timeFromInfo.from); + this.timeInfo = timeFromInfo.display; + this.rangeRaw.from = timeFromInfo.from; + this.rangeRaw.to = timeFromInfo.to; + this.range.from = timeFromDate; + this.range.to = dateMath.parse(timeFromInfo.to); + } + } + + if (this.panel.timeShift) { + var timeShiftInfo = rangeUtil.describeTextRange(this.panel.timeShift); + if (timeShiftInfo.invalid) { + this.timeInfo = 'invalid timeshift'; + return; + } + + var timeShift = '-' + this.panel.timeShift; + this.timeInfo += ' timeshift ' + timeShift; + this.range.from = dateMath.parseDateMath(timeShift, this.range.from, false); + this.range.to = dateMath.parseDateMath(timeShift, this.range.to, true); + + this.rangeRaw = this.range; + } + + if (this.panel.hideTimeOverride) { this.timeInfo = ''; - - // check panel time overrrides - if (this.panel.timeFrom) { - var timeFromInfo = rangeUtil.describeTextRange(this.panel.timeFrom); - if (timeFromInfo.invalid) { - this.timeInfo = 'invalid time override'; - return; - } - - if (_.isString(this.rangeRaw.from)) { - var timeFromDate = dateMath.parse(timeFromInfo.from); - this.timeInfo = timeFromInfo.display; - this.rangeRaw.from = timeFromInfo.from; - this.rangeRaw.to = timeFromInfo.to; - this.range.from = timeFromDate; - this.range.to = dateMath.parse(timeFromInfo.to); - } - } - - if (this.panel.timeShift) { - var timeShiftInfo = rangeUtil.describeTextRange(this.panel.timeShift); - if (timeShiftInfo.invalid) { - this.timeInfo = 'invalid timeshift'; - return; - } - - var timeShift = '-' + this.panel.timeShift; - this.timeInfo += ' timeshift ' + timeShift; - this.range.from = dateMath.parseDateMath(timeShift, this.range.from, false); - this.range.to = dateMath.parseDateMath(timeShift, this.range.to, true); - - this.rangeRaw = this.range; - } - - if (this.panel.hideTimeOverride) { - this.timeInfo = ''; - } - }; + } + }; issueQueries(datasource) { this.updateTimeRange(); diff --git a/public/app/features/panel/panel.ts b/public/app/features/panel/panel.ts deleted file mode 100644 index c2415aaa1da..00000000000 --- a/public/app/features/panel/panel.ts +++ /dev/null @@ -1,13 +0,0 @@ -/// - -import config from 'app/core/config'; - -import {PanelCtrl} from './panel_ctrl'; -import {MetricsPanelCtrl} from './metrics_panel_ctrl'; -import {QueryCtrl} from './query_ctrl'; - -export { - PanelCtrl, - MetricsPanelCtrl, - QueryCtrl, -} diff --git a/public/app/features/panel/panel_directive.ts b/public/app/features/panel/panel_directive.ts index 757f9f8e43d..2497732928b 100644 --- a/public/app/features/panel/panel_directive.ts +++ b/public/app/features/panel/panel_directive.ts @@ -5,10 +5,59 @@ import $ from 'jquery'; var module = angular.module('grafana.directives'); +var panelTemplate = ` +
          +
          + + + + + + + + + + +
          +
          + +
          + +
          + +
          + +
          +
          +
          +
          + + {{ctrl.name}} +
          + +
          +
          +
          +
          + + +
          + +
          +
          + +
          +
          +
          +
          +`; + module.directive('grafanaPanel', function() { return { restrict: 'E', - templateUrl: 'public/app/features/panel/partials/panel.html', + template: panelTemplate, transclude: true, scope: { ctrl: "=" }, link: function(scope, elem) { diff --git a/public/app/features/panel/partials/panel.html b/public/app/features/panel/partials/panel.html deleted file mode 100644 index b5228840687..00000000000 --- a/public/app/features/panel/partials/panel.html +++ /dev/null @@ -1,47 +0,0 @@ -
          -
          - - - - - - - - - - -
          -
          - -
          - -
          - -
          - -
          -
          -
          -
          - - {{ctrl.name}} -
          - -
          -
          -
          -
          - - -
          - -
          -
          - -
          -
          -
          -
          - diff --git a/public/app/headers/angular2/angular2.d.ts b/public/app/headers/angular2/angular2.d.ts deleted file mode 100644 index 133527dec3d..00000000000 --- a/public/app/headers/angular2/angular2.d.ts +++ /dev/null @@ -1,12213 +0,0 @@ -// Type definitions for Angular v2.0.0-alpha.37 -// Project: http://angular.io/ -// Definitions by: angular team -// Definitions: https://github.com/borisyankov/DefinitelyTyped - -// *********************************************************** -// This file is generated by the Angular build process. -// Please do not create manual edits or send pull requests -// modifying this file. -// *********************************************************** - -// angular2/angular2 depends transitively on these libraries. -// If you don't have them installed you can install them using TSD -// https://github.com/DefinitelyTyped/tsd - -/// -/// -// angular2/web_worker/worker depends transitively on these libraries. -// If you don't have them installed you can install them using TSD -// https://github.com/DefinitelyTyped/tsd - -/// -/// -// angular2/web_worker/ui depends transitively on these libraries. -// If you don't have them installed you can install them using TSD -// https://github.com/DefinitelyTyped/tsd - -/// -/// - - -interface Map {} -interface StringMap extends Map {} - - -declare module ng { - // See https://github.com/Microsoft/TypeScript/issues/1168 - class BaseException /* extends Error */ { - message: string; - stack: string; - toString(): string; - } - interface InjectableReference {} -} - -declare module ngWorker { - // See https://github.com/Microsoft/TypeScript/issues/1168 - class BaseException /* extends Error */ { - message: string; - stack: string; - toString(): string; - } - interface InjectableReference {} -} - -declare module ngUi { - // See https://github.com/Microsoft/TypeScript/issues/1168 - class BaseException /* extends Error */ { - message: string; - stack: string; - toString(): string; - } - interface InjectableReference {} -} - - - - - -/** - * The `angular2` is the single place to import all of the individual types. - */ -declare module ng { - - /** - * Bootstrapping for Angular applications. - * - * You instantiate an Angular application by explicitly specifying a component to use as the root - * component for your - * application via the `bootstrap()` method. - * - * ## Simple Example - * - * Assuming this `index.html`: - * - * ```html - * - * - * - * loading... - * - * - * ``` - * - * An application is bootstrapped inside an existing browser DOM, typically `index.html`. Unlike - * Angular 1, Angular 2 - * does not compile/process bindings in `index.html`. This is mainly for security reasons, as well - * as architectural - * changes in Angular 2. This means that `index.html` can safely be processed using server-side - * technologies such as - * bindings. Bindings can thus use double-curly `{{ syntax }}` without collision from Angular 2 - * component double-curly - * `{{ syntax }}`. - * - * We can use this script code: - * - * ``` - * @Component({ - * selector: 'my-app' - * }) - * @View({ - * template: 'Hello {{ name }}!' - * }) - * class MyApp { - * name:string; - * - * constructor() { - * this.name = 'World'; - * } - * } - * - * main() { - * return bootstrap(MyApp); - * } - * ``` - * - * When the app developer invokes `bootstrap()` with the root component `MyApp` as its argument, - * Angular performs the - * following tasks: - * - * 1. It uses the component's `selector` property to locate the DOM element which needs to be - * upgraded into - * the angular component. - * 2. It creates a new child injector (from the platform injector). Optionally, you can also - * override the injector configuration for an app by - * invoking `bootstrap` with the `componentInjectableBindings` argument. - * 3. It creates a new `Zone` and connects it to the angular application's change detection domain - * instance. - * 4. It creates a shadow DOM on the selected component's host element and loads the template into - * it. - * 5. It instantiates the specified component. - * 6. Finally, Angular performs change detection to apply the initial data bindings for the - * application. - * - * - * ## Instantiating Multiple Applications on a Single Page - * - * There are two ways to do this. - * - * - * ### Isolated Applications - * - * Angular creates a new application each time that the `bootstrap()` method is invoked. When - * multiple applications - * are created for a page, Angular treats each application as independent within an isolated change - * detection and - * `Zone` domain. If you need to share data between applications, use the strategy described in the - * next - * section, "Applications That Share Change Detection." - * - * - * ### Applications That Share Change Detection - * - * If you need to bootstrap multiple applications that share common data, the applications must - * share a common - * change detection and zone. To do that, create a meta-component that lists the application - * components in its template. - * By only invoking the `bootstrap()` method once, with the meta-component as its argument, you - * ensure that only a - * single change detection zone is created and therefore data can be shared across the applications. - * - * - * ## Platform Injector - * - * When working within a browser window, there are many singleton resources: cookies, title, - * location, and others. - * Angular services that represent these resources must likewise be shared across all Angular - * applications that - * occupy the same browser window. For this reason, Angular creates exactly one global platform - * injector which stores - * all shared services, and each angular application injector has the platform injector as its - * parent. - * - * Each application has its own private injector as well. When there are multiple applications on a - * page, Angular treats - * each application injector's services as private to that application. - * - * - * # API - * - `appComponentType`: The root component which should act as the application. This is a reference - * to a `Type` - * which is annotated with `@Component(...)`. - * - `componentInjectableBindings`: An additional set of bindings that can be added to the app - * injector - * to override default injection behavior. - * - `errorReporter`: `function(exception:any, stackTrace:string)` a default error reporter for - * unhandled exceptions. - * - * Returns a `Promise` of {@link ApplicationRef}. - */ - function bootstrap(appComponentType: /*Type*/ any, componentInjectableBindings?: Array) : Promise ; - - - /** - * Declare reusable UI building blocks for an application. - * - * Each Angular component requires a single `@Component` and at least one `@View` annotation. The - * `@Component` - * annotation specifies when a component is instantiated, and which properties and hostListeners it - * binds to. - * - * When a component is instantiated, Angular - * - creates a shadow DOM for the component. - * - loads the selected template into the shadow DOM. - * - creates all the injectable objects configured with `bindings` and `viewBindings`. - * - * All template expressions and statements are then evaluated against the component instance. - * - * For details on the `@View` annotation, see {@link ViewMetadata}. - * - * ## Example - * - * ``` - * @Component({ - * selector: 'greet' - * }) - * @View({ - * template: 'Hello {{name}}!' - * }) - * class Greet { - * name: string; - * - * constructor() { - * this.name = 'World'; - * } - * } - * ``` - */ - class ComponentMetadata extends DirectiveMetadata { - - - /** - * Defines the used change detection strategy. - * - * When a component is instantiated, Angular creates a change detector, which is responsible for - * propagating the component's bindings. - * - * The `changeDetection` property defines, whether the change detection will be checked every time - * or only when the component tells it to do so. - */ - changeDetection: ChangeDetectionStrategy; - - - /** - * Defines the set of injectable objects that are visible to its view dom children. - * - * ## Simple Example - * - * Here is an example of a class that can be injected: - * - * ``` - * class Greeter { - * greet(name:string) { - * return 'Hello ' + name + '!'; - * } - * } - * - * @Directive({ - * selector: 'needs-greeter' - * }) - * class NeedsGreeter { - * greeter:Greeter; - * - * constructor(greeter:Greeter) { - * this.greeter = greeter; - * } - * } - * - * @Component({ - * selector: 'greet', - * viewBindings: [ - * Greeter - * ] - * }) - * @View({ - * template: ``, - * directives: [NeedsGreeter] - * }) - * class HelloWorld { - * } - * - * ``` - */ - viewBindings: any[]; - } - - - /** - * Directives allow you to attach behavior to elements in the DOM. - * - * {@link DirectiveMetadata}s with an embedded view are called {@link ComponentMetadata}s. - * - * A directive consists of a single directive annotation and a controller class. When the - * directive's `selector` matches - * elements in the DOM, the following steps occur: - * - * 1. For each directive, the `ElementInjector` attempts to resolve the directive's constructor - * arguments. - * 2. Angular instantiates directives for each matched element using `ElementInjector` in a - * depth-first order, - * as declared in the HTML. - * - * ## Understanding How Injection Works - * - * There are three stages of injection resolution. - * - *Pre-existing Injectors*: - * - The terminal {@link Injector} cannot resolve dependencies. It either throws an error or, if - * the dependency was - * specified as `@Optional`, returns `null`. - * - The platform injector resolves browser singleton resources, such as: cookies, title, - * location, and others. - * - *Component Injectors*: Each component instance has its own {@link Injector}, and they follow - * the same parent-child hierarchy - * as the component instances in the DOM. - * - *Element Injectors*: Each component instance has a Shadow DOM. Within the Shadow DOM each - * element has an `ElementInjector` - * which follow the same parent-child hierarchy as the DOM elements themselves. - * - * When a template is instantiated, it also must instantiate the corresponding directives in a - * depth-first order. The - * current `ElementInjector` resolves the constructor dependencies for each directive. - * - * Angular then resolves dependencies as follows, according to the order in which they appear in the - * {@link ViewMetadata}: - * - * 1. Dependencies on the current element - * 2. Dependencies on element injectors and their parents until it encounters a Shadow DOM boundary - * 3. Dependencies on component injectors and their parents until it encounters the root component - * 4. Dependencies on pre-existing injectors - * - * - * The `ElementInjector` can inject other directives, element-specific special objects, or it can - * delegate to the parent - * injector. - * - * To inject other directives, declare the constructor parameter as: - * - `directive:DirectiveType`: a directive on the current element only - * - `@Host() directive:DirectiveType`: any directive that matches the type between the current - * element and the - * Shadow DOM root. - * - `@Query(DirectiveType) query:QueryList`: A live collection of direct child - * directives. - * - `@QueryDescendants(DirectiveType) query:QueryList`: A live collection of any - * child directives. - * - * To inject element-specific special objects, declare the constructor parameter as: - * - `element: ElementRef` to obtain a reference to logical element in the view. - * - `viewContainer: ViewContainerRef` to control child template instantiation, for - * {@link DirectiveMetadata} directives only - * - `bindingPropagation: BindingPropagation` to control change detection in a more granular way. - * - * ## Example - * - * The following example demonstrates how dependency injection resolves constructor arguments in - * practice. - * - * - * Assume this HTML template: - * - * ``` - *
          - *
          - *
          - *
          - *
          - *
          - *
          - *
          - *
          - *
          - * ``` - * - * With the following `dependency` decorator and `SomeService` injectable class. - * - * ``` - * @Injectable() - * class SomeService { - * } - * - * @Directive({ - * selector: '[dependency]', - * properties: [ - * 'id: dependency' - * ] - * }) - * class Dependency { - * id:string; - * } - * ``` - * - * Let's step through the different ways in which `MyDirective` could be declared... - * - * - * ### No injection - * - * Here the constructor is declared with no arguments, therefore nothing is injected into - * `MyDirective`. - * - * ``` - * @Directive({ selector: '[my-directive]' }) - * class MyDirective { - * constructor() { - * } - * } - * ``` - * - * This directive would be instantiated with no dependencies. - * - * - * ### Component-level injection - * - * Directives can inject any injectable instance from the closest component injector or any of its - * parents. - * - * Here, the constructor declares a parameter, `someService`, and injects the `SomeService` type - * from the parent - * component's injector. - * ``` - * @Directive({ selector: '[my-directive]' }) - * class MyDirective { - * constructor(someService: SomeService) { - * } - * } - * ``` - * - * This directive would be instantiated with a dependency on `SomeService`. - * - * - * ### Injecting a directive from the current element - * - * Directives can inject other directives declared on the current element. - * - * ``` - * @Directive({ selector: '[my-directive]' }) - * class MyDirective { - * constructor(dependency: Dependency) { - * expect(dependency.id).toEqual(3); - * } - * } - * ``` - * This directive would be instantiated with `Dependency` declared at the same element, in this case - * `dependency="3"`. - * - * ### Injecting a directive from any ancestor elements - * - * Directives can inject other directives declared on any ancestor element (in the current Shadow - * DOM), i.e. on the current element, the - * parent element, or its parents. - * ``` - * @Directive({ selector: '[my-directive]' }) - * class MyDirective { - * constructor(@Host() dependency: Dependency) { - * expect(dependency.id).toEqual(2); - * } - * } - * ``` - * - * `@Host` checks the current element, the parent, as well as its parents recursively. If - * `dependency="2"` didn't - * exist on the direct parent, this injection would - * have returned - * `dependency="1"`. - * - * - * ### Injecting a live collection of direct child directives - * - * - * A directive can also query for other child directives. Since parent directives are instantiated - * before child directives, a directive can't simply inject the list of child directives. Instead, - * the directive injects a {@link QueryList}, which updates its contents as children are added, - * removed, or moved by a directive that uses a {@link ViewContainerRef} such as a `ng-for`, an - * `ng-if`, or an `ng-switch`. - * - * ``` - * @Directive({ selector: '[my-directive]' }) - * class MyDirective { - * constructor(@Query(Dependency) dependencies:QueryList) { - * } - * } - * ``` - * - * This directive would be instantiated with a {@link QueryList} which contains `Dependency` 4 and - * 6. Here, `Dependency` 5 would not be included, because it is not a direct child. - * - * ### Injecting a live collection of descendant directives - * - * By passing the descendant flag to `@Query` above, we can include the children of the child - * elements. - * - * ``` - * @Directive({ selector: '[my-directive]' }) - * class MyDirective { - * constructor(@Query(Dependency, {descendants: true}) dependencies:QueryList) { - * } - * } - * ``` - * - * This directive would be instantiated with a Query which would contain `Dependency` 4, 5 and 6. - * - * ### Optional injection - * - * The normal behavior of directives is to return an error when a specified dependency cannot be - * resolved. If you - * would like to inject `null` on unresolved dependency instead, you can annotate that dependency - * with `@Optional()`. - * This explicitly permits the author of a template to treat some of the surrounding directives as - * optional. - * - * ``` - * @Directive({ selector: '[my-directive]' }) - * class MyDirective { - * constructor(@Optional() dependency:Dependency) { - * } - * } - * ``` - * - * This directive would be instantiated with a `Dependency` directive found on the current element. - * If none can be - * found, the injector supplies `null` instead of throwing an error. - * - * ## Example - * - * Here we use a decorator directive to simply define basic tool-tip behavior. - * - * ``` - * @Directive({ - * selector: '[tooltip]', - * properties: [ - * 'text: tooltip' - * ], - * host: { - * '(mouseenter)': 'onMouseEnter()', - * '(mouseleave)': 'onMouseLeave()' - * } - * }) - * class Tooltip{ - * text:string; - * overlay:Overlay; // NOT YET IMPLEMENTED - * overlayManager:OverlayManager; // NOT YET IMPLEMENTED - * - * constructor(overlayManager:OverlayManager) { - * this.overlay = overlay; - * } - * - * onMouseEnter() { - * // exact signature to be determined - * this.overlay = this.overlayManager.open(text, ...); - * } - * - * onMouseLeave() { - * this.overlay.close(); - * this.overlay = null; - * } - * } - * ``` - * In our HTML template, we can then add this behavior to a `
          ` or any other element with the - * `tooltip` selector, - * like so: - * - * ``` - *
          - * ``` - * - * Directives can also control the instantiation, destruction, and positioning of inline template - * elements: - * - * A directive uses a {@link ViewContainerRef} to instantiate, insert, move, and destroy views at - * runtime. - * The {@link ViewContainerRef} is created as a result of `