Commit Graph

108 Commits

Author SHA1 Message Date
Saturnino Abril
30011f67e8 [MM-10354] Add feature to remove team icon (#8684)
* set team.LastTeamIconUpdate to 0 when removing team icon

* add APIv4 for removing team icon

* removed comment and updated typo on AppError
2018-04-30 17:57:57 +08:00
Christopher Speller
2e6b3da1d3 Fixing some merge conflicts with structured logging. (#8685) 2018-04-27 13:31:50 -07:00
Christopher Speller
686c2fbab7 Structured logging (#8673)
* Implementing structured logging

* Changes to en.json to allow refactor to run.

* Fixing global logger

* Structured logger initalization.

* Add caller.

* Do some log redirection.

* Auto refactor

* Cleaning up l4g reference and removing dependancy.

* Removing junk.

* Copyright headers.

* Fixing tests

* Revert "Changes to en.json to allow refactor to run."

This reverts commit fd8249e99b.

* Fixing some auto refactor strangeness and typo.

* Making keys more human readable.
2018-04-27 12:49:45 -07:00
Jesse Hallam
7abd6176e9 modestly extend SQLite support (#8677)
This is primarily to unblock the advanced permissions merge. There's
still more work to do if we wanted this to be mainstream, but as it's
mainly for unit tests, I'm just focussing on the minimum.
2018-04-27 10:07:36 +01:00
George Goldberg
b13a228b04 MM-10121: CLI command to reset permissions system to default state. (#8637)
* MM-10121: CLI command to reset permissions system to default state.

* Review comment.
2018-04-18 10:18:07 +01:00
Jesse Hallam
3a4c7603b3 MM-10020: avoid duplicating unique indexes (#8587)
* unit test TestGet(Search)Replica

This adds partial, testing-focused support for SQLite, as well as removing
some translated log messages that required initializing i18n just for
testing.

* avoid returning master twice when no replicas are configured

* remove duplicate indexes

* unit test GetAllConns
2018-04-16 18:23:20 -07:00
Derrick Anderson
f9015a37f3 uncomment 4.9 upgrade code (#8582)
* set db to 4.9

* add prepatory code for 4.10

* bug from old example
2018-04-05 11:45:55 -04:00
Jesús Espino
014a3b6a60 Fixing misspell errors (#8544) 2018-03-29 10:04:54 -04:00
Jesse Hallam
d4c0494360 MM-9661: tweak post message size log (#8517)
Call out the number of supported characters explicitly, moving the byte
limit to parentheses.
2018-03-29 10:02:58 -04:00
Jesse Hallam
9c9a9ade12 Remove the index on Channels.DisplayName. (#8530)
As outlined in [this
discussion](https://pre-release.mattermost.com/core/pl/uw5bwmkb6irkbkn6pk9rkzpytr),
this index causes issues with MySQL's query planner, leading to full
table scans in a case where it would have made more sense to leverage a
filesort.
2018-03-28 09:22:55 -07:00
George Goldberg
a28f6e7748 Fix roles table ID max length so MySQL 5.6 doesn't fail. (#8532) 2018-03-28 09:58:00 +01:00
Martin Kraft
e13e64711f Merge remote-tracking branch 'origin/master' into advanced-permissions-phase-1 2018-03-27 09:01:42 -04:00
Jesse Hallam
8491ba5740 Relax 4k post message limit (#8478)
* MM-9661: rename POST_MESSAGE_MAX_RUNES to \0_v1

* MM-9661: s/4000/POST_MESSAGE_MAX_RUNES_V1/ in tests

* MM-9661: introduce POST_MESSAGE_MAX_RUNES_V2

* MM-9661: migrate Postgres Posts.Message column to TEXT from VARCHAR(4000)

This is safe to do in a production instance since the underyling type is
not changing. We explicitly don't do this automatically for MySQL, but
also don't need to since the ORM would have already created a TEXT column
for MySQL in that case.

* MM-9661: emit MaxPostSize in client config

This value remains unconfigurable at this time, but exposes the current
limit to the client. The limit remains at 4k in this commit.

* MM-9661: introduce and use SqlPostStore.GetMaxPostSize

Enforce a byte limitation in the database, and use 1/4 of that value as
the rune count limitation (assuming a worst case UTF-8 representation).

* move maxPostSizeCached, lastPostsCache and lastPostTimeCache out of the global context and onto the SqlPostStore

* address feedback from code review:

* ensure sqlstore unit tests are actually being run
* move global caches into SqlPostStore
* leverage sync.Once to address a race condition

* modify upgrade semantics to match new db semantics

gorp's behaviour on creating columns with a maximum length on Postgres
differs from MySQL:
* Postgres
  * gorp uses TEXT for string columns without a maximum length
  * gorp uses VARCHAR(N) for string columns with a maximum length of N
* MySQL
  * gorp uses TEXT for string columns with a maximum length >= 256
  * gorp uses VARCHAR(N) for string columns with a maximum length of N
  * gorp defaults to a maximum length of 255, implying VARCHAR(255)

So the Message column has been TEXT on MySQL but VARCHAR(4000) on
Postgres. With the new, longer limits of 65535, and without changes to
gorp, the expected behaviour is TEXT on MySQL and VARCHAR(65535) on
Postgres. This commit makes the upgrade semantics match the new database
semantics.

Ideally, we'd revisit the gorp behaviour at a later time.

* allow TestMaxPostSize test cases to actually run in parallel

* default maxPostSizeCached to POST_MESSAGE_MAX_RUNES_V1 in case the once initializer panics

* fix casting error

* MM-9661: skip the schema migration for Postgres

It turns out resizing VARCHAR requires a rewrite in some versions of
Postgres, but migrating VARCHAR to TEXT does not. Given the increasing
complexity, let's defer the migration to the enduser instead.
2018-03-26 17:55:35 -04:00
Martin Kraft
5fa1b35819 Merge branch 'master' into advanced-permissions-phase-1 2018-03-23 09:08:49 -04:00
Chris Duarte
74e703f58d Timezone feature (#8185)
* Add supported timezones into config

Remove Timezone list creation in favor of timezone from configs

Add Timezone field to Users table

Clean up format of SupportedTimezones in config

* Remove unwanted change

* Add test for updating user timezone

* Add empty map[string]string if Timezone is null

* Add EnableTimezoneSelection config

* Revert back to map[string]string for ClientConfig

* Refactor SupportedTimezones into timezones.json

* Include timezones.json in TestConfigFlag

* Add timezone api endpoint

* Bump varchar size to 256 and setMaxSize in user_store

* Refactor LoadConfig to LoadConfig and LoadTimezoneConfig

* Remove unnecessary argument in LoadConfig, mail_test

* Add test for timezone endpoint

* Add license header

* Refactor timezones endpoint to system.go

* Add system base route to timezone endpoint

* db timezone upgrade in db v4.9

* Avoid saving SupportedTimezones to config.json

* Add timezonePath support in config

* Remove EnableTimezoneSelection from config

* Use return statement without return parameter

* Refactor test for SupportedTimezones

* Check for supportedTimezone != nil instead of using len

* Decouple SupportedTimezones out of Config

* Fix failing test

* Add LastTeamIconUpdate back in upgrade.go

* Write timezone config in config_flag_test

* Add code fallback for default timezone support
2018-03-22 09:53:43 -04:00
George Goldberg
37f0e5e0eb Merge branch 'master' into advanced-permissions-phase-1 2018-03-19 10:53:37 +00:00
Jesse Hallam
4b675b347b MM-9770: rewrite getParentsPosts to improve performance (#8467)
* rename variables in testPostStoreGetPostsWithDetails

This helps to clarify the structure of the posts under test.

* clarify and expand existing testPostStoreGetPostsWithDetails assertions

* expand testPostStoreGetPostsWithDetails assertions

This verifies that replies to posts in the window, themselves not in the
window (because of a non-zero offset) are still fetched. They were
previously missing.

* MM-9770: rewrite getParentsPosts to improve performance

See discussion on ~developers-performance, but the basic idea here is to
force the database to use the `PRIMARY` index when fetching posts
instead of trying to filter down by channel and doing a scan.
2018-03-16 08:10:14 -07:00
George Goldberg
fadcdd271a Merge branch 'master' into advanced-permissions-phase-1 2018-03-13 13:36:23 +00:00
Martin Kraft
426796829a MM-8839: Adds ChannelType to compliance-related model. (#8424) 2018-03-09 10:30:05 -05:00
Joram Wilander
8ad99b4b1f Remove query to update channel extra_update_at field on user activation/deactivation (#8415) 2018-03-09 08:06:31 -05:00
Stephen Kiers
302dae5bb9 MM-9274- Sort Users in Channel by status (#8181)
* sort by lastActivity

* added status ordering to Users

* sort offline before dnd

* remove data not needed

* added seperate call for when order=‘status’ is on GetUser request

* remove PrintLn

* styling fix

* remove mistake

* mistake 2

* better comment

* explicit if statemnt

* writing tests

* removed manually added mocks

* generated mock

* ICU-668 Added unit tests

* style fix

* sort by lastActivity

* added status ordering to Users

* sort offline before dnd

* remove data not needed

* added seperate call for when order=‘status’ is on GetUser request

* remove PrintLn

* styling fix

* remove mistake

* mistake 2

* better comment

* explicit if statemnt

* writing tests

* removed manually added mocks

* generated mock

* ICU-668 Added unit tests

* style fix

* reverse dnd and offline

* Fixed app.SaveStatusAndBroadcast

* Fixed incorrect merge

* Fixing incorrect merge again
2018-03-09 07:48:30 -05:00
Joram Wilander
4081bb2a71 Moved team icon upgrade code to 4.9 since it didn't make it in for 4.8 (#8410) 2018-03-08 07:18:49 -05:00
Jonathan
d448a6bef3 XYZ-87: GlobalRelay DM Exports do not Include Channel Name (#8275)
* Adding ChannelMemberHistory table records when DM channels are created. This ensures that both participants in a DM are shown in compliance export, even if only one of them typed anything

* Direct/Group Message channels now get pretty display names for the purpose of compliance exports

* Fixed string formatting in t.Fatal calls

* Changed uses of ChannelMemberHistory over to ChannelMemberHistoryResult in tests. This should have been done as a part of the work in XYZ-110, but seems to have been missed in this branch for some reason
2018-03-07 15:54:51 +00:00
Joram Wilander
6b3c9a480a Add preparatory upgrade code for 4.9 (#8408) 2018-03-05 14:39:49 -05:00
Joram Wilander
4a1802c039 MM-9664 Add invalidation metrics for store caches (#8340)
* Add invalidation metrics for store caches

* Increment session invalidation metric

* Fix tests
2018-03-05 10:35:26 -05:00
George Goldberg
901acc9703 Merge branch 'master' into advanced-permissions-phase-1 2018-03-02 15:55:03 +00:00
Christian Hoff
2b3b6051d2 PLT-7567: Integration of Team Icons (#8284)
* PLT-7567: Integration of Team Icons

* PLT-7567: Read replica workaround, upgrade logic moved, more concrete i18n key

* PLT-7567: Read replica workaround, corrections

* PLT-7567: upgrade correction
2018-03-01 19:11:44 +00:00
George Goldberg
d2b70b8671 Simplification of a Channel Members query. (#8345) 2018-02-28 23:07:09 +00:00
Saturnino Abril
45d7c51d9f merge 4.7.1 to master 2018-02-19 23:47:34 +08:00
George Goldberg
f8289eb286 Merge branch 'master' into advanced-permissions-phase-1 2018-02-19 11:19:39 +00:00
JoramWilander
291432e254 Add upgrade code from 4.7.0 to 4.7.1 2018-02-16 17:14:22 -05:00
Jesús Espino
2930766c65 PLT-8723: Fix DeadLock on reactions insertions (#8225)
* PLT-8723: Fix DeadLock on reactions insertions

* Improved the HasReactions update SQL
2018-02-15 16:41:49 -08:00
Martin Kraft
642d78173a Merge remote-tracking branch 'origin/master' into advanced-permissions-phase-1 2018-02-14 08:53:25 -05:00
Jonathan
bf1fdf92c7 XYZ-110: Cherrypicking changes from release-4.7 to master (#8254)
* Cherry-picking 7b2861de3a to master because enterprise/master has code in it that enterprise/release-4.7 does not, and I don't want enterprise/master to break the next time somebody merges enterprise/release-4.7 to enterprise/master

* Renamed file to match existing scheme

(cherry picked from commit 8c22c5c6c6)
2018-02-13 15:17:29 -05:00
George Goldberg
0663f5f88d XYZ-78: Add note to upgrade code about advanced permissions migration. (#8267) 2018-02-13 18:03:09 +00:00
George Goldberg
5c101253c5 Merge branch 'master' into advanced-permissions-phase-1 2018-02-13 13:46:01 +00:00
Jonathan Fritz
18fa32567b Merge branch 'release-4.7' into XYZ-110 2018-02-13 08:39:56 -05:00
Jonathan Fritz
7b2861de3a Removed unused fields from ChannelMemberHistory table, introduced new model.ChannelMemberHistoryResult object that includes those fields that are selected from a join on other tables 2018-02-12 16:14:52 -05:00
Derrick Anderson
c209e44574 revert master changes 2018-02-12 16:01:02 -05:00
Joram Wilander
3e0c3eff9f ABC-228 Update GetPosts caching to work for non-60 limits (#8233)
* Update GetPosts caching to work for non-60 limits

* Only cache on limits of 30/60 and add test

* Add comments clarifying 30 and 60 limits
2018-02-09 14:47:22 -08:00
Derrick Anderson
e7b084842a Merge branch 'release-4.7' 2018-02-09 12:59:56 -05:00
Derrick Anderson
1b064c674a Add prepatory code for 4.8.0 (#8226)
* Add prepatory code for 4.8.0

* formatting issue

* build bug
2018-02-08 12:12:48 -05:00
George Goldberg
96ffde43dc XYZ-80: Add CreateAt/UpdateAt/DeleteAt fields to roles table. (#8223) 2018-02-08 17:08:47 +01:00
JoramWilander
840892ab88 Increase OAuth2 state parameter limit 2018-02-07 16:21:22 -05:00
Jesús Espino
a04b02081a Merge remote-tracking branch 'origin/master' into advanced-permissions-phase-1 2018-02-07 18:05:23 +01:00
Jonathan
d3e934d07a XYZ-35: Added Support for GlobalRelay Compliance Export Format
* Added username to ChannelMemberHistory struct in anticipation of supporting GlobalRelay in Compliance Export
* Removed translation from debug output - this makes it complicated to use utils functions from tests in the enterprise repo
* Added an advanced email function that allows for greater control over message details. Updated MessageExport config to support GlobalRelay. Added attachment support to InBucket unit tests
* Moving templates in from enterprise to solve test issues
* Added export format to diagnostics
* Changed email attachment code to use FileBackend so that S3 storage is properly supported
2018-02-07 09:02:46 -05:00
George Goldberg
7941c30117 Merge branch 'master' into advanced-permissions-phase-1 2018-02-06 17:25:53 +00:00
George Goldberg
e1cd646135 XYZ-37: Advanced Permissions Phase 1 Backend. (#8159)
* XYZ-13: Update Permission and Role structs to new design.

* XYZ-10: Role store.

* XYZ-9/XYZ-44: Roles API endpoints and WebSocket message.

* XYZ-8: Switch server permissions checks to store backed roles.

* XYZ-58: Proper validation of roles where required.

* XYZ-11/XYZ-55: Migration to store backed roles from policy config.

* XYZ-37: Update unit tests to work with database roles.

* XYZ-56: Remove the "guest" role.

* Changes to SetDefaultRolesFromConfig.

* Short-circuit the store if nothing has changed.

* Address first round of review comments.

* Address second round of review comments.
2018-02-06 15:34:08 +00:00
Joram Wilander
323d717a40 Uncomment upgrade code for 4.7 (#8201) 2018-02-05 18:11:31 -05:00
Chris
c89cf572f7 make channel autocomplete sort case-insensitive (#8176) 2018-01-31 14:57:08 -05:00