Commit Graph

99 Commits

Author SHA1 Message Date
Jesús Espino
49e0473753 MM-11567: Autocomplete search in: for DMs and GMs (#9430)
* MM-11567: Autocomplete search in: for DMs and GMs

* Adding unit tests

* Allowing to search Direct Messages in the autocompletion

* Fix it in TE
2018-09-27 16:15:41 +02:00
Harshil Sharma
af275fe924 #MM-12130 changes for custom service terms (#9450)
* #MM-12130 changes for custom service terms

* Fixed styling

* Added getServiceTerms API

* removed unnecessary panic

* removed custom service terms text from flat config

* reverted user sql store as those changes are no longer needed

* added tests

* Updated a config key to be more standard

* Added copyright info

* Loading service terms only if the feature is enabled

* Loading service terms only if the feature is enabled

* removed unused index

* added createservice termns API

* made a param to bool instead of string

* added createservice termns API

* review fixes

* fixed styling

* Minor refactoring

* removed saveConfig and loadConfig magic

* added empty service terms text check to createServiceTerms API

* refactoed some urls to be terms_of_service instead of service_terms

* removed check for support settings

* changed URLs in tests

* removed unused code

* fixed a bug

* added service termd id in conif

* fixed a test

* review fixes

* minor fixes

* Fixed TestCreateServiceTerms
2018-09-26 16:49:22 -04:00
Jesús Espino
15d64fb201 MM-7188: Cleaning push notification on every read, not only on channel switch (#9348)
* MM-7188: Cleaning push notification on every read, not only on channel switch

* Removed unnecesary goroutine

* Fixing tests

* Applying suggestion from PR
2018-09-26 16:34:12 +02:00
Jesús Espino
847f30a13b MM-11707: Change the default setting for EDIT_OTHERS_POSTS (#9447)
* MM-11707: Removes edit_others_posts permission from the team_admin role in MakeDefaultRoles().

* MM-11707: Tests fix.

* MM-11707: Update test store.

* MM-11707: Allow to change the permission for edit the others posts on TE

* Fixing tests
2018-09-24 15:35:39 +02:00
George Goldberg
ab99f0656f MM-11781: Basic Data Export Command Line. (#9296)
* MM-11781: Basic Data Export Command Line.

* ChannelStore new unit tests.

* TeamStore new unit tests.

* Unit test for new UserStore function.

* Unit tests for post store new methods.

* Review fixes.

* Fix duplicate command name.
2018-09-17 15:51:26 +01:00
Jesse Hallam
8b17bf9e42 MM-11886: materialize channel search (#9349)
* materialize PublicChannels table

Introduce triggers for each supported database that automatically maintain a subset of the Channels table corresponding to only public channels. This improves corresponding queries that no longer need to filter out 99% DM channels.

This initial commit modifies the channel store directly for easier code reviewing, but the next wraps an experimental version around it to enable a kill switch in case there are unforeseen performance regressions.

This addresses [MM-11886](https://mattermost.atlassian.net/browse/MM-11886) and [MM-11945](https://mattermost.atlassian.net/browse/MM-11945).

* extract the experimental public channels materialization

Wrap the original channel store with an experimental version that
leverages the materialized public channels, but can be disabled to
fallback to the original implementation.

This addresses MM-11947.

* s/ExperimentalPublicChannelsMaterialization/EnablePublicChannelsMaterialization/

* simplify error handling

* move experimental config listener until after store is initialized
2018-09-13 13:47:17 -04:00
Jesús Espino
e32581aef3 MM-11725: Add specific autocomplete endpoint for search autocomplete (#9337) 2018-09-11 22:45:31 +02:00
Carlos Tadeu Panato Junior
1f87596e75 Merge release-5.3 2018-09-10 16:13:21 +02:00
Christopher Speller
a8d116b381 Speed up search results post selection. (#9380) 2018-09-10 15:20:01 +02:00
cpanato
4cf6913c1c Merge remote-tracking branch 'upstream/release-5.3' into release-5.3-daily-merge-20180904 2018-09-04 23:27:39 +02:00
Jesús Espino
72560311a2 MM-11728: Avoid Archived channels editions throught Patch (#9335) 2018-09-03 17:07:51 +02:00
Daniel Schalla
531897b1f0 add megacheck as makefile target (#9288)
Fix code issues in channel_test.go

Fix Channel Test Issues detected by Megacheck

Fix API Emoji Test Issues detected by Megacheck

Fixed API Issues Reported by Megacheck

Fixed App issues reported by megacheck

Remaining fixes

removed test added by mistake from old HEAD

gofmt

Store Fixes

simplified returns

Fix test for multi member channel delete

revert to delete unused function
2018-09-03 14:08:40 +02:00
Jesús Espino
bba3bbd9f3 MM-11572: Force correct order on messages generated in the bulk (#9244) 2018-08-24 09:53:44 +02:00
George Goldberg
8bbee74ed8 MM-11348: Add telemetry for advanced permissions. (#9249) 2018-08-23 11:48:57 +01:00
Jesse Hallam
d8c8a19d35 avoid t.Fatal() in tests (#9189)
I've been burned a few times by tests that simply fatal, requiring me to
run another build to learn more about what the mismatch was. Avoid this.

This is part of a long running goal of mine to make testing "better".
2018-08-09 11:26:38 +02:00
Jesús Espino
6bf09e2c34 MM-11384: Add system install date information to the client config (#9218)
* MM-11384: Add system install date information to the client config

* Fixing translation text

* Fixes from Peer Review
2018-08-08 12:04:36 +02:00
Harrison Healey
c34b30a6e7 MM-11521/MM-11522 Fix being able to create users with invalid emails through API (#9199)
* MM-11522 Fix being able to create users with invalid emails through API

* Ensure store tests are using valid emails

* Add missing license header

* Remove invalid test case

* Fix TestUpdateOAuthUserAttrs
2018-08-01 15:18:14 -04:00
Martin Kraft
5872bf9c2f Pr 9039 (#9187)
* MM-11065: Allow to search and get archived channels from the API

* Fixing more tests

* Add some unit tests

* Add includeDeleted parameter to session permissions check function

* More test fixing

* Adding archive channels list in channels search

* Add restriction for archived channel edition

* Reverting permissions checks modification

* Changed the query parameter to include_deleted

* Enable search archive channels as true by default

* Adding tests for verify search on deleted channels

* Allowing to override archive channels during the imports

* Fixed test

* Search in archive channels from the API must be explicitly requested

* Removing includeDeleted parameter from GetChannelByName and GetChannelByNameForTeam

* Back to ViewArchivedChannels config

* Fixing tests

* Reverting GetChannelByName parameter

* Add include deleted parameter on GetChannel functions in plugins api

* Fixing tests
2018-07-30 15:06:08 -04:00
Pradeep Murugesan
b3c2ecd9b9 added the custom icon and username for the outgoing webhook and its response (#9141)
* 8272 added the username and icon as part of the model and persisted the same

* 8272 added the custome icon and name when set to the web hook response

* 8272 changed the infinte loop to timeout after 5 seconds

* 8272  fixed review comments
2018-07-25 08:31:41 -04:00
Joram Wilander
2362a5b7f7 Ignore | characters in postgres channel search (#9019) 2018-06-30 00:13:13 +02:00
Jerry Kurian
4b2a626354 add MaxLifetimeconns to server (#9004)
Update e.json string names and remove console log

reorder variables

update db.setConnMaxLifetime to take in *settings.MaxLifetimeConns

Add MaxLifetimeConns to test

Update variable names like MaxLifetimeConns to ConnMaxLifetimeMilliseconds

Update time passed to SetConnMaxLifetime

Revert "Update time passed to SetConnMaxLifetime"

This reverts commit dc1c1b6d5a6404d4ee11c65d4756c19f9316794f.

update en.json

Update supplier_test.go and docker.go ConnMaxLifetimeMilliseconds to 3600000
2018-06-28 09:45:20 -04:00
Joram Wilander
0d91bf323e Lock bulk importing to master database node (#9012) 2018-06-27 10:08:40 +01:00
Chris Duarte
41d2a9f435 Add Purpose as a searchable field (#8067)
* Add Purpose as a searchable field

* Add New Index and Remove old for Channels

* Include Purpose in FullTextSearch Clause

* Move removeIndex for Channels into upgrade.go
2018-06-26 13:47:39 -04:00
Daniel Schalla
ecefa6cdd1 Implementation of File Exists Function; Delete FileInfos upon Permanent User Delete (#8958)
Check if file was deleted on FS

Warning message if file couldnt be removed
2018-06-25 12:12:59 -04:00
George Goldberg
fc158fce90 MM-10570: Make permissions reset command clear custom role assignments. (#8976) 2018-06-25 14:34:59 +01:00
George Goldberg
1f65f0e3d6 MM-10233: Add scheme importing to bulk importer. (#8928) 2018-06-21 12:02:35 +01:00
Martin Kraft
260d7a0f85 MM-10412: Adds deleteBy prop to posts. (#8896) 2018-06-01 12:45:46 -04:00
George Goldberg
516a097e1a MM-10757: Default roles from scheme should be keyed by name not ID. (#8894)
* MM-10757: Default roles from scheme should be keyed by name not ID.

* Update permissions import/export.
2018-06-01 12:49:02 +01:00
Martin Kraft
69304fb54d Merge remote-tracking branch 'origin/master' into advanced-permissions-phase-2 2018-05-24 08:40:52 -04:00
Jesse Hallam
0a666a5656 MM-10649: soften Channels.ExtraUpdateAt deprecation (#8843)
The previous complete removal of this field resulted in an
incompatibility with 4.x servers that could not handle the now null
column field.

Instead, ensure this field is at least always set to 0, with a plan to
remove it altogether in a future release.
2018-05-23 10:28:19 -04:00
George Goldberg
eddb38bbef Fix tests. 2018-05-18 11:54:46 +01:00
Martin Kraft
e0390632b3 MM-10264: Adds CLI command to import and export permissions. (#8787)
* MM-10264: Adds CLI command to import and export permissions.

* MM-10264: Changes Scheme Name to DisplayName and adds Name slug field.

* MM-10264: Changes display name max size.

* MM-10264: Another merge fix.

* MM-10264: Changes for more Schemes methods checking for migration.

* MM-10264: More updates for Schemes migration checking.
2018-05-17 11:37:00 -04:00
George Goldberg
a09dc68e1d MM-10235: Make permissions reset CLI shcemes-aware. (#8773)
* MM-10235: Make permissions reset CLI shcemes-aware.

* Add i18n strings.
2018-05-17 16:28:14 +01:00
George Goldberg
319d61123a MM-10615: Reset teams/channels to default scheme on delete scheme. (#8811) 2018-05-17 12:48:31 +01:00
Martin Kraft
f82cc7896d Merge remote-tracking branch 'origin/master' into advanced-permissions-phase-2 2018-05-15 07:45:28 -04:00
George Goldberg
51bd710ecd MM-9728: Online migration for advanced permissions phase 2 (#8744)
* MM-9728: Online migration for advanced permissions phase 2

* Add unit tests for new store functions.

* Move migration specific code to own file.

* Add migration state function test.

* Style fixes.

* Add i18n strings.

* Fix mocks.

* Add TestMain to migrations package tests.

* Fix typo.

* Fix review comments.

* Fix up the "Check if migration is done" check to actually work.
2018-05-14 15:59:04 +01:00
Jesse Hallam
7e7c551987 MM-9739: deprecate the now unused Channels.ExtraUpdateAt (#8762) 2018-05-14 09:43:06 -04:00
Martin Kraft
91557bbd97 Merge remote-tracking branch 'origin/master' into advanced-permissions-phase-2 2018-05-11 08:05:23 -04:00
Christopher Speller
d8dd271e43 MM-4998 Adding LoginIdAttribute to allow LDAP users to change their login ID without losing their account (#8756)
* Adding LoginIdAttribute

* Modifying LDAP to use loginIDAttribute.

* Adding IDAttribute migration and AD objectGUID support.

* Removing unused idea.

* Fix typo.
2018-05-10 09:46:09 -07:00
George Goldberg
60cf74352f MM-10140: API Implementation for Schemes related Endpoints (#8615)
* Implement basic scheme CRUD endpoints.

* Get All Schemes (Paged) Endpoint and store plumbing.

* Add get teams/channels for schemes.

* Fix unit tests.

* Review fixes.

* More review fixes.
2018-05-03 09:00:26 -04:00
Martin Kraft
7d5e85e413 Merge remote-tracking branch 'origin/master' into advanced-permissions-phase-2 2018-05-02 07:45:20 -04:00
Martin Kraft
ff6c42309c Merge remote-tracking branch 'origin/master' into advanced-permissions-phase-2 2018-05-01 18:59:20 -04:00
Derrick Anderson
a5f006b8a9 Revert "MM-9770: rewrite getParentsPosts to improve performance (#8467)" (#8659) (#8694)
This reverts commit 4b675b347b.
2018-04-30 16:06:06 -04:00
Christopher Speller
686c2fbab7 Structured logging (#8673)
* Implementing structured logging

* Changes to en.json to allow refactor to run.

* Fixing global logger

* Structured logger initalization.

* Add caller.

* Do some log redirection.

* Auto refactor

* Cleaning up l4g reference and removing dependancy.

* Removing junk.

* Copyright headers.

* Fixing tests

* Revert "Changes to en.json to allow refactor to run."

This reverts commit fd8249e99b.

* Fixing some auto refactor strangeness and typo.

* Making keys more human readable.
2018-04-27 12:49:45 -07:00
George Goldberg
cd55c44c8f MM-8796: Full implementation of "Schemes" in Store/Model/App layers. (#8357)
* Add Scheme model and stub store.

* Port ChannelStore to be Scheme aware.

* Make almost all the API/APP layer work with ChannelSchemes.

Only thing still hacky is UpdateChannelMemberRoles().

* Add basic SchemeStore implementation.

* Migrate UpdateChannelMemberRoles properly and fix tests.

* Update store tests and mocks so they work.

* Include creating default roles in Scheme create store function.

* Implement role deletion and start scheme deletion.

* Only use non-deleted roles for authorization.

* Add GetByScheme method to Team store.

* Add GetChannelsByScheme.

* Update store mocks.

* Implement scheme deletion in the store.

* Rename is valid function.

* Add offset and limit to queries to fetch teams and channels by scheme.

* Fix queries.

* Implement scheme awareness in Team store and add a migration.

* Tidy up ChannelStore mapping functions and add exhaustive unit tests.

* Add all missing i18n.

* Proper tests for TeamStore internal functions and fix them.

* Make additional TeamMember fields nullable.

* Make new ChannelMember fields nullable.

* Create new nullable columns without defaults.

* Make new fields in large tables nullalble.

* Fix empty list of TeamMembers.

* Deduplicate SQL queries.

* Fix spelling.

* Fix review comment.

* More review fixes.

* More review fixes.
2018-04-20 14:49:13 -04:00
George Goldberg
b13a228b04 MM-10121: CLI command to reset permissions system to default state. (#8637)
* MM-10121: CLI command to reset permissions system to default state.

* Review comment.
2018-04-18 10:18:07 +01:00
Jesús Espino
014a3b6a60 Fixing misspell errors (#8544) 2018-03-29 10:04:54 -04:00
Jesse Hallam
1ccad749f1 Add unit test coverage for SqlStatusStore.GetAllFromTeam (#8520)
I added this as part of a separate PR that was deferred for now, but
figured the unit test coverage would still be useful.
2018-03-28 09:20:39 -07:00
Martin Kraft
e13e64711f Merge remote-tracking branch 'origin/master' into advanced-permissions-phase-1 2018-03-27 09:01:42 -04:00
Jesse Hallam
8491ba5740 Relax 4k post message limit (#8478)
* MM-9661: rename POST_MESSAGE_MAX_RUNES to \0_v1

* MM-9661: s/4000/POST_MESSAGE_MAX_RUNES_V1/ in tests

* MM-9661: introduce POST_MESSAGE_MAX_RUNES_V2

* MM-9661: migrate Postgres Posts.Message column to TEXT from VARCHAR(4000)

This is safe to do in a production instance since the underyling type is
not changing. We explicitly don't do this automatically for MySQL, but
also don't need to since the ORM would have already created a TEXT column
for MySQL in that case.

* MM-9661: emit MaxPostSize in client config

This value remains unconfigurable at this time, but exposes the current
limit to the client. The limit remains at 4k in this commit.

* MM-9661: introduce and use SqlPostStore.GetMaxPostSize

Enforce a byte limitation in the database, and use 1/4 of that value as
the rune count limitation (assuming a worst case UTF-8 representation).

* move maxPostSizeCached, lastPostsCache and lastPostTimeCache out of the global context and onto the SqlPostStore

* address feedback from code review:

* ensure sqlstore unit tests are actually being run
* move global caches into SqlPostStore
* leverage sync.Once to address a race condition

* modify upgrade semantics to match new db semantics

gorp's behaviour on creating columns with a maximum length on Postgres
differs from MySQL:
* Postgres
  * gorp uses TEXT for string columns without a maximum length
  * gorp uses VARCHAR(N) for string columns with a maximum length of N
* MySQL
  * gorp uses TEXT for string columns with a maximum length >= 256
  * gorp uses VARCHAR(N) for string columns with a maximum length of N
  * gorp defaults to a maximum length of 255, implying VARCHAR(255)

So the Message column has been TEXT on MySQL but VARCHAR(4000) on
Postgres. With the new, longer limits of 65535, and without changes to
gorp, the expected behaviour is TEXT on MySQL and VARCHAR(65535) on
Postgres. This commit makes the upgrade semantics match the new database
semantics.

Ideally, we'd revisit the gorp behaviour at a later time.

* allow TestMaxPostSize test cases to actually run in parallel

* default maxPostSizeCached to POST_MESSAGE_MAX_RUNES_V1 in case the once initializer panics

* fix casting error

* MM-9661: skip the schema migration for Postgres

It turns out resizing VARCHAR requires a rewrite in some versions of
Postgres, but migrating VARCHAR to TEXT does not. Given the increasing
complexity, let's defer the migration to the enduser instead.
2018-03-26 17:55:35 -04:00