Commit Graph

1390 Commits

Author SHA1 Message Date
Jesse Hallam
98ff5fab32 MM-16261: Synchronize plugins in HA (#11657)
* MM-16272 - Synchronize plugins across cluster (#11611)

* MM-16272 - Synchronize plugins across cluster

* Adding a test

* MM-16272 - Fixed tests

* MM-16272 - PR feedback

* MM-16270 - Plugin Sync (#11615)

* Initial implementation for plugin synch with file store. WIP

* Removed ListAll implementation. Used ListDirectory and change localstore to be consistent and return all items (files and folders) from directory

* Refactored plugin filestore operations out of main install/remove plugin

* Fixing error handling details

* Changes to use structured logging

* More logging fixes

* Wording and comments improvements

* Error handling and control flow improvements

* Changed managed flag check to use os.stat

* Added file store plugin dir and filename consts

* Replaced FileRead to use a the FileReader in PluginSync

* Minor styling and PR feedback changes

* Minor error handling improvements

* Added unit test for SyncPlugins. Changed SyncPlugins to use plugins environment to list available plugins

* PR Feedback improvements

* Minor err handling fix

* Removing FileStorePath from PluginEventData (#11644)

* Fix plugin path (#11654)

* tweak path, logging

Fix an issue not finding the plugins folder in S3. Tweak logging messages to add additional clarity.

* Removing FileExists check when Syncing plugins. Updated localstore to not return an error when directory does not exist

* PR Feedback

* Install prepackaged plugins locally only (#11656)

* s/uninstall/remove

* Updated ClusterMessage comment

* Updated PluginSync to test against s3 + local storage
2019-07-18 15:05:53 -03:00
jfrerich
a35adb3ae1 Filter bots out of notifications for members not in channel (#11655) 2019-07-17 21:12:29 -04:00
Saturnino Abril
aef5ef4ed0 MM-16949 Fix posts returned from unread API when the two different list contains parent post and comments (#11617)
* fix posts returned from unread API when the list contains parent post/s

* add ExtendAll to PostList and update test per feedback

* revert unintentional change to the other test and fix comment

* update the existing postlist.Extend, filter unique values and update unit tests
2019-07-18 02:05:43 +08:00
krjn
dd30488a09 [MM-16389] Add icon_emoji parameter to webhooks (#11586)
[MM-16389] Add icon_emoji field to posts from incoming webhooks

To be used to substitute the profile picture on posts.
2019-07-17 11:01:18 +02:00
Ali Farooq
5ed40a48c8 MM-16872 - Extend Plugin API to set LHS bot icon (#11601)
* MM-16872 - Extend Plugin API to set LHS bot icon

* MM-16872 - Using ReadSeeker as opposed to Reader for reading svg image file

* MM-16872 - PR feedback

* MM-16872 - Using userId rather than bot.UserId

* MM-16872 - Minor stylistic changes

* MM-16872 - Removing DriverName check
2019-07-11 12:00:12 -04:00
Jesús Espino
da6cb83f9b Allow to export global relay to zip (#11492)
* Allow to export global relay to zip

* Addressing review comments
2019-07-08 11:06:59 +02:00
Ali Farooq
2ecca12bed MM-15452 - Add ability to override LHS icon for bot accounts (#11423)
* MM-15452 - Add ability to override LHS icon for bot accounts

* MM-15452 - Added translations

* MM-15452 - Updated GetIconImage test to check returned image

* MM-15452 - Added Delete handler for /icon endpoint, invalidating user cache on set/delete

* MM-15452 - Moved /icon routes under bot/, addressed other pr feedback

* MM-15452 - More conflict resolutoin

* MM-15452 Restoring api4/user.go

* MM-15452 - Using require as opposed to t for test assertions

* MM-15452 - Updated as per PR feedback
2019-07-06 08:56:21 +02:00
Saturnino Abril
b832985f1d MM-11210 Add "GET /posts/unread" API to support landing on the last unread post (#11486)
* [MM-11210] Add API GET 'api/v4/channels/{channel_id:[A-Za-z0-9]+}/posts/unread' for scrolling overhaul (#9108)

* Add API GET 'api/v4/channels/{channel_id:[A-Za-z0-9]+}/posts/unread'

* add constants

* refactor GetPostSince and added more tests

* move constants to app package

* [MM-11528 &&  MM-11583] Add userId to in the "posts/unread" path and update test with time delay to fix intermittent failure (#9229)

* add userId to in the "posts/unread" path and update test with time delay to fix intermittent failure

* add limit before and after to query

* remove time delay on test and put pretermined value of Post.CreateAt

* Fix conflict

* [MM-11876] Add cursor to posts list such as next_post_id and previous_post_id (#9707)

* add cursor to posts list such as next_post_id and previous_post_id

add publish previous_post_id on WEBSOCKET_EVENT_POSTED and only get next or previous post IDs if necessary

revert change on adding previous_post_id in WEBSOCKET_EVENT_POSTED

add missing strings import

fix merge conflicts

* update per comment

* update per feedback

* corrected the logic in getting the next and previous post ID

* fix logic to determine next and post IDs, and rename function to have suffix of "Time"

* rearrange logics and add mote tests

* fix merge conflict

* fix missing message when using unread API (#10233)

* MM-15569 Fixes failing test on TestGetPostsForChannelAroundLastUnread (#11039)

* Fix missing posts when getting posts since

* revert changes to GetPostsSince

* migrate Post.GetPostAfterTime and Post.GetPostBeforeTime to sync by default

* revert change to cacheItem

* Fix post ID validation, build query on squirrel and only return post ID as necessary
2019-07-04 02:53:04 +05:30
jfrerich
95652da0b8 [MM-14720] Add Posts Per Day Analytics and Number Posts Previous Day Functionality (#11402)
* Add ability to get bot counts per day

Modify tests using AnalyticsPostCountsByDay to include extra input
(botsOnly)

Correctly generate the mock file with 'make store-mocks' target.  Didn't
see these comments earlier

* Initial commit for calculating total posts previous day and total posts
from bots previous day

* - Refactor to use asserts instead of if statements in tests
- Capture inputs to AnalyticsPostCountsByDay() function into an
  options struct
- Remove bot creation from diagnostics_test.go.

* Remove utils library

* create AnalyticsPostCountsOptions struct which is accepted as an input
to AnalyticsPostCountsByDay method

* Go vet fixes
2019-07-02 14:10:22 -07:00
Guillermo Vayá
b664291f21 [Mm-7854] [Backend] Add an endpoint to revoke sessions from all users (#11200)
* first steps towards revoke all sessions endpoint

* route added

* change permission into a more restrictive one

* fix url

* add store code

* testing & mocking

* fixing what merge broke

* remove sessions without retrieving them

* flush sessions from cache

* stop going through sessions to revoke caches, not needed anymore

* add test, fix func name

* fix tests

* remove unneeded code

* [MM-7854]remove access tokens, move to users

* fix docstring

* [MM-7854] improve readability by using require

* [MM-7854] fix tests

* [MM-7854]fix comment

* [MM-7854]improve testing logic
2019-07-01 22:28:46 +01:00
Jesse Hallam
ede426a806 Document markdown-enabled manifest fields. (#11426)
Markdown support for rendering plugin settings in the system console is inconsistent. Let's start by at least documenting where it is supported.
2019-06-28 15:57:25 -04:00
jfrerich
0cae57a6cd [MM-16146] When converting a bot to a user, set the role to system_user (#11361)
* when converting a bot to a user, set the role to system_user

* correct govet shadow error

* Change from inline error checking and fix shadow error

* Add SYSTEM_USER_ROLE_ID to the bot user model instead of updating the
role after the bot user is created

* modify UserFromBot test to include Roles field
2019-06-28 07:41:14 -07:00
Harrison Healey
4b96437370 MM-16477 Add api to get users modified since a given time (#11406)
* MM-16477 Add api to get users modified since a given time

* Address feedback
2019-06-27 09:37:03 -04:00
Michael Kochell
8cdf5ffe67 [MM-16376] Allow server to download and install a plugin from… (#11372)
* Initial implementation of plugin remote source

* Implement API route

* Test API route

* Add i18n

* Handle different error cases in API route

* Include missing i18n translation

* Include AllowInsecureDownloadUrl in telemetry capture

* Updates from PR feedback

* Use HTTPService instead of http.Get

* Remove InstallPluginFromUrlForced from client4

* Use net/url library to inspect url scheme

* remove PluginDownloadUrl from web/params.go

* Allow plugin downloads from internal sources
2019-06-26 15:45:07 -04:00
Jesús Espino
e53dba3848 MM-16400: Adding more control over bind and advertising address in cluster configuration (#11283) 2019-06-26 09:40:19 +02:00
Michael Kochell
b68194e035 [MM-15831] Improve system for storing status of available plug… (#11185)
* Move State property from activePlugin to PluginHealthStatus. env.activePlugins is now reserved for healthy running plugins.

* Add comments for function declarations

* Combine activePlugins and pluginHealthStatuses into a common structure, registeredPlugins

* Add check to see if plugin is active before deactivating it

* Make `Deactivate` set plugin status

* Add comment explaining the `registeredPlugins` map
* Give responsibility to set plugin disabled status upon deactivation back to `env.Deactivate`

* check if plugin needs to be deactivated before setting status
2019-06-25 17:44:08 -04:00
Michael Kochell
4e5f6fcfbc Remove enable plugin health check config option (#11369) 2019-06-25 09:56:31 -07:00
George Goldberg
130873c2f1 MM-16474: SMTP password should be redacted in system console. (#11350)
Fix Santize function to be the opposite of the Desanitize function for
the config file, and add a unit test for Sanitize.
2019-06-25 10:36:26 -04:00
Martin Kraft
d1b1b319cf MM-16499: Adds ability to retrieve channels with total count f… (#11375)
* MM-16499: Adds ability to retrieve channels with total count for pagination.

* MM-16499: Switches to custom package name for squirrel.
2019-06-25 15:18:48 +03:00
Martin Kraft
c07b7046ca MM-16500: Adds ability to retrieve the total count of teams via the API. (#11325) 2019-06-24 22:05:34 +01:00
Miguel de la Cruz
162a2a9903 [MM-16244] Add Elasticsearch SkipTLSVerification parameter to the config (#11189)
* [MM-16244] Add Elasticsearch SkipTLSVerification parameter to the config

* Add SkipTLSVerification config parameter to the diagnostics

* Update diagnostics key for consistency

Co-Authored-By: George Goldberg <george@gberg.me>
2019-06-23 12:03:55 +01:00
Miguel de la Cruz
9e9b008f3d [MM-13500] Adds channel /search_group endpoint (#10805)
* [MM-13500] Adds channel /search_group endpoint

* Add LIMIT to the queries

* Fix i18n extract

* Fix tests

* Add a new endpoint to get profiles by group channel ids

* Rebase fix
2019-06-22 00:14:21 +01:00
Donald Feury
66993e1fae MM-2954 - Add a separate post type for /me messages and update formatting (#11082)
* MM-2954

* Added new const post type for /me commands
* Modified me command to return response text without *__* wrapper
* Modified unit test for MeCommand to reflect changes
* Added unit test for the me command provider

* * Reverted change to text property in me command response
* Added original message in me command response props
* Updated unit tests

* gofmt changes
2019-06-20 17:16:36 -04:00
Gabe Jackson
41e5ec3c5e [MM-16032] Add system ping endpoint health checks (#11267)
* Add system ping endpoint health checks

This change adds the option for additional server health checks
to be performed when the system ping endpoint is hit. An additional
field 'getserverstatus' is required to run the enhanced health
checks to ensure previous default ping behavior is not modified.

* Use snake_casing
2019-06-20 16:06:04 -04:00
Harrison Healey
73a1bb80f1 MM-16465 Only enable NPS plugin by default if diagnostics are enabled (#11279) 2019-06-19 09:06:59 -04:00
George Goldberg
663977d71a MM-16115: Channel name max length in /rename to match UI. (#11114) 2019-06-18 10:44:29 +01:00
Carlos Tadeu Panato Junior
059eecc696 upgrade db to 5.13 and prepare upgrade to 5.14 (#11198)
* upgrade db to 5.13 and prepare upgrade to 5.14
2019-06-18 10:51:45 +02:00
Martin Kraft
e15a75a2ec MM-16224: Adds new API endpoint + (App & Client & Store) to retrieve the difference between the set of channel members and given group members. (#11186) 2019-06-17 11:04:27 -04:00
Martin Kraft
9d41c7a583 MM-16258: Adds new API endpoint + (App & Client & Store) to retrieve … (#11176)
* MM-16258: Adds new API endpoint + (App & Client & Store) to retrieve Users who would be removed from a list of hypothetical group IDs representing the synced groups.

* MM-16258: Adds roles to JSON response.

* MM-16258: Updates GetByIDs to use Squirrel.

* MM-16258: Puts as much as possible into Squirrel.

* MM-16258: Changes names of methods, functions, and route.

* MM-16258: Updates some comments.

* MM-16258: Extra validation of group_ids parameter.

* MM-16258: Changes validation of group_ids query param.

* MM-16258: Rename a variable and a constant.

* MM-16258: Fix test.
2019-06-17 09:51:56 -04:00
Eli Yukelzon
9ec99593eb MM-14721 - Add an identifier for compliance exports when a message is posted by a bot account (#11063)
* check from_bot post property to override IsBot
2019-06-14 18:36:38 +03:00
Harrison Healey
ff0696c667 MM-16228 Remove DisablePostMetadata setting (#11138)
* MM-16228 Remove DisablePostMetadata setting

* Remove DEPRECATED_DO_NOT_USE_DisablePostMetadata
2019-06-13 17:32:36 -04:00
Christopher Speller
1ca421472f Fix bots being unable to post when personal access tokens are disabled. (#11105) 2019-06-13 11:54:09 -07:00
Devin Binnie
c524ee544f [MM-14727] Modified App.Bot.CreateBot to create a new post to the bot… (#11062) 2019-06-13 13:36:13 +02:00
Jesús Espino
969c032a1e MM-15325: Allowing to configure the device for detect the IP in HA clusters (#10917) 2019-06-11 20:53:03 +02:00
Guillermo Vayá
ad69002f9e Truncate data from OpenGraph metadata (#10532)
* Truncate strings from OpenGraph metada

* remove unwanted opengraph fields, limit to 1 image

* test helper functions

* Add truncating test

* fix typo

* change test into not comparing for the pointer value

* truncate only once

* only shorten fields that were already present

* set maximum of 5 images

* fix original tests

* fix test

* limit to 5 images

* fix typo

* place functions below types, simplify function commentaries

* Rewrite how to empty opengraph's structure
2019-06-11 17:39:35 +02:00
Eli Yukelzon
c243b6640c MM-14712 - Add support for signing SAML requests (#11081)
* SAML Request signing added
* added telemetry for signrequest flag
2019-06-11 14:14:15 +03:00
Christopher Speller
69076934ec Adding bot description field to bot users. (#11066) 2019-06-07 08:09:34 -07:00
Eli Yukelzon
6a42ad2af5 [MM-14400] Programmatically generate default.json (#10551)
* create/update config.json using go generate

* added default config generator
added config-reset to Jenkins and make package, updated defaults to consider 'isNew' flag

* corrections after code review

* fixed Config.isValid to handle empty encryption keys

* fixed Config.isValid to handle empty encryption keys

* fixed Config.isValid to handle empty encryption keys

* isUpdate now only checks for nil

* Addressed review comments, added unit testing for default config generator

* err shadowing

* license

* provide output file for config generator via ENV variable, since go generate doesn't support arguments and we need two output paths (config-reset and package)

* cleanup

* proper defaults for PushNotificationServer and SendPushNotifications

* corrected generating defaults for TrustedProxyIPHeader to be consistent with default.json in master

* Check for empty SiteURL as well as nil

* corrected SiteURL settings and checking

* crazy typos fixed

* corrected tests to newly expected values

* relaxed the checks

* fixed formatting
2019-06-06 13:57:01 -04:00
Christopher Speller
f39dd28b72 Upgrading schema version to 5.12.0 (#11043) 2019-06-03 14:41:53 -07:00
Miguel de la Cruz
74997a126e Change ES trace config type from boolean to string (#11019) 2019-05-31 17:25:11 +01:00
Miguel de la Cruz
6aba17ff9d [MM-15637] Add Trace config option to the Elasticsearch block (#11008) 2019-05-30 16:48:19 +01:00
Jesús Espino
bb2e52ee68 Migrate Groups to not use the Layered store (#10946) 2019-05-28 15:17:10 +01:00
Jesús Espino
fb237cc624 Migrate User.Update method to sync by default (#10942) 2019-05-28 09:40:29 +02:00
Tsilavina Razafinirina
c05cf5b033 [MM-11287] Add support for plus sign and period/dot in custom URL schemes (#9155) (#10852) 2019-05-25 21:47:06 +02:00
Daniel Schalla
2d97f01781 [MM-15639] Add config setting to explicitly define which IP headers are trusted (#10907)
* Add config setting to explicitly define which IP headers are trusted

* fix variable shadowing

* Optimize code flow; Add Ratelimit test for header set

* Extend Ratelimit tests

* Add additional unit tests

* Structured logging
2019-05-24 20:22:13 +02:00
Martin Kraft
e8af4872c6 MM-15422: Adds new parameters for retrieving pages of channels. (#10903)
* MM-15422: Adds new parameters for retrieving pages of channels and searching channels.

* MM-15422: Appends excluded channel names with defaults. Adds separate struct for data-tier option parameter.
2019-05-24 11:28:42 -04:00
Jesús Espino
124b371312 Add support for tracing in LDAP (#10920) 2019-05-24 09:55:07 -04:00
Andres Orozco
88005fbf54 MM-15662: Rename "CreateBotAccounts" config.json setting to "EnableBotAccountCreation" (#10875)
* MM-15662 change conflig flag to EnableBotAccountCreation

* undo changes to i18n

* MM-15662: Change config flag for bot creation

adds newline to i18n

* MM-15662 Update diagnostics.go to use new config flag
2019-05-23 13:03:22 -07:00
Eli Yukelzon
e59674cfdd MM-14721 - Add an identifier for compliance exports when a message is posted by a bot account (#10908) 2019-05-23 13:33:18 +03:00
Daniel Schalla
d269891476 [MM-15490] Rework default password requirements (#10844)
* Rework default password requirements

* Update API Test Lib Default User PW

* Remove unused function; Disable password reqs in dev mode

* Disable strict password requirements for unit tests

* Update unit tests
2019-05-21 11:03:36 -07:00