* MM-34171: Fix racy test TestSentry
The NewServer call sets the global mlog Info variable.
But before that, if we call UpdateConfig with the functional options,
then the store.Set method will call mlog.Info before
it could be set.
To fix that, we prepare the updated config
and pass that directly to NewServer to avoid
having to call UpdateConfig.
https://mattermost.atlassian.net/browse/MM-34171
```release-note
NONE
```
* disable watcher
* Trying yet again
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-33893: Disable TCP_NO_DELAY for websocket connections
In very large installations, websocket messages cause too much
traffic congestion by sending too small packets and thereby cause
a drop in throughput.
To counter this, we disable the TCP_NO_DELAY flag for websocket
connections. This has shown to give noticeable improvements in
load tests.
We wrap this in a feature flag for now to let it soak in Community
first.
```release-note
NONE
```
https://mattermost.atlassian.net/browse/MM-33893
* fix gorilla specific conn
test-server-race wasn't using the same set of steps
that the test-server step did. Therefore one test was failing.
Refactored it such that scripts/test.sh can be used to run
normal and race tests as well
```release-note
NONE
```
* MM-34080: Removing sqlite entirely
The initial commit missed removing this blank import.
So the library still remained in our vendor directory.
Removing it for good now.
Bye bye sqlite.
https://mattermost.atlassian.net/browse/MM-34080
* fix go.mod
* MM-34000: Use non-epoll mode for TLS connections
A *crypto/tls.Conn does not expose the underlying TCP connection
or even a File method to get the underlying file descriptor
like the way a *net/TCPConn does. Therefore the netpoll code would
fail to get the file descriptor.
Relevant issue here: https://github.com/mailru/easygo/issues/3
It is indeed possible to use reflect black magic to get the unexported
member, but I have found unexpected errors during writing to the websocket
by getting the file descriptor this way. I do not want to spend time investigating
this especially since this is already released.
Once this is out, we can decide on the right way to fix this, most probably
by proposing to expose the File method or some other way.
https://mattermost.atlassian.net/browse/MM-34000
```release-note
Fix an issue where websockets wouldn't work with TLS connections.
In that case, we just fall back to the way it works for Windows machines,
which is to use a separate goroutine for reader connection.
```
* Ignore logging errors on non-epoll
On non-epoll systems, we needed to return an error
to break from the loop. But in that case, there is no
need to log the error
This is a deadlock due to reversed locking order of the SidebarChannels
and SidebarCategories table.
I could not find the exact culprit query from the deadlock output
because it only shows the last query a transaction is running. And from looking
at the code, the only query that runs "UPDATE SidebarCategories SET DisplayName = ?, Sorting = ? WHERE Id = ?"
is UpdateSidebarCategories. But for the deadlock to happen, it has to lock
SidebarChannels _first_, and then _then_ lock SidebarCategories.
Looking a bit more throughly, I found that DeleteSidebarCategory does indeed
lock the tables in an inverse way and if DeleteSidebarCategory runs concurrently with
UpdateSidebarCategories, they will deadlock.
Here's how it will happen.
```
tx1
DELETE FROM SidebarChannels WHERE CategoryId = 'xx';
tx2
UPDATE SidebarCategories SET DisplayName='dn' WHERE Id='xx';
tx2
DELETE FROM SidebarChannels WHERE (ChannelId IN ('yy') AND CategoryId = 'xx');
tx1
DELETE FROM SidebarCategories WHERE Id = 'xx';
```
And then we see:
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
To fix this, we simply reorder the Delete query to lock the SidebarCategories first,
and then SidebarChannels.
In fact, any transaction updating/deleting rows from those two tables should always operate on that order
if possible.
https://mattermost.atlassian.net/browse/MM-31396
```release-note
Fixed a database deadlock that can happen if a sidebar category is updated and deleted at the same time.
```
Our proxy configuration was historically incorrect, due to which
a lot of customers have that in their setups. As a result, strictly
following the websocket RFC results in a breaking change.
For now, we transparently upgrade the version header to 1.1, if we detect 1.0.
If a client was sending 1.0, it wouldn't have worked anyways because persistent
connections were introduced from 1.1 onwards.
https://mattermost.atlassian.net/browse/MM-33836
```release-note
WebSocket handshakes done with HTTP version lower than 1.1 will result in a warning,
and the server will transparently upgrade the version to 1.1 to comply with the
websocket RFC.
This is done to work around incorrect nginx (and other proxy) configs that do not set
the proxy_http_version directive to 1.1.
This facility will be removed in a future Mattermost version and it is strongly recommended
to fix the proxy configuration to correctly use the websocket protocol.
```
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-33789: Revert fallback to master for GetAllProfilesInChannel
This fixes a regression introduced in https://github.com/mattermost/mattermost-server/pull/16911.
It was causing problems with too many invalidations and overloading the writer instance for big installations.
Reverting this does not affect correctness at all because it was done out of abundance of caution and the
idea at that point was it was to be done for all caches.
https://mattermost.atlassian.net/browse/MM-33789
```release-note
NONE
```
* fix gofmt issues
* Extracting html templates into a library
* Moving tests to the right place
* Fixing tests
* Addressing PR review comments
* Addressing PR review comments
* Replacing attomic with RWMutex
* Returning errors as channel for Templates watcher
* Address PR review comments
* Other small fixes
* Simplifying NewWithWatcher
* Addressing PR review comments
* Making error handling on rendering templates more robust
* Fixing tests
* Changing how we return errors
* Fixing shadow variables
* Addressing PR review comments
* Logging errors from the outside of sendNotificationEmail
* Fixing lock in shutdown
* Fixing the resource copy for commands tests temporary directories
* Removing unused import
* A couple of tiny fixes
* MM-31094: Adds tooling to develop and test using a MySQL instance with replication lag. Adds some lazy lookups to fallback to master if results are not found.
* MM-31094: Removes mysql-read-replica from default docker services.
* MM-31094: Switches (store..SessionStore).Get and (store.TeamStore).GetMember to using context.Context.
* MM-31094: Updates (store.UsersStore).Get to use context.
* MM-31094: Updates (store.PostStore).Get to use context.
* MM-31094: Removes feature flag and config setting.
* MM-31094: Rolls back some master reads.
* MM-31094: Rolls a non-cache read.
* MM-31094: Removes feature flag from the store.
* MM-31094: Removes unused constant and struct field.
* MM-31094: Removes some old feature flag references.
* MM-31094: Fixes some tests.
* MM-31094: App layers fix.
* MM-31094: Fixes mocks.
* MM-31094: Don't reparse flag.
* MM-31094: No reparse.
* MM-31094: Removed unused FeatureFlags field.
* MM-31094: Removes unnecessary feature flags variable declarations.
* MM-31094: Fixes copy-paste error.
* MM-31094: Fixes logical error.
* MM-30194: Removes test method from store.
* Revert "MM-30194: Removes test method from store."
This reverts commit d5a6e8529b.
* MM-31094: Conforming to make's strange syntax.
* MM-31094: Configures helper for read replica with option.
* MM-31094: Adds some missing ctx's.
* MM-31094: WIP
* MM-31094: Updates test names.
* MM-31094: WIP
* MM-31094: Removes unnecessary master reads.
* MM-31094: ID case changes out of scope.
* MM-31094: Removes unused context.
* MM-31094: Switches to a helper. Removes some var naming changes. Fixes a merge error.
* MM-31094: Removes SQLITE db driver ref.
* MM-31094: Layer generate fix.
* MM-31094: Removes unnecessary changes.
* MM-31094: Moves test method.
* MM-31094: Re-add previous fix.
* MM-31094: Removes make command for dev.
* MM-31094: Fix for login.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* app/server: add pprof endpoint
* reflect review comments
* make metrics link conditional
* app/server: add metrics server setup to licence listeners
* refactor a bit
* trigger CI
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add field to config model
Config option for disabling link previews for given domains.
* Refactor functions and corresponding tests
* Expand logic for link preview
Newly added isLinkAllowedForPreview function determines whether a link
should display a preview. It gets corresponding config values consisting
of comma separated domain values, normalizes them and checks for
matches.
* Create tests for link preview restriction
* Fix formatting issue
* Add test cases where images are expected
* Apply suggestions from code review
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Apply remaining code suggestions
* Add RestrictLinkPreviews value to telemetries
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
