typographer can change " to ” leading to breakages in parser
At least codify this. Longer term we want to re-prioritize typographer so
it always runs after bbcode parsing.
Previously attributes such as `[test a='a"a' b="a'a"]` were not correctly
handled.
This amends the regex parser to ensure it correctly parses attributes
without breaking incorrectly on the first nested quote
Overall stubs lead to long term instability, this helps stabilize them.
Also fixed flaky spec around plugin hooks.
It was relying you `.posts` on system_user which could be loaded already
and invalid. Instead we now load it by hand.
In some cases Discourse admins may opt for sessions not to persist when a
browser is closed.
This is particularly useful in healthcare and education settings where
computers are shared among multiple workers.
By default `persistent_sessions` site setting is enabled, to opt out you
must disable the site setting.
This is a temporary fix since fonts may be cached without CORS
A larger scale change needed.
Our long term solution here is to have the SCSS compilation pipeline take
care of this by introducing some SCSS var that can be consumed in the various
stylesheets.
We do not want to write a file on boot
Previously in some cases the test suite could fail due to a bad entry in
redis from previous tests
This ensures the correct cache is expired when needed
Additionally improves performance of the redis check
After thinking about it, I worry that this will potentially leave a site
setting set when people hit ctrl-c ... feels a tiny bit risky, so leaving
it out.
- Introduces uploads:delete_missing_s3 which can be used to "give up" and
delete broken records from the database
- Fixes a bug in fix_missing_s3 - crashing on deleted posts
- Adds more info to analyze_missing_s3
Very large batches can take an enormous amount of time due to churn
Limiting to 200k changes at a time gives us a far larger chance of finishing
the job without timing out or deadlocking.
A giant transaction in a post migration can be very risky.
This splits the large amount of work this migration needs to do into 2 parts:
1. A re-runnable cleanup job prior to transaction
2. A minimally sized transaction to add the database constraint
This avoids large amounts of churn on the table
There is an fk to user_profile that can make destroying uploads fail
if they happen to be set as user profile.
This ensures we clear this information when destroying uploads.
There are more relationships, but this makes some more progress.
`rake uploads:analyze_missing` can be used get rich information regarding
uploads missing from s3 (where verified is false)
`rake uploads:fix_missing` is a work in progress task for automatically
correcting certain historic issues. At the moment it simply rebakes all
posts with missing uploads, but it will improve over time
This stops sync of tracking state when list is filtered, in the past this
would cause the tracking state to go off wack.
Additionally this introduces an alias for "filter=tracking", called "f=tracking"
This was done cause the term "filter" is used internally in 2 different ways
the main way is for /unread /new filtering.
Trying to also call a query param "filter" causes enormous amounts of
internal pain, this circumvents the issue.
`topicTrackingState.forEachTracked(topic,isNew,isUnread)` can be used to
iterate through tracking state efficiently.
This is handy for extension looking at subsets of tags and categories.
After visiting a topic list (by tag / category / top level) we track the list
Once a list is tracked the combo `g` `j` can be used to go to the next topic
in the list and `g` `k` to go to previous topic.
This allows you to quickly work through subsets of topics without having
to navigate back to the top level lists
The shortcut does not work in PM lists yet, or search results, both are
under consideration.
This code was intended to bypass iterating through tags in absence of a new
or unread topic.
Instead it always fired cause it was checking for function existence which
was clearly always true.
This pushes v8 from Chrome 73 (March 2019) -> 84 (July 14 2020)
Not expecting any user facing changes, but it is super nice to be on latest
v8 :confetti:
This adds a special filter to topic lists that will filter to tracked and
watched categories.
To use it you can visit:
`https://sitename/?filter=tracked`
`https://sitename/unread?filter=tracked`
and so on
Note, we do not include explicitly tracked and watched topics **outside** of
the tracked categories and tags.
We can consider a `filter=all_tracked` to cover this edge case.
Not ready for an upgrade due to: https://github.com/mdp/rotp/issues/98
The policy here is that for cases like this we pin the version and add
a comment explaining why it is pinned.
We can revisit in a few months depending on upstream.
This is very minor, see: https://github.com/advisories/GHSA-j6w9-fv6q-3q52
An attacker can elevate own cookie usage to bypass server cookie restrictions
Technically this is a security commit, but the surface area is extremely
low, we do not expect any real world impact.
This includes a fix for CVE-2020-8185 we are not vulnerable as we do not use
the impacted middleware. However it still makes sense to stay upgraded, other
small fixes exist in this release.
Previously we would include this section, unfortunately
1. It is usually elided in gmail
2. It can make the emails longer and more confusing
3. Omission is a feature, it means people need to visit site to get context
08044b4f regressed emoji auto complete logic since we (I) forgot to add the
space into the not capturing group at the beginning.
This meant that
`hello :+1:t` would not trigger an autocomplete to pick skin tone.
autocomplete resolving to [] was causing it to stop working.
Instead we have a special const (SKIP) which ensures it will
continue to be evaluated and only this instance is skipped.
There is a feature in search where we take over from the tokenizer
in postgres and attempt to inject more words into search.
So for example: sam.i.am will inject the words i and am.
This is not ideal cause there are many edge cases and this can
cause extreme index bloat.
This is an opening move commit to make it configurable, over the
next few weeks we will evaluate and decide if we disable this by
default or simply remove.
The logic of adding additional search results does not seem to be
needed anymore.
It appears to be a relic of an old implementation.
This saves an entire search query for every search made.
#b19dcac2 improved the serializer so it sends default notification
levels to users to work around cases where a category edit would
would result in clients being left with invalid notification state
Unfortunately this did not address the root issue.
When we edit categories we publish state to multiple users this
means that the serializer is executed unscoped with no user.
The client already handles this case per:
https://github.com/discourse/discourse/blob/dcad720a4ce0aa67e94299ff8370fafd20cc41b2/app/assets/javascripts/discourse/app/models/site.js#L119-L119
If a property is not shipped to it, it will leave it alone on the
existing category.
This fix ensures that these wide category info updates do not
include notification state to avoid corruption of local state.
If people wish to opt out they can use `DISABLE_BOOTSNAP = 1`
Bootsnap is production ready and was tested on our production
servers for safety.
Promoting it now so it is enabled globally.
Will result in faster application boot
This test lasted about 7 years prior to it becoming flaky.
Today ... for whatever reason the test suite created 100 users
prior to running this spec. So the new user becomes user id 101
And... lo and behold:
```
1) PostSerializer a hidden post with add_raw enabled a hidden post shows the raw post only if authorized to see it
Failure/Error: expect(serialized_post_for_user(Fabricate(:user))[:raw]).to eq(nil)
expected: nil
got: "Raw contents of the post."
(compared using ==)
# ./spec/serializers/post_serializer_spec.rb:127:in `block (4 levels) in <main>'
```
Previous to this fix the we were checking for non letters.
This was mismatching what pretty-text/addon/emoji.js was doing.
`ù:su`
and
`1:su`
Would lead to an emoji autocomplete popup in the composer.
Bootsnap is designed to work in production per:
https://github.com/Shopify/bootsnap
Over the years we have seen very few issues with it, none of which
were corruption.
This allows us to enable bootsnap in production
Having bootsnap enabled means that we can speed up deploys by
about 5 seconds per server. It also means a lot less waiting for
various production rake tasks and so on.
In some restricted setups all JS payloads need tight control.
This setting bans admins from making changes to JS on the site and
requires all themes be whitelisted to be used.
There are edge cases we still need to work through in this mode
hence this is still not supported in production and experimental.
Use an example like this to enable:
`DISCOURSE_WHITELISTED_THEME_REPOS="https://repo.com/repo.git,https://repo.com/repo2.git"`
By default this feature is not enabled and no changes are made.
One exception is that default theme id was missing a security check
this was added for correctness.
This introduces a new core API to get counts per tag from topic
tracking state
This API will only be useful if a plugin enable tags in topic
tracking state using
`TopicTrackingState.include_tags_in_report = true`
We previously did not account for completely untagged topics when
looking at muted tags, this caused new/unread counts to be off if
1. You had muted tags
2. You had an unread/new topic
3. This topic had no tags
This feature allows certain plugins to output tag information
to topic tracking state, this allows per tag stats which can be
used by sidebars and other plugins that need per tag stats
Not enabled by default cause this would add cost to a critical
query
This reverts commit 75f46ca632.
This unfortunately breaks
`RAILS_ENV=production bin/rake assets:precompile`
I don't think discourse-loader.js is transpiled
There was a race condition where drafts could be either saving
or queued to be saved and a user canceled draft leading to destroying
it.
This cancels debounce save and waits for save in the pipeline to
be over prior to firing off a DELETE on the draft
cancelComposer would leak a promise that never got resolved if
you aborted cancelling a composer.
This change ensured the promise will always be resolved
Previously we had a partial fix in place where non human users
were not allowed draft sequences, this left edges around where non
human users asked for drafts yet had none.
For example system could already have a few drafts in place.
This also removes and extensibility point we added that is not in use
This commit reorganises the delete dependencies on users and make sure
all are covered. We forgot some on bookmarks, security keys, anon users
and so on.
PG already handles English stop words, the list in cppjieba is
bigger than the list PG uses, which in turn causes confusion cause
words such as "volume" are stripped using cppijieba stop word list
We will follow up with another commit here to apply the Chinese
word stopwords, but for now to eliminate the confusion we are
skipping applying the stopword list when the dictionary in PG is
in English.
Prior to this change we would never clear memory from contexts and
rely on V8 reacting to pressure
This could lead to bloating of PrettyText and Transpiler contexts
This optimisations ensures that we will clear memory 2 seconds after
the last eval on the context
Previously we would raise a warning in the logs if downloading
a file (from s3) takes longer than 60 seconds.
At scale this happens reasonably frequently.
1. Raised the duration to 3 minutes
2. Pulled the resizing mutex out of the downloading mutex
so we have less and clearer error logs
Previously if saving a draft took longer than 2 seconds there
could be conditions where drafts could be saved concurrently.
This meant the composer could race with itself and raise conflicts.
This is likely to happen on bad internet connections or where
latency is really high.
Additionally a throttle was added so drafts save unconditionally
every 15 seconds.
Save draft in the model now properly and consistently returns a
promise.
Previously the code was very race condition prone leading to
odd failures in production
It was re-written in raw SQL to avoid conditions where rows
conflict on inserts
There is no clean way in ActiveRecord to do:
Insert, on conflict do nothing and return existing id.
This also increases test coverage, we were previously not testing
the code responsible for crawling external sites directly
Previously we only changed sequence on ownership change, this
cause a race condition between tabs where user could type for a
long time without being warned of an out of date draft.
This change is a radical change and we should watch closely.
Code was already in place to track sequence on the client so no
changes are needed there.
posts and topics tables can be huge, this migration change
ensures the migration performs less work in one giant transaction
reducing risk of deadlocks
Migration is re-runnable thanks to NOT EXISTS checks
In production `title` was set to undefined causing a
discrepancy between originalTitle and title
This attempts to work around the issue in the production bundle
Sam Saffron