Previously if saving a draft took longer than 2 seconds there
could be conditions where drafts could be saved concurrently.
This meant the composer could race with itself and raise conflicts.
This is likely to happen on bad internet connections or where
latency is really high.
Additionally a throttle was added so drafts save unconditionally
every 15 seconds.
Save draft in the model now properly and consistently returns a
promise.
Previously the code was very race condition prone leading to
odd failures in production
It was re-written in raw SQL to avoid conditions where rows
conflict on inserts
There is no clean way in ActiveRecord to do:
Insert, on conflict do nothing and return existing id.
This also increases test coverage, we were previously not testing
the code responsible for crawling external sites directly
Previously we only changed sequence on ownership change, this
cause a race condition between tabs where user could type for a
long time without being warned of an out of date draft.
This change is a radical change and we should watch closely.
Code was already in place to track sequence on the client so no
changes are needed there.
posts and topics tables can be huge, this migration change
ensures the migration performs less work in one giant transaction
reducing risk of deadlocks
Migration is re-runnable thanks to NOT EXISTS checks
In production `title` was set to undefined causing a
discrepancy between originalTitle and title
This attempts to work around the issue in the production bundle
Google insists on indexing pages so it can figure out if they
can be removed from the index.
see: https://support.google.com/webmasters/answer/6332384?hl=en
This change ensures the we have special behavior for Googlebot
where we allow indexing, but block the actual indexing via
X-Robots-Tag
Expand SiteSetting.allow_index_in_robots_txt so it also adds a
noindex header if set to false.
This makes sure that nothing is indexed even if it somehow reaches
Google.
Previously originalTitle was set to undefined leading to
titleDirty being unconditionally undefined for new topics
That mean you could not dismiss the composer via ESC
1. Total 6 attempts per day per user
2. Total of 5 per unique email/login that is not found per hour
3. If an admin blocks an IP that IP can not request a reset
Upgrades Rails to latest, this version has better compatibility
with Ruby 2.7
During the upgrade we needed a new cleaner mechanism for configuring
message bus.
All tests are green.
If anything weird pops up please revert.
The failover spec is very fragile and tests specific implementation
vs actual behavior
We rely on a different script during the build process to test
failover operates correctly
This ensures that at a minimum you are notified once a day of
repeat edits by the same user.
Long term we may consider winding this down to say 1 hour or
making it configurable.
Due to a refactor in e90f9e5cc4 we stopped notifying on edits if
a user liked a post and then edited.
The like could have happened a long time ago so this gets extra
confusing.
This change makes the suppression more deliberate. We only want
to suppress quite/link/mention if the user already got a reply
notification.
We can expand this suppression if it is not enough.
rspec-rails 4.0 was released so we no longer need to depend on a
beta version. Also updates minor on a bunch of rspec gems.
Thanks to @ryanwi for raising this.
We have the `# frozen_string_literal: true` comment on all our
files. This means all string literals are frozen. There is no need
to call #freeze on any literals.
For files with `# frozen_string_literal: true`
```
puts %w{a b}[0].frozen?
=> true
puts "hi".frozen?
=> true
puts "a #{1} b".frozen?
=> true
puts ("a " + "b").frozen?
=> false
puts (-("a " + "b")).frozen?
=> true
```
For more details see: https://samsaffron.com/archive/2018/02/16/reducing-string-duplication-in-ruby
We need to recreate the badge_posts view each time we change
column on the posts table.
The p.* is auto expanded on view create, cascade should not have
been used
avg_time on posts and topics have not been used in a year.
This uses a re-runnable ddl transaction diasabled migration to
drop the column, cause it touchs very high traffic table and may
deadlock
pry-nav is not yet supported on latest pry, this holds off on
upgrading pry, which in turn holds off on upgrading deps
Stripping pry-nav for now till it works with latest pry
rebuilding user_actions is not something that should be done.
Plugins such as solved and assigned extend it, there are tons of
little rules that were not captured in `user_actions:rebuild`
Previously we relied entirely on levenshtein_distance_spammer_emails site
setting to handle "similar looking" emails.
This commit improves the situation by always preferring to block (and check)
canonical emails.
This means that if:
`samevil+test@domain.com` is blocked the system will block `samevil@domain.com`
This means that `samevil+2@domain.com` (ad infinitum) will be blocked
This reverts commit 6f9177e2ed.
We decided on a completely different approach to the problem.
Instead we will let blocked emails be treated as canonical.
This reverts commit fa96054acf.
Sadly this fails a test case, we may have to hunt up all the
parentElements to check for the cooked class to perform the
bypass
onSelectionChanged fires a debounced event that calls window.getSelection()
window.getSelection() is reasonably expensive. There is no reason to do any
of this work if we have an input field focused, that is not how quote works
- Eliminate superfluous "author wrote" block
- Eliminate block-quote for all posts
- Move participant count and reply count to 1 line
- Prioritize name over username if forum requests
- Use fabrication in list controller spec to speed up spec
The new `enforce_canonical_emails` site setting ensures that emails in the
canonical form are unique.
This mean that if `s.a.m+1@gmail.com` is registered `sam@gmail.com` will
not be allowed.
The commit contains a blanket "tag strip" (stripping everything after +)
it also contains special handling of a "dot strip" for googlemail and gmail.
The setting only impacts new registrations after `enforce_canonical_emails`
The setting is default false so it will not impact any existing installs.
This reverts commit 8b46f14744.
It corrects the reason for the revert:
We rely on SafeMigrate existing cause we call it from migrations,
Zeitwerk will autoload it.
Instead of previous pattern we explicitly bypass all the hacks in
production mode.
We need to disable SafeMigrate cause it is not thread safe.
A thread safe implementation is possible but not worth the effort,
we catch the issues in dev and test.
SafeMigrate outputs text when we detect attempts to unsafely drop tables
and columns
It is unfortunately not thread safe
This is not needed in production as we would have already caught it by then
in our test suite.
Previously we were migrating multisites serially, this is extremely slow
especially when 200 dbs are involved.
The new implementation defaults to running 20 migrations concurrently, leading
to a 20x speedup.
We also amended it so errors are printed out last, something that makes
debugging failures easier.
This is code specific to Discourse cause we integrate SeedFu with our
migrations and can not include this in the multisite gem.
If a migration performs no changes it should not output stuff.
Previously we would output information about seeds which was very noisy.
On multisite this was particularly bad
Previously we were only updating group membership when a user record was
first created in an SSO setting.
This corrects it so we also update it if SSO changes the email
Previously we would load admin staff action logs unconditionally as soon
as the controller was instantiated, this is not desirable we only want to
run the query when you visit the route.
Previously we relied on race conditions to correctly open a draft, so this
broke.
New code is deliberate.
Also corrects missing observers on composer action
When editing a post we were incorrectly saving a draft prior to user typing
This caused a bloat in the amount of drafts saved per user and inconsistency
around behavior of "escape" button.
It also lead to lots of warnings about draft conflicts when copying stuff
between posts.
The code is improved to use promises more appropriately, however further
changes are needed to clean up internals so methods consistently return
promises.
Too many methods in the controller sometimes return a promise and sometimes
an object. Long term the methods will become async and all of this will be
corrected.
We have no way of detecting if a browser window is behind another window
or off screen on a virtual desktop.
In some cases we may want events to be delivered quicker to the browser.
Specifically a user may still have a window in view but is not interacting.
This gives users 20 minutes of extra "long polling time" prior to shifting
to short polling.
Previously we would consider a user "present" and "last seen" if the
browser window was visible.
This has many edge cases, you could be considered present and around for
days just by having a window open and no screensaver on.
Instead we now also check that you either clicked, transitioned around app
or scrolled the page in the last minute in combination with window
visibility
This will lead to more reliable notifications via email and reduce load of
message bus for cases where a user walks away from the terminal
This is a minor update to unicorn, in this update unicorn handles
chunked encoding a bit more correctly according to RFC
Should have no impact, but message bus will be validated in production
to confirm chunked encoding still works as expected
Previous to this change slugs for leaves in 3 level nestings would not work
Our UX picks only the last two levels
This also makes the results consistent for slugs as it enforces order.
On some sites when bootstrapping communities it is helpful to bootstrap
with a "light weight" invite code.
Use the site setting `invite_code` to set a global invite code.
In this case the administrator can share the code with
a community which is very easy to remember and then anyone who has
that code can easily register accounts.
People without the invite code are not allowed account registration.
Global invite codes are less secure than indevidual codes, in that they
tend to leak in the community however in some cases when starting a brand
new community the security guarantees of invites are not needed.
In some documentation heavy sites larger posts may be desirable.
This is not a setting to change without thinking, however where it is
needed it can make sense. Allow for a 50% increase in max over our
previous maximum.
The old limit of 99000 is arbitrary. Stuff gets riskier the longer posts
get especially with complex markdown, however 99000 is not a threshold
where stuff automatically breaks.
Due to unicorn env object recycling request.ip could point at the wrong
ip address by the time defer block is called. This usually would happen
under load.
This also avoids keeping the entire request object as referenced by the
closure.
Previously on boot we were always removing and adding the same pre-generated
files and symlinks.
This change attempts to avoid writing any automatically generated content if
it is exactly what it should be on disk.
This corrects issues where running a rails console can temporarily corrupt
internal state in production.
In some cases CTE caused pathologically bad query plans.
This optimises it so query runs by itself and caches for lifetime
of the topic query object.
This lightweight caching is done cause topic query will often
execute two queries (one for pinned and one for non pinned)
Introduces a new site setting `max_notifications_per_user`.
Out-of-the-box this is set to 10,000. If a user exceeds this number of
notifications, we will delete the oldest notifications keeping only 10,000.
To disable this safeguard set the setting to 0.
Enforcement happens weekly.
This is in place to protect the system from pathological states where a
single user has enormous amounts of notifications causing various queries
to time out. In practice nobody looks back more than a few hundred notifications.
Follows up #64b35120
This also corrects it so bytes used for internal storage counts all the space
used, previously it was only counting uploads not optimized images.
Additionally we now correctly count storage for optimized images.
This normalizes it so we only carry one place for grabbing disk space size
It also normalizes the command made so it uses Discourse.execute_command
which splits off params in a far cleaner way.
Previously we had many places in the app that called `hostname` to get
hostname of a server. This commit replaces the pattern in 2 ways
1. We cache the result in `Discourse.os_hostname` so it is only ever called once
2. We prefer to use Socket.gethostname which avoids making a shell command
This improves performance as we are not spawning hostname processes throughout
the app lifetime
byebug, ruby-prof, better_errors and rbtrace are very MRI specific, flag
them as such
This helps move forward on potential jruby and truffleruby experiments
Our current algorithm for picking the number of notifications to display
when expanding the notifications relies on magic numbers.
Previously we only allowed for header and an estimate of maximum height of
notification container, this is not ideal as there is padding at the bottom
and top of the notification container
This adds a special number for padding.
The longer term fix though is to render the notification panel off screen
then grab the correct count, finally adding it back into view with.
This would allow for large fonts, small fonts, custom themes and much more.
This reverts commit 7133fd8c89.
Unfortunately performCheckSize may have certain situations in topic
navigation where it is unsafe to call.
We need to add som more safety here prior to merging.
This setting is very high risk and can potentially break all
sorts of features.
To avoid complications and save people from themselves we are
hiding the site setting.
It can still be modified using the console if absolutely needed.
This contains a fix to avoid a certain exception that kept popping up on
page transitions when mini profiler was fighting with our app and our app
was removing the mini profiler timing.
Previously if somehow a user created a blank markdown document using tag
tricks (eg `<p></p><p></p><p></p><p></p><p></p><p></p>`) and so on, we would
completely strip the document down to blank on post process due to onebox
hack.
Needs a followup cause I am still unclear about the reason for empty p stripping
and it can cause some unclear cases when we re-cook posts.
Previously we would use the date the post was updated at as the grant date
this caused confusion.
This also tidies up the badges sql file which was using outdated patterns
for multi line strings.
Out-of-the-box Oj uses :object mode, this shifts us to use :compat mode
by default which is safer.
It means any de-serialization going forward will default to this mode.
If we wish to serialize or deserialize arbitrary objects going forward with
no json interfaces we will have to opt in.
Sam Saffron
