discourse/app
Martin Brennan faf5b4d3e9
PERF: Speed up secure media and ACL sync rake tasks (#16849)
Incorporates learnings from /t/64227:

* Changes the code to set access control posts in the rake
  task to be an efficient UPDATE SQL query.
  The original version was timing out with 312017 post uploads,
  the new query took ~3s to run.
* Changes the code to mark uploads as secure/not secure in
  the rake task to be an efficient UPDATE SQL query rather than
  using UploadSecurity. This took a very long time previously,
  and now takes only a few seconds.
* Spread out ACL syncing for uploads into jobs with batches of
  100 uploads at a time, so they can be parallelized instead
  of having to wait ~1.25 seconds for each ACL to be changed
  in S3 serially.

One issue that still remains is post rebaking. Doing this serially
is painfully slow. We have a way to do this in sidekiq via PeriodicalUpdates
but this is limited by max_old_rebakes_per_15_minutes. It would
be better to fan this rebaking out into jobs like we did for the
ACL sync, but that should be done in another PR.
2022-05-23 13:14:11 +10:00
..
assets FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
controllers FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
helpers DEV: allow running plugin tests for server_plugin_outlet (#16883) 2022-05-20 17:14:55 +05:30
jobs PERF: Speed up secure media and ACL sync rake tasks (#16849) 2022-05-23 13:14:11 +10:00
mailers FIX: Use our header value instead of custom header on duplicates (#16711) 2022-05-11 13:47:12 +10:00
models FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
serializers FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00
services FEATURE: Include participants in PN search data (#16855) 2022-05-18 10:34:01 -04:00
views FEATURE: Promote polymorphic bookmarks to default and migrate (#16729) 2022-05-23 10:07:15 +10:00