Commit Graph
100 Commits
Author SHA1 Message Date
Daniel Waterworth f7aefffea7 DEV: Concerns can use class_methods (#24875) 2023-12-13 14:12:03 -06:00
Daniel Waterworth 611acaa6bc FIX: Validate each value in an array custom field separately (#24659) 2023-12-07 14:24:04 -06:00
Daniel Waterworth a6c79aa27a DEV: Fix readonly admin login spec (#24714)
"prevents login by admins" should test with an admin.
2023-12-05 06:49:22 -06:00
Daniel Waterworth 434ae5bbe7 FIX: Allow setting an array custom field to a singleton value (#24636)
Also, validation happens per item in an array field.
2023-11-29 14:18:47 -06:00
Daniel Waterworth eef93ac926 DEV: Allow setting max_length for field types using the plugin API (#24635) 2023-11-29 14:17:12 -06:00
Daniel Waterworth 6aa69bdaea DEV: Allow setting different custom field length limits by key (#24505) 2023-11-22 12:00:42 -06:00
Daniel Waterworth 38abd471b6 DEV: Remove unnecessary fab! block (#24496) 2023-11-21 17:03:17 -06:00
Daniel Waterworth 0059e6807c DEV: Refactor save_custom_fields methods (#24495)
Operate a key at a time, to make it clearer what's going on.

This also fixes a bug where array integer fields would get re-written
even when there wasn't a change.
2023-11-21 12:40:15 -06:00
Daniel Waterworth ced21fa5c1 DEV: Deprecate array custom fields (#24492)
Array custom fields use separate rows for each value, but whenever we
update an array, we have always destroy the existing rows and create new
ones. Therefore, there's no benefit over using the json type.
2023-11-21 11:05:49 -06:00
Daniel Waterworth b3c67a78b8 FIX: Preserve custom field array order (#24491) 2023-11-21 10:55:22 -06:00
Daniel Waterworth 0c712eaa3a DEV: Don't define methods in an included block (#24433) 2023-11-17 12:22:32 -06:00
Daniel Waterworth e34d2cfde4 DEV: Remove unnecessary fab! blocks (#24419) 2023-11-16 13:22:08 -05:00
Daniel Waterworth 46e2523177 DEV: Remove custom field regexes (#24390)
As far as I can tell, this isn't used
2023-11-16 11:37:10 -06:00
Daniel Waterworth 0bc568f66d FIX: Preload the right fields on categories (#24396) 2023-11-15 16:34:03 -06:00
Daniel Waterworth e845138bc1 FIX: Updating presence status in readonly mode should fail gracefully (#24333) 2023-11-10 14:27:43 -06:00
Daniel Waterworth 6e161d3e75 DEV: Allow fab! without block (#24314)
The most common thing that we do with fab! is:

    fab!(:thing) { Fabricate(:thing) }

This commit adds a shorthand for this which is just simply:

    fab!(:thing)

i.e. If you omit the block, then, by default, you'll get a `Fabricate`d object using the fabricator of the same name.
2023-11-09 16:47:59 -06:00
Daniel Waterworth 5c92d7da22 FIX: Increase defer queue length (#24200)
It's important that there is a limit, but the current limit is too
restrictive.
2023-11-01 14:02:53 -05:00
Daniel WaterworthandPenar Musaraj 290306a932 SECURITY: Reduce maximum size of SVG sprite cache to prevent DoS
Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2023-09-12 15:31:28 -03:00
Daniel Waterworth 40acb9a111 DEV: Run core system tests by default in docker test image (#23517) 2023-09-11 16:04:33 -05:00
Daniel Waterworth 70d082584c DEV: Allow explicitly enabling/disabling system tests in bin/turbo_rspec (#23515)
This doesn't alter the default behavior.
2023-09-11 13:11:06 -05:00
Daniel Waterworth 82a56334a3 PERF: Cache each theme field value once (#23192)
Previously, theme fields from components would be cached for each of
their parent themes.
2023-08-31 11:24:02 -05:00
Daniel Waterworth 91f560a521 DEV: Make every DistributedCache lazily instantiated (#23147) 2023-08-18 10:59:11 -05:00
Daniel Waterworth 80a0d88e4c FIX: Ensure javascript caches are unique per theme/theme_field (#23126)
We were assuming that this was the case before, but not enforcing it.
2023-08-17 13:57:04 -05:00
Daniel Waterworth 50d527b80c DEV: Remove db_timeout setting (#22912)
It doesn't actually do anything.
2023-08-01 14:17:43 -05:00
Daniel Waterworth 26e267478d SECURITY: Don't allow a particular site to monopolize the defer queue 2023-07-28 12:53:51 +01:00
Daniel Waterworth b7404373cf FIX: Always clear caches after committing the current transaction (#22550)
Instead of having to remember every time, just always wait until the
current transaction (if it exists) has committed before clearing any
DistributedCache.

The only exception to this is caches that aren't caching things from
postgres.

This means we have to do the test setup after setting the test
transaction, because doing the test setup involves clearing caches.

Reapplying this - it now doesn't use after_commit if skip_db is set
2023-07-12 09:49:28 -05:00
Daniel Waterworth cb794275a7 Revert "FIX: Always clear caches after committing the current transaction" (#22493)
This reverts commit 8310c7842c.

This was breaking precompilation.
2023-07-07 18:05:38 -05:00
Daniel Waterworth 9dd01ca2ef FIX: Always clear caches after committing the current transaction (#22491)
Instead of having to remember every time, just always wait until the
current transaction (if it exists) has committed before clearing any
DistributedCache.

The only exception to this is caches that aren't caching things from
postgres.

This means we have to do the test setup after setting the test
transaction, because doing the test setup involves clearing caches.
2023-07-07 14:24:56 -05:00
Daniel Waterworth 5e2edeb2f7 FIX: Set test transaction to before_all transaction (#22445)
So that after_commit hooks work correctly for pre-fabricated objects.
2023-07-06 12:23:28 -05:00
Daniel Waterworth 6f8056572e DEV: Refactor updating parent/child relationships between themes (#22244)
The prior add_relative_themes! method was confusing and cleared the
cache repeatedly instead of once for the whole operation.
2023-06-22 13:57:39 -05:00
Daniel Waterworth 50bded7b47 FIX: Force ordering to make things more consistent and fix flaky test (#22228) 2023-06-21 13:16:10 -05:00
Daniel Waterworth 0cec84206e DEV: Fix interactions between tests via plugin svg sprite cache (#22047) 2023-06-09 13:53:21 -05:00
Daniel Waterworth 67afd85aae Revert "DEV: Use runtime info to split test files for parallel testing (#21896)" (#22016)
This reverts commit 14ed971db6.

This prevented the core backend tests from running in GitHub CI
2023-06-08 15:13:26 -05:00
Daniel Waterworth 3d3e8c8fbd FIX: json_schemer no longer allows nil keys (#22015) 2023-06-08 15:12:47 -05:00
Daniel Waterworth 353c8ea51f DEV: Don't pass in extra, ignored parameters when rendering sass (#21591)
As far as I can tell, these parameters do nothing.
2023-05-16 12:14:06 -05:00
Daniel Waterworth 53d99d1a26 DEV: Skip flaky tests (#21534) 2023-05-15 07:51:25 +02:00
Daniel WaterworthandPenar Musaraj 0a96579cef DEV: Allow site setting errors to be displayed as html if they are safe (#21342)
This is only useful for plugins currently

Co-authored-by: Penar Musaraj <pmusaraj@gmail.com>
2023-05-04 15:01:19 -05:00
Daniel Waterworth bc236119d3 DEV: Increase docker.rake system test timeout time to match CI (#21339) 2023-05-02 12:12:03 -05:00
Daniel Waterworth a4dd758f5c DEV: Add 20m timeout to system tests (#21323) 2023-05-01 13:23:48 -05:00
Daniel Waterworth ae63d9ea09 PERF: Remove unnecessary queries from Theme#set_field (#21281) 2023-04-27 13:04:58 -05:00
Daniel Waterworth 38cebd3ed5 Revert "FEATURE: Better thread reply counter cache (#21108)" (#21192)
This reverts commit 180e3e11d1.

Per internal discussions, this is a temporary revert, to investigate if this is causing a performance regression.
2023-04-20 15:09:47 -05:00
Daniel Waterworth a4d51810e2 DEV: Correct ambiguous name (#21189)
This file would do different things depending on whether
Migration::Migration existed beforehand.
2023-04-20 11:28:08 -05:00
Daniel Waterworth c68497159f FIX: Use string for postgres recently readonly DistributedCache (#21040)
Since DistributedCaches don't marshal timestamps
2023-04-10 13:54:55 -05:00
Daniel Waterworth fcc73b441d FIX: Make emoji cache store marshalled objects (#21029)
We only marshal arrays, hashes and sets, which meant that the emojis
here were just getting `to_s`ed.

This is a hack.
2023-04-10 10:44:11 -05:00
Daniel Waterworth 0ff86feb96 DEV: Rely on helpers for ajax error handling to reduce repetition (#20988) 2023-04-06 11:25:24 -05:00
Daniel Waterworth 355b44472b DEV: Allow HTML errors whenever a popup is generated (#20989)
Follow-up-to: 6bbf832400
2023-04-06 10:00:54 -05:00
Daniel Waterworth cd6d47e012 FIX: Handle errors when we show the edit user directory columns modal (#20986) 2023-04-05 14:31:32 -05:00
Daniel Waterworth 79c87b5003 FIX: Show better default error when failing to delete all user posts (#20985) 2023-04-05 14:26:00 -05:00
Daniel Waterworth 852475dca4 DEV: Add yarn-app shim to run yarn using the app modules (#20983) 2023-04-05 13:12:58 -05:00
Daniel Waterworth 2df0eca39a DEV: Make postgres_readonly cache work like other caches (#20879)
We didn't have an authoritative source for this data previously, so now
it's stored in redis.
2023-03-30 09:14:59 -05:00
Daniel Waterworth d563b73202 DEV: Make category topic_ids cache work like other caches (#20814) 2023-03-27 09:23:29 -05:00
Daniel Waterworth 3e80ab3143 DEV: Make category subcategory_ids cache work like other caches (#20815) 2023-03-27 09:23:23 -05:00
Daniel Waterworth b2554e1490 DEV: Make hex cache work like other caches (#20816) 2023-03-27 09:22:53 -05:00
Daniel Waterworth 9d34b73a5e DEV: Make csp cache work like other caches (#20818) 2023-03-27 09:22:38 -05:00
Daniel Waterworth 0f4bb19524 DEV: make developer cache work like other caches (#20817) 2023-03-27 09:22:33 -05:00
Daniel Waterworth 0728fa055b DEV: Make theme cache work like other caches (#20813) 2023-03-24 12:37:16 -05:00
Daniel Waterworth 7e23e6c1e8 DEV: Make banner cache work like other caches (#20811) 2023-03-24 12:36:55 -05:00
Daniel Waterworth bc8ab33708 DEV: Disable test to see if this is what's causing ci failures (#20809) 2023-03-24 11:15:30 -05:00
Daniel Waterworth 52999d1422 DEV: Show which system tests are being executed in CI (#20795)
... to see if it's always the same test causing timeouts
2023-03-23 15:12:14 -05:00
Daniel Waterworth 7b01576c8d DEV: Remove emoji cache dead code (#20764)
The cache is already not shared between app servers that have different
app_versions, so this check was redundant.
2023-03-21 12:33:12 -05:00
Daniel Waterworth a0a6f6d71b FIX: Fix the emoji toned regexes (#20763) 2023-03-21 11:48:55 -05:00
Daniel Waterworth da0d20d4a9 DEV: Refactor svg sprite parsing (#20727)
There was a lot of duplication in the svg parsing and coercion code. This reduces that duplication and causes svg sprite parsing to happen earlier so that more computation is cached.
2023-03-20 11:41:23 -05:00
Daniel Waterworth 293cb7bde2 FIX: An ember build is required to run the system tests (#20725) 2023-03-17 13:20:49 -05:00
Daniel Waterworth 164b60cd07 DEV: Optionally, run system tests in docker:test 2023-03-15 16:46:48 -05:00
Daniel Waterworth 5324216740 DEV: Use rspec instead of turbo_rspec with one core 2023-03-15 16:46:48 -05:00
Daniel Waterworth 84f590ab83 DEV: Store theme sprites in the DB (#20501)
Let's avoid fetching sprites from the CDN during page rendering.
2023-03-14 13:11:45 -05:00
Daniel Waterworth b96a23978a PERF: Add index to speed up ReindexSearch job (#20151) 2023-02-21 13:16:43 -05:00
Daniel Waterworth 666536cbd1 DEV: Prefer \A and \z over ^ and $ in regexes (#19936) 2023-01-20 12:52:49 -06:00
Daniel Waterworth 3030a53819 FIX: Prevent concurrent updates to top_topics (#19854)
to prevent lock timeouts
2023-01-12 14:03:26 -06:00
Daniel Waterworth dfc449a32e FIX: Continue if checking themes if updating one fails (#19665) 2023-01-09 11:56:43 -06:00
Daniel Waterworth 97c5e19537 FIX: Don't assume a clean slate in tests (#19499)
since plugins can have fixtures
2022-12-19 12:12:03 -06:00
Daniel Waterworth d244eb0d7c FIX: Don't validate channel slug if it hasn't changed (#19498)
This is generating lots of log noise for operations that aren't even
touching the slug.
2022-12-19 12:11:51 -06:00
Daniel Waterworth 2b4009c6bc DEV: Fake sleeping in distributed mutex tests (#19472)
otherwise they take a looong time
2022-12-14 15:11:46 -06:00
Daniel Waterworth 3a4ac3a7c0 FIX: Don't update url in GitImporter (#19273)
Since it's fetched and used elsewhere and expected to stay the same.
2022-12-01 10:50:06 -06:00
Daniel Waterworth d9364a272e FIX: When following redirects before cloning, use the first git request (#19269)
This is closer to git's redirect following behaviour. We prevented git
following redirects when we clone in order to prevent SSRF attacks.

Follow-up-to: 291bbc4fb9
2022-11-30 14:21:09 -06:00
Daniel Waterworth aab4ea1849 Version bump to v2.9.0.beta14 (#19242) 2022-11-29 10:59:53 -06:00
Daniel Waterworth 84c83e8d4a SECURITY: Filter tags in user notifications for visibility (#19239) 2022-11-29 10:35:41 -06:00
Daniel Waterworth f895f27b02 FIX: When filtering tags for visibility, respect tag group permissions (#19152) 2022-11-22 12:55:57 -06:00
Daniel Waterworth 47709c6d49 DEV: Reinstate --single-branch when cloning themes (#19026) 2022-11-15 13:23:57 -06:00
Daniel Waterworth 6f48502dd2 DEV: Add rake task to install a theme from an archive (#18890) 2022-11-04 16:42:07 -05:00
Daniel Waterworth 291bbc4fb9 FIX: When cloning themes via HTTP, try the original URI too (#18870)
This should fix fetching from gitlab.

In order to get SSRF protection, we had to prevent redirects when cloning via git, but some repos are behind redirects and we want to support those too. We use `FinalDestination` before cloning to try to simulate git with redirects, but this isn't quite how git works, so there's some discrepancies between our SSRF protected cloning behavior and normal git behavior that I'm trying to work around.

This is temporary fix. It would be better to use `FinalDestination` to simulate the first request that git makes. I aim to make it work like that in the not too distant future, but this is better for now.
2022-11-03 15:19:08 -05:00
Daniel Waterworth 1398bd5f1f FIX: Theme import error handling needs to happen inside the hijack block (#18866)
Otherwise the errors don't get caught.
2022-11-03 14:02:26 -05:00
Daniel Waterworth 167181f4b7 DEV: Quote values when constructing SQL (#18827)
All of these cases should already be safe, but still good to quote for
"defense in depth".
2022-11-01 14:05:13 -05:00
Daniel Waterworth 563ec624b2 FIX: Allow email login for admins in staff-writes-only-mode (#18443) 2022-09-30 14:12:49 -05:00
Daniel Waterworth c1a7fa6b5d FIX: Allow logout for admins in staff-writes-only-mode (#18441) 2022-09-30 13:03:20 -05:00
Daniel Waterworth cb922ca8c8 DEV: update .ruby-version.sample (#18426) 2022-09-29 13:38:44 -05:00
Daniel Waterworth 69d74ae508 DEV: Differentiate staff-writes-only banner (#18364) 2022-09-26 13:16:29 -05:00
Daniel Waterworth e3d7253199 DEV: Only lock rows that need to change in migration (#18319) 2022-09-21 14:14:45 -05:00
Daniel Waterworth e37ced96bf DEV: Don't interpret user field names as HTML (#18317)
This isn't a security bug, because only admins can create user fields
and we have to trust admins, because they can change themes, which are
shown site-wide and can contain unrestricted JS.
2022-09-21 12:21:58 -05:00
Daniel Waterworth e0ece3a77e Revert "DEV: Improve multisite db scripts in dev (#17337)" (#17801)
This reverts commit 2e4056d185.

Unfortunately, this broke db:create and db:drop when multisite config
wasn't present.
2022-08-04 16:15:06 -05:00
Daniel Waterworth 83d3543e33 DEV: Guardians aren't active record objects, so shouldn't use fab! (#17789) 2022-08-03 19:17:40 -05:00
Daniel Waterworth e91cc340e6 FIX: Add ember redirect for tags/:tag_id -> tag/:tag_id (#17397)
/tags/:tag_id is deprecated, but links may exist in posts.
2022-07-08 14:39:43 -05:00
Daniel Waterworth d7886e4d18 FIX: Allow all subdomains of localhost in development (#17089)
domains can contain numbers.

Follow-up-to: 00e756e358
2022-06-14 14:54:49 -05:00
Daniel Waterworth 00e756e358 DEV: Allow all subdomains of localhost in development (#17018)
By default in rails, it looks like only one level deep is allowed.
2022-06-06 16:02:51 -05:00
Daniel Waterworth d7bc395182 FIX: Seed multisite dbs after migrating in development (#17017)
Dumping the schema cache reset the current_db and we only need to do
this once.
2022-06-06 14:43:43 -05:00
Daniel WaterworthandDavid Taylor f00e282067 FIX: Delete extraneous topic bookmarks (#16926)
Otherwise the migration can fail

Co-authored-by: David Taylor <david@taylorhq.com>
2022-05-26 11:58:15 -05:00
Daniel Waterworth 6e53f4d913 DEV: New readonly mode. Only applies to non-staff (#16243) 2022-05-17 13:06:08 -05:00
Daniel Waterworth 9eadabe9fc DEV: Let's deny access to sso endpoints when disabled consistently (#16752) 2022-05-13 14:33:28 -05:00
Daniel Waterworth 6a4696eec8 DEV: Add helper method for repeated sso logging pattern (#16749) 2022-05-13 12:19:44 -05:00
Daniel Waterworth 66a04c5cfe FIX: Prevent all kinds of login in readonly mode (#16743) 2022-05-13 10:52:01 -05:00