Commit Graph
817 Commits
Author SHA1 Message Date
David Taylor 0f1c9574f2 DEV: Update stage_security_fixes rake task (#37346)
- Moves it from `version_bump.rake` to the new `release.rake`. This is
the last thing which was pending upgrade, so we can now delete the whole
`version_bump.rake` file & spec

- Adds support for release/* branches

- Adds an interactive prompt to choose which security PRs to include

- Creates & merges the PR using `gh` CLI. Less manual work.
2026-01-28 15:12:29 +00:00
Michael Brown e935afbb62 DEV: always load the excon gem
We ran into trouble with MethodProfiler referencing excon without having loaded
it as it was relying on an initialiser to load it and depending on the side
effect.

If the excon gem is going to be loaded anyways, it doesn't make sense to have
it not loaded by default; this will be more robust.
2026-01-15 17:13:11 -05:00
Martin BrennanandRégis Hanol d2252b5cd3 DEV: Remove IMAP support in Discourse (#37002)
per
https://meta.discourse.org/t/imap-support-for-group-inboxes/160588/39?u=martin
we have been planning to remove IMAP support for a while,
because of its low usage and adoption, high complexity, and maintenance
burden.
This commit removes all IMAP-related code, including models,
jobs, services, and frontend components.

---------

Co-authored-by: Régis Hanol <regis@hanol.fr>
2026-01-12 10:07:26 +10:00
SamandMark McClure d31ca17c57 FEATURE: update mathjax to version 4.1 (#36814)
This upgrades to Mathjax 4.1 and latest katex
Implement rich text composer support for math
Adds support for /( )/ and /[ /] which was missing and very common now 
Removes math javascript from our repo

---------

Co-authored-by: Mark McClure <mcmcclur@unca.edu>
2026-01-09 08:24:31 +11:00
Loïc Guitaut a3fd7eb462 DEV: Add ruby-lsp-rspec (#35735)
We already have `ruby-lsp` and `ruby-lsp-rails`, so let’s add
`ruby-lsp-rspec` too, as it helps navigate specs quite a lot.
2025-10-31 12:20:20 +01:00
Loïc Guitaut 154224f109 DEV: Add Pitchfork alongside Unicorn (#35370)
This PR adds Pitchfork, as we want to move away from Unicorn ultimately.

Unicorn still boots by default, so there should be no disruption for
anyone.

To use Pitchfork instead of Unicorn, the `RUN_PITCHFORK` environment
variable must be set.
This will make `bin/rails s` and `config/unicorn_launcher` boot
Pitchfork. `unicorn_launcher` was patched because that way we can easily
switch between Unicorn and Pitchfork without having to change too many
things on the infra side.

The upgrader from the `docker_manager` plugin doesn’t work yet with
Pitchfork. This will be addressed in a future PR.
2025-10-24 11:08:23 +02:00
David Taylor e5ca38d2be DEV: Add ruby-lsp to development gems (#35170)
This significantly simplifies and speeds up using the ruby LSP in
development, since a separate 'composed bundle' is no longer needed.
2025-10-03 13:17:01 +02:00
Loïc Guitaut 2e47ee8a9e DEV: Fix Unicorn reloading (#34980)
Recently we replaced the `pry-byebug` gem by the `debug` one. It broke
the auto-reload mechanism we have in the development environment for
Unicorn.

Indeed, the `debug` gem by default will add an `at_exit` hook that will
wait for all its children to exit. It clashes with our own mechanism.

The workaround is to require `debug/prelude` instead of `debug`: the
`debugger` command and breakpoints will still work but without the hook
being set.
2025-09-25 15:51:02 +02:00
Rafael dos Santos Silva f608e0cd7e DEV: Move debugging gem from byebug to debug (#34827)
The official `debug` gem provides more modern debugging capabilities,
better syntax highlighting, the native ability to use vscode / chrome
debuggers and is maintained alongside ruby itself.
2025-09-17 10:08:52 -03:00
Loïc Guitaut eedda1f809 DEV: Enable Goldiloader by default
As things are going well with Goldiloader enabled, we can now enable it
by default.
2025-08-26 09:48:56 +02:00
Loïc Guitaut 1dc7295bba DEV: Add Goldiloader behind a setting (#34006)
This will allow us to try out Goldiloader and see how it performs.
2025-08-04 12:05:02 +02:00
Loïc Guitaut 3aa6842a5c DEV: Pin Rack to < 3 (#33891)
Upgrading Rack to version 3 will break Unicorn. When we don’t use Unicorn anymore, we’ll be able to upgrade to Rack 3.
2025-07-28 11:43:52 +02:00
aa2fb29fa6 DEV: Use rollup for theme JS compilation (#33103)
This commit is a complete reimplementation of our theme JS compilation
system.

Previously, we compiled theme JS into AMD `define` statements on a
per-source-file basis, and then concatenated them together for the
client. These AMD modules would integrate with those in Discourse core,
allowing two way access between core/theme modules. Going forward, we'll
be moving away from AMD, and towards native ES modules in core. Before
we can do that, we need to stop relying on AMD as the 'glue' between
core and themes/plugins.

This change introduces Rollup (running in mini-racer) as a compiler for
theme JS. This is configured to generate a single ES Module which
exports a list of 'compat modules'. Core `import()`s the modules for
each active theme, and adds them all to AMD. In future, this consumption
can be updated to avoid AMD entirely.

All module resolution within a theme is handled by Rollup, and does not
use AMD.

Import of core/plugin modules from themes are automatically transformed
into calls to a new `window.moduleBroker` interface. For now, this is a
direct interface to AMD. In future, this can be updated to point to real
ES Modules in core.

Despite the complete overhaul of the internals, this is not a breaking
change, and should have no impact on existing themes. If any
incompatibilities are found, please report them on
https://meta.discourse.org.

---------

Co-authored-by: Jarek Radosz <jarek@cvx.dev>
Co-authored-by: Chris Manson <chris@manson.ie>
2025-07-25 12:02:29 +01:00
Blake Erickson af3abb54e3 FEATURE: Add support for aws MediaConvert (#33092)
When enabled this will convert uploaded videos to a standard format that should
be playable on all devices and browsers.

The goal of this feature is to prevent codec playback issues that
sometimes can occur with video uploads.

It uses an adapter pattern, so that other services for video conversion
could be easily added in the future.
2025-07-23 11:58:33 -06:00
Sam fcd022b5b3 DEV: revert hiredis upgrade (#33785)
Revert hiredis upgrade due to .info failing on some environments

https://meta.discourse.org/t/rebuild-error/375387
2025-07-23 18:25:05 +10:00
Sam 471f0d0cd4 PERF: switch redis driver to hiredis (#33773)
Adds high performance redis driver, see:
https://github.com/redis-rb/redis-client

It is automatically configured.
2025-07-23 12:27:39 +10:00
Jarek Radosz 7d4974b2bb DEV: Unlock gem versions (#33753)
let's see what shakes out in the next dependabot run
2025-07-22 19:49:20 +02:00
Jarek Radosz 1821e2b900 DEV: Move discourse-ai to core (#33749)
https://meta.discourse.org/t/373574

Internal `/t/-/156778`
2025-07-22 15:07:59 +02:00
Jarek Radosz 97b1f04f8d DEV: Move discourse-github to core (#33749)
https://meta.discourse.org/t/373574

Internal `/t/-/156778`
2025-07-22 15:07:59 +02:00
Loïc Guitaut 0eab7daea4 DEV: Upgrade Rails to version 8.0.2
- Migrated from annotate to annotaterb as the former is not maintained
  anymore.
- Dropped our `fast_pluck` patch as the default `pluck` implementation
  seems now faster.
2025-07-22 09:59:44 +02:00
Jarek Radosz ff7a0f2391 DEV: Move discourse-subscriptions to core (#33570)
https://meta.discourse.org/t/373574

Internal `/t/-/156778`
2025-07-15 16:38:05 +02:00
Jarek Radosz 5e86f2c21f DEV: Move discourse-zendesk-plugin to core (#33570)
https://meta.discourse.org/t/373574

Internal `/t/-/156778`
2025-07-15 16:38:05 +02:00
David Taylor 60b3a216e9 DEV: Drop fakeweb dependency (#33069)
This was only be `require`'d by one official plugin
(discourse-perspective), but was causing failures in specs of unrelated
plugins. See
https://github.com/discourse/discourse-perspective-api/pull/110
2025-06-04 14:29:25 +01:00
Ted Johansson 2eb459116c DEV: Update Bullet to latest version (#32904)
A few minor versions of Bullet were incompatible with Discourse because we use our own content security policy middleware.

This has now been fixed upstream and released in 8.0.7.
2025-06-04 09:40:06 +08:00
Ted Johansson 29eb2eb096 FIX: Lock Bullet gem to a compatible version (#32724)
Because we use a custom ContentSecurityPolicy middleware, the latest versions (> 8.0.3) of Bullet error out on load.

This PR locks the gem to the latest compatible version.
2025-05-15 10:03:24 +08:00
Joffrey JAFFEUX b6aad28ccf DEV: replace selenium driver with playwright (#31977)
This commit is replacing the system specs driver (selenium) by
Playwright: https://playwright.dev/

We are still using Capybara to write the specs but they will now be run
by Playwright. To achieve this we are using the non official ruby
driver: https://github.com/YusukeIwaki/capybara-playwright-driver

### Notable changes

- `CHROME_DEV_TOOLS` has been removed, it's not working well with
playwright use `pause_test` and inspect browser for now.

- `fill_in` is not generating key events in playwright, use `send_keys`
if you need this.

### New spec options

#### trace

Allows to capture a trace in a zip file which you can load at
https://trace.playwright.dev or locally through `npx playwright
show-trace /path/to/trace.zip`

_Example usage:_

```ruby
it "shows bar", trace: true do
  visit("/")

  find(".foo").click

  expect(page).to have_css(".bar")
end
```

#### video

Allows to capture a video of your spec.

_Example usage:_

```ruby
it "shows bar", video: true do
  visit("/")

  find(".foo").click

  expect(page).to have_css(".bar")
end
```

### New env variable

#### PLAYWRIGHT_SLOW_MO_MS

Allow to force playwright to wait DURATION (in ms) at each action.

_Example usage:_

```
PLAYWRIGHT_SLOW_MO_MS=1000 rspec foo_spec.rb
```

#### PLAYWRIGHT_HEADLESS

Allow to be in headless mode or not. Default will be headless.

_Example usage:_

```
PLAYWRIGHT_HEADLESS=0 rspec foo_spec.rb # will show the browser
```

### New helpers

#### with_logs

Allows to access the browser logs and check if something specific has
been logged.

_Example usage:_

```ruby
with_logs do |logger|
  # do something

  expect(logger.logs.map { |log| log[:message] }).to include("foo")
end
```

#### add_cookie

Allows to add a cookie on the browser session.

_Example usage:_

```ruby
add_cookie(name: "destination_url", value: "/new")
```

#### get_style

Get the property style value of an element.

_Example usage:_

```ruby
expect(get_style(find(".foo"), "height")).to eq("200px")
```

#### get_rgb_color

Get the rgb color of an element.

_Example usage:_

```ruby
expect(get_rgb_color(find("html"), "backgroundColor")).to eq("rgb(170, 51, 159)")
```
2025-05-06 10:44:14 +02:00
David Taylor e7450cc6da DEV: Migrate from sprockets to propshaft for assets (#32475)
We are no longer using any of the transpilation/bundling features of
Sprockets. We only use it to serve assets in development, and then
collect & fingerprint them in production. This commit switches us to use
the more modern "Propshaft" gem for that functionality.

Propshaft is much simpler than Sprockets. Instead of taking a
combination of paths + "precompile" list, Propshaft simply assumes all
files in the configured directory are required in production. Previously
we had some base paths configured quite high in the directory structure,
and then only precompiled selected assets within the directory. That's
no longer possible, so this commit refactors those places (mostly
plugin-related) to use dedicated directories under
`app/assets/generated/`.

Another difference is that Propshaft applies asset digests in
development as well as production. This is great for caching & dev/prod
consistency, but does mean some small changes were required in tests.

We previously had some freedom-patches applied to Sprockets. Some of
those had to be ported across to Propshaft. We now have three patches:

1. Skip adding digest hashes to webpack-generated chunks (which are
already digested, and referred to from other js files)

2. Avoid raising errors for missing assets in test mode. We don't always
compile assets before running basic RSpec tests.

3. Maintain relative paths for sourcemap URLs, so that files don't need
to be recompiled depending on their CDN path

Significant refactors are made to the `assets.rake` and `s3.rake` tasks,
which rely on implementation details of Sprockets/Propshaft.
2025-04-30 08:59:32 +01:00
David Taylor c62a4a4759 PERF: Compile main locale bundles just-in-time (#32335)
Previously all locale bundles would be built & compressed during
assets:precompile. For most sites, only one of these languages was
actually used, so this is fairly wasteful.

This commit moves the main locale bundle into the
ExtraLocalesController, which has recently undergone many improvements
to make it more efficient. This allows locale files to be bundled "just
in time" when they're first accessed.

Now that brotli level=6 is enabled for these assets in our nginx config,
this change should have no impact on the locale bundle size.
2025-04-28 10:31:27 +01:00
Loïc Guitaut 2ed31fea64 DEV: Upgrade the Redis gem to v5.4 2025-03-19 14:34:00 +01:00
Ted Johansson e87bfad23b Revert "DEV: Replace Rinku native gem with PrettyText" (#31692)
Reverts discourse/discourse#31557

This is causing excessive spacing due to the addition of empty `<p>`
tags. Revert first while we fix that.
2025-03-07 12:01:22 +10:00
Sam 503645dc34 DEV: bump to major release of mini_racer (#31640)
Only notable changes is that we added
internal support for TimeWithZone which
was absent from previous release

We also improved error messages for T_OBJECT
2025-03-05 12:06:36 +11:00
Gerhard Schlager 0c818fea50 DEV: Relax Ruby version constraint to ~> 3.3 (#31610)
This change allows updates to any 3.x version, while still restricting
major updates to 4.x.
2025-03-04 14:23:49 +01:00
Martin Brennan 76e58a55ed DEV: Bump ruby version requirement to 3.3.0 (#31538)
This bumps the core ruby version requirement
to at least 3.3.0, and moves this version check
into the Gemfile per
https://bundler.io/guides/gemfile_ruby.html

We are doing this because generally our gems
are using a higher version, it's about time
for a bump anyway.
2025-03-04 12:23:32 +11:00
Joffrey JAFFEUXandDavid Taylor d38acc5df1 DEV: discourse-emojis gem (#31408)
This commit moves most of emoji logic into the discourse-emojis gem:
https://github.com/discourse/discourse-emojis/

Most notably:
- images are now symlinked from the gem
- the gem provides path to the json files

Search aliases have also been made asynchronous and memoized. When you
will search for an emoji we will now load the aliases and store the list
for future use.

---------

Co-authored-by: David Taylor <david@taylorhq.com>
2025-03-03 13:09:08 +01:00
Ted Johansson 258dfab8d7 DEV: Replace Rinku native gem with PrettyText (#31557)
We have a native dependency, Rinku, that's used only to make links in one place. We can get rid of this and use PrettyText instead.

This is almost a one-for-one replacement, but PrettyText adds rel="noopener nofollow ugc" to external links, which I suspect is actually what we want. It also wraps the result in a <p> tag, which we strip out for parity with Rinku.
2025-03-03 09:19:17 +08:00
Jarek Radosz c592edbe34 DEV: Update discourse_dev_assets and faker (#31436) 2025-02-22 10:25:56 +01:00
Jarek Radosz 59406fb286 DEV: Remove an obsolete gem (#31459) 2025-02-22 10:25:33 +01:00
Sam 8d3a35e25b DEV: update mini_racer to node 23.6.1 (#31251)
This brings us extremely close to latest
2025-02-10 12:58:34 +11:00
Sam 0b35f72e6a DEV: upgrade mini_racer (#31154)
Previous upgrade had a runaway CPU issue due to
overly aggressive GC running.

MiniRacer was running V8 GC every 2 seconds.

New change fixes the parameter so it only issues a GC
if 2 seconds past since last MiniRacer eval.
2025-02-04 12:33:19 +11:00
Sam b76c5406bd DEV: revert mini_racer upgrade (#31119)
There are reports of high CPU usage, so we will revert for now
2025-02-03 17:17:35 +11:00
Sam be5b12c906 DEV: update mini_racer (#30656)
Mini_racer was redesigned to keep all V8 vm integration on a dedicated
thread.

Previous attempts at upgrades had lots of stack poisoning across V8
owned
and Ruby owned threads that were leading to segfaults.

We will monitor the new design in production to see how robust the new
solution is.
2025-01-23 09:08:00 +11:00
Gerhard Schlager 2d05a82f52 DEV: Add Upload to IntermediateDB (#29780) 2025-01-20 23:16:30 +01:00
Loïc Guitaut d6bec460a8 DEV: Upgrade Rails to version 7.2 2024-11-27 10:48:47 +01:00
David Taylor b3b0695bb1 PERF: Optimize themes:update task (#29290)
- Add concurrency when running on multisite clusters (default 10, configurable via THEME_UPDATE_CONCURRENCY env)

- Add a version cache for the duration of the rake task. This avoids duplicating work when many sites in the cluster have the same theme installed, and it is already up-to-date

- Updates output to be more concurrent friendly (all `puts`, no `print`)
2024-10-21 12:36:40 +01:00
Sam f6df0f8789 Revert "DEV: upgrade mini_racer (#29023)" (#29027)
This reverts commit 042bfeac66.

as expected still segfaulting
2024-09-30 14:34:04 +10:00
Sam 042bfeac66 DEV: upgrade mini_racer (#29023)
This is attempt 5 at upgrading mini racer, fixes some edge conditions in
mini_racer that can lead to segfaults.
2024-09-30 09:12:14 +10:00
Sam Saffron f22316d0cd Revert "DEV: attempt another upgrade on mini_racer (#28951)"
This reverts commit 68e017c324.

Still seeing faults ... we had 1 segfault in 7 hours.
2024-09-19 09:48:01 +10:00
Loïc Guitaut 05b8ff436c DEV: Introduce a Service::ActionBase class for service actions
This will help to enforce a consistent pattern for creating service
actions.

This patch also namespaces actions and policies, making everything
related to a service available directly in
`app/services/<concept-name>`, making things more consistent at that
level too.
2024-09-18 17:02:46 +02:00
Sam 68e017c324 DEV: attempt another upgrade on mini_racer (#28951)
This time the compile disables strict aliasing to see if it resolves the issue
2024-09-18 08:44:26 +10:00
Sam 991227337a Revert "DEV: attempt a mini_racer upgrade (#28924)" (#28925)
This reverts commit dc8620a491.

Still segfaulting - missing artifact for ARM darwin
2024-09-16 12:10:32 +10:00