This commit is a complete reimplementation of our theme JS compilation
system.
Previously, we compiled theme JS into AMD `define` statements on a
per-source-file basis, and then concatenated them together for the
client. These AMD modules would integrate with those in Discourse core,
allowing two way access between core/theme modules. Going forward, we'll
be moving away from AMD, and towards native ES modules in core. Before
we can do that, we need to stop relying on AMD as the 'glue' between
core and themes/plugins.
This change introduces Rollup (running in mini-racer) as a compiler for
theme JS. This is configured to generate a single ES Module which
exports a list of 'compat modules'. Core `import()`s the modules for
each active theme, and adds them all to AMD. In future, this consumption
can be updated to avoid AMD entirely.
All module resolution within a theme is handled by Rollup, and does not
use AMD.
Import of core/plugin modules from themes are automatically transformed
into calls to a new `window.moduleBroker` interface. For now, this is a
direct interface to AMD. In future, this can be updated to point to real
ES Modules in core.
Despite the complete overhaul of the internals, this is not a breaking
change, and should have no impact on existing themes. If any
incompatibilities are found, please report them on
https://meta.discourse.org.
---------
Co-authored-by: Jarek Radosz <jarek@cvx.dev>
Co-authored-by: Chris Manson <chris@manson.ie>
When enabled this will convert uploaded videos to a standard format that should
be playable on all devices and browsers.
The goal of this feature is to prevent codec playback issues that
sometimes can occur with video uploads.
It uses an adapter pattern, so that other services for video conversion
could be easily added in the future.
- Migrated from annotate to annotaterb as the former is not maintained
anymore.
- Dropped our `fast_pluck` patch as the default `pluck` implementation
seems now faster.
A few minor versions of Bullet were incompatible with Discourse because we use our own content security policy middleware.
This has now been fixed upstream and released in 8.0.7.
Because we use a custom ContentSecurityPolicy middleware, the latest versions (> 8.0.3) of Bullet error out on load.
This PR locks the gem to the latest compatible version.
This commit is replacing the system specs driver (selenium) by
Playwright: https://playwright.dev/
We are still using Capybara to write the specs but they will now be run
by Playwright. To achieve this we are using the non official ruby
driver: https://github.com/YusukeIwaki/capybara-playwright-driver
### Notable changes
- `CHROME_DEV_TOOLS` has been removed, it's not working well with
playwright use `pause_test` and inspect browser for now.
- `fill_in` is not generating key events in playwright, use `send_keys`
if you need this.
### New spec options
#### trace
Allows to capture a trace in a zip file which you can load at
https://trace.playwright.dev or locally through `npx playwright
show-trace /path/to/trace.zip`
_Example usage:_
```ruby
it "shows bar", trace: true do
visit("/")
find(".foo").click
expect(page).to have_css(".bar")
end
```
#### video
Allows to capture a video of your spec.
_Example usage:_
```ruby
it "shows bar", video: true do
visit("/")
find(".foo").click
expect(page).to have_css(".bar")
end
```
### New env variable
#### PLAYWRIGHT_SLOW_MO_MS
Allow to force playwright to wait DURATION (in ms) at each action.
_Example usage:_
```
PLAYWRIGHT_SLOW_MO_MS=1000 rspec foo_spec.rb
```
#### PLAYWRIGHT_HEADLESS
Allow to be in headless mode or not. Default will be headless.
_Example usage:_
```
PLAYWRIGHT_HEADLESS=0 rspec foo_spec.rb # will show the browser
```
### New helpers
#### with_logs
Allows to access the browser logs and check if something specific has
been logged.
_Example usage:_
```ruby
with_logs do |logger|
# do something
expect(logger.logs.map { |log| log[:message] }).to include("foo")
end
```
#### add_cookie
Allows to add a cookie on the browser session.
_Example usage:_
```ruby
add_cookie(name: "destination_url", value: "/new")
```
#### get_style
Get the property style value of an element.
_Example usage:_
```ruby
expect(get_style(find(".foo"), "height")).to eq("200px")
```
#### get_rgb_color
Get the rgb color of an element.
_Example usage:_
```ruby
expect(get_rgb_color(find("html"), "backgroundColor")).to eq("rgb(170, 51, 159)")
```
We are no longer using any of the transpilation/bundling features of
Sprockets. We only use it to serve assets in development, and then
collect & fingerprint them in production. This commit switches us to use
the more modern "Propshaft" gem for that functionality.
Propshaft is much simpler than Sprockets. Instead of taking a
combination of paths + "precompile" list, Propshaft simply assumes all
files in the configured directory are required in production. Previously
we had some base paths configured quite high in the directory structure,
and then only precompiled selected assets within the directory. That's
no longer possible, so this commit refactors those places (mostly
plugin-related) to use dedicated directories under
`app/assets/generated/`.
Another difference is that Propshaft applies asset digests in
development as well as production. This is great for caching & dev/prod
consistency, but does mean some small changes were required in tests.
We previously had some freedom-patches applied to Sprockets. Some of
those had to be ported across to Propshaft. We now have three patches:
1. Skip adding digest hashes to webpack-generated chunks (which are
already digested, and referred to from other js files)
2. Avoid raising errors for missing assets in test mode. We don't always
compile assets before running basic RSpec tests.
3. Maintain relative paths for sourcemap URLs, so that files don't need
to be recompiled depending on their CDN path
Significant refactors are made to the `assets.rake` and `s3.rake` tasks,
which rely on implementation details of Sprockets/Propshaft.
Previously all locale bundles would be built & compressed during
assets:precompile. For most sites, only one of these languages was
actually used, so this is fairly wasteful.
This commit moves the main locale bundle into the
ExtraLocalesController, which has recently undergone many improvements
to make it more efficient. This allows locale files to be bundled "just
in time" when they're first accessed.
Now that brotli level=6 is enabled for these assets in our nginx config,
this change should have no impact on the locale bundle size.
Only notable changes is that we added
internal support for TimeWithZone which
was absent from previous release
We also improved error messages for T_OBJECT
This bumps the core ruby version requirement
to at least 3.3.0, and moves this version check
into the Gemfile per
https://bundler.io/guides/gemfile_ruby.html
We are doing this because generally our gems
are using a higher version, it's about time
for a bump anyway.
This commit moves most of emoji logic into the discourse-emojis gem:
https://github.com/discourse/discourse-emojis/
Most notably:
- images are now symlinked from the gem
- the gem provides path to the json files
Search aliases have also been made asynchronous and memoized. When you
will search for an emoji we will now load the aliases and store the list
for future use.
---------
Co-authored-by: David Taylor <david@taylorhq.com>
We have a native dependency, Rinku, that's used only to make links in one place. We can get rid of this and use PrettyText instead.
This is almost a one-for-one replacement, but PrettyText adds rel="noopener nofollow ugc" to external links, which I suspect is actually what we want. It also wraps the result in a <p> tag, which we strip out for parity with Rinku.
Previous upgrade had a runaway CPU issue due to
overly aggressive GC running.
MiniRacer was running V8 GC every 2 seconds.
New change fixes the parameter so it only issues a GC
if 2 seconds past since last MiniRacer eval.
Mini_racer was redesigned to keep all V8 vm integration on a dedicated
thread.
Previous attempts at upgrades had lots of stack poisoning across V8
owned
and Ruby owned threads that were leading to segfaults.
We will monitor the new design in production to see how robust the new
solution is.
- Add concurrency when running on multisite clusters (default 10, configurable via THEME_UPDATE_CONCURRENCY env)
- Add a version cache for the duration of the rake task. This avoids duplicating work when many sites in the cluster have the same theme installed, and it is already up-to-date
- Updates output to be more concurrent friendly (all `puts`, no `print`)
This will help to enforce a consistent pattern for creating service
actions.
This patch also namespaces actions and policies, making everything
related to a service available directly in
`app/services/<concept-name>`, making things more consistent at that
level too.
* Updates GitHub Actions
* Switches from `bundler/inline` to an optional group in the `Gemfile` because the previous solution didn't work well with rspec
* Adds the converter framework and tests
* Allows loading private converters (see README)
* Switches from multiple CLI tools to a single CLI
* Makes DB connections reusable and adds a new abstraction for the `IntermediateDB`
* `IntermediateDB` acts as an interface for IPC calls when a converter steps runs in parallel (forks). Only the main process writes to the DB.
* Includes a simple example implementation of a converter for now.
The latest node 22.7.0 based release of mini_racer disabled
Maglev (https://v8.dev/blog/maglev) based optimisations
From previous segfaults it appears the failures happened in the maglev code
paths.
Node release of v8 lags latest release of v8 so, whatever is happening, may
be resolved in latest.
Loïc Guitaut
Blake Erickson
Sam
Jarek Radosz
Ted Johansson
Joffrey JAFFEUX
Gerhard Schlager
Martin Brennan