discourse/script/check_reproducible_assets.rb

60 lines
1.8 KiB
Ruby
Raw Normal View History

DEV: introduce Embroider behind a flag, and start testing in CI (#23005) Discourse core now builds and runs with Embroider! This commit adds the Embroider-based build pipeline (`USE_EMBROIDER=1`) and start testing it on CI. The new pipeline uses Embroider's compat mode + webpack bundler to build discourse code, and leave everything else (admin, wizard, markdown-it, plugins, etc) exactly the same using the existing Broccoli-based build as external bundles (<script> tags), passed to the build as `extraPublicTress` (which just means they get placed in the `/public` folder). At runtime, these "external" bundles are glued back together with `loader.js`. Specifically, the external bundles are compiled as AMD modules (just as they were before) and registered with the global `loader.js` instance. They expect their `import`s (outside of whatever is included in the bundle) to be already available in the `loader.js` runtime registry. In the classic build, _every_ module gets compiled into AMD and gets added to the `loader.js` runtime registry. In Embroider, the goal is to do this as little as possible, to give the bundler more flexibility to optimize modules, or omit them entirely if it is confident that the module is unused (i.e. tree-shaking). Even in the most compatible mode, there are cases where Embroider is confident enough to omit modules in the runtime `loader.js` registry (notably, "auto-imported" non-addon NPM packages). So we have to be mindful of that an manage those dependencies ourselves, as seen in #22703. In the longer term, we will look into using modern features (such as `import()`) to express these inter-dependencies. This will only be behind a flag for a short period of time while we perform some final testing. Within the next few weeks, we intend to enable by default and remove the flag. --------- Co-authored-by: David Taylor <david@taylorhq.com>
2023-09-07 07:15:43 -05:00
#! /usr/bin/env ruby
# frozen_string_literal: true
# It's important that our JS asset builds are reproducible so that users aren't forced to re-download
# assets after every deploy. This script runs two builds and compares the output to ensure that they
# are identical.
require "digest"
DIST_DIR = File.expand_path("#{__dir__}/../app/assets/javascripts/discourse/dist")
def collect_asset_info
files =
Dir.glob("**/*", base: DIST_DIR).reject { |path| File.directory? "#{DIST_DIR}/#{path}" }.sort
puts "Found #{files.length} files"
raise "No files found" if files.empty?
digests = files.map { |file| Digest::MD5.file("#{DIST_DIR}/#{file}").hexdigest }
{ files: files, digests: digests }
end
puts "Running first build..."
system "#{__dir__}/../bin/yarn-app ember build -prod", exception: true
first_build_info = collect_asset_info
puts "Running second build..."
system "#{__dir__}/../bin/yarn-app ember build -prod", exception: true
second_build_info = collect_asset_info
puts nil, nil, "Comparing builds...", nil, nil
if first_build_info[:files] != second_build_info[:files]
puts "Set of files is different"
new_assets = first_build_info[:files].difference(second_build_info[:files])
puts "Second build had additional assets: #{new_assets.inspect}"
missing_assets = second_build_info[:files].difference(first_build_info[:files])
puts "Second build was missing assets: #{missing_assets.inspect}"
exit 1
else
puts "Both builds produced the same file names"
end
if first_build_info[:digests] != second_build_info[:digests]
puts "File digests are different"
first_build_info[:files].each_with_index do |file, index|
if first_build_info[:digests][index] != second_build_info[:digests][index]
puts "File #{file} has different digest"
end
end
exit 1
else
puts "Files in both builds had identical digests"
end
puts nil, "Success!"