mirror of
https://github.com/discourse/discourse.git
synced 2025-02-25 18:55:32 -06:00
b652f66378
We were running into errors running `ember build` on machines with high CPU counts. It was then noted that `thread-loader`, which embroider uses, defaults to spinning up x workers where x is number of physical CPU cores - 1. That is probably too much so we set out to find out an optimial count to set for the `JOBS` env which embroider will use to set the number of `thread-loader` workers. I first built an image using the following Dockerfile. ``` FROM discourse/base:release RUN cd /var/www/discourse && sudo -EH -u discourse bundle exec rake plugin:install_all_official RUN cd /var/www/discourse && sudo -EH -u discourse bundle exec rake assets:precompile:prereqs ``` I then ran the following command on my M3 Max Macbook Pro that has 14 phyisal CPU cores. ``` for j in 1 2 4 8 14; do echo "JOBS=$j"; time docker run --rm -it -e JOBS=$j test:latest /bin/bash -c "su discourse -c 'cd /var/www/discourse && bundle exec rake assets:precompile:build'"; done ``` These are the results I got: ``` JOBS=1 0.04s user 0.03s system 0% cpu 1:01.92 total JOBS=2 0.04s user 0.02s system 0% cpu 42.605 total JOBS=4 0.04s user 0.02s system 0% cpu 37.012 total JOBS=8 0.04s user 0.02s system 0% cpu 35.199 total JOBs=14 0.04s user 0.02s system 0% cpu 37.941 total ``` We think JOBS=2 is a good default when the `JOBS` env has not been set. Anything above just consumes more resources for little benefit.