sphinx-build: Add support for "-j auto"

Adjust -j argument so it can accept special keyword "auto", which
will set -j to the number of CPUs on machine.

Change was inspired by headaches while trying to find the best way
to get sphinx-build to use the max number of cores from any machine
we build on (Ubuntu, Mac, AWS Linux, etc). Rather than write
external scripts to set this (sysctl calls, grepping /proc/cpuinfo,
etc), it made more sense to build the functionality into
sphinx-build itself.

Change Summary:
* New custom type in sphinx.cmdline for handling -j argument
* Adjusted help string for -j flag
This commit is contained in:
mark floyd 2017-12-01 18:34:33 -08:00
parent eff13bc4c4
commit 1db38e0eca

View File

@ -11,6 +11,7 @@
from __future__ import print_function
import argparse
import multiprocessing
import sys
import traceback
from os import path
@ -83,6 +84,19 @@ def handle_exception(app, args, exception, stderr=sys.stderr):
file=stderr)
def jobs_argument(value):
# type (int, str) -> int
"""
Special type to handle 'auto' flags passed to 'sphinx-build' via -j flag. Can
be expanded to handle other special scaling requests, such as setting job count
to (cpu_count / 2)
"""
if value == 'auto':
return multiprocessing.cpu_count()
else:
return value
def get_parser():
# type: () -> argparse.ArgumentParser
parser = argparse.ArgumentParser(
@ -129,10 +143,9 @@ files can be built by specifying individual filenames.
group.add_argument('-d', metavar='PATH', dest='doctreedir',
help='path for the cached environment and doctree '
'files (default: OUTPUTDIR/.doctrees)')
group.add_argument('-j', metavar='N', default=1, type=int, dest='jobs',
group.add_argument('-j', metavar='N', default=1, type=jobs_argument, dest='jobs',
help='build in parallel with N processes where '
'possible')
'possible (special value "auto" will set N to number of CPUs')
group = parser.add_argument_group('build configuration options')
group.add_argument('-c', metavar='PATH', dest='confdir',
help='path where configuration file (conf.py) is '