Install docutils in seperate file

If a Requirement specifier includes a pre-release or development
    version (e.g. >=0.0.dev0) then pip will allow pre-release and development
    versions for that requirement. This does not include the != flag.

From pip.pypa.io/en/latest/cli/pip_install/#pre-release-versions

.readthedocs.yml

@@ -6,6 +6,7 @@ python:
     - requirements: docs/requirements.txt
     - method: pip
       path: .
+    - requirements: docs/docutils-test.txt
 
 sphinx:
   configuration: docs/conf.py

docs/docutils-test.txt

@@ -0,0 +1 @@
+docutils==0.17

docs/requirements.txt

@@ -1,2 +1,2 @@
-sphinx>=3.0
+sphinx>=4.0.dev0
 sphinxcontrib-httpdomain