docs: Add sphinx_selective_exclude extension suite.
Designed specifically to workaround issues we were facing with generating multiple conditionalized output docsets from a single master doctree. Extensions were factored out into a separate project, based on the fact that many other Sphinx users experience similar or related problems: https://github.com/pfalcon/sphinx_selective_exclude Corresponds to the 182f4a8da57 upstream revision.
This commit is contained in:
Paul Sokolovsky
parent
9de5eb278d
commit
f6d01b8b67
|
|
@ -0,0 +1,25 @@
|
|||
Copyright (c) 2016 by the sphinx_selective_exclude authors.
|
||||
All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are
|
||||
met:
|
||||
|
||||
* Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
|
||||
* Redistributions in binary form must reproduce the above copyright
|
||||
notice, this list of conditions and the following disclaimer in the
|
||||
documentation and/or other materials provided with the distribution.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
|
@ -0,0 +1,138 @@
|
|||
Sphinx eager ".. only::" directive and other selective rendition extensions
|
||||
===========================================================================
|
||||
|
||||
Project home page: https://github.com/pfalcon/sphinx_selective_exclude
|
||||
|
||||
The implementation of ".. only::" directive in Sphinx documentation
|
||||
generation tool is known to violate principles of least user surprise
|
||||
and user expectations in general. Instead of excluding content early
|
||||
in the pipeline (pre-processor style), Sphinx defers exclusion until
|
||||
output phase, and what's the worst, various stages processing ignore
|
||||
"only" blocks and their exclusion status, so they may leak unexpected
|
||||
information into ToC, indexes, etc.
|
||||
|
||||
There's multiple issues submitted upstream on this matter:
|
||||
|
||||
* https://github.com/sphinx-doc/sphinx/issues/2150
|
||||
* https://github.com/sphinx-doc/sphinx/issues/1717
|
||||
* https://github.com/sphinx-doc/sphinx/issues/1488
|
||||
* etc.
|
||||
|
||||
They are largely ignored by Sphinx maintainers.
|
||||
|
||||
This projects tries to rectify situation on users' side. It actually
|
||||
changes the way Sphinx processes "only" directive, but does this
|
||||
without forking the project, and instead is made as a standard
|
||||
Sphinx extension, which a user may add to their documentation config.
|
||||
Unlike normal extensions, extensions provided in this package
|
||||
monkey-patch Sphinx core to work in a way expected by users.
|
||||
|
||||
eager_only
|
||||
----------
|
||||
|
||||
The core extension provided by the package is called `eager_only` and
|
||||
is based on the idea by Andrea Cassioli (see bugreports above) to
|
||||
process "only" directive as soon as possible during parsing phase.
|
||||
This approach has some drawbacks, like producing warnings like
|
||||
"WARNING: document isn't included in any toctree" if "only" is used
|
||||
to shape up a toctree, or the fact that changing a documentation
|
||||
builder (html/latex/etc.) will almost certainly require complete
|
||||
rebuild of documentation. But these are relatively minor issues
|
||||
comparing to completely broken way "only" works in upstream Sphinx.
|
||||
|
||||
modindex_exclude
|
||||
----------------
|
||||
|
||||
"only" directive allows for fine-grained conditional exclusion, but
|
||||
sometimes you may want to exclude entire module(s) at once. Even if
|
||||
you wrap an entire module description in "only" directive, like:
|
||||
|
||||
.. only: option1
|
||||
.. module:: my_module
|
||||
|
||||
...
|
||||
|
||||
You will still have an HTML page generated, albeit empty. It may also
|
||||
go into indexes, so will be discoverable by users, leading to less
|
||||
than ideal experience. `modindex_exclude` extension is design to
|
||||
resolve this issue, by making sure that any reference of a module
|
||||
is excluded from Python module index ("modindex"), as well as
|
||||
general cross-reference index ("genindex"). In the latter case,
|
||||
any symbol belong to a module will be excluded. Unlike `eager_only`
|
||||
extension which appear to have issued with "latexpdf" builder,
|
||||
`modindex_exclude` is useful for PDF, and allows to get cleaner
|
||||
index for PDF, just the same as for HTML.
|
||||
|
||||
search_auto_exclude
|
||||
-------------------
|
||||
|
||||
Even if you exclude soem documents from toctree:: using only::
|
||||
directive, they will be indexed for full-text search, so user may
|
||||
find them and get confused. This plugin follows very simple idea
|
||||
that if you didn't include some documents in the toctree, then
|
||||
you didn't want them to be accessible (e.g. for a particular
|
||||
configuration), and so will make sure they aren't indexed either.
|
||||
|
||||
This extension depends on `eager_only` and won't work without it.
|
||||
Note that Sphinx will issue warnings, as usual, for any documents
|
||||
not included in a toctree. This is considered a feature, and gives
|
||||
you a chance to check that document exclusions are indeed right
|
||||
for a particular configuration you build (and not that you forgot
|
||||
to add something to a toctree).
|
||||
|
||||
Summary
|
||||
-------
|
||||
|
||||
Based on the above, sphinx_selective_exclude offers extension to let
|
||||
you:
|
||||
|
||||
* Make "only::" directive work in an expected, intuitive manner, using
|
||||
`eager_only` extension.
|
||||
* However, if you apply only:: to toctree::, excluded documents will
|
||||
still be available via full-text search, so you need to use
|
||||
`search_auto_exclude` for that to work as expected.
|
||||
* Similar to search, indexes may also require special treatment, hence
|
||||
there's the `modindex_exclude` extension.
|
||||
|
||||
Most likely, you will want to use all 3 extensions together - if you
|
||||
really want build subsets of docimentation covering sufficiently different
|
||||
configurations from a single doctree. However, if one of them is enough
|
||||
to cover your usecase, that's OK to (and why they were separated into
|
||||
3 extensions, to follow KISS and "least surprise" principles and to
|
||||
not make people deal with things they aren't interested in). In this case,
|
||||
however remember there're other extensions, if you later hit a usecase
|
||||
when they're needed.
|
||||
|
||||
Usage
|
||||
-----
|
||||
|
||||
To use these extensions, add https://github.com/pfalcon/sphinx_selective_exclude
|
||||
as a git submodule to your project, in documentation folder (where
|
||||
Sphinx conf.py is located). Alternatively, commit sphinx_selective_exclude
|
||||
directory instead of making it a submodule (you will need to pick up
|
||||
any project updates manually then).
|
||||
|
||||
Add following lines to "extensions" settings in your conf.py (you
|
||||
likely already have some standard Sphinx extensions enabled):
|
||||
|
||||
extensions = [
|
||||
...
|
||||
'sphinx_selective_exclude.eager_only',
|
||||
'sphinx_selective_exclude.search_auto_exclude',
|
||||
'sphinx_selective_exclude.modindex_exclude',
|
||||
]
|
||||
|
||||
As discussed above, you may enable all extensions, or one by one.
|
||||
|
||||
Please note that to make sure these extensions work well and avoid producing
|
||||
output docs with artifacts, it is IMPERATIVE to remove cached doctree if
|
||||
you rebuild documentation with another builder (i.e. with different output
|
||||
format). Also, to stay on safe side, it's recommended to remove old doctree
|
||||
anyway before generating production-ready documentation for publishing. To
|
||||
do that, run something like:
|
||||
|
||||
rm -rf _build/doctrees/
|
||||
|
||||
A typical artificat when not following these simple rules is that content
|
||||
of some sections may be missing. If you face anything like that, just
|
||||
remember what's written above and remove cached doctrees.
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
#
|
||||
# This is a Sphinx documentation tool extension which makes .only::
|
||||
# directives be eagerly processed early in the parsing stage. This
|
||||
# makes sure that content in .only:: blocks gets actually excluded
|
||||
# as a typical user expects, instead of bits of information in
|
||||
# these blocks leaking to documentation in various ways (e.g.,
|
||||
# indexes containing entries for functions which are actually in
|
||||
# .only:: blocks and thus excluded from documentation, etc.)
|
||||
# Note that with this extension, you may need to completely
|
||||
# rebuild a doctree when switching builders (i.e. completely
|
||||
# remove _build/doctree dir between generation of HTML vs PDF
|
||||
# documentation).
|
||||
#
|
||||
# This extension works by monkey-patching Sphinx core, so potentially
|
||||
# may not work with untested Sphinx versions. It tested to work with
|
||||
# 1.2.2 and 1.4.2
|
||||
#
|
||||
# Copyright (c) 2016 Paul Sokolovsky
|
||||
# Based on idea by Andrea Cassioli:
|
||||
# https://github.com/sphinx-doc/sphinx/issues/2150#issuecomment-171912290
|
||||
# Licensed under the terms of BSD license, see LICENSE file.
|
||||
#
|
||||
import sphinx
|
||||
from docutils.parsers.rst import directives
|
||||
|
||||
|
||||
class EagerOnly(sphinx.directives.other.Only):
|
||||
|
||||
def run(self, *args):
|
||||
# Evaluate the condition eagerly, and if false return no nodes right away
|
||||
env = self.state.document.settings.env
|
||||
env.app.builder.tags.add('TRUE')
|
||||
#print(repr(self.arguments[0]))
|
||||
if not env.app.builder.tags.eval_condition(self.arguments[0]):
|
||||
return []
|
||||
|
||||
# Otherwise, do the usual processing
|
||||
nodes = super(EagerOnly, self).run()
|
||||
if len(nodes) == 1:
|
||||
nodes[0]['expr'] = 'TRUE'
|
||||
return nodes
|
||||
|
||||
|
||||
def setup(app):
|
||||
directives.register_directive('only', EagerOnly)
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
#
|
||||
# This is a Sphinx documentation tool extension which allows to
|
||||
# exclude some Python modules from the generated indexes. Modules
|
||||
# are excluded both from "modindex" and "genindex" index tables
|
||||
# (in the latter case, all members of a module are exlcuded).
|
||||
# To control exclusion, set "modindex_exclude" variable in Sphinx
|
||||
# conf.py to the list of modules to exclude. Note: these should be
|
||||
# modules (as defined by py:module directive, not just raw filenames).
|
||||
# This extension works by monkey-patching Sphinx core, so potentially
|
||||
# may not work with untested Sphinx versions. It tested to work with
|
||||
# 1.2.2 and 1.4.2
|
||||
#
|
||||
# Copyright (c) 2016 Paul Sokolovsky
|
||||
# Licensed under the terms of BSD license, see LICENSE file.
|
||||
#
|
||||
import sphinx
|
||||
|
||||
|
||||
#org_PythonModuleIndex_generate = None
|
||||
org_PyObject_add_target_and_index = None
|
||||
org_PyModule_run = None
|
||||
|
||||
EXCLUDES = {}
|
||||
|
||||
# No longer used, PyModule_run() monkey-patch does all the job
|
||||
def PythonModuleIndex_generate(self, docnames=None):
|
||||
docnames = []
|
||||
excludes = self.domain.env.config['modindex_exclude']
|
||||
for modname, (docname, synopsis, platforms, deprecated) in self.domain.data['modules'].items():
|
||||
#print(docname)
|
||||
if modname not in excludes:
|
||||
docnames.append(docname)
|
||||
|
||||
return org_PythonModuleIndex_generate(self, docnames)
|
||||
|
||||
|
||||
def PyObject_add_target_and_index(self, name_cls, sig, signode):
|
||||
if hasattr(self.env, "ref_context"):
|
||||
# Sphinx 1.4
|
||||
ref_context = self.env.ref_context
|
||||
else:
|
||||
# Sphinx 1.2
|
||||
ref_context = self.env.temp_data
|
||||
modname = self.options.get(
|
||||
'module', ref_context.get('py:module'))
|
||||
#print("*", modname, name_cls)
|
||||
if modname in self.env.config['modindex_exclude']:
|
||||
return None
|
||||
return org_PyObject_add_target_and_index(self, name_cls, sig, signode)
|
||||
|
||||
|
||||
def PyModule_run(self):
|
||||
env = self.state.document.settings.env
|
||||
modname = self.arguments[0].strip()
|
||||
excl = env.config['modindex_exclude']
|
||||
if modname in excl:
|
||||
self.options['noindex'] = True
|
||||
EXCLUDES.setdefault(modname, []).append(env.docname)
|
||||
return org_PyModule_run(self)
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.add_config_value('modindex_exclude', [], 'html')
|
||||
|
||||
# global org_PythonModuleIndex_generate
|
||||
# org_PythonModuleIndex_generate = sphinx.domains.python.PythonModuleIndex.generate
|
||||
# sphinx.domains.python.PythonModuleIndex.generate = PythonModuleIndex_generate
|
||||
|
||||
global org_PyObject_add_target_and_index
|
||||
org_PyObject_add_target_and_index = sphinx.domains.python.PyObject.add_target_and_index
|
||||
sphinx.domains.python.PyObject.add_target_and_index = PyObject_add_target_and_index
|
||||
|
||||
global org_PyModule_run
|
||||
org_PyModule_run = sphinx.domains.python.PyModule.run
|
||||
sphinx.domains.python.PyModule.run = PyModule_run
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
#
|
||||
# This is a Sphinx documentation tool extension which allows to
|
||||
# automatically exclude from full-text search index document
|
||||
# which are not referenced via toctree::. It's intended to be
|
||||
# used with toctrees conditional on only:: directive, with the
|
||||
# idea being that if you didn't include it in the ToC, you don't
|
||||
# want the docs being findable by search either (for example,
|
||||
# because these docs contain information not pertinent to a
|
||||
# particular product configuration).
|
||||
#
|
||||
# This extension depends on "eager_only" extension and won't work
|
||||
# without it.
|
||||
#
|
||||
# Copyright (c) 2016 Paul Sokolovsky
|
||||
# Licensed under the terms of BSD license, see LICENSE file.
|
||||
#
|
||||
import sphinx
|
||||
|
||||
|
||||
org_StandaloneHTMLBuilder_index_page = None
|
||||
|
||||
|
||||
def StandaloneHTMLBuilder_index_page(self, pagename, doctree, title):
|
||||
if pagename not in self.env.files_to_rebuild:
|
||||
if pagename != self.env.config.master_doc and 'orphan' not in self.env.metadata[pagename]:
|
||||
print("Excluding %s from full-text index because it's not referenced in ToC" % pagename)
|
||||
return
|
||||
return org_StandaloneHTMLBuilder_index_page(self, pagename, doctree, title)
|
||||
|
||||
|
||||
def setup(app):
|
||||
global org_StandaloneHTMLBuilder_index_page
|
||||
org_StandaloneHTMLBuilder_index_page = sphinx.builders.html.StandaloneHTMLBuilder.index_page
|
||||
sphinx.builders.html.StandaloneHTMLBuilder.index_page = StandaloneHTMLBuilder_index_page
|
||||
Loading…
Reference in New Issue