Align behavior of pre-commit hook and Sphinx extension (#476)

* Extract check selection logic from sphinx side into utility function.

* Add test cases for get_validation_checks().

* Change pre-commit hook behavior from check exclusion to check selection like sphinx side.

* Update example configs for pre-commit hook in docs.

* Add global exclusion option to pre-commit hook.

* Add test cases for global exclusion pre-commit option.

* Update docs for global exclusion option.

* Deprecate override_GL08 since global exclusion covers this.

* Expand override logic for pre-commit hook to any check.

* Update override examples in docs.

* Check for exclusion before running checks.

* Port inline comments for ignoring checks to sphinx side.

* Move note in docs on inline comments to its own section now that both sides support it.

* Remove ignore comment logic from hook now that it is in validate().

* Add note on inline comment usage for --validate.

* Add numpydoc_validation_overrides option to Sphinx extension side.

* Add numpydoc_validation_overrides option to the docs.

* Refine additions to docs.

* Clarify comment.

* Switch to typing for Python 3.8

* Fix check on type.

Author: stefmolin

doc/install.rst

@@ -138,3 +138,17 @@ numpydoc_validation_exclude : set
     validation.
     Only has an effect when docstring validation is activated, i.e.
     ``numpydoc_validation_checks`` is not an empty set.
+numpydoc_validation_overrides : dict
+    A dictionary mapping :ref:`validation checks <validation_checks>` to a
+    container of strings using :py:mod:`re` syntax specifying patterns to
+    ignore for docstring validation.
+    For example, the following skips the ``SS02`` check for docstrings
+    starting with the word ``Process``::
+
+        numpydoc_validation_overrides = {"SS02": [r'^Process ']}
+
+    The default is an empty dictionary meaning no overrides.
+    Only has an effect when docstring validation is activated, i.e.
+    ``numpydoc_validation_checks`` is not an empty set. Use
+    :ref:`inline ignore comments <inline_ignore_comments>` to turn off
+    specific checks for parts of your code.

doc/validation.rst

@@ -2,6 +2,8 @@
 Validation
 ==========
 
+.. _pre_commit_hook:
+
 Docstring Validation using Pre-Commit Hook
 ------------------------------------------
 
@@ -22,44 +24,55 @@ command line options for this hook:
 
     $ python -m numpydoc.hooks.validate_docstrings --help
 
-Using a config file provides additional customization. Both
-``pyproject.toml`` and ``setup.cfg`` are supported; however, if the
-project contains both you must use the ``pyproject.toml`` file.
-The example below configures the pre-commit hook to ignore three checks
-and specifies exceptions to the checks ``SS05`` (allow docstrings to
-start with "Process ", "Assess ", or "Access ") and ``GL08`` (allow
-the class/method/function with name "__init__" to not have a docstring).
+Using a config file provides additional customization. Both ``pyproject.toml``
+and ``setup.cfg`` are supported; however, if the project contains both
+you must use the ``pyproject.toml`` file. The example below configures
+the pre-commit hook as follows:
+
+* ``checks``: Report findings on all checks except ``EX01``, ``SA01``, and
+  ``ES01`` (using the same logic as the :ref:`validation during Sphinx build
+  <validation_during_sphinx_build>` for ``numpydoc_validation_checks``).
+* ``exclude``: Don't report issues on objects matching any of the regular
+  regular expressions ``\.undocumented_method$`` or ``\.__repr__$``. This
+  maps to ``numpydoc_validation_exclude`` from the
+  :ref:`Sphinx build configuration <validation_during_sphinx_build>`.
+* ``override_SS05``: Allow docstrings to start with "Process ", "Assess ",
+  or "Access ". To override different checks, add a field for each code in
+  the form of ``override_<code>`` with a collection of regular expression(s)
+  to search for in the contents of a docstring, not the object name. This
+  maps to ``numpydoc_validation_overrides`` from the
+  :ref:`Sphinx build configuration <validation_during_sphinx_build>`.
 
 ``pyproject.toml``::
 
     [tool.numpydoc_validation]
-    ignore = [
+    checks = [
+        "all",   # report on all checks, except the below
         "EX01",
         "SA01",
         "ES01",
     ]
-    override_SS05 = '^((Process|Assess|Access) )'
-    override_GL08 = '^(__init__)$'
+    exclude = [  # don't report on objects that match any of these regex
+        '\.undocumented_method$',
+        '\.__repr__$',
+    ]
+    override_SS05 = [  # override SS05 to allow docstrings starting with these words
+        '^Process ',
+        '^Assess ',
+        '^Access ',
+    ]
 
 ``setup.cfg``::
 
     [tool:numpydoc_validation]
-    ignore = EX01,SA01,ES01
-    override_SS05 = ^((Process|Assess|Access) )
-    override_GL08 = ^(__init__)$
-
-For more fine-tuned control, you can also include inline comments to tell the
-validation hook to ignore certain checks:
+    checks = all,EX01,SA01,ES01
+    exclude = \.undocumented_method$,\.__repr__$
+    override_SS05 = ^Process ,^Assess ,^Access ,
 
-.. code-block:: python
-
-    class SomeClass:  # numpydoc ignore=EX01,SA01,ES01
-        """This is the docstring for SomeClass."""
+In addition to the above, :ref:`inline ignore comments <inline_ignore_comments>`
+can be used to ignore findings on a case by case basis.
 
-        def __init__(self):  # numpydoc ignore=GL08
-            pass
-
-If any issues are found when commiting, a report is printed out and the
+If any issues are found when commiting, a report is printed out, and the
 commit is halted:
 
 .. code-block:: output
@@ -77,7 +90,9 @@ commit is halted:
     | src/pkg/module.py:33 | module.MyClass.parse | RT03    | Return value has no description      |
     +----------------------+----------------------+---------+--------------------------------------+
 
-See below for a full listing of checks.
+See :ref:`below <validation_checks>` for a full listing of checks.
+
+.. _validation_via_cli:
 
 Docstring Validation using Python
 ---------------------------------
@@ -94,12 +109,16 @@ This will validate that the docstring can be built.
 For an exhaustive validation of the formatting of the docstring, use the
 ``--validate`` parameter. This will report the errors detected, such as
 incorrect capitalization, wrong order of the sections, and many other
-issues.
+issues. Note that this will honor :ref:`inline ignore comments <inline_ignore_comments>`,
+but will not look for any configuration like the :ref:`pre-commit hook <pre_commit_hook>`
+or :ref:`Sphinx extension <validation_during_sphinx_build>` do.
+
+.. _validation_during_sphinx_build:
 
 Docstring Validation during Sphinx Build
 ----------------------------------------
 
-It is also possible to run docstring validation as part of the sphinx build
+It is also possible to run docstring validation as part of the Sphinx build
 process.
 This behavior is controlled by the ``numpydoc_validation_checks`` configuration
 parameter in ``conf.py``.
@@ -109,7 +128,7 @@ following line to ``conf.py``::
 
     numpydoc_validation_checks = {"PR01"}
 
-This will cause a sphinx warning to be raised for any (non-module) docstring
+This will cause a Sphinx warning to be raised for any (non-module) docstring
 that has undocumented parameters in the signature.
 The full set of validation checks can be activated by::
 
@@ -125,6 +144,48 @@ special keyword ``"all"``::
     # Report warnings for all validation checks except GL01, GL02, and GL05
     numpydoc_validation_checks = {"all", "GL01", "GL02", "GL05"}
 
+In addition, you can exclude any findings on certain objects with
+``numpydoc_validation_exclude``, which maps to ``exclude`` in the
+:ref:`pre-commit hook setup <pre_commit_hook>`::
+
+    # don't report on objects that match any of these regex
+    numpydoc_validation_exclude = [
+        '\.undocumented_method$',
+        '\.__repr__$',
+    ]
+
+Overrides based on docstring contents are also supported, but the structure
+is slightly different than the :ref:`pre-commit hook setup <pre_commit_hook>`::
+
+    numpydoc_validation_overrides = {
+        "SS02": [  # override SS05 to allow docstrings starting with these words
+            '^Process ',
+            '^Assess ',
+            '^Access ',
+        ]
+    }
+
+.. _inline_ignore_comments:
+
+Ignoring Validation Checks with Inline Comments
+-----------------------------------------------
+
+Sometimes you only want to ignore a specific check or set of checks for a
+specific piece of code. This level of fine-tuned control is provided via
+inline comments:
+
+.. code-block:: python
+
+    class SomeClass:  # numpydoc ignore=EX01,SA01,ES01
+        """This is the docstring for SomeClass."""
+
+        def __init__(self):  # numpydoc ignore=GL08
+            pass
+
+This is supported by the :ref:`CLI <validation_via_cli>`,
+:ref:`pre-commit hook <pre_commit_hook>`, and
+:ref:`Sphinx extension <validation_during_sphinx_build>`.
+
 .. _validation_checks:
 
 Built-in Validation Checks