Merge pull request #13839 from bluetech/strict

Add `strict` configuration option to enable all other strictness options

Author: bluetech

changelog/13823.feature.rst

@@ -0,0 +1,10 @@
+Added a :confval:`strict` configuration option to enable all strictness-related options.
+
+When set to ``True``, the :confval:`strict` option currently enables :confval:`strict_config`,
+:confval:`strict_markers`, :confval:`strict_xfail`, and :confval:`strict_parametrization_ids`.
+
+The individual strictness options can be explicitly set to override the global :confval:`strict` setting.
+
+If new strictness options are added in the future, they will also be automatically enabled by :confval:`strict`.
+Therefore, we only recommend setting ``strict=True`` if you're using a locked version of pytest,
+or if you want to proactively adopt new strictness options as they are added.

changelog/13823.improvement.rst

@@ -0,0 +1,4 @@
+Added :confval:`strict_xfail` as an alias to the ``xfail_strict`` option,
+:confval:`strict_config` as an alias to the ``--strict-config`` flag,
+and :confval:`strict_markers` as an alias to the ``--strict-markers`` flag.
+This makes all strictness options consistently have configuration options with the prefix ``strict_``.

doc/en/deprecations.rst

@@ -589,18 +589,20 @@ removed in pytest 8 (deprecated since pytest 2.4.0):
 - ``parser.addoption(..., type="int/string/float/complex")`` - use ``type=int`` etc. instead.
 
 
-The ``--strict`` command-line option
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The ``--strict`` command-line option (reintroduced)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. deprecated:: 6.2
-.. versionremoved:: 8.0
+.. versionchanged:: 9.0
 
-The ``--strict`` command-line option has been deprecated in favor of ``--strict-markers``, which
+The ``--strict`` command-line option had been deprecated in favor of ``--strict-markers``, which
 better conveys what the option does.
 
-We have plans to maybe in the future to reintroduce ``--strict`` and make it an encompassing
-flag for all strictness related options (``--strict-markers`` and ``--strict-config``
-at the moment, more might be introduced in the future).
+In version 8.1, we accidentally un-deprecated ``--strict``.
+
+In version 9.0, we changed ``--strict`` to make it set the new :confval:`strict`
+configuration option. It now enables all strictness related options (including
+:confval:`strict_markers`).
 
 
 .. _cmdline-preparse-deprecated:

doc/en/example/markers.rst

@@ -286,8 +286,7 @@ For an example on how to add and work with markers from a plugin, see
 
     * Asking for existing markers via ``pytest --markers`` gives good output
 
-    * Typos in function markers are treated as an error if you use
-      the ``--strict-markers`` option.
+    * Typos in function markers are treated as an error if you use the :confval:`strict_markers` configuration option.
 
 .. _`scoped-marking`:
 

doc/en/how-to/mark.rst

@@ -80,14 +80,14 @@ surprising due to mistyped names. As described in the previous section, you can
 the warning for custom marks by registering them in your ``pytest.ini`` file or
 using a custom ``pytest_configure`` hook.
 
-When the ``--strict-markers`` command-line flag is passed, any unknown marks applied
+When the :confval:`strict_markers` ini option is set, any unknown marks applied
 with the ``@pytest.mark.name_of_the_mark`` decorator will trigger an error. You can
-enforce this validation in your project by adding ``--strict-markers`` to ``addopts``:
+enforce this validation in your project by setting :confval:`strict_markers` in your configuration:
 
 .. code-block:: ini
 
     [pytest]
-    addopts = --strict-markers
+    strict_markers = True
     markers =
         slow: marks tests as slow (deselect with '-m "not slow"')
         serial

doc/en/how-to/skipping.rst

@@ -331,12 +331,12 @@ You can change this by setting the ``strict`` keyword-only parameter to ``True``
 This will make ``XPASS`` ("unexpectedly passing") results from this test to fail the test suite.
 
 You can change the default value of the ``strict`` parameter using the
-``xfail_strict`` ini option:
+``strict_xfail`` ini option:
 
 .. code-block:: ini
 
     [pytest]
-    xfail_strict=true
+    strict_xfail=true
 
 
 Ignoring xfail

doc/en/reference/reference.rst

@@ -261,7 +261,7 @@ pytest.mark.xfail
 
 Marks a test function as *expected to fail*.
 
-.. py:function:: pytest.mark.xfail(condition=False, *, reason=None, raises=None, run=True, strict=xfail_strict)
+.. py:function:: pytest.mark.xfail(condition=False, *, reason=None, raises=None, run=True, strict=strict_xfail)
 
     :keyword Union[bool, str] condition:
         Condition for marking the test function as xfail (``True/False`` or a
@@ -286,7 +286,7 @@ Marks a test function as *expected to fail*.
           that are always failing and there should be a clear indication if they unexpectedly start to pass (for example
           a new release of a library fixes a known bug).
 
-        Defaults to :confval:`xfail_strict`, which is ``False`` by default.
+        Defaults to :confval:`strict_xfail`, which is ``False`` by default.
 
 
 Custom marks
@@ -1774,25 +1774,21 @@ passed multiple times. The expected format is ``name=value``. For example::
 
 .. confval:: markers
 
-    When the ``--strict-markers`` or ``--strict`` command-line arguments are used,
+    When the :confval:`strict_markers` configuration option is set,
     only known markers - defined in code by core pytest or some plugin - are allowed.
 
     You can list additional markers in this setting to add them to the whitelist,
-    in which case you probably want to add ``--strict-markers`` to ``addopts``
+    in which case you probably want to set :confval:`strict_markers` to ``True``
     to avoid future regressions:
 
     .. code-block:: ini
 
         [pytest]
-        addopts = --strict-markers
+        strict_markers = True
         markers =
             slow
             serial
 
-    .. note::
-        The use of ``--strict-markers`` is highly preferred. ``--strict`` was kept for
-        backward compatibility only and may be confusing for others as it only applies to
-        markers and not to other options.
 
 .. confval:: minversion
 
@@ -2070,7 +2066,33 @@ passed multiple times. The expected format is ``name=value``. For example::
     "auto" can be used to explicitly use the global verbosity level.
 
 
-.. confval:: xfail_strict
+.. confval:: strict
+
+    If set to ``True``, enables all strictness options:
+
+    * :confval:`strict_config`
+    * :confval:`strict_markers`
+    * :confval:`strict_xfail`
+    * :confval:`strict_parametrization_ids`
+
+    Plugins may also enable their own strictness options.
+
+    If you explicitly set an individual strictness option, it takes precedence over ``strict``.
+
+    .. note::
+        If new strictness options are added to pytest in the future, they will also be enabled by ``strict``.
+        We therefore only recommend using this option when using a locked version of pytest,
+        or if you want to proactively adopt new strictness options as they are added.
+
+    .. code-block:: ini
+
+        [pytest]
+        strict = True
+
+    .. versionadded:: 9.0
+
+
+.. confval:: strict_xfail
 
     If set to ``True``, tests marked with ``@pytest.mark.xfail`` that actually succeed will by default fail the
     test suite.
@@ -2080,7 +2102,38 @@ passed multiple times. The expected format is ``name=value``. For example::
     .. code-block:: ini
 
         [pytest]
-        xfail_strict = True
+        strict_xfail = True
+
+    You can also enable this option via the :confval:`strict` option.
+
+    .. versionchanged:: 9.0
+        Renamed from ``xfail_strict`` to ``strict_xfail``.
+        ``xfail_strict`` is accepted as an alias for ``strict_xfail``.
+
+
+.. confval:: strict_config
+
+    If set to ``True``, any warnings encountered while parsing the ``pytest`` section of the configuration file will raise errors.
+
+    .. code-block:: ini
+
+        [pytest]
+        strict_config = True
+
+    You can also enable this option via the :confval:`strict` option.
+
+
+.. confval:: strict_markers
+
+    If set to ``True``, markers not registered in the ``markers`` section of the configuration file will raise errors.
+
+    .. code-block:: ini
+
+        [pytest]
+        strict_markers = True
+
+    You can also enable this option via the :confval:`strict` option.
+
 
 .. confval:: strict_parametrization_ids
 
@@ -2094,6 +2147,8 @@ passed multiple times. The expected format is ``name=value``. For example::
           [pytest]
           strict_parametrization_ids = True
 
+    You can also enable this option via the :confval:`strict` option.
+
     For example,
 
     .. code-block:: python

pyproject.toml

@@ -355,7 +355,7 @@ max_supported_python = "3.14"
 
 [tool.pytest.ini_options]
 minversion = "2.0"
-addopts = "-rfEX -p pytester --strict-markers"
+addopts = "-rfEX -p pytester"
 python_files = [
     "test_*.py",
     "*_test.py",
@@ -378,8 +378,7 @@ norecursedirs = [
     "build",
     "dist",
 ]
-xfail_strict = true
-strict_parametrization_ids = true
+strict = true
 filterwarnings = [
     "error",
     "default:Using or importing the ABCs:DeprecationWarning:unittest2.*",

src/_pytest/config/__init__.py

@@ -1515,7 +1515,10 @@ def _validate_plugins(self) -> None:
             )
 
     def _warn_or_fail_if_strict(self, message: str) -> None:
-        if self.known_args_namespace.strict_config:
+        strict_config = self.getini("strict_config")
+        if strict_config is None:
+            strict_config = self.getini("strict")
+        if strict_config:
             raise UsageError(message)
 
         self.issue_config_time_warning(PytestConfigWarning(message), stacklevel=3)

src/_pytest/config/argparsing.py

@@ -547,3 +547,40 @@ def _split_lines(self, text, width):
         for line in text.splitlines():
             lines.extend(textwrap.wrap(line.strip(), width))
         return lines
+
+
+class OverrideIniAction(argparse.Action):
+    """Custom argparse action that makes a CLI flag equivalent to overriding an
+    option, in addition to behaving like `store_true`.
+
+    This can simplify things since code only needs to inspect the ini option
+    and not consider the CLI flag.
+    """
+
+    def __init__(
+        self,
+        option_strings: Sequence[str],
+        dest: str,
+        nargs: int | str | None = None,
+        *args,
+        ini_option: str,
+        ini_value: str,
+        **kwargs,
+    ) -> None:
+        super().__init__(option_strings, dest, 0, *args, **kwargs)
+        self.ini_option = ini_option
+        self.ini_value = ini_value
+
+    def __call__(
+        self,
+        parser: argparse.ArgumentParser,
+        namespace: argparse.Namespace,
+        *args,
+        **kwargs,
+    ) -> None:
+        setattr(namespace, self.dest, True)
+        current_overrides = getattr(namespace, "override_ini", None)
+        if current_overrides is None:
+            current_overrides = []
+        current_overrides.append(f"{self.ini_option}={self.ini_value}")
+        setattr(namespace, "override_ini", current_overrides)