Skip to content

Commit 863ca1a

Browse files
committed
Remove the now crummy support for subprocesses. Users will need to set patch = subprocess or similar in their coverage config to get equivalent behavior. Minium required coverage is set the current patch release to avoid unexpected and untested problems.
Also cleanup some old tests and support code.
1 parent a69d1ab commit 863ca1a

File tree

10 files changed

+166
-668
lines changed

10 files changed

+166
-668
lines changed

docs/subprocess-support.rst

Lines changed: 9 additions & 177 deletions
Original file line numberDiff line numberDiff line change
@@ -2,189 +2,21 @@
22
Subprocess support
33
==================
44

5-
Normally coverage writes the data via a pretty standard atexit handler. However, if the subprocess doesn't exit on its
6-
own then the atexit handler might not run. Why that happens is best left to the adventurous to discover by waddling
7-
through the Python bug tracker.
8-
9-
pytest-cov supports subprocesses, and works around these atexit limitations. However, there are a few pitfalls that need to be explained.
10-
11-
But first, how does pytest-cov's subprocess support works?
12-
13-
pytest-cov packaging injects a pytest-cov.pth into the installation. This file effectively runs this at *every* python startup:
14-
15-
.. code-block:: python
16-
17-
if 'COV_CORE_SOURCE' in os.environ:
18-
try:
19-
from pytest_cov.embed import init
20-
init()
21-
except Exception as exc:
22-
sys.stderr.write(
23-
"pytest-cov: Failed to setup subprocess coverage. "
24-
"Environ: {0!r} "
25-
"Exception: {1!r}\n".format(
26-
dict((k, v) for k, v in os.environ.items() if k.startswith('COV_CORE')),
27-
exc
28-
)
29-
)
30-
31-
The pytest plugin will set this ``COV_CORE_SOURCE`` environment variable thus any subprocess that inherits the environment variables
32-
(the default behavior) will run ``pytest_cov.embed.init`` which in turn sets up coverage according to these variables:
33-
34-
* ``COV_CORE_SOURCE``
35-
* ``COV_CORE_CONFIG``
36-
* ``COV_CORE_DATAFILE``
37-
* ``COV_CORE_BRANCH``
38-
* ``COV_CORE_CONTEXT``
39-
40-
Why does it have the ``COV_CORE`` you wonder? Well, it's mostly historical reasons: long time ago pytest-cov depended on a cov-core package
41-
that implemented common functionality for pytest-cov, nose-cov and nose2-cov. The dependency is gone but the convention is kept. It could
42-
be changed but it would break all projects that manually set these intended-to-be-internal-but-sadly-not-in-reality environment variables.
43-
44-
Coverage's subprocess support
45-
=============================
46-
47-
Now that you understand how pytest-cov works you can easily figure out that using
48-
`coverage's recommended <https://coverage.readthedocs.io/en/latest/subprocess.html>`_ way of dealing with subprocesses,
49-
by either having this in a ``.pth`` file or ``sitecustomize.py`` will break everything:
50-
51-
.. code-block::
52-
53-
import coverage; coverage.process_startup() # this will break pytest-cov
54-
55-
Do not do that as that will restart coverage with the wrong options.
56-
57-
If you use ``multiprocessing``
58-
==============================
59-
60-
Builtin support for multiprocessing was dropped in pytest-cov 4.0.
61-
This support was mostly working but very broken in certain scenarios (see `issue 82408 <https://github.com/python/cpython/issues/82408>`_)
62-
and made the test suite very flaky and slow.
63-
64-
However, there is `builtin multiprocessing support in coverage <https://coverage.readthedocs.io/en/latest/config.html#run-concurrency>`_
65-
and you can migrate to that. All you need is this in your preferred configuration file (example: ``.coveragerc``):
5+
Subprocess support was removed in pytest-cov 7.0 due to various complexities resulting from coverage's own subprocess support.
6+
To migrate you should change your coverage config to have at least this:
667

678
.. code-block:: ini
689
6910
[run]
70-
concurrency = multiprocessing
71-
parallel = true
72-
sigterm = true
73-
74-
Now as a side-note, it's a good idea in general to properly close your Pool by using ``Pool.join()``:
75-
76-
.. code-block:: python
77-
78-
from multiprocessing import Pool
79-
80-
def f(x):
81-
return x*x
82-
83-
if __name__ == '__main__':
84-
p = Pool(5)
85-
try:
86-
print(p.map(f, [1, 2, 3]))
87-
finally:
88-
p.close() # Marks the pool as closed.
89-
p.join() # Waits for workers to exit.
90-
91-
92-
.. _cleanup_on_sigterm:
93-
94-
Signal handlers
95-
===============
96-
97-
pytest-cov provides a signal handling routines, mostly for special situations where you'd have custom signal handling that doesn't
98-
allow atexit to properly run and the now-gone multiprocessing support:
99-
100-
* ``pytest_cov.embed.cleanup_on_sigterm()``
101-
* ``pytest_cov.embed.cleanup_on_signal(signum)`` (e.g.: ``cleanup_on_signal(signal.SIGHUP)``)
102-
103-
If you use multiprocessing
104-
--------------------------
105-
106-
It is not recommanded to use these signal handlers with multiprocessing as registering signal handlers will cause deadlocks in the pool,
107-
see: https://bugs.python.org/issue38227).
108-
109-
If you got custom signal handling
110-
---------------------------------
111-
112-
**pytest-cov 2.6** has a rudimentary ``pytest_cov.embed.cleanup_on_sigterm`` you can use to register a SIGTERM handler
113-
that flushes the coverage data.
114-
115-
**pytest-cov 2.7** adds a ``pytest_cov.embed.cleanup_on_signal`` function and changes the implementation to be more
116-
robust: the handler will call the previous handler (if you had previously registered any), and is re-entrant (will
117-
defer extra signals if delivered while the handler runs).
118-
119-
For example, if you reload on SIGHUP you should have something like this:
120-
121-
.. code-block:: python
122-
123-
import os
124-
import signal
125-
126-
def restart_service(frame, signum):
127-
os.exec( ... ) # or whatever your custom signal would do
128-
signal.signal(signal.SIGHUP, restart_service)
129-
130-
try:
131-
from pytest_cov.embed import cleanup_on_signal
132-
except ImportError:
133-
pass
134-
else:
135-
cleanup_on_signal(signal.SIGHUP)
136-
137-
Note that both ``cleanup_on_signal`` and ``cleanup_on_sigterm`` will run the previous signal handler.
138-
139-
Alternatively you can do this:
140-
141-
.. code-block:: python
142-
143-
import os
144-
import signal
145-
146-
try:
147-
from pytest_cov.embed import cleanup
148-
except ImportError:
149-
cleanup = None
150-
151-
def restart_service(frame, signum):
152-
if cleanup is not None:
153-
cleanup()
154-
155-
os.exec( ... ) # or whatever your custom signal would do
156-
signal.signal(signal.SIGHUP, restart_service)
157-
158-
If you use Windows
159-
------------------
160-
161-
On Windows you can register a handler for SIGTERM but it doesn't actually work. It will work if you
162-
`os.kill(os.getpid(), signal.SIGTERM)` (send SIGTERM to the current process) but for most intents and purposes that's
163-
completely useless.
164-
165-
Consequently this means that if you use multiprocessing you got no choice but to use the close/join pattern as described
166-
above. Using the context manager API or `terminate` won't work as it relies on SIGTERM.
167-
168-
However you can have a working handler for SIGBREAK (with some caveats):
169-
170-
.. code-block:: python
11+
patch = subprocess
17112
172-
import os
173-
import signal
13+
Or if you use pyproject.toml:
17414

175-
def shutdown(frame, signum):
176-
# your app's shutdown or whatever
177-
signal.signal(signal.SIGBREAK, shutdown)
15+
.. code-block:: toml
17816
179-
try:
180-
from pytest_cov.embed import cleanup_on_signal
181-
except ImportError:
182-
pass
183-
else:
184-
cleanup_on_signal(signal.SIGBREAK)
17+
[tool.coverage.run]
18+
patch = ["subprocess"]
18519
186-
The `caveats <https://stefan.sofa-rockers.org/2013/08/15/handling-sub-process-hierarchies-python-linux-os-x/>`_ being
187-
roughly:
20+
Note that if you enable the subprocess patch then ``parallel = true`` is automatically set.
18821

189-
* you need to deliver ``signal.CTRL_BREAK_EVENT``
190-
* it gets delivered to the whole process group, and that can have unforeseen consequences
22+
If it still doesn't produce the same coverage as before you may need to enable more patches, see the `coverage config <https://coverage.readthedocs.io/en/latest/config.html#run-patch>`_ and `subprocess <https://coverage.readthedocs.io/en/latest/subprocess.html>`_ documentation.

setup.py

Lines changed: 1 addition & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -1,80 +1,17 @@
11
#!/usr/bin/env python
22

33
import re
4-
from itertools import chain
54
from pathlib import Path
65

7-
from setuptools import Command
86
from setuptools import find_packages
97
from setuptools import setup
108

11-
try:
12-
# https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html
13-
from setuptools.command.build import build
14-
except ImportError:
15-
from distutils.command.build import build
16-
17-
from setuptools.command.develop import develop
18-
from setuptools.command.easy_install import easy_install
19-
from setuptools.command.install_lib import install_lib
20-
219

2210
def read(*names, **kwargs):
2311
with Path(__file__).parent.joinpath(*names).open(encoding=kwargs.get('encoding', 'utf8')) as fh:
2412
return fh.read()
2513

2614

27-
class BuildWithPTH(build):
28-
def run(self, *args, **kwargs):
29-
super().run(*args, **kwargs)
30-
path = str(Path(__file__).parent / 'src' / 'pytest-cov.pth')
31-
dest = str(Path(self.build_lib) / Path(path).name)
32-
self.copy_file(path, dest)
33-
34-
35-
class EasyInstallWithPTH(easy_install):
36-
def run(self, *args, **kwargs):
37-
super().run(*args, **kwargs)
38-
path = str(Path(__file__).parent / 'src' / 'pytest-cov.pth')
39-
dest = str(Path(self.install_dir) / Path(path).name)
40-
self.copy_file(path, dest)
41-
42-
43-
class InstallLibWithPTH(install_lib):
44-
def run(self, *args, **kwargs):
45-
super().run(*args, **kwargs)
46-
path = str(Path(__file__).parent / 'src' / 'pytest-cov.pth')
47-
dest = str(Path(self.install_dir) / Path(path).name)
48-
self.copy_file(path, dest)
49-
self.outputs = [dest]
50-
51-
def get_outputs(self):
52-
return chain(super().get_outputs(), self.outputs)
53-
54-
55-
class DevelopWithPTH(develop):
56-
def run(self, *args, **kwargs):
57-
super().run(*args, **kwargs)
58-
path = str(Path(__file__).parent / 'src' / 'pytest-cov.pth')
59-
dest = str(Path(self.install_dir) / Path(path).name)
60-
self.copy_file(path, dest)
61-
62-
63-
class GeneratePTH(Command):
64-
user_options = ()
65-
66-
def initialize_options(self):
67-
pass
68-
69-
def finalize_options(self):
70-
pass
71-
72-
def run(self):
73-
with Path(__file__).parent.joinpath('src', 'pytest-cov.pth').open('w') as fh:
74-
with Path(__file__).parent.joinpath('src', 'pytest-cov.embed').open() as sh:
75-
fh.write(f'import os, sys;exec({sh.read().replace(" ", " ")!r})')
76-
77-
7815
setup(
7916
name='pytest-cov',
8017
version='6.3.0',
@@ -125,7 +62,7 @@ def run(self):
12562
python_requires='>=3.9',
12663
install_requires=[
12764
'pytest>=6.2.5',
128-
'coverage[toml]>=7.5',
65+
'coverage[toml]>=7.10.6',
12966
'pluggy>=1.2',
13067
],
13168
extras_require={
@@ -142,11 +79,4 @@ def run(self):
14279
'pytest_cov = pytest_cov.plugin',
14380
],
14481
},
145-
cmdclass={
146-
'build': BuildWithPTH,
147-
'easy_install': EasyInstallWithPTH,
148-
'install_lib': InstallLibWithPTH,
149-
'develop': DevelopWithPTH,
150-
'genpth': GeneratePTH,
151-
},
15282
)

src/pytest-cov.embed

Lines changed: 0 additions & 13 deletions
This file was deleted.

src/pytest-cov.pth

Lines changed: 0 additions & 1 deletion
This file was deleted.

src/pytest_cov/compat.py

Lines changed: 0 additions & 15 deletions
This file was deleted.

0 commit comments

Comments
 (0)