-
-
Notifications
You must be signed in to change notification settings - Fork 445
Description
Describe the bug
Launching a job to retrieve files on 715 host gives this error:
RuntimeError: Set changed size during iteration
[snip].venv/lib/python3.13/site-packages/pyinfra_cli/util.py", line 221, in _parallel_load_hosts
state.pool.spawn(load_file, host): host for host in state.inventory.iter_active_hosts()
To Reproduce
I'm using ssh connector on 715 hosts (not all hosts are reachable) , the inventory is dynamic:
from my_password import password
def make_hosts():
with open("hostlist.csv", "r") as f:
hosts = list(map(lambda x: x.strip().lower(), f.readlines()))
return {
"hosts": (
hosts,
{
"ssh_user": "user",
"ssh_password": password,
"ssh_allow_agent": False,
"ssh_known_hosts_file": "/dev/null",
"_parallel": 10,
"_sudo": True,
},
)
}I've tried with or without _parallel
these are the operations:
from pyinfra import host
from pyinfra.facts import server, files as files_facts
from pyinfra.operations import files
from pathlib import Path
hostname: str = host.get_fact(server.Hostname)
output_path = Path(f"sudoers/{hostname}")
(output_path / "sudoers.d").mkdir(parents=True, exist_ok=True)
files.get(
src="/etc/sudoers",
dest=f"sudoers/{hostname}/sudoers",
)
sudoers_d = host.get_fact(
files_facts.FindFiles,
"/etc/sudoers.d",
quote_path=True,
)
for f in sudoers_d:
files.get(
src=f,
dest=f"sudoers/{hostname}/sudoers.d/{Path(f).name}",
)Expected behavior
I'd like to have the files on my machine 😄
Meta
-
Include output of
pyinfra --support.
System: Linux
Platform: Linux-6.17.7-arch1-1-x86_64-with-glibc2.42
Release: 6.17.7-arch1-1
Machine: x86_64
pyinfra: v3.5.1
click: v8.3.0
distro: v1.9.0
gevent: v25.9.1
jinja2: v3.1.6
packaging: v25.0
paramiko: v3.5.1
python-dateutil: v2.9.0.post0
typeguard: v4.4.4
typing-extensions: v4.15.0
Executable: [snip]/.venv/bin/pyinfra
Python: 3.13.2 (CPython, Clang 19.1.6 ) -
How was pyinfra installed (source/pip)?
uv pip -
Include pyinfra-debug.log (if one was created)
File " [snip]/.venv/lib/python3.13/site-packages/pyinfra_cli/cli.py", line 243, in cli
_main(*args, **kwargs)
~~~~~^^^^^^^^^^^^^^^^^
File " [snip]/.venv/lib/python3.13/site-packages/pyinfra_cli/cli.py", line 378, in _main
can_diff, state, config = _handle_commands(
~~~~~~~~~~~~~~~~^
state, config, command, original_operations, operations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File " [snip]/.venv/lib/python3.13/site-packages/pyinfra_cli/cli.py", line 705, in _handle_commands
state, config, operations = _prepare_deploy_operations(state, config, operations)
~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
File " [snip]/.venv/lib/python3.13/site-packages/pyinfra_cli/cli.py", line 766, in _prepare_deploy_operations
load_deploy_file(state, filename)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File " [snip]/.venv/lib/python3.13/site-packages/pyinfra_cli/util.py", line 235, in load_deploy_file
_parallel_load_hosts(state, lambda: exec_file(filename), filename)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File " [snip]/.venv/lib/python3.13/site-packages/pyinfra_cli/util.py", line 221, in _parallel_load_hosts
state.pool.spawn(load_file, host): host for host in state.inventory.iter_active_hosts()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
RuntimeError: Set changed size during iteration
- Consider including output with
-vvand--debug.
no other relevant informations seems added