Summary
On CodSpeed GitHub Action v4 using mode: walltime on codspeed-macro runners (Linux ARM64), our benchmarks complete successfully (pytest reports all benchmarked tests passed), but then the benchmark process crashes with exit status 139 (SIGSEGV).
This started after upgrading pytest from 9.0.2 to 9.0.3 via a Renovate lockfile update.
Links
Environment / configuration
- Repo: cyndilib/cyndilib
- Workflow:
.github/workflows/codspeed.yml
- Runner:
runs-on: codspeed-macro (appears as Linux ARM64)
- CodSpeed action:
CodSpeedHQ/action@v4
- Mode:
walltime (matrix also runs simulation; issue observed in walltime)
- Python: 3.13 (
actions/setup-python@v6)
- Install:
uv sync --frozen --group test
- Command:
uv run py.test tests/ --codspeed -n=0
- pytest plugin:
pytest-codspeed>=4.0.0 (installed via test dependency group)
Regression window
- Works with: pytest 9.0.2
- Fails with: pytest 9.0.3
Lockfile diff in PR #75 updates pytest 9.0.2 -> 9.0.3 (and other packages, but pytest is the notable change correlated with the new crash).
What happens (log excerpt)
- pytest output ends normally:
======================== 9 passed, 269 deselected ... ========================
- immediately after:
failed to execute the benchmark process: exit status: 139
Request
- Is this a known compatibility issue with pytest 9.0.3 on CodSpeed walltime macro runners (ARM64)?
- Is there a recommended pytest version pin/range for
pytest-codspeed + walltime right now?
- If you want more debugging info: what’s the preferred way to collect a backtrace/core dump or additional diagnostics on macro runners?
Possible relevant pytest 9.0.3 changelog item
pytest 9.0.3 mentions at least one “crash fix” related to ExceptionGroup/tracebackhide handling. Not sure if related, but mentioning in case it helps triage.
Summary
On CodSpeed GitHub Action v4 using
mode: walltimeoncodspeed-macrorunners (Linux ARM64), our benchmarks complete successfully (pytest reports all benchmarked tests passed), but then the benchmark process crashes with exit status 139 (SIGSEGV).This started after upgrading pytest from 9.0.2 to 9.0.3 via a Renovate lockfile update.
Links
Environment / configuration
.github/workflows/codspeed.ymlruns-on: codspeed-macro(appears as Linux ARM64)CodSpeedHQ/action@v4walltime(matrix also runssimulation; issue observed inwalltime)actions/setup-python@v6)uv sync --frozen --group testuv run py.test tests/ --codspeed -n=0pytest-codspeed>=4.0.0(installed via test dependency group)Regression window
Lockfile diff in PR #75 updates pytest 9.0.2 -> 9.0.3 (and other packages, but pytest is the notable change correlated with the new crash).
What happens (log excerpt)
======================== 9 passed, 269 deselected ... ========================failed to execute the benchmark process: exit status: 139Request
pytest-codspeed+ walltime right now?Possible relevant pytest 9.0.3 changelog item
pytest 9.0.3 mentions at least one “crash fix” related to ExceptionGroup/tracebackhide handling. Not sure if related, but mentioning in case it helps triage.