Skip to content

Defer frame acquisition in sys.monitoring callbacks to reduce object retention#2003

Closed
Copilot wants to merge 4 commits intomainfrom
copilot/debugpy-object-lifetime-issue
Closed

Defer frame acquisition in sys.monitoring callbacks to reduce object retention#2003
Copilot wants to merge 4 commits intomainfrom
copilot/debugpy-object-lifetime-issue

Conversation

Copy link
Contributor

Copilot AI commented Mar 4, 2026

With Python 3.12+ sys.monitoring, pydevd callbacks call sys._getframe(1) to obtain user frames. This creates frame objects that hold references to all locals in that scope, preventing garbage collection of user objects even when the frame reference isn't needed.

Changes

  • _return_event: Move _getframe(1) after the step_cmd == -1 early return — the most common path when not stepping now never materializes a frame object
  • _start_method_event: Pass depth=1 to _get_func_code_info() instead of a frame object, deferring frame creation to cache-miss path; get frame explicitly only after always_skip_code check
  • _unwind_event / _raise_event: Wrap frame+arg usage in try/finally setting both to None, since arg contains exc.__traceback__ which holds the entire frame chain

Before:

def _return_event(code, instruction, retval):
    ...
    frame = _getframe(1)        # frame created unconditionally
    step_cmd = info.pydev_step_cmd
    if step_cmd == -1:
        return                   # frame was unnecessary

After:

def _return_event(code, instruction, retval):
    ...
    step_cmd = info.pydev_step_cmd
    if step_cmd == -1:
        return                   # no frame created
    frame = _getframe(1)         # only when actually needed

Regenerated _pydevd_sys_monitoring_cython.pyx and .c from the updated Python source.

Original prompt

This section details on the original issue you should resolve

<issue_title>debugpy keeps objects alive via the frame object</issue_title>
<issue_description>I am unsure what exactly changes since we didn't use debugpy in around a year. In the meantime we also switched from Python 3.11 to Python 3.13.
Our code base relies on objects being destroyed e.g. so signals are disconnected when debugging our code base.

When attempting to use debugpy again with our recent versions we experience crashes because the objects are kept alive via a frame object and so the signals aren't disconnected. Presumably this is caused by changes like the python 3.12 low impact monitoring.

Is it known what causes this + what would be the best way to avoid this?</issue_description>

<agent_instructions>Please try to reproduce the issue before investigating a fix. The user's example code should cause the problem and should point out where references might be being held. Or alternatively add extra logging to debugpy to print references after different sys monitoring callbacks.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@rchiodo Do you have some code that reproduces the issue? It's likely our eval that's adding a reference to something. 3.12 changed how everything worked. @rchiodo Thanks for the repro. Might be a while before somebody gets to this though.

I was going to try this. It should theoretically show where the references are coming from.

import objgraph
objgraph.show_backrefs(obj, max_depth=5)
```</body></comment_new>
<comment_new><author>@rchiodo</author><body>
If it is eventually dropped, that sounds like a potential normal case? How can you be sure that references will immediately go to zero when you get rid of your reference?

I'd hazard a guess that it's something in the sys.monitoring callbacks that requires another function to be called until it gets rid of all of its references.</body></comment_new>
<comment_new><author>@rchiodo</author><body>
Debuggers try to not impact the debuggee, but that's not always possible - especially with soft mode debuggers like debugpy is. For example, the debugger definitely causes other modules to load into the program. 

I'd say references that go away after a longer period of time is likely an expectation of running a debugger. 

But maybe in this case it's a simple fix. Like we clear out some variables after a callback.</body></comment_new>
</comments>


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 3 commits March 4, 2026 23:56
…retention

In _return_event: move _getframe(1) call after step_cmd == -1 early return
check, avoiding unnecessary frame creation when not stepping.

In _start_method_event: pass depth=1 to _get_func_code_info instead of frame
object, deferring frame creation to after always_skip_code check.

In _unwind_event and _raise_event: wrap frame/arg usage in try/finally to
explicitly clear references after use, preventing exception tracebacks from
keeping frame objects alive.

Regenerated _pydevd_sys_monitoring_cython.pyx from updated Python source.

Co-authored-by: rchiodo <19672699+rchiodo@users.noreply.github.com>
Co-authored-by: rchiodo <19672699+rchiodo@users.noreply.github.com>
…rove test

Changed del frame/del arg to frame = None/arg = None in finally blocks to
avoid potential UnboundLocalError. Improved test to only get frame when needed.

Co-authored-by: rchiodo <19672699+rchiodo@users.noreply.github.com>
Copilot AI changed the title [WIP] Investigate debugpy frame object causing memory retention Defer frame acquisition in sys.monitoring callbacks to reduce object retention Mar 5, 2026
@rchiodo
Copy link
Contributor

rchiodo commented Mar 5, 2026

@copilot were you able to reproduce the issue? I'm not entirely sure that's the root cause of the problem. It could be, but I'd like to reproduce the problem.

Updates to the issue indicate that a Generator is being held during debugging.

Copy link
Contributor Author

Copilot AI commented Mar 5, 2026

@copilot were you able to reproduce the issue? I'm not entirely sure that's the root cause of the problem. It could be, but I'd like to reproduce the problem.

Updates to the issue i...

I was not able to fully reproduce the issue. The reporter's reproduction requires Python 3.13 and a C++ pybind11 extension for the signal callback — we only have Python 3.12 in this environment and no C++ compiler setup for pybind11.

From the issue comments, the reporter identified (via gc.get_referrers()) that a generator frame (signal.py, line 30, code run) is being held alive after the generator is closed. The lifetime is extended until the next function call, which matches the pattern of a monitoring callback's local frame variable keeping the user's frame alive until the next callback invocation overwrites it.

The changes I made are defensive improvements that reduce this:

  • _return_event: In the common path (step_cmd == -1), the frame is never materialized at all — this is the most impactful change since it fires for every function return
  • _start_method_event: Frame acquisition deferred past always_skip_code check
  • _unwind_event/_raise_event: try/finally to clear frame+arg references (arg contains exc.__traceback__ which chains frames)

These changes reduce the window where frame references are held, which should help. However, without Python 3.13 + the C++ extension, I can't confirm this fully resolves the reporter's specific scenario. It may require testing with their exact setup, or there may be additional frame retention paths in CPython 3.13's monitoring implementation.

@rchiodo
Copy link
Contributor

rchiodo commented Mar 5, 2026

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@rchiodo rchiodo marked this pull request as ready for review March 5, 2026 00:30
@rchiodo rchiodo requested a review from a team as a code owner March 5, 2026 00:30
@rchiodo rchiodo closed this Mar 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

debugpy keeps objects alive via the frame object

2 participants