Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-93696: Locate frozen module source with __file__ #93697

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

SnoopJeDi
Copy link
Contributor

@SnoopJeDi SnoopJeDi commented Jun 10, 2022

See #93696, this patch allows pdb to locate the source for frozen stdlib modules. With the patch applied, the output of pdb is as expected:

$ git describe && ./python -V
v3.11.0b1-358-gc91afc1c0a
Python 3.12.0a0
$ ./python -m pdb repro.py 
> /home/snoopjedi/repos/cpython/repro.py(2)<module>()
-> import importlib._bootstrap
(Pdb) n
> /home/snoopjedi/repos/cpython/repro.py(5)<module>()
-> importlib._bootstrap._resolve_name("os", ".", 1)
(Pdb) s
--Call--
> <frozen importlib._bootstrap>(1037)_resolve_name()
(Pdb) l
1032        def __exit__(self, exc_type, exc_value, exc_traceback):
1033            """Release the import lock regardless of any raised exceptions."""
1034            _imp.release_lock()
1035 
1036 
1037 -> def _resolve_name(name, package, level):
1038        """Resolve a relative module name to an absolute one."""
1039        bits = package.rsplit('.', level - 1)
1040        if len(bits) < level:
1041            raise ImportError('attempted relative import beyond top-level package')
1042        base = bits[0]

I am not particularly familiar with the frozen module system so I'm not sure if there are edge cases where this will produce undesirable behavior, but thought that I would send it upstream just in case 😅

@SnoopJeDi
Copy link
Contributor Author

@SnoopJeDi SnoopJeDi commented Jun 10, 2022

Edit: I have now added a test to this PR, see follow-up comment

I did not include an additional test with this PR because I don't think there's a good way to generate a dummy frozen module for use in the test and it seems inappropriate to test against one of the stdlib modules. All the same, here's what the test would look like for the current version of importlib._bootstrap, in case there's some clever testing trick I missed out here.

click to see patch
diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py
index 0141b739c2..894f3e9099 100644
--- a/Lib/test/test_pdb.py
+++ b/Lib/test/test_pdb.py
@@ -532,6 +532,40 @@ def test_list_commands():
     (Pdb) continue
     """
 
+def test_frozen_list():
+    '''Test the list command on frozen stdlib modules
+
+    >>> def test_function():
+    ...     import importlib._bootstrap
+    ...     import pdb; pdb.Pdb(nosigint=True, readrc=False).set_trace()
+    ...     importlib._bootstrap._resolve_name("os", ".", 1)
+
+    >>> with PdbTestInput([  # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
+    ...     'step',
+    ...     'list',
+    ...     'continue',
+    ... ]):
+    ...     test_function()
+    > <doctest test.test_pdb.test_frozen_list[0]>(4)test_function()
+    -> importlib._bootstrap._resolve_name("os", ".", 1)
+    (Pdb) step
+    --Call--
+    > <frozen importlib._bootstrap>(1037)_resolve_name()
+    (Pdb) list
+    1032            def __exit__(self, exc_type, exc_value, exc_traceback):
+    1033                """Release the import lock regardless of any raised exceptions."""
+    1034                _imp.release_lock()
+    1035
+    1036
+    1037 ->     def _resolve_name(name, package, level):
+    1038            """Resolve a relative module name to an absolute one."""
+    1039            bits = package.rsplit('.', level - 1)
+    1040            if len(bits) < level:
+    1041                raise ImportError('attempted relative import beyond top-level package')
+    1042            base = bits[0]
+    (Pdb) continue
+    '''
+
 def test_pdb_whatis_command():
     """Test the whatis command

@SnoopJeDi
Copy link
Contributor Author

@SnoopJeDi SnoopJeDi commented Jun 12, 2022

Updated with a test that creates a module that suitably fakes the behavior of functions inside of frozen modules, in the sense that co_filename on the contained function starts with '<frozen', and the __file__ attribute of the module will point to the "real" source file that pdb should use for resolving list

@kumaraditya303 kumaraditya303 self-requested a review Jul 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants