Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

linecache should remember files it failed to read and not try again #94436

Open
iritkatriel opened this issue Jun 30, 2022 · 5 comments
Open
Labels
3.12 performance Performance or resource usage

Comments

@iritkatriel
Copy link
Member

iritkatriel commented Jun 30, 2022

When linecache doesn't find a source file or fails to read it due to OSError, SyntaxError or UnicodeDecodeError, it returns an empty result. In subsequent attempts it would try to find and read the file again.

We should instead cache the negative lookup result and bypass the search in future queries.

@iritkatriel iritkatriel added the performance Performance or resource usage label Jun 30, 2022
@iritkatriel iritkatriel self-assigned this Jun 30, 2022
@iritkatriel
Copy link
Member Author

iritkatriel commented Jun 30, 2022

CC @gpshead .

@iritkatriel iritkatriel removed their assignment Sep 3, 2022
@corona10
Copy link
Member

corona10 commented Oct 3, 2022

We may need to understand the following decision that cache was removed for this case.
@vstinner, Is this different behavior from what @iritkatriel intended?

41a64a5

@iritkatriel
Copy link
Member Author

iritkatriel commented Oct 3, 2022

I have doubts about this issue as well. It came up in #94410 (comment), where it seemed to make sense, but I've seen other issues where we do/don't want various things cached, and I don't know if there's an obviously correct thing to do here.

@vstinner
Copy link
Member

vstinner commented Oct 3, 2022

We may need to understand the following decision that cache was removed for this case.

Sorry, I don't recall the rationale for this change in Python 3.

updatecache() changes:

  • commit 921c824 (1992, initial commit adding Lib/linecache.py): except IOError: (...) return []
  • commit c341c62 (1992) iterates on sys.path and return [] if the file is not found
  • commit aada7b8 (2010) adds except IOError: pass -- introduce a BUG! lines doesn't exist in this case (NameError)
  • commit 78f43a7 (2010) replaces pass with lines = [] -- fix the NameError
  • commit 41a64a5 (2010) replaces lines = [] with return []

@vstinner
Copy link
Member

vstinner commented Oct 3, 2022

We should instead cache the negative lookup result and bypass the search in future queries.

It sounds reasonable to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.12 performance Performance or resource usage
Projects
None yet
Development

No branches or pull requests

3 participants