New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial tensorflow stubs #8974
base: main
Are you sure you want to change the base?
Initial tensorflow stubs #8974
Conversation
This comment has been minimized.
This comment has been minimized.
You're missing a layer of nesting. It should be For the questions you ask, my non-authoritative opinions are:
|
This comment has been minimized.
This comment has been minimized.
Diff from mypy_primer, showing the effect of this PR on open source code: jax (https://github.com/google/jax)
- jax/collect_profile.py:25: error: Cannot find implementation or library stub for module named "tensorflow.python.profiler" [import]
- jax/experimental/jax2tf/tests/model_harness.py:35: error: Cannot find implementation or library stub for module named "tensorflow" [import]
- jax/experimental/jax2tf/examples/tf_js/quickdraw/quickdraw.py:30: error: Cannot find implementation or library stub for module named "tensorflow" [import]
- jax/tools/jax_to_ir.py:86: error: Cannot find implementation or library stub for module named "tensorflow" [import]
- jax/experimental/jax2tf/tests/converters.py:20: error: Cannot find implementation or library stub for module named "tensorflow" [import]
+ jax/experimental/jax2tf/tests/call_tf_test.py:36: error: Incompatible types in assignment (expression has type "None", variable has type Module) [assignment]
streamlit (https://github.com/streamlit/streamlit)
+ lib/tests/streamlit/runtime/caching/hashing_test.py:48: note: ... from here:
+ lib/tests/streamlit/runtime/caching/hashing_test.py:48: note: ... from here:
+ lib/tests/streamlit/runtime/caching/hashing_test.py:48: note: In module imported here:
|
@hauntsaninja, an advantage of including default values is that tools like pyright or pylance will use stubs (if available) for the signatures they show to users when hovering over a function call, and users want to see the default values. I'd like to understand your point of view though as to why you would prefer them removed? |
I took a look at primer hits. The jax hit mypy's redefinition rule and they have code like, try:
import tensorflow
except:
tensorflow = None I'm less sure about what streamlit hit means about note ... here. The code they have is, try:
import tensorflow as tf
except ImportError:
pass The remaining errors look to be expected from lack of numpy in environment. |
@gramster I'm personally okay in theory with default values in stubs if we restrict to simple literals, but it should be its own discussion — my comment here mostly just reflected the status quo, changing of which would need discussion. I also figured we're not really losing that much work on this PR: simple literal default values should be easy to automatically reintroduce to all non-overload stubs and we'd need to do so for all existing stubs anyway. |
Anyway, opened #8988 to discuss default values |
Summary
Initial tensorflow stubs to start with. These are very incomplete given that tensorflow's api is very large. I'm reviving this now as supporting non-types dependencies has made progress and is a requirement for these stubs as they strongly rely on numpy. Similar to opencv stubs this should not merge until non type dependencies work in CI.
These are all manually made based on tensorflow documentation, testing in repl, and usage in large codebase reliant on tensorflow. mypy stubgen last I tried, crashes on tensorflow's codebase and pyright works but produces a large number of false positives. Both mypy and pyright share a weakness in that tensorflow relies on a lot of dynamic module magic with imports. It uses a decorator(tf_export) to re-export class in a different module sometimes even though module may not exist by itself. A secondary issue is most tensorflow classes are defined in internal locations without any private prefixing, and re-exported elsewhere that's documented. And some core tensorflow classes rely on dynamic patching of methods. As an example tf.Tensor class one of the most commonly used types in tensorflow many of it's basic methods like
__add__
and__mul__
are dynamically patched on.I consider this ready for discussion as there are a couple questions I have before moving forward.
getattr(name: str) -> Incomplete
pattern to minimize false positives users would experience. If I do make a pr with full stubs I have currently I'll need to be careful of that as my current local stubs often missed that because at work I followed a philosophy of false positives are better then missing an error and fix stub as needed. A full version of what I have would look roughly like this pr.Default values in stub files should be specified as "..."
The files/classes added here are a couple of the most commonly used ones. If small pr approach is good I'll likely continue by filling in more of
tensorflow/__init__.pyi
andtensorflow/math.pyi
followed by adding in other commonly used files.