Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-36635: Add _testinternalcapi module #12841

Merged
merged 1 commit into from Apr 18, 2019

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Apr 15, 2019

Add a new _testinternalcapi module to test the internal C API.

Move _Py_GetConfigsAsDict() function to the internal C API:
_testembed now uses _testinternalcapi to access the function.

https://bugs.python.org/issue36635

@vstinner
Copy link
Member Author

vstinner commented Apr 15, 2019

I chose to name the C file _testinternalcapi.c rather _testinternalcapimodule.c, just because "_testinternalcapimodule.c" is too long :-)

@vstinner
Copy link
Member Author

vstinner commented Apr 15, 2019

I'm not sure about the module name. Maybe "_testinternal" is better? (shorter, contains enough info?)

@vstinner
Copy link
Member Author

vstinner commented Apr 15, 2019

I copied the _testcapi project of PCbuild/ and I tried to adapt it, but it seems like I made a mistake. Importing _testinternalcapi fails with:

ImportError: dynamic module does not define module export function (PyInit__testinternalcapi)

@vstinner
Copy link
Member Author

vstinner commented Apr 15, 2019

I copied the _testcapi project of PCbuild/ and I tried to adapt it, but it seems like I made a mistake. Importing _testinternalcapi fails with: ...

@zooba @zware: I did something wrong for the Windows project. Any idea? Should I use the GUI to create the project instead?

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Apr 15, 2019

Perhaps you need to generate new unique identifiers.

@vstinner
Copy link
Member Author

vstinner commented Apr 15, 2019

Perhaps you need to generate new unique identifiers.

Now done in the 2 project files.

@zooba
Copy link
Member

zooba commented Apr 15, 2019

The GUIDs shouldn't matter outside of Visual Studio, and the only one that has to change is the ProjectGuid (which was updated initially, I believe). The ones in the filters file are referring to known values for the category, so they were fine.

(There is an update missing from Tools/msi/test/test_files.wxs, but that won't be the cause of this issue.)

Based on not seeing the following line in the build logs, I think it's decided not to produce a DLL for some reason:

Creating library C:\projects\cpython\PCbuild\amd64\_testinternalcapi.lib and object C:\projects\cpython\PCbuild\amd64\_testinternalcapi.exp

Possibly the Py_BUILD_CORE_BUILTIN directive is causing one of the dllexport macros to change? I'd have to dig into the headers more closely to figure it out, so perhaps you'll get it first.

@vstinner
Copy link
Member Author

vstinner commented Apr 15, 2019

Hum. The problem should be that PyMODINIT_FUNC is defined differently if Py_BUILD_CORE_BUILTIN is defined:

#define PyMODINIT_FUNC PyObject*

instead of:

#define PyMODINIT_FUNC __declspec(dllexport) PyObject*

The following change works around the issue on Windows:

diff --git a/Modules/_testinternalcapi.c b/Modules/_testinternalcapi.c
index c326d5c97d..13acf377fa 100644
--- a/Modules/_testinternalcapi.c
+++ b/Modules/_testinternalcapi.c
@@ -35,7 +35,7 @@ static struct PyModuleDef _testcapimodule = {
 };


-PyMODINIT_FUNC
+__declspec(dllexport) PyMODINIT_FUNC
 PyInit__testinternalcapi(void)
 {
     PyObject *m;

@zooba
Copy link
Member

zooba commented Apr 15, 2019

Yeah, that sounds like it. The "BUILTIN" part of the macro implies that it's going to be merged into the CPython shared library and loading using inittab rather than dynamically.

Perhaps there's another macro you could use for this module? Or else just hack around it as you showed.

@vstinner
Copy link
Member Author

vstinner commented Apr 16, 2019

Yeah, that sounds like it. The "BUILTIN" part of the macro implies that it's going to be merged into the CPython shared library and loading using inittab rather than dynamically. Perhaps there's another macro you could use for this module? Or else just hack around it as you showed.

There is already a Py_BUILD_CORE_MODULE define which is defined in the Visual Studio project when a C extension is built as a .pyd file (DLL):

PCbuild/pyproject.props:26:    <_PydPreprocessorDefinition Condition="$(TargetExt) == '.pyd'">Py_BUILD_CORE_MODULE;</_PydPreprocessorDefinition>

This define is only used at once place:

PC/pyconfig.h:146:#if defined(Py_BUILD_CORE) || defined(Py_BUILD_CORE_BUILTIN) || defined(Py_BUILD_CORE_MODULE)

I wrote PR #12853 to change PyAPI_FUNC(type), PyAPI_DATA(type) and PyMODINIT_FUNC macros
of pyport.h when Py_BUILD_CORE_MODULE is defined. I should fix this issue.

@vstinner
Copy link
Member Author

vstinner commented Apr 16, 2019

The PR #12853 fix this change: I tested manually, it works.

zooba
zooba approved these changes Apr 17, 2019
Copy link
Member

@zooba zooba left a comment

If the fix for dllexport works here (and it sounds like it will), I'm happy with this change.

Add a new _testinternalcapi module to test the internal C API.

Move _Py_GetConfigsAsDict() function to the internal C API:
_testembed now uses _testinternalcapi to access the function.
@vstinner vstinner force-pushed the WIP_testinternalcapimodule branch from 3d80dd4 to 942a874 Compare Apr 18, 2019
@vstinner
Copy link
Member Author

vstinner commented Apr 18, 2019

The PR #12853 fix this change: I tested manually, it works.

I merged PR #12853. I rebased this PR on top of it, I squashed commits and I made minor changes (I fixed Modules/Setup).

Modules/_testinternalcapi.c now requires that a define is set explicitly:

#if !defined(Py_BUILD_CORE_BUILTIN) && !defined(Py_BUILD_CORE_MODULE)
#  error "Py_BUILD_CORE_BUILTIN or Py_BUILD_CORE_MODULE must be defined"
#endif

@vstinner vstinner merged commit 23bace2 into python:master Apr 18, 2019
@vstinner vstinner deleted the WIP_testinternalcapimodule branch Apr 18, 2019
@vstinner
Copy link
Member Author

vstinner commented Apr 18, 2019

Thanks for the review @zooba, and thanks @serhiy-storchaka and @zware for hints to debug the compilation issue on Windows :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants