PyCore: Re-implementation of embedded Python as a separate module (Major change)
This MR introduces a re-implementation of embedded Python as a new module. This is the second step of the major issue described in #223 (closed).
The Python-related parts of the codebase are gathered into a new module, PyCore
.
The new module provides the Python and Numpy C-API to the rest of the codebase, so that the other modules remain strictly Python-free.
Previously, the parts of the codebase which used Python C-API explicitly, were Base/Py
, Device/Data/Datafield
, and Sample/Multilayer/PyImport
.
The ‘stable’ part of the Python C-API is marked explicitly in the embedded Python interpreter (see PyCore/Embed/PythonInterpreter.h
). The compatibility across Python 3.7–3.9 is ensured; see https://docs.python.org/3/c-api/stable.html. Note that the Numpy C-API has no official guarantee for stability; see https://numpy.org/doc/stable/reference/c-api and https://numpy.org/doc/stable/reference/c-api/array.html .
Moreover, as the first step, the embedded Python can use the fabio Python package to read different file formats (see issue #156). This is yet to be tested thoroughly.
Further changes:
- The interface of
MakeLib
function is simplified to avoid global variables. -
SwigLib
is cleaned up and improved. - Tests for embedded Python are turned off and should be re-implemented.
Related to #223 (closed) and #156.
Merge request reports
Activity
requested review from @j.wuttke
assigned to @a.nejati
added 316 commits
-
a042db02...94d8a3e9 - 259 commits from branch
main
- 4e3c8e15 - PyCore: The header needed for using Python and Numpy C-API
- 1cf631a5 - PyObjectDecl: Forward declaration of PyObject
- bf1c5285 - Result: A container for returned results which may fail
- ee8b5443 - PyObjectPtr: Container for the results from the embedded Python interpreter
- 1e6024d1 - PythonInterpreter: Embedded Python functionality
- 0e5ddf8b - PythonInterpreter: Add Fabio functionality
- 1cf3ecfc - PythonInterpreter: Remove unused functions or variables
- e74df853 - Amend PythonInterpreter for Windows compiler MSVC 19
- 786f8c2f - Add PyTools/CMakeLists.txt
- 8e73fd75 - Add PyTools/Sample/CMakeLists.txt
- d812604d - Add PyTools/Embed/Makefile [DEBUG]
- 7a2d4ebd - Sample/Multilayer/PyImport: Import a MultiLayer instance from a Python code
- c4515e4f - GUI/View/Project/PyImportAssistant: Use PythonInterpreter functionality
- b9d0f388 - Adapt GUI/CMakeLists.txt
- 601f9b7d - Adapt Device/Data/Datafield module
- 4fa7b122 - Rename 'PyTools' to 'PyCore'
- 00e18178 - rm unneeded PyCore/Sample/CMakeLists.txt
- c091c9d7 - CMakeLists.txt<root>: add PyCore module
- 6e4b6d53 - rm old Python-based modules
- 78612f25 - fix PyCore/CMakeLists.txt
- 78a170c6 - PyCore/Embed/PythonInterpreter.cpp: fix the return type of `_init_numpy` function
- b32426b9 - Adapt MakeLib.cmake
- 140937ee - Adapt MakePythonWheel
- 589ca747 - Adapt Device/CMakeLists.txt
- cecd57eb - Adapt Device/Data/Datafield module
- 5a766087 - MakeLib: simplify interface and avoid global variables
- 67369d95 - SwigLib: move compile-flag setting and Windows post-build and installation commands to MakeLib
- c1f27b82 - SwigLib: minor improvements, add comments and TODO
- b40cf63d - Apply clang-format
- df7ee30c - PyCore/Embed/PythonInterpreter: rm unneeded functions and variables
- e0991fe7 - Turn off PyEmbedded tests [DEBUG]
- 584f5151 - Move 'PyCore/Sample/Multilayer/' to 'PyCore/Sample/'
- edda1da8 - Move Status implementation to PyObjectPtr
- fb783379 - rm Result module
- 219ee2cf - PyCore/Sample/ImportMultiLayer: avoid cyclic dependence on MultiLayer module;...
- 06bf782e - PythonInterpreter: rm usages of stringstream and Result class
- 3976648f - PythonInterpreter: add an explanation for "Python stable ABI" + cleanup
- 921f17f1 - Datafield::npArray: rm usage of Result class
- 21f555fa - Datafield::npArray: add assertion to disallow rank() > 2
- ca8eb9e4 - PyImportAssistant: rm usage of Result class
- 791437d9 - PyObjectPtr: Minor improvements
- e713f4b7 - PyObjectPtr: define Status as a class, instead of a struct
- 3d62c932 - PyObjectPtr: rename NpArrayDescr to NumpyArrayDescriptor
- 19b614ad - PythonInterpreter: rename NpArrayDescr to NumpyArrayDescriptor
- 04a8c62f - PyObjectDecl: rm USER_API
- 996f3549 - PyCore: add the standard initial comment blocks to all header files
- 8651f194 - PyCore: use Doxygen style for comments
- 27ca76f8 - PyObjectPtr: make the raw pointer to the Python object a private member
- 8b9e5165 - PyCore/Sample/ImportMultiLayer: fix the check for SWIG macro definition
- cd33d691 - PyCore: fix include statements
- 3ff62454 - PyCore: rm Makefile (used only for debugging)
- 988998e1 - Apply clang-format
- 6bea4228 - PyCore: rm Status class
- 4f463aab - PythonInterpreter: rm unneeded auxiliary function
- 1c44f1d3 - Wrap/Swig/commons.i: Minor improvement to Numpy usage
- 9e97b46b - Update SWIG-produced wrappers for Python API
- 73e7c501 - Apply clang-format
Toggle commit list-
a042db02...94d8a3e9 - 259 commits from branch
added 10 commits
- 983e7c8b - rm unused NumpyArrayDescriptor
- 346365f5 - PythonInterpreter: standardize comments
- e79465a6 - rm duplicate includes
- a289dce0 - shorten name -> PyInterpreter
- ce04f783 - rm unused include
- 906c3341 - get && reset => release
- 94247bc1 - code formatting
- 9ee78637 - define np_size_t where it is used
- 7c62ff14 - ImportMultilayer: add/simplify file header
- e4fc5429 - correct include
Toggle commit listadded 8 commits
Toggle commit listmentioned in commit cf5c84e1