Skip to content

PyCore: Re-implementation of embedded Python as a separate module (Major change)

Ammar Nejati requested to merge newEmbeddedPython into main

This MR introduces a re-implementation of embedded Python as a new module. This is the second step of the major issue described in #223.

The Python-related parts of the codebase are gathered into a new module, PyCore. The new module provides the Python and Numpy C-API to the rest of the codebase, so that the other modules remain strictly Python-free.

Previously, the parts of the codebase which used Python C-API explicitly, were Base/Py, Device/Data/Datafield, and Sample/Multilayer/PyImport.

The ‘stable’ part of the Python C-API is marked explicitly in the embedded Python interpreter (see PyCore/Embed/PythonInterpreter.h). The compatibility across Python 3.7–3.9 is ensured; see https://docs.python.org/3/c-api/stable.html. Note that the Numpy C-API has no official guarantee for stability; see https://numpy.org/doc/stable/reference/c-api and https://numpy.org/doc/stable/reference/c-api/array.html .

Moreover, as the first step, the embedded Python can use the fabio Python package to read different file formats (see issue #156). This is yet to be tested thoroughly.

The class Result is introduced as a container for returned results which may fail (see PyCore/Embed/Result). A Result<T> instance contains a value of type T (or none) and a Status which denotes the status of the value (OK/Warning/Error). This is used to avoid throwing exceptions in a core module (the problems of which are addressed in detail before), leaving the decision to the callers (or the higher interface) whether an exception should be raised or not. The class Result is conceived after Rust's Result type; see https://doc.rust-lang.org/std/result.

Further changes:
  • The interface of MakeLib function is simplified to avoid global variables.
  • SwigLib is cleaned up and improved.
  • Tests for embedded Python are turned off and should be re-implemented.

Related to #223 and #156.

Edited by Ammar Nejati

Merge request reports