Produce Python packages for multiple versions of Python 3 (Major change)

The Python packages ('wheels') are produced automatically for a given set of Python 3 versions and stored as artifacts.

• An option BORNAGAIN_PYTHON_PACKAGE is introduced in the build scripts to enable building of BornAgain Python packages ('wheels') for arbitrary versions of Python 3. Currently, Python 3.7–3.9 is supported. The required versions are stored in BORNAGAIN_PYTHON_PACKAGE_VERSIONS.
  The root path of the Python platforms are stored in the cached string variable BORNAGAIN_PYTHON_PLATFORMS_PATH. Different versions of Python are expected to be found in sub-directories named Python37, Python38, etc.

• The required configuration and setup files for building a Python package are stored in Wrap/PythonPackage folder. The wheel configuration follows the latest Python standards, PEP 518. An empty C module is added to enforce Python setuptools to add the proper tags to the wheel file for each platform according to PEP 425. This tag is essential for the PyPI repository and the automatic installation mechanism via pip.

• The SWIG-produced C++ and Python wrappers are modified so that the wrapper be configurable via CMake to produce the shared libraries for multiple versions of Python. For instance, the produced libraries will have the name _libBornAgain*_py38 for Python 3.8.

  • In the C++ wrappers, _libBornAgain* is replaced by @_libBornAgain*_PYTAG@.
  • In the Python wrappers, import _libBornAgain* is replaced by import @_libBornAgain*_PYTAG@ as _libBornAgain*.

• For each version of Python, some build directories are defined in multipython/PyDirectories module to produce the packages. The structure of the output subdirectory for a specific version, say Python 3.8, is:

  python_packages
  |
  +--py38
     |...{Python setup config files}
     +--src
        |
        +--bornagain
           |...{Python init files}
           +--lib
              |...{BornAgain libraries and their Python API}
              +--extra
                 |...{Extra dependencies}
                 +--wrap
                    |...{cpp wrappers to produce the libraries}

• multipython/FindCustomPython3 module (find_custom_python3function) is introduced to find a given Python3 platform in a custom non-standard folder. The resulting variables are suffixed with a given ‘Python tag’; e.g., Python3_FOUND_py37 or Python3_NumPy_FOUND_py37 for Python 3.7. Note that the Python tag is the py_version_nodot defined in PEP 425.

• For each required version of Python, the development platform is found in Dependences module via calls to find_custom_python3. The main version of Python installed on the system is automatically added to the required versions.

• multipython/MakeSharedLib module (make_shared_lib function) is introduced to define a BornAgain shared library for a given Python version with the necessary compile and link flags and dependencies. The 'main' version of the library corresponds to the library which will be installed via CMake install.

  • Linux: The RPATH flag is set for BornAgain shared libraries under Linux so that they could find their dependencies in $ORIGIN/extra directory. This is needed for the self-contained Python packages, so that the required external libraries (like GSL) could be packed within the Python package. In this way, the Python user does not need to care about such dependencies.
  • Windows: Under Windows such a flag does not exist. The libraries are found via searching the directories in PATH environmental variable. In Python >= 3.8, os.add_dll_directory is used to set this path.

• MakeSharedLib uses multipython/MakeSWIGLib module (make_SWIG_lib function) to define a SWIG API for a shared library with a given Python version. Python-related properties of the libraries are set in make_SWIG_lib. make_SWIG_lib is an internal function, called only from the function make_shared_lib.
  An extra internal function _ConfigureSWIG is used to update and modify the SWIG API, whenever needed. The proper changes to the SWIG-produced C++ and Python wrappers are:

  • C++ wrappers: _libBornAgain* => @_libBornAgainBase_PYTAG@
  • Python wrappers import _libBornAgain* => import @_libBornAgain*_PYTAG@ as _libBornAgain*

• multipython/MakePythonWheel module (make_python_wheel function) is introduced to build the Python package (wheel) for a given Python version. For each version of Python, the wheel is built via Python setuptools with all the extra libraries added (like libgsl). The wheel configuration follows the latest Python standards, PEP 518. An empty C module is added to enforce Python setuptools to add the proper tags to the wheel file according to PEP 425.
  make_python_wheel function is called in make_SWIG_lib.

  Current extra libraries are:

  • Boost iostreams
  • GSL + GSLCBLAS
  • FFTW3
  • Cerf
  • TIFF + TIFFXX, only if BORNAGAIN_TIFF_SUPPORT is ON

  NOTE: For the Python wheel we do not necessarily need Boost iostreams and the tiff libraries since Python has already well-established I/O modules for many different data formats.

• multipython/MakeMultiPythonLibs module (make_multipython_libs function) is a higher-level interface to make_shared_lib which simplifies the definition and configuration of BornAgain shared libraries along with their Python API for all required Python versions.
  make_multipython_libs simply calls make_shared_lib several times to build the libraries and the package for a Python version.

• For each BornAgain module, CMakeLists.txt is modified to build the shared libraries and the package for each version of Python.
  Each module is decomposed into Python-dependent and Python-independent parts. The Python-independent parts are compiled only once into object files (a CMake OBJECT library). The Python-dependent parts are compiled for each version of Python due to the incompatibility of the Python headers and libraries across different versions -- ABI incompatibility. This scheme is required for build efficiency, otherwise one has to build the whole library several times. The produced object files are finally used to make the shared library which is then linked with a given version of the Python libraries.

  Currently, Base, Sample and Device modules have Python-dependent parts. Later, the Python-dependent part of these modules should be extracted as a separate module to make the build mechanism easier.

  The CMake script for each module only declares the properties of that module (e.g., defines the sources and include directories). Furthermore, a function is defined which sets the module's specific dependencies (which could be Python-dependent). The declarations and this function are used later when building multiple versions of the library (corresponding to each Python version).

• The GitLab CI scripts for Linux, MacOS and MS-Windows are modified to make the Python wheels. The wheels are stored as artifacts in the build subdirectory python_packages/wheels for each version of Python; e.g., python_packages/wheels/py38.

• The new CMake modules are separated into cmake/BornAgain/multipython folder.

• No changes are made to the source code or BornAgain module structure.

• The standard build procedure for a user who does not wish to build Python packages remains the same as before (as long as BORNAGAIN_PYTHON_PACKAGE is OFF).

Closes #96 (closed)