Write Python with the Ease of FORTRAN!

 Posted on July 3, 2018

Write Python with the Ease of FORTRAN!

 Posted on July 3, 2018

Using Forpy to write a FORTRAN program that links in the Python runtime. This is called “embedding” and is the opposite of what is typically done through “extending” with something like f2py.

Have you ever been writing Python code and wished the white-space requirements1 were more strict? Maybe you belong in the camp of developers who need the fastest possible vector and matrix operations you can get on a CPU and don’t want to figure out how to do that in C or whatever the kids are using these days. Perhaps you have decades of highly-tuned, well-tested and stable FORTRAN code that you can’t just up and rewrite in Go, and you want some way to leverage new libraries written in, say, Python. If this sounds like you, and let’s be honest, this does sound like you, then you’ll be amazed at how easy it is to embed Python right inside your FORTRAN code with the help of Forpy.

Seriously speaking, I’d advise against doing what I present in this post. It is best to follow Python’s core developer’s advice and do the following:

  1. start with a high-level scripting language
  2. push performance-critical code to native binaries
  3. optimize native code, using assembly if needed

This makes sense for the vast majority of software, but major projects can not always be accomplished this way for many reasons technical or otherwise.2 So without any further excuses, here’s how to get started with Forpy.

Python and FORTRAN3

A brief search around the internet (circa 2018) reveals a lot of discussion on extending Python with compiled native binary libraries. Most of the approaches to do the reverse, that is, to embed Python into a compiled program revolve around doing so from the C programming language or creating a Python script at runtime and executing it through a system call. The latter method incurs the overhead of bringing up the Python interpreter every time you want to run some Python code, including any imports of Python modules. If the scripts do a substantial amount of work, then this overhead may be ignored, though it does not address moving data to and from the Python script which is typically done through data files on disk.

What if the script’s business logic is so fast and you are making these system calls at such a high rate that you are dominated by the initialization time for each call? This is where Forpy can come in real handy. I’ll assume you have already run the “Hello, World!” example found in Forpy README. I’m going to extend this a bit to make use of something in Python that is generally hard to implement in FORTRAN: the str.format method.

The source for this article can be found here. Here is our first “Hello, Python!” example which is a single file: hello_python.F90. To compile and run it using gfortran, the GNU Fortran compiler, is as simple as:

gfortran -c forpy_mod.F90
gfortran -ohello-python hello_python.F90 forpy_mod.o `python3-config --ldflags`
./hello-python

#define pyerrcheck if(pyerr/=0) then;call err_print;stop;endif

module PyUtilities
  use forpy_mod
  implicit none

  contains

  function format_str(fmt, value) result(message)
    implicit none

    character(len=:), allocatable, intent(in) :: fmt, value
    character(len=:), allocatable :: message

    integer :: pyerr
    type(str) :: py_hello_fmt
    type(object) :: py_ret
    type(tuple) :: py_args

    pyerr = str_create(py_hello_fmt, fmt)
    pyerr = tuple_create(py_args, 1)
    pyerr = py_args%setitem(0, value)
    pyerr = call_py(py_ret, py_hello_fmt, "format", py_args)
    call py_args%destroy
    pyerr = cast(message, py_ret)
    pyerrcheck
  end function format_str
end module PyUtilities

program hello_python
  use forpy_mod, only: forpy_initialize, forpy_finalize, err_print
  use PyUtilities
  implicit none

  integer :: pyerr
  character(len=:), allocatable :: hello_fmt, who, message

  ! starts up the Python interpreter
  pyerr = forpy_initialize()
  pyerrcheck

  hello_fmt = "Hello, {}!"
  who = "Python"

  ! Equivalent Python:
  ! message = hello_fmt.format(who)
  message = format_str(hello_fmt, who)

  print'(a)', message

  ! closes down Python
  call forpy_finalize
end program

You’ll notice the format_str method was placed in a module called PyUtilities. This isolates most of the python related marshalling that has to be done when calling methods using Forpy. The initialize and finalize methods could have been put into format_str, but then the Python interpreter would be brought up every time it’s called. To avoid that, these methods (along with err_print) are brought into the main program. Now the Python interpreter is only brought up once and we can call format_str to our heart’s content.

A Qt5 Application Written in… FORTRAN!?

The last example shown here uses the PyQt5 module (from FORTRAN) to bring up a window that presents our now totally cliché message. But think about what is going on here: A program written in FORTRAN, is interfacing with the Python core library (via Forpy) written in C, which uses the PyQt5 module written in Python, which uses the Qt5 engine written in C++ to present a window to the screen. Notice, there’s no need to link the Qt5 libraries when compiling the FORTRAN program – they are already available in the shared object file found in the PyQt5 Python module. Here is hello_qt.F90:

#define pyerrcheck if(pyerr/=0) then;call err_print;stop;endif

module PyQt
  use forpy_mod
  implicit none

  contains

  function show_window(message) result(pyerr)
    implicit none

    character(len=:), allocatable, intent(in) :: message
    integer :: pyerr

    type(module_py) :: QtWidgets
    type(object) :: app, window, label
    type(list) :: empty_list
    type(tuple) :: args

    ! import PyQt5.QtWidgets as QtWidgets
    pyerr = import_py(QtWidgets, "PyQt5.QtWidgets")
    pyerrcheck

    ! empty_list = []
    ! app = QtWidgets.QApplication([])
    pyerr = list_create(empty_list)
    pyerr = tuple_create(args, 1)
    pyerr = args%setitem(0, empty_list)
    pyerr = call_py(app, QtWidgets, "QApplication", args)
    pyerrcheck
    call args%destroy
    call empty_list%destroy

    ! window = QtWidgets.QMainWindow()
    pyerr = call_py(window, QtWidgets, "QMainWindow")
    pyerrcheck

    ! label = QtWidgets.QLabel(message)
    pyerr = tuple_create(args, 1)
    pyerr = args%setitem(0, message)
    pyerr = call_py(label, QtWidgets, "QLabel", args)
    pyerrcheck
    call args%destroy

    ! window.setCentralWidget(label)
    pyerr = tuple_create(args, 1)
    pyerr = args%setitem(0, label)
    pyerr = call_py_noret(window, "setCentralWidget", args)
    pyerrcheck
    call args%destroy

    ! window.show()
    pyerr = call_py_noret(window, "show")
    pyerrcheck

    ! app.exec_()
    pyerr = call_py_noret(app, "exec_")
  end function show_window
end module PyQt

program hello_qt
  use forpy_mod, only: forpy_initialize, forpy_finalize, err_print
  use PyQt
  implicit none

  integer :: pyerr
  character(len=:), allocatable :: message

  ! starts up the Python interpreter
  pyerr = forpy_initialize()
  pyerrcheck

  message = "Hello, PyQt5!"
  pyerr = show_window(message)
  pyerrcheck

  ! closes down Python
  call forpy_finalize
end program

The window that shows up is less than impressive I’ll admit:

hello_qt_from_fortran

And perhaps this is an absurd use of FORTRAN, but the fact that this chain of technologies works and can be leveraged so easily is, in my opinion, nothing short of astounding! Of course, credit for the ease of doing this goes to the author of Forpy, Elias Rabel who used Balint Aradi’s Python powered Fortran preprocessor (fypp) to generate the FORTRAN module from a template file. To learn more about using Forpy, take a look at the excellent API reference which proved invaluable in the creation of the examples presented above.

Notes from the Author of Forpy

I contacted Elias Rabel, the author of Forpy, and he graciously responded with some history and motivation behind his creation.

The question about my motivation is a really good one and has made me think. I’m interested in many topics in physics and computer science, but choosing a topic to specialize in, is quite hard for me. But there is one topic I think about a lot: programming languages. I started with QBASIC as a kid then Visual Basic, then much later C, C++, Java and Matlab, in general experimenting a lot with different languages. I learned Fortran and Python in 2011, Fortran for a job, Python for fun - going to a local Python user group and meeting people with open source projects surely made an impact, learning about f2py, cython and many other Python projects.

I’d been, like many people, skeptical of Fortran, and during my studies (physics) I never had to touch any Fortran code. Although being a skeptical about Fortran, I took a Fortran job [for a time, even though I realized that] Fortran 90 and later was not used very much and - to my surprise - disliked a lot. [After] some time passed, and [I was able to view] Fortran in a historical context, and the fact that you can write nice, scientific code in modern Fortran (if just more people would use it) I started to think more positively about it.

So my motivation for forpy was to improve the “Fortran experience.” Initially, I did not think about full interoperability with Python, I just wanted to experiment with Python datastructures in Fortran, but why not add a way to import modules and to call objects.

I started to do some experiments (creating a Python list, appending some integers and reading them again) in February 2017. I also looked for a solution for array interoperability early and it worked out. I always wanted to have generic functions for setting and getting items and I knew I planned to use a script to generate Fortran code. I looked for existing tools and I found “fypp”.

The biggest difficulty was to design the API and the countless design decisions one has to make. I challenged myself to do all of Forpy in Fortran and at some points I thought I’d have to add some C helper code, but I could find workarounds that are not too hacky. Also doing such a project makes you think about the benefits of open-sourcing and so on…

– Elias Rabel

References

  1. Yes, yes… FORTRAN 90 does not have the white-space requirements of FORTRAN 77. ↩︎

  2. The power of social momentum is strong. This is true virtually everywhere and software development is no exception. ↩︎

  3. Throughout this article, I assume the use of FORTRAN 90 or later. ↩︎