H2O Programming

Announcing chemfiles 0.9

Wed, 03 Apr 2019 00:00:00 +0000

We are very happy to announce the release of the 0.9 version of Chemfiles. Chemfiles is a C++ library providing write and read access to chemistry file formats. Chemfiles also has bindings to other languages and can be used from C, Fortran, Python, Julia and Rust.

You can download and install this release from your favorite package manager:

# conda
conda install -c conda-forge chemfiles

# pip
pip install chemfiles

# cargo
[dependencies]
chemfiles = "0.9"

# Julia Pkg
> Pkg.add("Chemfiles")

We also have pre-built binaries for various Linux distributions in OpenSuse build service, and code source is available on Github.

What is chemfiles?

As I discussed in the first public presentation of chemfiles on this blog, there are a lot of different file formats used in computational chemistry. Most simulation and analysis softwares define their own format, but at the same time most formats contains the same kind of information.

This multiplication of formats means that there is less interoperability between different software. To run analysis on the results of a simulation wee have to either use readily available algorithms, or learn how to read the format of our current software. And when we try to use another software (because a specific method is available there, or a different force-field); we need to rewrite most of my analysis scripts. There is a lot of friction to switch between software and just try to use different methods.

Chemfiles is an attempt to provide an unified and simple interface for programmers to work with all these file formats. It does not try to create a new file format to unify all uses cases, instead it provides interoperability between existing formats. In order to do so, chemfiles defines an internal representation for the data that can exist in various formats, and transparently read and write files from this internal format. Which means that you only need to learn how to use chemfiles, and you can use it to read or write every supported file format!

What’s new in version 0.9

We added support for three new formats, bringing the total number of supported formats to 18! The new formats are the MacroMolecule Transmission Format (MMTF), created by the people behind the protein data bank and PDB format. It is a format designed for fast transmission over Internet, compact storage of huge molecules, and fast loading. The two other new formats are the Structure-Data File (SDF) format; and the Cambridge Structure Search and Retrieval (CSSR) format.

Compressed files are now transparently read and written. Supported compression algorithms include lzma (.xz) and gzip (.gz). Reading a compressed file is as easy as:

from chemfiles import Trajectory

file = Trajectory("name.xyz.gz")
frame = file.read()

The atoms selection language have been rewritten and extended to support more complex expressions, such as mathematical constrains:

from chemfiles import Selection

# Get the atoms in a certain region of space
selection = Selection("x^2 - y ^2 < sqrt(z^2 + 25)")
atoms = selection.evaluate(frame)

Other improvements include topology and bonding constrains; as well as constrains on atomic properties:

# Get the atoms bonded to an oxygen atom
selection = Selection("bonded(#1, name O)")

# Get the atoms defined with HETATM in PDB format
selection = Selection("[is_hetatm] and name S")

On a more technical side, the C API functions accessing data such as atoms, unit cells, residues inside a frame or a topology no longer make a copy. This allows to directly read and write inside the containing frame or topology, both from C and from all the languages with a binding for chemfiles. This means that using chemfiles from Python, Julia, Fortran or Rust should now be faster and more ergonomic.

This release would not have been possible without all the contributions by @frodofine, many thanks to him!

You too can contribute to chemfiles! All backgrounds and skills are welcomed and needed, from documentation improvements to C++ code changes. Testing chemfiles and giving your feedback is very valuable too.

Going further

To learn more, you can go have a look at the documentation, go to the Github page of the project, or the the chat room. Happy science!

Announcing chemfiles 0.8

Mon, 26 Mar 2018 00:00:00 +0000

We are very happy to announce the release of the 0.8.0 version of Chemfiles. Chemfiles is a C++ library providing write and read access to chemistry file formats. Chemfiles also has bindings to other languages and can be used from C, Fortran, Python, Julia and Rust.

If you don’t know Chemfiles, you can learn more about why we are writing it, what problem it solves and what are some of its use cases here.

You can download and install this release from your favorite package manager:

# conda
conda install -c conda-forge chemfiles

# pip
pip install chemfiles

# cargo
[dependencies]
chemfiles = "0.8"

# Julia Pkg
> Pkg.add("Chemfiles")

We also have pre-built binaries for various Linux distributions in OpenSuse build service, and code source is available on Github.

What’s new in version 0.8

We added support for four new formats: LAMMPS data files, Tinker .arc files (also called tinker XYZ format), MOL2 files and Molden files. LAMMPS Data files are used to specify starting configuration for the LAMMPS molecular dynamics package, and .arc files are used as input and output by the Tinker familly of molecular dynamics software. Molden is a software for visualizing the output from quantum calculations, and a lot of ab initio software support creating Molden files.

We also switched from Mozilla Public License to BSD license, in order to make it easier to use chemfiles in all projects, commercial and open source.

Chemfiles also gained support for storing and retrieving arbitrary properties on atoms and frames. Properties are values associated with a name and an atom/a frame. For example, the forces acting on an atom or its beta factors can be atomic properties, while the author or the instrument used to generate a file can be set as Frame properties. These properties will help keeping the interface of Chemfiles small and easy to use, by having a single way to specify any additional data that can be in a specific format.

This release also adds support for reading configuration files (by default located in .chemfilesrc). For now, only type renaming is implemented in the configuration files, but new features are coming! Type renaming is useful for some formats where numbers (2, 4, 5) or long strings (Hw, Ca1, Oxc) are used instead of actual atomic types (H, C, O). As Chemfiles uses the atomic types to get the radii and assign bonds, type renaming allow chemfiles to get the right type and perform bonds detections or files conversion.

This release would not have been possible without many contributors, both to the core C++ library and the languages bindings! Many thanks to all of them: @pgbarletta, @lscalfi, @frodofine and @g-bauer.

Going further

To learn more, you can go the website, to the Github page of the project, or the the chat room. See you soon with the chemfiles project!

Announcing chemfiles 0.7

Thu, 06 Apr 2017 00:00:00 +0000

I am very happy to announce the release of the version 0.7 of the free and open-source chemfiles library. Chemfiles is a modern library for reading and writing theoretical chemistry trajectory files.

You can install the new version with conda in the conda-forge channel, using pre-compiled binaries on Linux, or from the code source.

The computation chemistry files mess

A recurrent pain point for anyone working in the theoretical and computational chemistry field is the multiplication of file formats. Every software (would it be GAUSSIAN, VASP, CP2K, LAMMPS, GROMACS, AMBER, or any other one really) comes with its own set of files formats. And it is up to the user to adapt, and learn how to work with the set of formats used by a specific software.

But every format contains basically the same information: a variation around atomic positions, velocities, forces, names and bonding. They still have different use cases: PDB for proteins and bio-molecules, as an easy to edit file; NetCDF or TNG for storing trajectories with thousands of atoms to the disk, XYZ for an easy to read and write format, etc.

But this multiplication of formats means that there is less interoperability between different software. To run analysis on the results of a simulation I have either to use a readily available algorithm, or need to learn how to read the format of my current software. And when I try to use another software (because a specific method is available there, or a different force-field); I need to rewrite most of my analysis scripts. There is a lot of friction to switch between software and just try to use different methods.

Chemfiles is an attempt to provide an unified and simple interface for programmers to work with all these file formats. It does not try to create a new file format to unify all uses cases, it provides interoperability between existing formats. In order to do so, chemfiles defines an internal representation for the data that can exist in various formats (a Frame), and transparently read and write files from this internal format. Which means that you only need to learn how to use a Frame, and you can use it to read or write every supported file format.

Use cases for chemfiles

You can use chemfiles everywhere you need to either read or write theoretical chemistry data, and you would like to support multiple formats. Specifically, I can see three major area where chemfiles can be useful to you:

Analysis of trajectory data. You can use chemfiles to write your code only once, and support multiple file formats without more work. I am working on a set of analysis and data manipulation algorithms in the cfiles project, using chemfiles for reading and writing the files.
Writing simulation software. You are writing your own simulation software, and want to be able to provide multiple format as output format. Or maybe you want to be able to read multiple input formats for the initial configuration.
Molecular viewer programs. If you are working on a molecular viewer program, chemfiles can provide you the ability to read various formats by implementing a single and well-defined interface.

Features

Here is a set of the features chemfiles offer, and what make this library useful:

chemfiles offer interface to multiple languages. It is written in C++, but you can use if from C, C++, Python, Fortran, Julia and Rust;
chemfiles can read both text (XYZ, PDB) and binary file formats (NetCDF, XTC, TNG);
chemfiles support a varying number of atoms along a trajectory – for example for Grand-Canonical Monte-Carlo simulations;
chemfiles support separated files for topology and trajectories. You can give the topology in PDB format, and then read the positions from an XTC file.
chemfiles support periodic boundary conditions for orthorhombic and triclinic unit cells;
chemfiles provides a rich atom selection language. It is up to my knowledge the only software with support for multiple selections, so I’ll talk a bit more about this later.
chemfiles is cross platform. It is continuously tested on GNU/Linux, OS X and Windows;

Concerning the multiple selections, I believe they are an unique feature of chemfiles. Selections are a way to get a list of atoms in the system matching a given constrain: for example "name H or mass < 1.2" or "type Zn and (x > 22.7 or y < 12)" are selections. With multiple selections, you can select multiple atoms at once: "pairs: name(#1) H and name(#2) O" will give you all the O-H pairs in the system. You can get more than two atoms at once, and even work with bonding information: for example "angles: type(#2) C" will give you all the angles (set of 3 atoms linked together) with a carbon in the middle.

What is new in chemfiles 0.7

The 0.7 version of chemfiles adds support for residues. Residues are group of atoms that belong to the same logical unit. They can be amino-acids in proteins, small molecules or something else. This version also add support for the TNG format from the GROMACS project, as well as a bunch of improvements to the interface of the library.

If you want to know more about this release, please have a look at the release notes!

Getting started

We have a complete documentation about how to use chemfiles here. You can read the installation instructions here. We provide pre-compiled binaries for Linux systems, or through the conda package manager for Linux and OS X. If you have any question, or if you need help, we have a chat room using gitter, feel free to ask anything there!

Contributing

The chemfiles project is open to your contributions! You can help in a lot of different ways:

The first and main way to help chemfiles is to use it for your use case, and tell us what is easy and what is not, so that we can improve the code;
The second way to help is to share the word! If you think chemfiles is nice and useful, share it with your colleagues, students, friends, …
you can read the documentation and signal to us parts you don’t understand so that we can improve them; or event better proposing improvements;
By fixing bug in the existing code. We maintain a list of easy issues to get started with the E-easy label on Github. We also offer to mentor you through to fix these issues. If you are interested, leave a message on the issue or on gitter.
Implementing new formats to enlarge the list of supported formats;

Beginners and newbies are welcome at all levels: your insight is very important to improve the documentation; and we offer to help you getting started with the code and the tools we use.

We also have some documentation for contributors here, and do not hesitate to ask questions on gitter!

Now, you can go the website, to the source code, or the the chat room. See you soon with the chemfiles project!

Julia, some criticism

Thu, 10 Sep 2015 00:00:00 +0000

Relations with new technologies are like love: you will go through three phases:

Passion: At first you hear about it, and someday you start spending time with it. And you fall for it: Oh my god, this is so nice! Why did I not used that before?. You tell everyone about it, and want everyone to try it;
Disillusion: Then you start finding that the technology also has bad sides, that world is not a pink warm blanket in which you can hide forever. You see things you do not like, and everything you try to overcome it do not seems to have an effect.
Acceptance: You start knowing what is the fit for this technology, and what is not. You still use it for some stuff, but also rely on other to get things done.

I am in the second part of my relation with the Julia language: after being enthusiastic about it, I am now seeing some of its flaw. I believe some of them can be fixed, and the other are inherently part of the language or the design decisions.

What will not be fixed

These issues will not be fixed, because there are part of the language design. They are issues for me for two reasons: - They make it harder to maintain a big codebase; - They make it harder to use Julia as a general purpose language.

That is totally fine for most of the scientists (who are the main targeted audience), who use Julia for everyday data exploration and small scientific scripts. But these issues arise when trying (as I was doing) to write large simulation software using Julia. Simulation softwares are not throw away scripts, they will hopefully live in the wild for more than 10 years. They must be extendibles, and easy to maintain.

Include-based code organization

There is no one to one correspondence between Julia modules and file organization on the filesystem. One has to explicitly declare a new module module Bar ... end, and use the include function to include code in the current module. This has two consequences:

modules are big;
code organization do not follow file organization.

Modules are big because it is up to the programmer to declare new module. So most of them (and I include myself) will not do it and have a big god-module.

The fact that code organization does not follow file organization is bad because you have to read through all the files to learn what is going on. This add another entry barrier for open source collaborations where newcomers have to spend time figuring what is going on, and make maintenance of old code harder.

This is the same model as C/C++ #include for header files. But this model is bad, and adds a cognitive charge on top of the programmer head. C++ folks want to add a proper module system since the C++11 TS discussions.

Very bad namespacing system

Namespaces are one honking great idea – let’s do more of those!

– The Zen of Python

In Julia, when you do using Package, you bring all exported names from the package inside your current namespace. This is (for all exported symbols) the corresponding operation to Python from package import *. But what happens if we already have another variable with the same name? Let’s see (julia 0.4):

julia> module Foo
         export bar
         bar = 4
       end
Foo

julia> bar = 7
7

julia> using Foo
WARNING: using Foo.bar in module Main conflicts with an existing identifier.

julia> bar
7

Yay, nice, we get a warning here. And the value of our variable is not changed. But what if we accidentally declare a variable with an already existing identifier?

julia> module Foo
         export bar
         bar = 4
       end
Foo

julia> using Foo  # bar == 4

julia> bar = 7
7

julia> bar 7

Here, we silently replaced bar without any warning. What is a desirable feature within a new scope (we do want to be able to define variables with already used names in functions), is not desirable in the global scope. Replace variables with functions in the upper example, and add to this the fact that Julia encourage you to use short variables and function names, and your code starts breaking randomly. Other functions get called in the middle of a script, without having explicitly replaced them (as in Python, when we do monkey-patching).

This is bad for maintenance. You cannot easily refactor your code, or it will start breaking like this. As there is a late binding for variables, you will not see errors until the runtime of your functions. So you better have a lot of unit tests to check your code. But scientists notoriously don’t use enough unit tests.

On the top of that, Base and Core export a lot of symbols and you do not know what is or not in scope.

$ julia -e "whos(Base); whos(Core)" | wc -l
1515

Too much magic

I think there is too much magic in Julia code, in the sense that you do no get full control over what happens at the system level. Coming from the C++ and Fortran side, this can be annoying when it comes to memory management (where do allocations take place? No idea without an external tool); or explicit control over SIMD. What do the @simd macro do? No one know without looking the generated LLVM IR, and we have to hope for the best when using it.

Automagic generation of code is nice for high-level construct, but when you want to squeeze the last bit of performance from the hardware it is not really the best we can do.

What can be fixed

This is a list of smaller flaws, that will be fixed or are being fixed currently.

Enormous Base module. There are too much things in Base. I mean, why is the airy function part of the standard library? This is being worked on;
Installing native dependencies is hard, but this is true for most of the languages. Julia BinDeps package is already a good step forward;
No debugger. You have to use the good ol’ println everywhere, and when you have used lldb once, it is hard to go back. This may change when Galium.jl becomes usable

What is still nice and shiny

For a scientific computing language, Julia is still very nice: (multi-dimmensionals) arrays are first-class citizen; multiple dispatch is a very nice way to implement code reuse; battery included for scientific needs; a nice community. Another very nice feature is the code generation introspection tools, using @code_llvm and @code_native macros.

I will still use Julia, but not for large codebases: maintaining integrity, and forcing myself to respect a code organization it is just too much work that could be done by a compiler.

EDIT: You can also see the comment thread on Reddit.

First impressions on Rust

Thu, 27 Aug 2015 00:00:00 +0000

Rust is a new programming language, created by the mozilla foundation. All programming languages are just tools, and as an hammer can not be use to drill holes, not all programming languages target the same areas of application. Rust’s focus on safety, zero-cost abstractions and parallel computing without data races; and target system programming with C efficiency, and C++ expressiveness.

I’ve been playing with Rust for a month, and here I would like to share some thoughts on Rust design and usability.

Rust model just feels right

There is more than one way to do this.

That could be the moto of programming. In addition to the hundred of existing languages out there, we have multiple paradigms to express our algorithms: from procedural and structured programming to functional programming, including the omniscient object-oriented paradigm. But in all these paradigms, one thing remains constant: we try as much as possible to separate data and data structures from code and behavior.

This is not yet another programmer caprice, but a nice way to get easy to maintain, and easy to adapt code.

And Rust is one of the first language I use which enforce this distinction at language level. In Rust, we have struct to pack data together, and taits to express behavior. And these two concepts are completely orthogonal, i.e. we can implement any trait we want for any structure.

Packing data and adding behavior

Let’s take a little example by defining a 3D vector type: Vect3D. The only field will be data, a Rust native array of float on 64-bits and of dimension 3.

struct Vect3D {
    data: [f64; 3]
}

This feels like C, but we can jump over and add some methods to Vect3D in an ‘impl’ block.

impl Vect3D {
    // 'new' is the conventional name for constructors
    fn new(x: f64, y: f64, z: f64) -> Vect3D {
        Vect3D{
            data: [x, y, z]
        } // No return statement, return the last expression if there is no ';'
    }

    // Scale the vector by the scalar value `s`
    fn scale(&self, s: f64) -> Vect3D {
        let x = self.data[0] * s; // Let introduce a variable, which type is inferred.
        let y = self.data[1] * s;
        let z = self.data[2] * s;
        Vect3D{
            data: [x, y, z] // Create and return a new scaled Vect3D
        }
    }
}

Which is already closer to C++.

A basic usage of our Vect3D type will looks like this:

fn main() {
    let v = Vect3D::new(1.0, 2.0, 3.0);
    // {:?} is the debug format statement
    println!("{:?}", v); // -> Vect3D { data: [1, 2, 3] }
    let u = v.scale(2.0);
    println!("{:?}", u); // -> Vect3D { data: [2, 4, 6] }
}

So the impl block allow us to add some specific behavior to our types, but what if we want to abstract this behavior? How do we specify the way to index an object? How do we abstract a whole class of “objects”, like specifying the behavior of animals? In Rust, trait are the answer this problematic.

Zero-cost abstractions: traits defines behavior

In addition to data, behavior — and especially abstract behavior — provides the building blocks for all programming task. We can find a way to define an abstract behavior and let the users implement it in all languages: we have interfaces in Java, class inheritance with pure abstract classes in C++ or Python, specials methods (like __init__ or __add__) in Python, methods definitions in Julia (call, getindex, …).

Rust uses traits to perform this. A trait in Rust is a group of functions with specific names and signature and some associated data (associated types or constants for example). This trait specify a behavior, and must be implemented for every specific struct. Here, I implement the Index trait (the trait called by the my_vector[3] operation) for the Vect3D type we defined earlier.

use std::ops::Index; // Bring the trait into scope

impl Index<usize> for Vect3D {
    type Output = f64;  // Indexing will return a 64-bit float

    // The index function is the only one that needs to be implemented.
    fn index(&self, index: usize) -> f64 {
        if index < 3 {
            self.data[index]
        } else {
            panic!("Out of bounds indexing!")
        }
    }
}

It is also easy to define our own trait, for example the Animal trait define the behavior of some abstract animal.

trait Animal {
    // Declaring a function which will have to be implemented
    fn noise(&self) -> String;

    // Declaring a function with default implementation
    fn pet(&self) -> String {
        self.noise()
    }
}

We can then implement this animal trait for a Cat and a Dog struct, and play with the implementations.

struct Cat;
struct Dog;

impl Animal for Cat {
    fn noise(&self) -> String {"meow"}
    fn pet(&self) -> String {"purrr"}
}

impl Animal for Dog {
    fn noise(&self) -> String {"bark, bark!"}
}

fn main() {
    let cat = Cat{};
    let dog = Dog{};

    cat.noise(); // -> "meow"
    dog.noise(); // -> "bark, bark!"

    cat.pet(); // -> "purr"
    dog.pet(); // -> "bark, bark!"
}

Rust also have support generic programming, like in C++ templates. Rust genericity is based on trait, and use compile-time dispatch to create all the needed functions. Another interesting point is that generic functions are checked at definition and not at instantiation, removing all the horror of C++ template error messages.

Use traits, they are good for you

I think this data vs behavior separation is a nice way to go, and gives us lot of things at once:

clean interface separation, without the class inheritance nightmare (where does this function come from already?);
easy to maintain code. This is a consequence of the previous point, and of the data/behavior separation;
zero-cost abstractions, because these traits are still plain old function, and the compiler is LLVM-baked.
RAII based memory management, which free us from doing lot of resource management by hand.

Rust also have some other nice points:

safety as a default: everything is immutable by default, and you have to ask for a mutable version is you need it. This is the other way around in C/C++ where you should to over-use the const keyword;
nice modules (not based on the #include trick!) system, where you can write code in the order you want;
all code is private by default;
generic programming is supported in a clever way, based on traits;
All the code is statically checked, and only runtime bugs are still present.

I think that Rust is what C++ is going to look like in 20 years, with a proper module system, and more and more static verification of the code.

Rust is not for beginners …

Rust still have some downsides:

the current lack of some libraries, or the presence of libraries, but only in the unstable channel. This point should improve with time, as more and more people start to use the language;
Rust is very complex, even more than C++.

You have to learn concepts like moving, borrowing, references, lifetime even before the “hello world” example. And then you try to add generic, traits, and parallelism, and you just go mad.

The compiler is not permissive at all, and yell at you every time when you start writing code. At least, it provides useful suggestions, and clear explanations.

For me, Rust is great for big codebase, where parallelism is needed. And it can replace C++ here, being far more easy to maintain and to keep alive. It also remove all bugs from memory error, and even if I do not have years of experience in C++, this kind of bug have already bitten me often!

Calling C++ from C and Fortran

Sun, 15 Mar 2015 00:00:00 +0000

All the code in this article is available as a gist, if you want to use it.

The number of available languages nowadays is very big, and growing every day. As a library writer, I would like to make sure that all the cool code I write today will still be runnable in 10 or 20 years, if it is still useful. Ideally, this very good library (let’s call it libfoo) would be language agnostic, such that any programmer can use it, whatever his favorite language be.

Picking a language to write a library

In real world, I will always need to write some glue code between languages. In order to make it easier to write this glue code, I can use C to write my library. Any existing language does has a C interface, and can call C code in one way or another.

But I don’t really like C. I find that for my particular library, an object-oriented (OO) language will help me, and will allow me to be more efficient. So, I need an OO language with a very straightforward C interface. This interface will then be called by all the bindings. And here comes the C++ !

As usual, when writing a C++ library, we should start by writing down our main interface specification, in an header file. This library will be the Foo library, built around the following Foo.hpp file:

#include <string>

class Foo {
public:
    Foo(int a, int b);
    ~Foo();

    int bar(int c) const;
    double baz(double d) const;
private:
    int a;
    int b;
};

void foo_speaker(std::string s);

This is the most boring library ever, with only one class, two useless function member and a global function. The library implementation can be found in the corresponding Foo.cpp file:

#include "Foo.hpp"
#include <iostream>
using namespace std;


Foo::Foo(int _a, int _b): a(_a), b(_b){
    cout << "C++ side, constructor" << endl;
}

Foo::~Foo(){
    cout << "C++ side, destructor" << endl;
}

int Foo::bar(int c) const{
    return a + c;
}

double Foo::baz(double d) const{
    return d + b;
}

void foo_speaker(string s){
    Foo f(4, 2);
    cout << s << " Foo(4, 2).bar(3) is: " <<  f.bar(3) << endl;
}

Creating the C API for our code

The good thing with C++, is its very straightforward C interface. All we have to do is use an extern "C" statement around the functions we need to wrap. So let’s create a mixed C/C++ header containing the exported C interface to libfoo.

We will wrap all the public functions, together with the constructor and the destructor. The foo.h file contains the C interface, and can be parsed by either a C compiler or a C++ compiler.

#ifdef __cplusplus // Are we compiling this with a C++ compiler ?
extern "C" {
    class Foo;
    typedef Foo FOO;
#else
    // From the C side, we use an opaque pointer.
    typedef struct FOO FOO;
#endif

// Constructor
FOO* create_foo(int a, int b);

// Destructor
void delete_foo(FOO* foo);

// The const qualificators maps from the member function to pointers to the
// class instances.
int foo_bar(const FOO* foo, int c);
double foo_baz(const FOO* foo, double d);

void foo_speaker(const char* s);

#ifdef __cplusplus
}
#endif

This C wrapper will be implemented in C++. All we need is to use a pointer to our Foo class, creating it dynamically with new. So let’s put this code to the foo_capi.cpp file.

#include "foo.h"
#include "Foo.hpp"

#include <iostream>
using namespace std;

FOO* create_foo(int a, int b){
    cout << "C API, create_foo" << endl;
    return new Foo(a, b);
}

void delete_foo(FOO* foo){
    cout << "C API, delete_foo" << endl;
    delete foo;
}

int foo_bar(const FOO* foo, int c){
    return foo->bar(c);
}

double foo_baz(const FOO* foo, double d){
    return foo->baz(d);
}

void foo_speaker(const char* s) {
    foo_speaker(string(s));
}

Wrapping the C API to Fortran

Why fortran ? Why this old and dying language ? You may ask. Well, three reasons here: first, Fortran is still very used in the high performance computing (HPC) world. You know, these guys running calculations for days on thousand of processor. And this is not a metaphor: super-computers frequently have at least 1024 processor per computing node. Second, the last standard is from 2008, and the previous one in 2003 introduced object-oriented programming. And third, I like this language, because it is as simple as C for simple tasks, but a lot more safer and easier to use.

I’ll split the fortran interface in two file, because wrapping C can become very verbose. The first one is foo_cdef.f90, and contains the interface declaration for the external C function. This is the equivalent of the foo.h file.

! C functions declaration
interface
    function create_foo_c(a, b) bind(C, name="create_foo")
        use iso_c_binding
        implicit none
        type(c_ptr) :: create_foo_c
        integer(c_int), value :: a
        integer(c_int), value :: b
    end function

    subroutine delete_foo_c(foo) bind(C, name="delete_foo")
        use iso_c_binding
        implicit none
        type(c_ptr), value :: foo
    end subroutine

    function foo_bar_c(foo, c) bind(C, name="foo_bar")
        use iso_c_binding
        implicit none
        integer(c_int) :: foo_bar_c
        ! The const qualification is translated into an intent(in)
        type(c_ptr), intent(in), value :: foo
        integer(c_int), value :: c
    end function

    function foo_baz_c(foo, c) bind(C, name="foo_baz")
        use iso_c_binding
        implicit none
        real(c_double) :: foo_baz_c
        type(c_ptr), intent(in), value :: foo
        real(c_double), value :: c
    end function

    ! void functions maps to subroutines
    subroutine foo_speaker_c(str) bind(C, name="foo_speaker")
        use iso_c_binding
        implicit none
        character(len=1, kind=C_CHAR), intent(in) :: str(*)
    end subroutine
end interface

What is interesting here ? We use the iso_c_binding module from fortran 2003, and bind the function to their C name. Another point is the value attribute for all the parameter: in Fortran, all the arguments are passed by reference, so we have to precise that we will use the pass-by-value convention.

And that’s all we need to have something functional ! But let’s make this interface feels a bit more like fortran code. For this, we will use the object-oriented (OO) capacities of the Fortran 2003 standard. Their is something funny in wrapping C++ OO code to plain old C to build and OO module in fortran … All this code goes to the foo_mod.f90 file:

module libfoo
    use iso_c_binding

    private
    public :: foo, foo_speaker

    ! Yes, include is a keyword in Fortran !
    include "foo_cdef.f90"

    ! We'll use a Fortan type to represent a C++ class here, in an opaque maner
    type foo
        private
        type(c_ptr) :: ptr ! pointer to the Foo class
    contains
        ! We can bind some functions to this type, allowing for a cleaner syntax.
        final :: delete_foo ! Destructor
        procedure :: delete => delete_foo_polymorph ! Destructor for gfortran
        ! Function member
        procedure :: bar => foo_bar
        procedure :: baz => foo_baz
    end type

    ! This function will act as the constructor for foo type
    interface foo
        procedure create_foo
    end interface

contains ! Implementation of the functions. We just wrap the C function here.
    function create_foo(a, b)
        implicit none
        type(foo) :: create_foo
        integer, intent(in) :: a, b
        create_foo%ptr = create_foo_c(a, b)
    end function

    subroutine delete_foo(this)
        implicit none
        type(foo) :: this
        call delete_foo_c(this%ptr)
    end subroutine

    ! Bounds procedure needs to take a polymorphic (class) argument
    subroutine delete_foo_polymorph(this)
        implicit none
        class(foo) :: this
        call delete_foo_c(this%ptr)
    end subroutine

    integer function foo_bar(this, c)
        implicit none
        class(foo), intent(in) :: this
        integer, intent(in) :: c
        foo_bar = foo_bar_c(this%ptr, c)
    end function

    double precision function foo_baz(this, c)
        implicit none
        class(foo), intent(in) :: this
        double precision, intent(in) :: c
        foo_baz = foo_baz_c(this%ptr, c)
    end function

    subroutine foo_speaker(str)
        implicit none
        character(len=*), intent(in) :: str
        character(len=1, kind=C_CHAR) :: c_str(len_trim(str) + 1)
        integer :: N, i

        ! Converting Fortran string to C string
        N = len_trim(str)
        do i = 1, N
            c_str(i) = str(i:i)
        end do
        c_str(N + 1) = C_NULL_CHAR

        call foo_speaker_c(c_str)
    end subroutine
end module

This library can now be used in a fortran program! As you can see, the library interface is really easy to use, and looks a lot like the C++ interface.

program test
    use libfoo
    implicit none
    type(foo) :: f

    ! Create an object of type foo
    f = foo(3, 4)

    ! Call bound procedures (member functions)
    write(*,*) f%bar(60), " should be ", 63
    write(*,*) f%baz(10d0), " should be ", 14.0d0

    call foo_speaker("From Fortran!")

    ! The destructor should be called automatically here, but this is not yet
    ! implemented in gfortran. So let's do it manually.
#ifdef __GNUC__
    call f%delete
#endif
end program

Theoretically, the destructor should be automatically called when a variable goes out of scope, enabling the RAII idiom in Fortran, but the gfortran implementation (as of 4.9.2) is not complete for this point.

We can now compile the code, using this Makefile:

FC = gfortran
CXX = g++

UNAME := $(shell uname -s)

FCFLAGS = -Wall -Wextra
CCFLAGS = -Wall -Wextra
ifeq ($(UNAME_S),Darwin)
	LDFLAGS = -lstdc++
else
	LDFLAGS = -lc++
endif

all: test.x
test.o : foo_mod.o

%.x : %.o foo_mod.o foo_capi.o Foo.o
	${FC} $^ -o $@ ${LDFLAGS}

%.o : %.f90
	${FC} ${FCFLAGS} -c $< -o $@

%.o : %.cpp
	${CXX} ${CCFLAGS} -c $^ -o $@

.PHONY : clean

clean :
	${RM} -rf *.o *.mod test.x

You should note that as gfortran is the program which will call the linker, we have to explicitly link to the C++ standard library. And now we can run our program!

$ make
$ ./test.x
C API, create_foo
C++ side, constructor
          63  should be           63
   14.000000000000000       should be    14.000000000000000
C++ side, constructor
From Fortran! Foo(4, 2).bar(3) is: 7
C++ side, destructor
C API, delete_foo
C++ side, destructor

So, what is missing for this interface? We should add a couple of try/cacth statements in the CAPI implementation, and replace it by a status code, in order to play nicely with the C++ exceptions. Another amelioration point is for the foo_cdef.f90 and foo_mod.f90 files, which are rather repetitive, boring and error prone to write. It should be possible to use a C parser to generate these files from the foo.h file. This would make another article!

Hello world

Thu, 26 Feb 2015 00:00:00 +0000

So, here I am. Starting a programming blog. I’ve been told many time that this is a good idea, that writing on the net can only have benefits. Let’s see what will append!

Who am I?

I am known on the Interet with the pseudonyme of Luthaf, and I am a french Master student in theoretical chemistry. I am also a scientist who enjoy programming, and likes making good, fun and clean code. My experience is built around two very oposite domains: web programming, and scientific programming.

As an other part of my life, I am also a musician, playing lot of random intruments from around the world, like a tin whistle, a Khene, an hammered dulcimer and a mandoline.

Hello everyone !

Some programming languages I enjoy and I may write about:

#!/usr/bin/env julia

println("Hello from Julia!")

#!/usr/bin/env python3

print("Hello from Python!")

#include <iostream>

int main() {
   std::cout << "Hello from C++!" << std::endl;
   return 0;
}

program hello
   implicit none
   write(*, *) "Hello from Fortran!"
end program

See you soon on this blog !