MPI "lite" library?

[aolofsson]

We have been searching for a message passing parallel programming framework for the Epiphany without great results for years. MPI is great because it has a strong community of thousands of expert programmers and a well understood programming model but it's a non-starter for the Epiphany b/c it's just too heavy.

Does anyone know of any efforts currently underway to create a really light-weight open source message passing library for the many manycore architectures currently in development (Epiphany, Intel MIC, Kalray, P2012, Tilera, etc)? A light weight "standard" transport layer could make developing higher level message passing/streaming languages and frameworks much easier. (similar to how OpenCL is pushing heterogeneous computing forward)

Here's the best alternative we have found so far written for the Intel SCC.
http://marcbug.scc-dc.com/svn/repository/trunk/

[dar]

Well, its amusing to communicate this over the forum, but I have looked into this already. In talking with a colleague involved with openmpi, I pointed out that the early OpenCL kernel extensions looked like MPI "lite" - and he then pointed out that something called precisely that had been the subject of discussions within MPI. There is also a package with that name for multithreaded architectures, I beleive. This is mostly an issue of what to throw out of the MPI standard, and then mapping the syntax that is left. The MPI "lite" library you are looking for can be build on top of libcoprthr. I have experimented with the concept using a toy neutron fission code where I created a program that worked semantically like an MPI code being launched on the Epiphany cores. I believe the biggest issue will be partitioning code into "kernel size" pieces.

[ed2k]

maybe even the MPI "lite" is too heavy for 32K memory. why not using a simple c-struct based memory copy model (strip down FIFO, ring buffer etc), leave the heavy lifting to the host (endian conversion, parsing etc)
Erlang's c-port could be the right direction.

[dar]

I cannot speak to Erlang c-port since I do not know enough about it. The term MPI "lite" is used here, I believe, to capture an idea or objective, and not implying a specific level of complexity or support, i.e., it remains to be fully defined and is more notional. What is definitely true is that it should be a memory copy based implementation. And the 32K limit is a stringent one, as you suggest, and will limit what is realistic to achieve.

The motive from my perspective is based on a belief that there is benefit in presenting familiar APIs with potentially different under-the-hood implementations, as opposed to always attempting to construct something new for the programmer to learn, and then rewrite there code from scratch. The motive here is to allow, if possible, MPI programmers to use there code with modifications, to adapt it to this parallel architecture, possibly having to deal with some limitations. Alternative is to rewrite code. Can something meaningful be done here with the concept of MPI "lite"? Not sure, but its interesting enough to take a look, I believe. I am looking at some proof-of-concept code now to test the idea.

[ed2k]

I was taking it literally via google search, http://pcl.cs.ucla.edu/projects/sesame/ ... ull97.html