Jonathan Dursi recently posted a fairly controversial blog entry entitled "HPC is dying, and MPI is killing it."
Some people immediately dismissed the blog post (and its followups) as trolling. Others praised Jonathan for bringing up the issues.
Brock Palen and I recently chatted with Jonathan about this blog entry on RCE-Cast.
Here's my take on Jonathan's writings:
Truth be told, there's lots of great things about the HPC community, but there's also some not-great things. Things that could be improved. Indeed, whyisn'tthe HPC community drawing as many new people as other communities, such as the Big Data community?
I think that this has a lot to do with the "missing middle" that has been discussed many times - which can be at least partially characterized by the fact that HPC tends to be at the very top end of the computational spectrum (and therefore a somewhat small group). Some view that as a good thing - that HPC should really only be the very bleeding edge of research, and the very largest (national-scale) supercomputers.
I disagree. The rest of the world can benefit from HPC technologies, too - you shouldn't need to be a die-hard HPC expert to enjoy the fruits of its labor.
Case in point: ISVs have done an excellent job of capitalizing on HPC technologies (e.g., MPI). They incorporate parallel technologies that are under the covers. The end user doesn't know or care how the parallelization works; they just know that when they point the software that a cluster of servers, they get their results faster.
So what's my point?
I'm certainlynotsaying that we need to "dumb down" MPI and/or general HPC approaches.
I think, loosely speaking, it is worthwhile to take a good look around MPI and the overall HPC community and ask ourselves: how can we be better?