USFMPI: A Multi-threaded Implementation of MPI for Linux Clusters

S.G. Caglar, G.D. Benson, Q. Huang, and C.-W. Chu (USA)


: Distributed software systems and applications, Compiler and run-time support, MPI, Threads


USFMPI is an MPI 1.2 compliant implementation for Linux clusters of symmetric multiprocessors and switched networks. USFMPI was designed and implemented from scratch based on the MPI standard. Compared to exist ing open source or vendor supplied implementations, the source code for USFMPI is compact and easy to under stand. Therefore, our implementation is well suited for ex ploring alternate designs and optimizations. Central to the design of USFMPI is the use of a separate communication thread to facilitate the overlap of computation and commu nication and to take advantage of multiple processors. We have implemented USFMPI on top of TCP for FastEthernet and GM for Myrinet. This paper describes the design and implementation of USFMPI and presents performance re sults from experiments executed on Linux cluster. We show that our implementation compares favorably to MPICH and LAM for point-to-point communication and much better for the allgather collective function. In addition, our multi threaded design allows for better overlapping of computa tion and communication when compared to single-threaded implementations.

Important Links:

Go Back