Overview
MRNet is a software overlay network that provides efficient multicast and reduction communications for parallel and distributed tools and systems. MRNet uses a tree of processes between the tool's front-end and back-ends to improve group communication performance. These internal processes are also used to distribute many important tool activities, reducing data analysis time and keeping tool front-end loads manageable.
MRNet-based tool components communicate across logical channels called streams. At MRNet internal processes, filters are bound to these streams to synchronize and aggregate dataflows. Using filters, MRNet can efficiently compute averages, sums, and other more complex aggregations and analyses on tool data. MRNet also supports facilities that allow tool developers dynamically load new tool-specific filters into the system.
Key Features:
- Flexible Organization: MRNet process tree organization is specified in a configuration file that can specify common network layouts like k-ary and k-nomial trees, or custom layouts tailored to the system(s) running the tool.
- Scalable, Flexible Data Aggregation: MRNet's built-in filters provide efficient computation of averages, sums, concatenation, and other common data reductions. Custom filters can be loaded dynamically into the network to perform tool-specific aggregation operations.
- High-bandwidth Communication: MRNet transfers data within the tool system using an efficient, packed binary representation. Zero-copy data paths are used whenever possible to reduce the cost of transferring data through internal processes.
- Scalable Multicast: MRNet supports efficient message multicast to reduce the cost of issuing control requests from the tool front-end to its back-ends.
- Multiple Concurrent Data Channels: MRNet supports multiple logical streams of data between tool components. Data aggregation and message multicast takes place within the context of a data stream, and multiple operations (both upward and downward) can be active simultaneously.
- Open Source Licensing.
You can download MRNet here.
