Editorial Applications of Distributed Computing Environments
This special issue of Concurrency: Practice and Experience focuses on papers that report on projects, which are providing environments for applications exploiting distributed systems.
Computer systems that could support distributed applications have been in existence for some time. It is only more recently that these systems have been identified as computational resources worthy of serious consideration by applications developers. This realisation can be partially explained by the advances made with commodity processors and network technologies, and partially by the limited availability and high cost of supercomputing CPU cycles. In addition to these factors, the standardisation of tools and libraries has helped make distributed systems a more appealing computational target for applications developers. An example of where standardisation has pushed forward the usage of distributed system is MPI. The MPI library is available for most computer's systems: from the SGI-Cray T3E and IBM SP2 through to PC-based machines running Linux or NT. An implication of the widespread availability of MPI is that a distributed system based on PCs could potentially be used to prototype and test an HPC application that will eventually run on one of the high-end supercomputers. Alternatively, the same PC-based system itself could be used effectively to produce worthwhile results.
A developer of distributed applications still has a relatively immature environment to deal with, even though the software to configure and manage distributed systems is maturing rapidly. The Distributed Computing Environment (DCE) from the Open Software Foundation, for example, has been in existence since 1990.
One major problem encountered by developers is the computational model used by the current generation of distributed environments. This model is based on the client-server paradigm which is not used by most HPC applications and consequently creates additional problems for the developer. A number of other problems exist that need to be overcome, these include:
β’ Allowing applications to span the administrative boundaries of systems.
β’ Efficient utilisation of heterogeneous systems, ranging from PC's to Supercomputers. β’ Effective load balancing, checkpointing and migration of processes.
β’ Minimisation of the impact of communications latencies and maximisation of the available bandwidth. β’ Provision of transparent mechanisms that enable parallel debugging and profiling. HPC applications developers using a distributed system need, therefore, to enrich their run-time environments with additional tools to enable their applications to run successfully. The papers published in this special issue are all concerned with the development and implementation of tools and utilities that enhance commonly used distributed environments so that they can be more effectively and efficiently used by distributed HPC applications.