A major problem in parallel computing has been, and still is, moving applications from sequential computers to a parallel computer, and between different parallel architectures. This problem is caused, mainly, by the differences between algorithms written for sequential computers and algorithms written for parallel machines, and that there are only a few languages that can be used for both sequential and parallel computers without extensive modification of the code.
In addition, there is also a great variety of different processor- and network-structures available. And the vendors have been, still are, and probably will continue to be tailoring their languages to their specific architectures. These are major reasons why moving applications from sequential architecture to parallel architectures, and between parallel architectures is so difficult. You may still port programs that do not use the machine-specific features, but such programs are seldom as efficient as the tailored programs. Therefore, developers are often ``forced'' to tailor their programs to the specific machine they are using, if they want an efficient application.
This situation is reminiscent of the situation that existed on mainframes for many years. The problem now is not just hardware and language barriers, as it was on mainframes, but the very algorithms used to implement the applications.
A partial solution to this problem is to define libraries that contain routines which solve many of the most prevalent problems. These libraries define a standard interface to a specific set of problems, and can often be implemented for several specific architectures using the local machine-specific attributes to produce efficient code. Source codes that use such libraries, and which do not use any machine dependent language features, may then be ported to a variety of machines. There exist several widely used libraries for numerical computing, such as BLAS, LAPACK and lately ScaLAPACK.
A very important advantage to such libraries is the fact that when better ways to implement a routine are found, changes may be made in only a few locations, which may then be incorporated into the application, something which can be (or should be) done automatically. Porting applications from sequential computers to parallel computers may still be difficult, even if they do use such libraries, because the algorithms of the application may need adjustments, the interface of the library may change when moving to the parallel environment, as may the storage of data, both of which happen when LAPACK-applications are moved to the ScaLAPACK library.
In recent years, object-oriented languages such as C++ have become more and more popular with developers because of their abilities to ease programming, make the use of structures more intuitive, hide information about the implementation of the system from the programmer, and the authorized-routine-only way of manipulating the data-structures, which makes safe programming easier. These have the potential to make design and debugging a much easier task. Object-oriented languages also make reuse of code and libraries easier, because you can design an often used structure as a class that may be used again and again, and it may also have an interface that is similar to its real world equivalent. In C++ and other languages, it is also possible to implement more versatile classes, called templates, in which the important datatype(s) may easily be changed at compile-time, and which can be used to solve large classes of general problems.
Object-oriented development on parallel systems is currently not widespread, because few compilers for these languages on parallel architectures are available. This project intends to bridge this gap by investigating techniques that may be used to build applications that do not depend on knowledge about the architecture used to run the application. The project is focused on numerical applications using matrix algebra, but the techniques presented could be applicable in other areas of computation as well. While this library, and these techniques, cannot solve the problems with old software, it may help to make new software more transportable.
The main technique investigated in this project uses a sequential computer to execute the object-oriented part of the program, while one (or possibly more) parallel computers supply the computational power needed for the task, in a frontend/backend environment. Other techniques presented involve how to organize the class hierarchy.
The main target of this project was to investigate how to use C++ in numerical programming on parallel computers, in a way that encourages portability. The target architectures were a MasPar MP-2 SIMD machine with 16384 processors and an Intel Paragon XP with 98 processors. As the project progressed, sequential and distributed versions for Sparc and DEC Alpha workstations were also tested.
Figure: The frontend/backend implementation of SymPLA
This document presents the techniques which have been used to
implement a prototype of the Symbolic Parallel
Linear Algebra library (SymPLA), a partial, portable, double
precision matrix class library in C++ with a symbolic interface to
linear algebra. The parallel versions use a sequential frontend,
parallel backend implementation (Figure 
), in which all
matrix data are stored on the parallel computer and the backend is
used to perform the operations requested by the frontend. These
methods may easily be used to implement a full library for parallel
applications, and may also be of use in other areas of
computation. The upper levels of the prototype library do not need any
knowledge about how the actual computations and storage of the matrix
data are accomplished, because this is handled by architecture
specific classes further down in the library.
The library has been designed to make the use of matrices easy, including submatrices, transpose, factorization and solution. In many cases, the syntax of these operations are similar to the syntax used in symbolic math.
The rest of this chapter discusses the background and the intentions of this project. In Chapter 2, some of the present possibilities in parallel computing are briefly presented. In Chapter 3, the overall design and the interface of the matrix class library SymPLA are presented as they were implemented in the prototype. Chapter 4 presents the three implementations used in this project, and sketches how they were implemented. Chapter 5 shows an example routine, presents performance data, and discusses the library's portability. Then, in Chapter 6, some ideas on how to proceed in the further development of the library are discussed.