shpcc94 (1158318), страница 3
Текст из файла (страница 3)
FArrays become normalC++ arrays since there is no vector unit attached tothe i860 processor. Nevertheless, the overloaded operators make array element access conceptually easier.The Fortran90 style subroutine has to be changed withvector expressions being converted to do-loops.6 Expirement with a galactic dynamicsapplicationWe use a galactic dynamics application to testour Fortran interface. The application uses the selfconsistent-eld (SCF) method to evolve collisionlessstellar systems [5]. The original code was writtenin Fortran77 by Lars Hernquist in 1991.
In 1993,the code was converted to CMF by Greg Bryan.Expirements conducted on the 512-node CM-5 atthe National Center for Supercomputing Applications(NCSA) by Greg Bryan indicate that with 10 millionparticles the CMF code can achieve 14.4 Gops on 512nodes of the CM-5 (private communication).6.1 Brief summary of the SCF codeThe algorithm used in the SCF code is described indetail in Hernquist and Ostriker 1992 [5]. The codesolves the coupled Vlasov and Poisson equation forcollisionless stellar systems using the N-body approximation approach. Poisson's equation for gravitationalpotential is solved by expanding the density andpotential in a set of basis functions.
The basisset is constructed so that the lowest order memberswell-approximate a galaxy obeying the de VaucouleursR1=4 projected density prole law.The expansions of the density and potential takethe following forms:X Anlmnlm(~r);nlmX(~r) = Anlm nlm (~r);(~r) =nlmwhere n is the radial \quantum" number and l and mare quantum numbers for the angular variables. Generally, the two sums will involve dierent expansioncoecients. But the assumption of bi-orthogonalityensures a one-to-one relationship between terms in theexpansions for the density and potential. The basissets nlm and nlm also satisfy Poisson's equation:r2 nlm (~r) = 4nlm (~r):The basis sets nlm and nlm are given bylpnlm (~r) = K2nl r(1 +rr)2l+3 Cn2l+3=2 () 4Ylm (; );lpnlm (~r) = (1 + rr)2l+1 Cn2l+3=2 () 4Ylm (; );where Knl is a number related only to n and l, andCn2l+3=2() and Ylm (; ) are ultrasperical polynomials and spherical harmonics, respectively.
After somealgebra, the expansion coecients becomeAnlm = I1mk [nlm (rk ; k ; k)] ;nl kXwhere Inl is a number and mk is the mass of the kthparticle. Once the gravitational potential is found, thegravitational force per unit mass can be obtained bytaking the gradient of the potential and the particlescan be accelerated accordingly.6.2 The pC++/Fortran Version of theSCF CodeWe design a C++ class called Segment to representa subgroup of the N particles used in the simulation.As we have discussed earlier, the major procedure inthe SCF code is to compute the sums for the expansioncoecients Anlm . Our approach is to rst computelocal sums within each Segment object.
After this,global sums are formed by a global reduction. Theglobal sums are then broadcast back to each Segmentobject where the particles are accelerated by the gravitational force. Fortran subroutines in the originalFortran code now become member functions of theSegment class, although subroutines involving interelement communication and I/O need to be modied.The Segment class is declared as follows with manyless important variables and member functions omitted:class Segment {public:FArrayDouble x,y,z,vx,vy,vz,ax,ay,az,mass,plm,clm,dlm,elm,flm,dplm;double sinsum[lmax+1][lmax+1][nmax+1],cossum[lmax+1][lmax+1][nmax+1];Segment();void compute_polynomial();void compute_acceleration();void update_position();void update_velocity();};In the above declaration, x, y, z, vx, vy, vz, ax,, ,are one-dimensional arrays that contain the positions, the velocities, the accelerations,and the masses of particles belonging to a Segmentobject, and plm, clm, dlm, elm, flm, dplm containthe expansion coecients and the values of the polynomial.
The sinsum and cossum contain the localsums and eventually the global sums of expansion coecients. They are serial arrays on the CM-5 andare allocated in the Sparc chip's memory. Amongthe class member functions (Fortran subroutines),compute polynomial computes the polynomials andlocal sums, compute acceleration computes the acceleration for each particle, update position andupdate velocity update the positions and velocitiesof particles.The collection that distributes the elements, allocates memory, and manages inter-element communication is declared as below. Again, many less important member functions are omitted for brevity:ay az massCollection SelfConsistField:public Fortran {public:SelfConsistField(Distribution *D,Align *A);void InParticles();void InParameters();void OutParticles(int nsnap);MethodOfElement:virtual void compute_polynomial();virtual void compute_acceleration();virtual void update_position();virtual void update_velocity();void read_segment();void write_segment();};Functions declared here are pC++ functions.Their main functionalities are to handle the I/Oand inter-element communication.
InParticles,InParameters, and OutParticles read the inputles and write to the output les, and read segmentand write segment are called by InParticles andOutParticles to perform parallel I/O on both theCM-5 and Paragon. Functions that are already dened in element class Segment are declared as virtualfunctions in this collection declaration.The main program is listed below:void Processor_Main() {Processors P;Distribution D(elem_count,&P,BLOCK);Align A(elem_count,"[ALIGN(X[i],D[i])]");SelfConsistField<Segment> X(&D,&A);// read initial modelX.InParameters();X.InParticles();X.compute_polynomial();X.ReduceDoubleAdd(offset,variable_count);X.compute_acceleration();// main loopfor(n = 1; n <= nsteps; n++) {X.update_position();X.compute_polynomial();X.ReduceDoubleAdd(offset,variable_count);X.compute_acceleration();X.update_velocity();X.OutParticles(n);}}where ReduceDoubleAdd is a reduction function inherited from SuperKernel.
offset is measured from thebeginning of the class Segment to the beginning of theeld sinsum, and variable count is the total numberof array elements is sinsum and cossum. A leapfrogintegration scheme is used to advance particles.6.3 Timing resultsOur experiments with the pC++/CMF SCF codewas conducted on the 32-node CM-5 at the University of Maryland and the 88-node Paragon at IndianaUniversity. For comparison, we also ran the CMF SCFcode on the CM-5. The results of these experimentsare listed in Table 1.CPU time in seconds spent in main loop without I/O.No.
of pC++/CMFCMFpC++/F77Particleson CM5on CM5on Paragon5125.4 (76.6)6.6 (40.9)5.851208.6 (75.1)9.9 (40.9)25.05120045.8 (89.1) 50.3 (57.4)332.5Total CPU time in seconds with parallel I/O5120049.1 (89.1) 58.2 (57.4)396.3Table 1: Execution time in seconds for 100 time stepson 32-node partitions with the expansions truncated atnmax = 6 and lmax = 4. Numbers in parentheses areamount of memory used in Mbytes. The total time includes reading the input model (2.87 Mbytes), initializing the system, evolving the system for 100 evolutionsteps, and writing two output les (2.87 Mbytes and2.46 Mbytes).As can be seen in Table 1, the pC++/CMF SCFcode is, in fact, slightly faster than the CMF SCF codein all cases.
This can be explained by the fact thatthe pC++/CMF code is able to use a faster globalvector reduction function. The timing results showthat our Fortran interface implementation is very ecient. Given the results shown in Table 1 and experiments conducted on NCSA's CM-5 using larger partition sizes, we conclude that the pC++/CMF SCFcode can achieve at least the 14.4 Gop speed of theCMF SCF code. However, the memory overhead inpC++ is quite large, as the pC++/CMF code requires on an average about 1:8 times of memory toexecute.
The slow speed on the Paragon is due to thefact that no optimization was done by Intel Fortran77compiler. For the Paragon porting, we simply translated Fortran90 vector expressions back to do-loops.If we use Kuck & Associates math library [6], which isoptimized for the i860 processor, we can expect to gainmore speedup. But using the Kuck & Associates mathlibrary would make the Fortran77 code less portableto other machines.7 Summary and conclusionsIn this paper, we have discussed the design of a Fortran interface to the parallel C++ (pC++) programming language. Our prototype Fortran interface whichallows linking pC++ control programs to CM Fortran and Fortran77 subroutines proves to be ecient.Our experiments demonstrate that our portable mixof Fortran and C++ enables programmers to write ef-cient programs, and, at the same time, utilize boththe fast vector and array computation properties ofFortran and the data structure manipulation and encapsulation properties of C++.One of the shortcomings of our current implementation is that Fortran COMMON blocks cannot beaccessed by pC++ control programs.
Communication between Fortran subroutines through COMMONblocks is allowed, though potential race-condition mayarise because pC++ programming model allows morethan one pC++ collection element to be allocatedon one processor. In addition, our current compilercannot generate the wrapper functions required forFortran subroutines.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
