6

DVM debugger

User’s guide

1. Introduction

The following uniform approach is used to debug all DVM-programs.

First, the program is debugged on a workstation as a sequential program using ordinary debugging methods and tools.

Then the program is executed at the same workstation in the special mode of checking DVM-directives named dynamic control. Dynamic control allows detecting incorrect DVM-directive usage, which is impossible or very difficult to detect at compilation time or at runtime when the program is executed on a parallel computer.

At the next step the program may be executed on one workstation or workstation cluster in the mode of accumulation and comparison of traces obtained during parallel and sequential execution. It allows to localize the program point and moment, when the results are beginning to differ.

Dynamic control, trace accumulation and comparison are performed by DVM-debugger.

1. Types of errors in parallel DVM-program

There are following three groups of errors that can occur in parallel DVM-program:

• Syntax errors in DVM directives (incorrect usage of an operator, missing of brackets and so on). Violation of static semantics belongs to this class also.

• Incorrect order of DVM directives or invalid parameters of DVM directives.

• Wrong computation due to DVM directive incorrectness and errors, not detected when program was executed in sequential mode.

A compiler detects the errors of the first group.

The errors of the second group are detected by Lib-DVM library when the program is executed in parallel mode. Lib-DVM functions check the correctness of DVM-directives order and parameters.

The errors of third group are detected by DVM-debugger when

• DVM-program is executed on a single processor in dynamic control mode.

• DVM-program is executed on a single or several processors in the mode of sequential (reference) and parallel trace comparison.

2.Dynamic control of DVM-directives

Dynamic control mode allows to check correctness of the program parallization by DVM-directives.

The dynamic control is based on simulation of DVM-program parallel execution on a single processor. The information about Lib-DVM function calls and references to variables is gathered and analyzed.

To trace references to variables the program is compiled with inserting DVM debugger function calls. To start compilation and execution of DVM-program the following commands are provided:

dvmcerr < filename of C-DVM program (without extension)>
dvmferr <filename of Fortran-DVM program(without extension)>

It is necessary to take into account, that execution of the program in such a mode requires considerable additional memory resources and processor time. Therefore the program is recommended to debug using test data at first (choosing number and size of arrays, number of iterations or accuracy of calculations) to minimize memory resources and processor time.

3.1. Kinds of detected errors

The dynamic control allows to detect the following kinds of errors:

• Undeclared cross-iteration dependencies of data in a parallel loop. The checking is performed for distributed array elements only. If array element is modified inside one iteration but used in another, the error is detected.

• Using of non-initialized private variables inside or outside of a parallel loop.

• Modification of non-distributed variables not specified as reduction or private.

• Using reduction variables after the asynchronous reduction start but before its completion.

• Undeclared access to non-local elements of the distributed array.

• Writing to shadow edges of the distributed array.

• Reading shadow elements before their update completion.

In the case wrong DVM-directives are found, diagnostics about type of error, line number from the source text and numbers of iterations of all nested loops is output in stderr stream.

3.2. Dynamic control diagnostics.

Writing to read-only variable	Writing to read-only variable is detected.
Using of non-initialized private variable	Access to non-initialized variable or to array element is detected.
Using variable before asynchronous reduction competed	Access to reduction variable before reduction operation completion.
Access to nun-local element	Access to non-local element of distributed array.
Writing to shadow element of array	Writing to shadow element of array.
Shadow element was not updated	Access to shadow elements before completion of shadow renew operation.
Data dependence in loop	Data dependence in parallel loop is detected.
WAIT for shadow renew without START	Operation of waiting for shadow renewal is issued without start of shadow renew operation.
Double WAIT for shadow renew	Repeated operation of waiting for shadow renewal.
Write to exported element before shadow renewal	Updating exported element of array before completion of shadow renewal.
Writing to remote data buffer	Writing to remote data buffer.
Write to remote element in sequential branch	Access to array element in sequential branch of program without preliminary testing that the element is located on current processor.
WAIT for reduction without START	Waiting for asynchronous reduction completion is issued without Start asynchonous reduction.
Using an element outside of array	Access to element of array beyond its limits.

3.Trace comparison.

The absence of the errors of dynamic control does not guarantee correct parallel program execution due to the following reasons:

• Correctness of reduction operation descriptions is not checked.

• Errors can occur in procedures called from DVM-programs, which are written in other languages and not controlled.

• The tested sequential program can contain errors not appeared during sequential execution, but these errors could be detected during parallel execution.

To find such errors the method of comparison of sequential execution trace and parallel execution trace is provided. The traces are obtained in the following modes:

• Obtaining reference trace file as a result of sequential execution of the program and comparing it with the trace of parallel execution of the program on a single processor. At this step the correctness of reduction operation descriptions is checked. The following commands are provided for compilation and execution of DVM-program in this mode:

dvmcred <filename>
dvmfred <filename>

If differences are detected in compared trace results diagnostics is output to stderr stream.

• Start up parallel program in a mode of emulation of multiprocessor system on a workstation and compare the execution trace with the reference one. The following commands are provided for compilation and execution of DVM-program in this mode:

dvmcdif N1 N2 N3 <filename>
dvmfdif N1 N2 N3 <filename>
Where N1,N2,N3 – sizes of processor matrix (it is possible to specify <N1 N2> or N1 only).

If differences are detected in compared results diagnostics is output to stderr stream.

These two steps allow to localize the program point and moment, when the results are beginning to differ. The differences may occur due to wrong usage of DVM-directives or sequential algorithm error, appearing only during parallel execution.

When tracing, the information about the following events is gathered: all variables reads and writes, entering and exiting parallel loops, start and end of each loop iteration.

As tracing as well as dynamic control result in considerable overhead, the means are provided to control trace level to achieve more high efficiency or to reduce size of trace files.

4.1. Loop description file.

A special file, named loops.ctr is provided for tracing. The file contains description of all program loops. The file allows to determine trace detailing.

The file is created by following commands:

dvmcsize <filename>
dvmfsize <filename>

After creation the file loops.ctr may be modified by a user to decrease trace size. The user may set the mode of selective accumulation of the reference trace, completely or partially canceled accumulation of tracing for some loops. Then the command dvmcsize or dvmfsize should be performed again to evaluate the size of a reference trace. If results are not acceptable the process should be repeated.

The trace entity is a program loop. The following accumulation levels may be specified both for whole program and for each loop:

• Only the information about loops and iterations are accumulated (NONE level).

• In addition to the previous level the information about variable modifications are accumulated (MODIF level).

• In addition to the previous level the information about usage of all variables are accumulated. (ALL level).

Furthermore, you can specify an iteration range for each loop which information about variable usage will be accumulated for.

Parameters for each loop are specified in loop description file. The following information is written to the file: a number of trace lines, trace size in bytes for ALL level, trace size in bytes for MODIF level. This information is specified both for whole trace file and for each loop. If the same loop is performed several times, the total information is specified. Besides it, loop number, source file with lines of a beginning and end of the loop, and trace accumulation level and tracing iterations are resulted for each loop in the trace file. The specified level can be changed, by putting other parameters in the file.

Structure of loop description file is the following.

FILESIZE = <trace size in bytes for whole file for ALL level in all loops>

MODIFYSIZE = < trace size in bytes for whole file for MODIF level in all loops>

STRINGCOUNT = < a number of trace lines for the file for ALL level in all loops>

<S,P>LOOP <loop number> [<parent loop number>], <loop rank> = < NONE, MODIF, ALL> {(<dimension>:<first iteration>, <last iteration>) … }

SOURCE = <file name>, <line number>

FILESIZE = < trace size in bytes for the loop for ALL level >

MODIFYSIZE = < trace size in bytes for the loop for MODIF level>

STRINGCOUNT = <a number of trace lines for the loop for ALL level>

ITERCOUNT = <a number of loop iterations>

END_LOOP = <loop number>

<S,P>LOOP <N> [<M>] <R>= < NONE, MODIF, ALL> (I:<F>, <L>) (I:<F>, <L>) …

SOURCE = <file name>, <line number>

FILESIZE = < trace size in bytes for the loop for ALL level >

MODIFYSIZE = < trace size in bytes for the loop for MODIF level>

STRINGCOUNT = <a number of trace lines for the loop for ALL level>

ITERCOUNT = <a number of loop iterations>