Descriptor data structure

3.1 Descriptor data structure

All the general matrix informations and elements to be exchanged among processes are stored within a data structure of the type psb_desc_type. Every structure of this type is associated with a discretization pattern and enables data communications and other operations that are necessary for implementing the various algorithms of interest to us.

The data structure itself psb_desc_type can be treated as an opaque object handled via the tools routines of Sec. 6 or the query routines detailed below; nevertheless we include here a description for the curious reader.

First we describe the psb_indx_map type. This is a data structure that keeps track of a certain number of basic issues such as:

The value of the communication/MPI context;
The number of indices in the index space, i.e. global number of rows and columns of a sparse matrix;
The local set of indices, including:
- The number of local indices (and local rows);
- The number of halo indices (and therefore local columns);
- The global indices corresponding to the local ones.

There are many different schemes for storing these data; therefore there are a number of types extending the base one, and the descriptor structure holds a polymorphic object whose dynamic type can be any of the extended types. The methods associated with this data type answer the following queries:

For a given set of local indices, find the corresponding indices in the global numbering;
For a given set of global indices, find the corresponding indices in the local numbering, if any, or return an invalid
Add a global index to the set of halo indices;
Find the process owner of each member of a set of global indices.

All methods but the last are purely local; the last method potentially requires communication among processes, and thus is a synchronous method. The choice of a specific dynamic type for the index map is made at the time the descriptor is initially allocated, according to the mode of initialization (see also 6).

The descriptor contents are as follows:

indxmap

A polymorphic variable of a type that is any extension of the indx_map type described above.

halo_index

A list of the halo and boundary elements for the current process to be exchanged with other processes; for each processes with which it is necessary to communicate:

Process identifier;
Number of points to be received;
Indices of points to be received;
Number of points to be sent;
Indices of points to be sent;

Specified as: a vector of integer type, see 3.3.

ext_index

A list of element indices to be exchanged to implement the mapping between a base descriptor and a descriptor with overlap.
Specified as: a vector of integer type, see 3.3.

ovrlap_index

A list of the overlap elements for the current process, organized in groups like the previous vector:

Process identifier;
Number of points to be received;
Indices of points to be received;
Number of points to be sent;
Indices of points to be sent;

Specified as: a vector of integer type, see 3.3.

ovr_mst_idx

A list to retrieve the value of each overlap element from the respective master process.
Specified as: a vector of integer type, see 3.3.

ovrlap_elem

For all overlap points belonging to th ecurrent process:

Overlap point index;
Number of processes sharing that overlap points;
Index of a “master” process:

Specified as: an allocatable integer array of rank two.

bnd_elem

A list of all boundary points, i.e. points that have a connection with other processes.

The Fortran 2003 declaration for psb_desc_type structures is as follows:

type psb_desc_type
    class(psb_indx_map), allocatable :: indxmap
    type(psb_i_vect_type) :: v_halo_index
    type(psb_i_vect_type) :: v_ext_index
    type(psb_i_vect_type) :: v_ovrlap_index
    type(psb_i_vect_type) :: v_ovr_mst_idx
    integer, allocatable  :: ovrlap_elem(:,:)
    integer, allocatable  :: bnd_elem(:)
end type psb_desc_type

Figure 3: The PSBLAS defined data type that contains the communication descriptor.

A communication descriptor associated with a sparse matrix has a state, which can take the following values:

Build:: State entered after the first allocation, and before the first assembly; in this state it is possible to add communication requirements among different processes.
Assembled:: State entered after the assembly; computations using the associated sparse matrix, such as matrix-vector products, are only possible in this state.

3.1.1 Descriptor Methods

3.1.2 get_local_rows — Get number of local rows

nr = desc%get_local_rows()

Type:: Asynchronous.
On Entry
desc: the communication descriptor.
Scope: local.

On Return
Function value: The number of local rows, i.e. the number of rows owned by the current process; as explained in 1, it is equal to |_i|+|_i|. The returned value is specific to the calling process.

3.1.3 get_local_cols — Get number of local cols

nc = desc%get_local_cols()

Type:: Asynchronous.
On Entry
desc: the communication descriptor.
Scope: local.

On Return
Function value: The number of local cols, i.e. the number of indices used by the current process, including both local and halo indices; as explained in 1, it is equal to |_i| + |_i| + |_i|. The returned value is specific to the calling process.

3.1.4 get_global_rows — Get number of global rows

nr = desc%get_global_rows()

Type:: Asynchronous.
On Entry
desc: the communication descriptor.
Scope: local.

On Return
Function value: The number of global rows, i.e. the size of the global index space.

3.1.5 get_global_cols — Get number of global cols

nr = desc%get_global_cols()

Type:: Asynchronous.
On Entry
desc: the communication descriptor.
Scope: local.

On Return
Function value: The number of global cols; usually this is equal to the number of global rows.

3.1.6 get_global_indices — Get vector of global indices

myidx = desc%get_global_indices([owned])

Type:: Asynchronous.
On Entry
desc: the communication descriptor.
Scope: local.
Type: required.
owned: Choose if you only want owned indices (owned=.true.) or also halo indices (owned=.false.). Scope: local.
Type: optional; default: .true..

On Return
Function value: The global indices, returned as an allocatable integer array of kind psb_lpk_ and rank 1.

3.1.7 get_context — Get communication context

ictxt = desc%get_context()

Type:: Asynchronous.
On Entry
desc: the communication descriptor.
Scope: local.

On Return
Function value: The communication context.

3.1.8 Clone — clone current object

call desc%clone(descout,info)

Type:: Asynchronous.
On Entry
desc: the communication descriptor.
Scope: local.

On Return
descout: A copy of the input object.
info: Return code.

3.1.9 CNV — convert internal storage format

call desc%cnv(mold)

Type:: Asynchronous.
On Entry
desc: the communication descriptor.
Scope: local.
mold: the desired integer storage format.
Scope: local.
Specified as: a object of type derived from (integer) psb_T_base_vect_type.

The mold arguments may be employed to interface with special devices, such as GPUs and other accelerators.

3.1.10 psb_cd_get_large_threshold — Get threshold for index mapping switch

ith = psb_cd_get_large_threshold()

Type:: Asynchronous.
On Return
Function value: The current value for the size threshold.

3.1.11 psb_cd_set_large_threshold — Set threshold for index mapping switch

call psb_cd_set_large_threshold(ith)

Type:: Synchronous.
On Entry
ith: the new threshold for communication descriptors.
Scope: global.
Type: required.
Intent: in.
Specified as: an integer value greater than zero.

Note: the threshold value is only queried by the library at the time a call to psb_cdall is executed, therefore changing the threshold has no effect on communication descriptors that have already been initialized. Moreover the threshold must have the same value on all processes.

3.1.12 get_p_adjcncy — Get process adjacency list

list = desc%get_p_adjcncy()

Type:: Asynchronous.
On Return
Function value: The current list of adjacent processes, i.e. processes with which the current one has to exchange halo data.

3.1.13 set_p_adjcncy — Set process adjacency list

call desc%set_p_adjcncy(list)

Type:: Asynchronous.
On Entry
list: the list of adjacent processes.
Scope: local.
Type: required.
Intent: in.
Specified as: a one-dimensional array of integers of kind psb_ipk_.

Note: this method can be called after a call to psb_cdall and before a call to psb_cdasb. The user is specifying here some knowledge about which processes are topological neighbours of the current process. The availability of this information may speed up the execution of the assembly call psb_cdasb.

3.1.14 fnd_owner — Find the owner process of a set of indices

call desc%fnd_owner(idx,iprc,info)

Type:: Synchronous.
On Entry
idx: the list of global indices for which we need the owning processes.
Scope: local.
Type: required.
Intent: in.
Specified as: a one-dimensional array of integers of kind psb_lpk_.
On Return
iprc: the list of processes owning the indices in idx.
Scope: local.
Type: required.
Intent: in.
Specified as: an allocatable one-dimensional array of integers of kind psb_ipk_.

Note: this method may or may not actually require communications, depending on the exact internal data storage; given that the choice of storage may be altered by runtime parameters, it is necessary for safety that this method is called by all processes.

3.1.15 Named Constants

psb_none_: Generic no-op;
psb_root_: Default root process for broadcast and scatter operations;
psb_nohalo_: Do not fetch halo elements;
psb_halo_: Fetch halo elements from neighbouring processes;
psb_sum_: Sum overlapped elements
psb_avg_: Average overlapped elements
psb_comm_halo_: Exchange data based on the halo_index list;
psb_comm_ext_: Exchange data based on the ext_index list;
psb_comm_ovr_: Exchange data based on the ovrlap_index list;
psb_comm_mov_: Exchange data based on the ovr_mst_idx list;

[next] [front] [up]