3.1 Descriptor data structure

All the general matrix informations and elements to be exchanged among processes are stored within a data structure of the type psb_desc_type. Every structure of this type is associated with a discretization pattern and enables data communications and other operations that are necessary for implementing the various algorithms of interest to us.

The data structure itself psb_desc_type can be treated as an opaque object handled via the tools routines of Sec. 6 or the query routines detailed below; nevertheless we include here a description for the curious reader.

First we describe the psb_indx_map type. This is a data structure that keeps track of a certain number of basic issues such as:

There are many different schemes for storing these data; therefore there are a number of types extending the base one, and the descriptor structure holds a polymorphic object whose dynamic type can be any of the extended types. The methods associated with this data type answer the following queries:

All methods but the last are purely local; the last method potentially requires communication among processes, and thus is a synchronous method. The choice of a specific dynamic type for the index map is made at the time the descriptor is initially allocated, according to the mode of initialization (see also 6).

The descriptor contents are as follows:

indxmap
A polymorphic variable of a type that is any extension of the indx_map type described above.
halo_index
A list of the halo and boundary elements for the current process to be exchanged with other processes; for each processes with which it is necessary to communicate:
  1. Process identifier;
  2. Number of points to be received;
  3. Indices of points to be received;
  4. Number of points to be sent;
  5. Indices of points to be sent;

Specified as: a vector of integer type, see 3.3.

ext_index
A list of element indices to be exchanged to implement the mapping between a base descriptor and a descriptor with overlap.
Specified as: a vector of integer type, see 3.3.
ovrlap_index
A list of the overlap elements for the current process, organized in groups like the previous vector:
  1. Process identifier;
  2. Number of points to be received;
  3. Indices of points to be received;
  4. Number of points to be sent;
  5. Indices of points to be sent;

Specified as: a vector of integer type, see 3.3.

ovr_mst_idx
A list to retrieve the value of each overlap element from the respective master process.
Specified as: a vector of integer type, see 3.3.
ovrlap_elem
For all overlap points belonging to th ecurrent process:
  1. Overlap point index;
  2. Number of processes sharing that overlap points;
  3. Index of a “master” process:

Specified as: an allocatable integer array of rank two.

bnd_elem
A list of all boundary points, i.e. points that have a connection with other processes.

The Fortran 2003 declaration for psb_desc_type structures is as follows:


type psb_desc_type  
    class(psb_indx_map), allocatable :: indxmap  
    type(psb_i_vect_type) :: v_halo_index  
    type(psb_i_vect_type) :: v_ext_index  
    type(psb_i_vect_type) :: v_ovrlap_index  
    type(psb_i_vect_type) :: v_ovr_mst_idx  
    integer, allocatable  :: ovrlap_elem(:,:)  
    integer, allocatable  :: bnd_elem(:)  
end type psb_desc_type


Figure 3: The PSBLAS defined data type that contains the communication descriptor.


A communication descriptor associated with a sparse matrix has a state, which can take the following values:

Build:
State entered after the first allocation, and before the first assembly; in this state it is possible to add communication requirements among different processes.
Assembled:
State entered after the assembly; computations using the associated sparse matrix, such as matrix-vector products, are only possible in this state.

3.1.1 Descriptor Methods

3.1.2 get_local_rows — Get number of local rows
nr = desc%get_local_rows()

Type:
Asynchronous.
On Entry
desc
the communication descriptor.
Scope: local.

On Return
Function value
The number of local rows, i.e. the number of rows owned by the current process; as explained in 1, it is equal to |Ii|+|Bi|. The returned value is specific to the calling process.

3.1.3 get_local_cols — Get number of local cols
nc = desc%get_local_cols()

Type:
Asynchronous.
On Entry
desc
the communication descriptor.
Scope: local.

On Return
Function value
The number of local cols, i.e. the number of indices used by the current process, including both local and halo indices; as explained in 1, it is equal to |Ii| + |Bi| + |Hi|. The returned value is specific to the calling process.

3.1.4 get_global_rows — Get number of global rows
nr = desc%get_global_rows()

Type:
Asynchronous.
On Entry
desc
the communication descriptor.
Scope: local.

On Return
Function value
The number of global rows, i.e. the size of the global index space.

3.1.5 get_global_cols — Get number of global cols
nr = desc%get_global_cols()

Type:
Asynchronous.
On Entry
desc
the communication descriptor.
Scope: local.

On Return
Function value
The number of global cols; usually this is equal to the number of global rows.

3.1.6 get_global_indices — Get vector of global indices
myidx = desc%get_global_indices([owned])

Type:
Asynchronous.
On Entry
desc
the communication descriptor.
Scope: local.
Type: required.
owned
Choose if you only want owned indices (owned=.true.) or also halo indices (owned=.false.). Scope: local.
Type: optional; default: .true..

On Return
Function value
The global indices, returned as an allocatable integer array of kind psb_lpk_ and rank 1.

3.1.7 get_context — Get communication context
ictxt = desc%get_context()

Type:
Asynchronous.
On Entry
desc
the communication descriptor.
Scope: local.

On Return
Function value
The communication context.

3.1.8 Clone — clone current object
call  desc%clone(descout,info)

Type:
Asynchronous.
On Entry
desc
the communication descriptor.
Scope: local.

On Return
descout
A copy of the input object.
info
Return code.

3.1.9 CNV — convert internal storage format
call  desc%cnv(mold)

Type:
Asynchronous.
On Entry
desc
the communication descriptor.
Scope: local.
mold
the desired integer storage format.
Scope: local.
Specified as: a object of type derived from (integer) psb_T_base_vect_type.

The mold arguments may be employed to interface with special devices, such as GPUs and other accelerators.

3.1.10 psb_cd_get_large_threshold — Get threshold for index mapping switch
ith = psb_cd_get_large_threshold()

Type:
Asynchronous.
On Return
Function value
The current value for the size threshold.

3.1.11 psb_cd_set_large_threshold — Set threshold for index mapping switch
call psb_cd_set_large_threshold(ith)

Type:
Synchronous.
On Entry
ith
the new threshold for communication descriptors.
Scope: global.
Type: required.
Intent: in.
Specified as: an integer value greater than zero.

Note: the threshold value is only queried by the library at the time a call to psb_cdall is executed, therefore changing the threshold has no effect on communication descriptors that have already been initialized. Moreover the threshold must have the same value on all processes.

3.1.12 get_p_adjcncy — Get process adjacency list
list =  desc%get_p_adjcncy()

Type:
Asynchronous.
On Return
Function value
The current list of adjacent processes, i.e. processes with which the current one has to exchange halo data.

3.1.13 set_p_adjcncy — Set process adjacency list
call desc%set_p_adjcncy(list)

Type:
Asynchronous.
On Entry
list
the list of adjacent processes.
Scope: local.
Type: required.
Intent: in.
Specified as: a one-dimensional array of integers of kind psb_ipk_.

Note: this method can be called after a call to psb_cdall and before a call to psb_cdasb. The user is specifying here some knowledge about which processes are topological neighbours of the current process. The availability of this information may speed up the execution of the assembly call psb_cdasb.

3.1.14 fnd_owner — Find the owner process of a set of indices
call desc%fnd_owner(idx,iprc,info)

Type:
Synchronous.
On Entry
idx
the list of global indices for which we need the owning processes.
Scope: local.
Type: required.
Intent: in.
Specified as: a one-dimensional array of integers of kind psb_lpk_.
On Return
iprc
the list of processes owning the indices in idx.
Scope: local.
Type: required.
Intent: in.
Specified as: an allocatable one-dimensional array of integers of kind psb_ipk_.

Note: this method may or may not actually require communications, depending on the exact internal data storage; given that the choice of storage may be altered by runtime parameters, it is necessary for safety that this method is called by all processes.

3.1.15 Named Constants

psb_none_
Generic no-op;
psb_root_
Default root process for broadcast and scatter operations;
psb_nohalo_
Do not fetch halo elements;
psb_halo_
Fetch halo elements from neighbouring processes;
psb_sum_
Sum overlapped elements
psb_avg_
Average overlapped elements
psb_comm_halo_
Exchange data based on the halo_index list;
psb_comm_ext_
Exchange data based on the ext_index list;
psb_comm_ovr_
Exchange data based on the ovrlap_index list;
psb_comm_mov_
Exchange data based on the ovr_mst_idx list;