\section{Parallel environment routines}
\label{sec:parenv}

\clearpage\subsection*{psb\_init --- Initializes PSBLAS parallel environment}
\addcontentsline{toc}{subsection}{psb\_init}

\begin{verbatim}
call psb_init(icontxt, np, basectxt, ids)
\end{verbatim}

This subroutine initializes the PSBLAS parallel environment, defining
a virtual parallel machine.
\begin{description}
\item[Type:] Synchronous.
\item[\bf  On Entry ]
\item[np] Number of processes in the PSBLAS virtual parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf optional}.\\
Intent: {\bf in}.\\
Specified as: an integer value. \
Default: use all available processes.
\item[basectxt] the initial communication context. The new context
  will be defined from the processes participating in the initial one.\\
  Scope: {\bf global}.\\
Type: {\bf optional}.\\
Intent: {\bf in}.\\
Specified as: an integer value. \
Default: use MPI\_COMM\_WORLD.
\item[ids] Identities of the processes to use for the new context; the
  argument is ignored when \verb|np| is not specified. This allows the
  processes in the new environment to be in an order different from the
  original one.\\
  Scope: {\bf global}.\\
Type: {\bf optional}.\\
Intent: {\bf in}.\\
Specified as: an integer array. \
Default: use the indices $(0\dots np-1)$.  
\end{description}

\begin{description}
\item[\bf On Return]
\item[icontxt] the communication context identifying the virtual
  parallel machine. Note that this is always a duplicate of
  \verb|basectxt|, so that library communications are completely
  separated from other communication operations.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf out}.\\
Specified as: an integer variable.
\end{description}


{\par\noindent\large\bfseries Notes}
\begin{enumerate}
\item A call to this routine must precede any other PSBLAS call. 
\item It is an error to specify a value for $np$ greater than the
  number of processes available in the underlying base parallel
  environment. 
\end{enumerate}


\clearpage\subsection*{psb\_info --- Return information about  PSBLAS parallel
  environment}
\addcontentsline{toc}{subsection}{psb\_info}

\begin{verbatim}
call psb_info(icontxt, iam, np)
\end{verbatim}

This subroutine returns information about  the PSBLAS parallel environment, defining
a virtual parallel machine.
\begin{description}
\item[Type:] Asynchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\end{description}

\begin{description}
\item[\bf On Return]
\item[iam] Identifier of current  process in the PSBLAS virtual parallel machine.\\
Scope: {\bf local}.\\
Type: {\bf required}.\\
Intent: {\bf out}.\\
Specified as: an integer value. $-1 \le iam \le np-1$\
\item[np] Number of processes in the PSBLAS virtual parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf out}.\\
Specified as: an integer variable. \
\end{description}


{\par\noindent\large\bfseries Notes}
\begin{enumerate}
\item For processes in the virtual parallel machine  the identifier
  will satisfy $0 \le iam \le np-1$;
\item If the user has requested on \verb|psb_init| a number of
  processes less than the total available in the parallel execution
  environment, the remaining processes will have on return $iam=-1$;
  the only call involving \verb|icontxt| that any such process may
  execute is  to \verb|psb_exit|. 
\end{enumerate}


\clearpage\subsection*{psb\_exit --- Exit from  PSBLAS parallel environment}
\addcontentsline{toc}{subsection}{psb\_exit}

\begin{verbatim}
call psb_exit(icontxt)
call psb_exit(icontxt,close)
\end{verbatim}

This subroutine exits from the  PSBLAS parallel virtual  machine.
\begin{description}
\item[Type:] Synchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\item[close] Whether to close all data structures related to the
  virtual parallel machine, besides those associated with icontxt.\\
Scope: {\bf global}.\\
Type: {\bf optional}.\\
Intent: {\bf in}.\\
Specified as: a logical  variable, default value: true.
\end{description}

{\par\noindent\large\bfseries Notes}
\begin{enumerate}
\item This routine may be called even if a previous call to
  \verb|psb_info| has returned with $iam=-1$; indeed, it it is the only
  routine that may be called with argument \verb|icontxt| in this
  situation.
\item A call to this routine with \verb|close=.true.| implies a call
  to \verb|MPI_Finalize|, after which no parallel routine may be called.
\item If the user whishes to use multiple communication contexts in the
  same program, or to enter and exit multiple times into the parallel
  environment, this routine may be called to 
  selectively close the contexts with \verb|close=.false.|, while on
  the last call it should be called with \verb|close=.true.| to
  shutdown in a clean way the entire parallel environment.
\end{enumerate}


\clearpage\subsection*{psb\_get\_mpicomm --- Get the MPI communicator}
\addcontentsline{toc}{subsection}{psb\_get\_mpicomm}

\begin{verbatim}
call psb_get_mpicomm(icontxt, icomm)
\end{verbatim}

This subroutine returns the MPI communicator associated with a PSBLAS context
\begin{description}
\item[Type:] Asynchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\end{description}

\begin{description}
\item[\bf On Return]
\item[icomm] The MPI communicator associated with the  PSBLAS virtual parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf out}.\\
\end{description}


\clearpage\subsection*{psb\_get\_rank --- Get the MPI rank}
\addcontentsline{toc}{subsection}{psb\_get\_rank}

\begin{verbatim}
call psb_get_rank(rank, icontxt, id)
\end{verbatim}

This subroutine returns the MPI rank of the  PSBLAS process $id$
\begin{description}
\item[Type:] Asynchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\item[id] Identifier of a   process in the PSBLAS virtual parallel machine.\\
Scope: {\bf local}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer value. $0 \le id \le np-1$\
\end{description}

\begin{description}
\item[\bf On Return]
\item[rank] The MPI rank associated with the  PSBLAS process $id$.\\
Scope: {\bf local}.\\
Type: {\bf required}.\\
Intent: {\bf out}.\\
\end{description}




\clearpage\subsection*{psb\_wtime --- Wall clock timing}
\addcontentsline{toc}{subsection}{psb\_wtime}

\begin{verbatim}
time = psb_wtime()
\end{verbatim}

This function returns a wall clock timer. The resolution of the timer
is dependent on the underlying parallel environment implementation.
\begin{description}
\item[Type:] Asynchronous.
\item[\bf  On Exit ]
\item[Function value] the elapsed time in seconds.\\
Returned  as: a  \verb|real(psb_dpk_)|  variable.
\end{description}


\clearpage\subsection*{psb\_barrier --- Sinchronization point  parallel
  environment}
\addcontentsline{toc}{subsection}{psb\_barrier}

\begin{verbatim}
call psb_barrier(icontxt)
\end{verbatim}

This subroutine acts as an explicit synchronization point for  the  PSBLAS
parallel virtual  machine. 
\begin{description}
\item[Type:] Synchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\end{description}


\clearpage\subsection*{psb\_abort --- Abort a computation}
\addcontentsline{toc}{subsection}{psb\_abort}

\begin{verbatim}
call psb_abort(icontxt)
\end{verbatim}

This subroutine aborts computation on the parallel virtual machine. 
\begin{description}
\item[Type:] Asynchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\end{description}





\clearpage\subsection*{psb\_bcast --- Broadcast data}
\addcontentsline{toc}{subsection}{psb\_bcast}

\begin{verbatim}
call psb_bcast(icontxt, dat, root)
\end{verbatim}

This subroutine implements a broadcast operation based on the
underlying communication library. 
\begin{description}
\item[Type:] Synchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\item[dat] On the root process, the data to be broadcast.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer, real or complex variable, which may be a
scalar, or a rank 1 or 2 array, or a character or logical variable,
which may be a scalar or rank 1 array. \
Type, kind, rank and size must agree on all processes.
\item[root] Root process holding data to be broadcast.\\
Scope: {\bf global}.\\
Type: {\bf optional}.\\
Intent: {\bf in}.\\
Specified as: an integer value $0<= root <= np-1$, default 0 \
\end{description}


\begin{description}
\item[\bf On Return]
\item[dat] On processes other than  root, the data to be broadcast.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer, real or complex variable, which may be a
scalar, or a rank 1 or 2 array, or a character or logical scalar. \
Type, kind, rank and size must agree on all processes.
\end{description}


\clearpage\subsection*{psb\_sum --- Global sum}
\addcontentsline{toc}{subsection}{psb\_sum}

\begin{verbatim}
call psb_sum(icontxt, dat, root)
\end{verbatim}

This subroutine implements a sum reduction  operation based on the
underlying communication library. 
\begin{description}
\item[Type:] Synchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\item[dat] The local contribution to the global sum.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer, real or complex variable, which may be a
scalar, or a rank 1 or 2 array. \
Type, kind, rank and size must agree on all processes.
\item[root] Process to hold the final sum, or $-1$ to make it available
  on all processes.\\
Scope: {\bf global}.\\
Type: {\bf optional}.\\
Intent: {\bf in}.\\
Specified as: an integer value $-1<= root <= np-1$, default -1. \
\end{description}


\begin{description}
\item[\bf On Return]
\item[dat] On destination process(es), the result of the sum operation.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer, real or complex variable, which may be a
scalar, or a rank 1 or 2 array. \\
Type, kind, rank and size must agree on all processes.
\end{description}

{\par\noindent\large\bfseries Notes}
\begin{enumerate}
\item The \verb|dat| argument is both input and output, and its
  value may be changed even on processes different from the final
  result destination.
\item The \verb|dat| argument may also be a long integer scalar.
\end{enumerate}



\clearpage\subsection*{psb\_max --- Global maximum}
\addcontentsline{toc}{subsection}{psb\_max}

\begin{verbatim}
call psb_max(icontxt, dat, root)
\end{verbatim}

This subroutine implements a maximum valuereduction
operation based on the underlying communication library. 
\begin{description}
\item[Type:] Synchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\item[dat] The local contribution to the global maximum.\\
Scope: {\bf local}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer or  real variable, which may be a
scalar, or a rank 1 or 2 array. \
Type, kind, rank and size must agree on all processes.
\item[root] Process to hold the final maximum, or $-1$ to make it available
  on all processes.\\
Scope: {\bf global}.\\
Type: {\bf optional}.\\
Intent: {\bf in}.\\
Specified as: an integer value $-1<= root <= np-1$, default -1. \\
\end{description}


\begin{description}
\item[\bf On Return]
\item[dat] On destination process(es), the result of the maximum operation.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer or  real variable, which may be a
scalar, or a rank 1 or 2 array. \
Type, kind, rank and size must agree on all processes.
\end{description}


{\par\noindent\large\bfseries Notes}
\begin{enumerate}
\item The \verb|dat| argument is both input and output, and its
  value may be changed even on processes different from the final
  result destination.
\item The \verb|dat| argument may also be a long integer scalar.
\end{enumerate}

\clearpage\subsection*{psb\_min --- Global minimum}
\addcontentsline{toc}{subsection}{psb\_min}

\begin{verbatim}
call psb_min(icontxt, dat, root)
\end{verbatim}

This subroutine implements a minimum value reduction
operation based on the underlying communication library. 
\begin{description}
\item[Type:] Synchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\item[dat] The local contribution to the global minimum.\\
Scope: {\bf local}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer  or real variable, which may be a
scalar, or a rank 1 or 2 array. \
Type, kind, rank and size must agree on all processes.
\item[root] Process to hold the final value, or $-1$ to make it available
  on all processes.\\
Scope: {\bf global}.\\
Type: {\bf optional}.\\
Intent: {\bf in}.\\
Specified as: an integer value $-1<= root <= np-1$, default -1. \\
\end{description}


\begin{description}
\item[\bf On Return]
\item[dat] On destination process(es), the result of the minimum operation.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer  or  real variable, which may be a
scalar, or a rank 1 or 2 array. \\
Type, kind, rank and size must agree on all processes.
\end{description}


{\par\noindent\large\bfseries Notes}
\begin{enumerate}
\item The \verb|dat| argument is both input and output, and its
  value may be changed even on processes different from the final
  result destination.
\item The \verb|dat| argument may also be a long integer scalar.
\end{enumerate}

\clearpage\subsection*{psb\_amx --- Global maximum absolute value}
\addcontentsline{toc}{subsection}{psb\_amx}

\begin{verbatim}
call psb_amx(icontxt, dat, root)
\end{verbatim}

This subroutine implements a maximum absolute value reduction
operation based on the underlying communication library. 
\begin{description}
\item[Type:] Synchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\item[dat] The local contribution to the global maximum.\\
Scope: {\bf local}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer, real or complex variable, which may be a
scalar, or a rank 1 or 2 array. \
Type, kind, rank and size must agree on all processes.
\item[root] Process to hold the final value, or $-1$ to make it available
  on all processes.\\
Scope: {\bf global}.\\
Type: {\bf optional}.\\
Intent: {\bf in}.\\
Specified as: an integer value $-1<= root <= np-1$, default -1. \\
\end{description}


\begin{description}
\item[\bf On Return]
\item[dat] On destination process(es), the result of the maximum operation.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer, real or complex variable, which may be a
scalar, or a rank 1 or 2 array. \
Type, kind, rank and size must agree on all processes.
\end{description}


{\par\noindent\large\bfseries Notes}
\begin{enumerate}
\item The \verb|dat| argument is both input and output, and its
  value may be changed even on processes different from the final
  result destination.
\item The \verb|dat| argument may also be a long integer scalar.
\end{enumerate}

\clearpage\subsection*{psb\_amn --- Global minimum absolute value}
\addcontentsline{toc}{subsection}{psb\_amn}

\begin{verbatim}
call psb_amn(icontxt, dat, root)
\end{verbatim}

This subroutine implements a minimum absolute value reduction
operation based on the underlying communication library. 
\begin{description}
\item[Type:] Synchronous.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\item[dat] The local contribution to the global minimum.\\
Scope: {\bf local}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer, real or complex variable, which may be a
scalar, or a rank 1 or 2 array. \
Type, kind, rank and size must agree on all processes.
\item[root] Process to hold the final value, or $-1$ to make it available
  on all processes.\\
Scope: {\bf global}.\\
Type: {\bf optional}.\\
Intent: {\bf in}.\\
Specified as: an integer value $-1<= root <= np-1$, default -1. \\
\end{description}


\begin{description}
\item[\bf On Return]
\item[dat] On destination process(es), the result of the minimum operation.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer, real or complex variable, which may be a
scalar, or a rank 1 or 2 array. \\
Type, kind, rank and size must agree on all processes.
\end{description}


{\par\noindent\large\bfseries Notes}
\begin{enumerate}
\item The \verb|dat| argument is both input and output, and its
  value may be changed even on processes different from the final
  result destination.
\item The \verb|dat| argument may also be a long integer scalar.
\end{enumerate}


\clearpage\subsection*{psb\_snd --- Send data}
\addcontentsline{toc}{subsection}{psb\_snd}

\begin{verbatim}
call psb_snd(icontxt, dat, dst, m)
\end{verbatim}

This subroutine sends a packet of data to a destination.
\begin{description}
\item[Type:] Synchronous: see usage notes. 
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\item[dat] The data to be sent.\\
Scope: {\bf local}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer, real or complex variable, which may be a
scalar, or a rank 1 or 2 array, or a character or logical scalar. \
Type, kind and  rank must agree on sender and receiver process; if $m$ is
not specified, size must agree as well. 
\item[dst] Destination process.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer value $0<= dst <= np-1$. \\
\item[m] Number of rows.\\
Scope: {\bf global}.\\
Type: {\bf Optional}.\\
Intent: {\bf in}.\\
Specified as: an integer value $0<= m <= size(dat,1)$. \\
When $dat$ is a rank 2 array, specifies the number of rows to be sent
independently of the leading dimension $size(dat,1)$; must have the
same value on sending and receiving processes.
\end{description}


\begin{description}
\item[\bf On Return]
\end{description}

{\par\noindent\large\bfseries Notes}
\begin{enumerate}
\item This subroutine implies a synchronization, but only between the
  calling process and the destination  process $dst$.
\end{enumerate}


\clearpage\subsection*{psb\_rcv --- Receive data}
\addcontentsline{toc}{subsection}{psb\_rcv}

\begin{verbatim}
call psb_rcv(icontxt, dat, src, m)
\end{verbatim}

This subroutine receives a packet of data to a destination.
\begin{description}
\item[Type:] Synchronous: see usage notes.
\item[\bf  On Entry ]
\item[icontxt] the communication context identifying the virtual
  parallel machine.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer variable.
\item[src] Source process.\\
Scope: {\bf global}.\\
Type: {\bf required}.\\
Intent: {\bf in}.\\
Specified as: an integer value $0<= src <= np-1$. \\
\item[m] Number of rows.\\
Scope: {\bf global}.\\
Type: {\bf Optional}.\\
Intent: {\bf in}.\\
Specified as: an integer value $0<= m <= size(dat,1)$. \\
When $dat$ is a rank 2 array, specifies the number of rows to be sent
independently of the leading dimension $size(dat,1)$; must have the
same value on sending and receiving processes.
\end{description}


\begin{description}
\item[\bf On Return]
\item[dat] The data to be received.\\
Scope: {\bf local}.\\
Type: {\bf required}.\\
Intent: {\bf inout}.\\
Specified as: an integer, real or complex variable, which may be a
scalar, or a rank 1 or 2 array, or a character or logical scalar. \
Type, kind and  rank must agree on sender and receiver process; if $m$ is
not specified, size must agree as well. 
\end{description}

{\par\noindent\large\bfseries Notes}
\begin{enumerate}
\item This subroutine implies a synchronization, but only between the
  calling process and the source  process $src$.
\end{enumerate}