Accelerated Filtering on Graphs using Lanczos method
\\\largeScientific Computing project report}
Accelerated filtering on graphs using Lanczos method
\\\largeRelazione del progetto di Calcolo Scientifico}
\author{Alberto Defendi}
\date{}
@ -85,170 +73,24 @@
{\setlength{\parskip}{0em}
\tableofcontents}
\section{Introduction}
We introduce basic graph theory concepts and briefly overview the results used in the project experiments.
\subsection{Signal processing on graphs}
We consider a weighted undirected graph $\G=(\V, \E, \mathcal{W})$,
where $\V\subset\R^N $ is the set of vertices, $\E\subset\R^M $ is the set of edges, and $\mathcal{W}
: \V\times\mathcal{V}\to\R$ is a weight function. The weight function can be
represented with a $ N \times N $ matrix $ W $ such that
$W_{i,j}=
\begin{cases}
W(v_i, v_j), &\text{if } (v_i, v_j) \in\E\\
0, &\text{otherwise}
\end{cases}
\quad\text{for all } i,j = 1, \dots, |\V|
$,
and for our needs, we assume $ W $ to be symmetric, that is $ W_{i,j}= W_{j,i}$.
In the study of signal processing on graph, we model the idea of sending a signal to a node as
assigning a value to a vertex with a function $ s : \V\to\R$, whose values can be represented as a vector
$ s = s(\V)=[s_1, \dots, s_N]\in\R^N $ where each entry $ s_i \defeq s(v_i)$ represents
the signal sent over a node $ v_i \in\V$. We also keep track of the weight of each vertex
with a function $ d(i)\defeq\sum_{j=1}^N W_{i,j}$, that we represent with the matrix $D =
\text{diag}(d(1), \dots, d(N))$.
In this setting, we introduce the graph Laplacian $\L$ defined as
$\L= D - W$, where $D $ is the diagonal degree matrix with entries $D_{ii}= d(i)$. By
construction, $\L^T =\mathcal{L}$, thus the Lanczos method can be safely applied to our
problem.
\begin{remark}
The Laplacian $\L$ is symmetric and semi-definite (it is diagonally dominant), hence by the
spectral theorem it admits the decomposition
\begin{equation*}
\L = U \Lambda U^{*},
\end{equation*}
where $ U =\left[ u_0,\dots,u_{N-1}\right]\in O(N)$ is called Fourier basis, and $\Lambda=
\text{diag}\left(\lambda_0, \dots, \lambda_{N-1}\right)$ is the matrix of eigenvalues of $\L$,
without loss of generality we can assume $0=\lambda_0\leq\lambda_1\dots\leq\lambda_{N-1}$
and the vectors in $ U $ to be in the same order that the eigenvalues.
\footnote{this assumption is aligned with the PyGSP $\text{compute\_fourier\_basis()}$ function
implementation, used in this work to compute the matrix $\Lambda$. Clarifying this assumption is
outside our scope.}.
\end{remark}
\begin{definition}[Graph signal]
A graph signal is a continuous function $ g : \R^+\to\R$.
\end{definition}
Diagonalising the Laplacian would give an easy way to compute the function $ g(\L)$ by
evaluating it on the eigenvalues of $\L$, which means computing $ g(\L)= U g(\Lambda) U^{*}$.
In matrix notation, applying a filter to a graph $\G$ corresponds to the operation
\begin{equation*}
s^\prime\defeq g(\L)s = U g(\L) U^{*}s.
\end{equation*}
\begin{remark}[Computational cost]
Computing the Fourier basis for $\L$ is computationally expensive for large graphs, thus the
choice of using the Lanczos method for $ g(\L)$, with its computational cost of $ O(M \dot |\E|)$,
it offers an efficient alternative in computing $ g(\L)$. However, storing the basis $ V_M $ costs MN
additional memory, that could be avoided using a two-step implementation, that we leave for future
work.
\end{remark}
\subsection{The Lanczos method}
\begin{definition}
Given a matrix $ A \in\R^{N \times N}$ and a vector $ b \in R^N $, the Krylov subspace of order $j$ is defined as the set $\mathcal{K}_j (A,b)=\{ b, Ab, A^2b, \dots,
A^{j-1}b\}$. We represent the basis of this subspace in a matrix $ V_M = \left[ v_1,\dots,v_M
alberto