Matrix Theory From Eigenvalues to Data Insights

We can store an image in a matrix. Every image consists of a set of pixels which are the building blocks of that image. Each pixel represents the color or the intensity of light in a specific location in the image. In a grayscale image with PNG format, each pixel has a value between 0 and 1, where zero corresponds to black and 1 corresponds to white. So a grayscale image with \(m \times n\) pixels can be stored in an \(m \times n\) matrix.

Using SVD, we can have a good approximation of the original image and save a lot of memory. If we use the first 30 singular values to reconstruct the image, we only need to keep the first 30 \(\sigma_i\text{,}\) \(\mathbf{u}_i\text{,}\) and \(\mathbf{v}_i\) which means storing \(30 \times (1 + m + n)\) values instead of the original \(m \times n\) values.

SVD can be used to reduce the noise in images. Noise usually corresponds to smaller singular values because they don’t represent major structures or patterns in the image. By reconstructing the image using only the first few singular values, we can effectively filter out the noise.

SVD assigns most of the noise (but not all of that) to the vectors represented by the lower singular values. If we reconstruct a low-rank matrix (ignoring the lower singular values), the noise will be reduced, however, the correct part of the matrix changes too. The result is a matrix that is only an approximation of the noiseless matrix that we are looking for.

SVD is the foundation of Principal Component Analysis (PCA). PCA is an important dimensionality reduction technique widely used in data science and machine learning. Through SVD, we can find the main directions of variation in the data and project high-dimensional data into a low-dimensional space while retaining the most important information.