CSC Format for Sparse Matrices

• CSC Format is a sparse matrix representation using three arrays (val, rowidx, colptr) that enable efficient column-oriented access and traversal.
• Extensions such as VCSC and IVCSC leverage redundancy in column data to reduce memory usage significantly, with reductions up to 7.5× in high-redundancy scenarios.
• CSC supports fast operations like SpMV with O(nnz) time complexity, balancing storage requirements with computational performance in scientific and numerical applications.

The Compressed Sparse Column (CSC) format is a canonical data structure for representing sparse matrices in memory, particularly designed to optimize storage efficiency and computational performance for column-oriented access patterns. Widely adopted in numerical linear algebra and scientific computing, CSC is foundational for high-performance sparse matrix-vector and matrix-matrix operations. Extensions of CSC, such as Value-Compressed Sparse Column (VCSC) and Index- and Value-Compressed Sparse Column (IVCSC), have been developed to address scenarios with additional data redundancy, relevant in domains like genomics and recommender systems (Ruiter et al., 2023).

1. CSC Format: Structure and Semantics

CSC encodes a sparse $m \times n$ matrix $A$ with $nnz$ nonzero elements using three contiguous arrays:

• $\texttt{val}[0\ldots nnz-1]$: the nonzero values, stored column-wise.
• $\texttt{rowidx}[0\ldots nnz-1]$: the corresponding row indices for each value.
• $\texttt{colptr}[0\ldots n]$: a pointer array marking the boundaries of each column in $\texttt{val}$ and $\texttt{rowidx}$, where $\texttt{colptr}[j]$ is the start index in column $j$ and $\texttt{colptr}[n]=nnz$ (Ruiter et al., 2023).

Example:

Given

$$A = \begin{bmatrix} 0 & 5 & 0 \ 3 & 0 & 4 \ 0 & 5 & 6 \end{bmatrix}$$

with $nnz=5$, CSC arrays are:

• $\texttt{val} = [3, 5, 5, 4, 6]$
• $\texttt{rowidx} = [1, 0, 2, 1, 2]$
• $\texttt{colptr} = [0, 1, 3, 5]$

This structure enables efficient traversal and access for operations (e.g., SpMV) and balances storage needs and random-access efficiency.

2. Storage Analysis and Memory Footprint

Memory usage in CSC is determined by the data types used to represent matrix entries and indices:

$$\text{CSC\_size} = \mathtt{val\_size}\cdot nnz + \mathtt{idx\_size}\cdot nnz + \mathtt{idx\_size}\cdot (n+1)$$

where $\mathtt{val\_size}$ is the byte size per value (e.g., 8 bytes for double), and $\mathtt{idx\_size}$ is the byte size per index (e.g., 4 bytes for 32-bit integer) (Ruiter et al., 2023).

The first two terms account for storage of nonzero values and their indices, while the final term encodes the $n+1$ column pointers. Compared with the Coordinate (COO) format, which uses $$(\texttt{val\_size} + 2\times\texttt{idx\_size})\cdot nnz$$, CSC reduces storage overhead by omitting explicit storage of column indices per nonzero.

3. Computational Operations and Complexity

CSC is optimized for column-access patterns, particularly efficient for:

• Sparse Matrix-Vector Multiplication (SpMV):

Iteration proceeds column-by-column.

for j in range(n): for k in range(colptr[j], colptr[j+1]): y[rowidx[k]] += val[k] * x[j]

This yields $O(nnz)$ time complexity, with contiguous memory access patterns for $\texttt{colptr}$ and $\texttt{rowidx}$, advantageous for cache efficiency (Ruiter et al., 2023).

• Element Lookup $A[i,j]$:

Requires scanning $k$ in $[\texttt{colptr}[j], \texttt{colptr}[j+1])$ for $\texttt{rowidx}[k]=i$, yielding $O(nnz_j)$ per lookup, where $nnz_j$ is the count of nonzeros in column $j$.

4. Extensions: VCSC and IVCSC

Conventional CSC captures only sparsity, not redundancy in the value distribution within matrix columns. Two recent extensions have addressed this limitation (Ruiter et al., 2023):

• Value-Compressed Sparse Column (VCSC):
  • $\texttt{uniqVals}_j[0 \ldots u_j - 1]$: unique nonzero values.
  • $\texttt{counts}_j[0 \ldots u_j - 1]$: occurrence counts for each unique value.
  • $\texttt{idx}_j[0 \ldots nnz_j - 1]$: row indices partitioned by value group.
  • Offsets to flatten $\texttt{uniqVals}, \texttt{counts}, \texttt{idx}$ arrays.

The memory usage is:

$$\text{VCSC\_size} = \sum_{j=1}^n \left[\mathtt{val\_size}\cdot u_j + \mathtt{idx\_size}\cdot u_j + \mathtt{idx\_size}\cdot nnz_j\right] + \mathtt{idx\_size}\cdot (n+1)$$

where $u_j$ is the count of unique values in column $j$. When redundancy $r_j=1-(u_j/nnz_j)$ is large, this achieves substantial compression.

• Index- and Value-Compressed Sparse Column (IVCSC):
  • Positive deltas $\delta_k = r_k - r_{k-1}$ are stored using the minimal possible byte-width.
  • Groups are prefixed with a per-group byte width, minimizing index storage to the theoretical minimum, especially when deltas are small.

In practice, IVCSC reaches up to $7.5\times$ memory reduction over CSC in redundancy-dominated matrices (e.g., genomics data, mean redundancy ratio MMR $\approx 0.99$) (Ruiter et al., 2023).

5. Compression Ratios and Empirical Results

Compression capability of CSC, VCSC, and IVCSC formats is strongly influenced by the redundancy of nonzero values:

• In high-redundancy regimes (column-wise repeated values), VCSC reduces memory usage by up to $2.8\times$ vs. CSC, IVCSC by up to $7.5\times$.
• On a $1.3 \times 10^9$ nonzero single-cell genomics matrix (MMR $0.987$), memory usage as a percentage of the uncompressed COO size was: CSC $75\%$, VCSC $26\%$, IVCSC $9\%$.
• When redundancy is low (e.g., unique floats per entry), VCSC and IVCSC offer no benefit and may increase memory usage (Ruiter et al., 2023).

Dataset	$nnz$	Sparsity	MMR	CSC (%)	VCSC (%)	IVCSC (%)
Single-cell	$1.3 \times 10^9$	91.9%	0.987	75	26	9
MovieLens	$2.85 \times 10^7$	97.0%	0.616	76	33	18
Simulated unique	$9.99 \times 10^7$	90.0%	0.0	75	100	105

A plausible implication is that the choice of format should be matched to the data’s redundancy profile.

6. Performance Trade-Offs and Applicability

• CSC/COO: Offer the fastest SpMV and SpMM, with $O(nnz)$ storage. Construction from COO is fastest when redundancy is low.
• VCSC: Incurs $O(u_j)$ per-column looping overhead and offers its largest gains when $r_j > 0.3$. In benchmarked ML/genomics workloads, SpMV is within $10\%$ of CSC's runtime.
• IVCSC: Maximizes memory savings—up to $10\times$ vs. COO, $7\times$ vs. CSC—at the cost of $2-4\times$ slower SpMV (e.g., 420 ms for IVCSC vs. 140 ms for CSC in a $10^6\times10^4$ matrix with 90% mean redundancy). Construction time remains tractable ($<50$ ms on $10^4\times10^2$ matrices at low redundancy).

Selection guidelines for practitioners:

• Use CSC/COO for general sparse data with little redundancy.
• Use VCSC when columns aggregate repeated values (count data, ratings, genomics); minor performance cost, significant memory reduction.
• Use IVCSC where memory capacity is limiting and reduced I/O bandwidth or slower decompression can be tolerated (e.g., offline post-processing of very large matrices) (Ruiter et al., 2023).

7. Contextual Relevance and Limitations

CSC remains a preferred default for a broad spectrum of sparse linear algebra workloads due to low overhead and compatibility with major computing frameworks. However, its inability to exploit nonzero value redundancy has motivated the creation of VCSC and IVCSC, particularly for modern data modalities with structured repetition. The effectiveness of these extensions is empirically bounded by the actual redundancy and local row-index proximity in the data. Deployment of VCSC and IVCSC should explicitly consider their overheads in construction time, per-column uniqueness ratio, and compatibility with existing computational routines—factors that delimit their applicability for certain classes of problems (Ruiter et al., 2023).