Rank deficiency of Bernoulli random matrices for growing corank

Abstract: Let A be an n x n Bernoulli random matrix whose entries are i.i.d. Bernoulli(p) random variables. In this paper, we determine the probability that the corank of A is at least k when k is of order O(sqrt(log n)): P(corank A >= k) = (1-p+o_n(1))^kn.

• The paper establishes sharp asymptotics for the probability that a Bernoulli random matrix attains a corank of at least k = O(√(log n)) using rigorous probabilistic methods.
• It refines earlier results by leveraging the inversion of randomness and the compressible/incompressible vector framework to derive optimal small ball probability bounds.
• The findings impact random matrix theory, computational complexity, and coding theory, while highlighting current limits in extending the analysis for larger k.

Rank Deficiency of Bernoulli Random Matrices for Growing Corank

Problem Formulation and Background

The paper investigates the probability that a square $n \times n$ random Bernoulli matrix with i.i.d. $\operatorname{Ber}(p)$ entries exhibits a corank (nullity) at least $k$, with $k$ allowed to grow as $O(\sqrt{\log n})$. This problem refines the classical random matrix theory question on the singularity of $\{0,1\}$ random matrices, extending the focus from the event $\mathrm{rank}(A) < n$ to the more general regime $\mathrm{rank}(A) \le n-k$ for nonconstant $k$.

Historic works by Komlós and others yielded upper bounds on the singularity probability and eventually established that $\mathbb{P}(\det A=0) = o_n(1)$, with a conjectured asymptotic of $\operatorname{Ber}(p)$0 for the uniform Bernoulli case. This analysis extends naturally to questions about higher corank, with the corresponding conjecture by Vu being that for fixed $\operatorname{Ber}(p)$1, $\operatorname{Ber}(p)$2. While this has been resolved for fixed $\operatorname{Ber}(p)$3 [Jain et al.], the regime where $\operatorname{Ber}(p)$4 grows slowly with $\operatorname{Ber}(p)$5 was open, and existing results only provided weaker bounds, such as exponential decay in $\operatorname{Ber}(p)$6 [Rudelson].

Main Results

The authors prove that for $\operatorname{Ber}(p)$7 and $\operatorname{Ber}(p)$8,

$\operatorname{Ber}(p)$9

effectively confirming the conjectural asymptotic in the polynomially growing $k$0 regime. The principal upper bound is established for all $k$1, where $k$2 depends on $k$3 and an accuracy parameter $k$4. The bound is sharp in the sense that $k$5 arises from the probability that $k$6 columns are identically zero, yielding the dominant contribution asymptotically.

Technical Framework and Methods

The argument adapts and refines several modern approaches in the invertibility analysis of random matrices, especially the "inversion of randomness" principle of Tikhomirov and the compressible/incompressible dichotomy of Rudelson and Vershynin.

Reduction to Kernel Structure:

If the corank of $k$7 is at least $k$8, there exist $k$9 linearly independent vectors in the kernel of a submatrix formed by removing $k$0 columns. Analyzing the probability that such vectors exist, and their structure, enables a union bound formulation.

Compressible and Incompressible Vectors:

The kernel vectors are partitioned into compressible (essentially sparse or close-to-sparse) and incompressible vectors. The key is to show:

• The probability that the kernel contains $k$1 orthogonal compressible vectors is at most $k$2—essentially matching the lower bound from zero columns.
• If $k$3 linearly independent kernel vectors exist, with high probability, at least one is incompressible. Controlling the small ball probabilities for inner products with incompressible vectors is crucial in bounding the overall probability.

Key Innovations:

• For compressible vectors, the proof establishes optimal small ball probabilities via refined net arguments and explicit small ball estimates analogous to the Littlewood-Offord theory, but generalized to vector systems.
• For the incompressible regime, the analysis employs Tikhomirov's "inversion of randomness" method, which connects concentration of measure with arithmetic structure, and is refined here to handle the regime $k$4.
• The authors demonstrate that pushing beyond this $k$5 requires a fundamentally new approach, as the high-dimensional Littlewood-Offord problem becomes the bottleneck.

Numerical Bounds and Limiting Regimes

The work provides both upper and lower bounds that match up to $k$6 terms, elucidating the sharpness of the probability asymptotics. The result encompasses all $k$7 but is strongest in the uniform case ($k$8). The upper bound for $k$9 is dictated by the inherent limitations in the incompressible vector counting argument, currently capped at $O(\sqrt{\log n})$0 due to limits in high-dimensional arithmetic combinatorics.

Implications, Limitations, and Future Directions

This result provides a rigorous and precise characterization of the rare-event regime for moderate-large corank in Bernoulli random matrices, sharpening our understanding of the spectrum near the boundary of singularity. The methods consolidate the modern probabilistic-combinatorial toolkit of random matrix theory in the non-asymptotic regime. The explicit probabilistic bounds have direct implications for computational complexity (e.g., the analysis of algorithms based on matrix invertibility) and coding theory.

The main limitation is that the methodology does not extend to $O(\sqrt{\log n})$1, due to the absence of sufficiently strong high-dimensional inverse Littlewood-Offord theorems. Overcoming this barrier—likely requiring a breakthrough in inverse arithmetic combinatorics—remains a significant open problem.

Future developments may involve:

• High-dimensional inverse Littlewood-Offord theorems for structured vector systems.
• Extensions to non-binary, more general discrete, or even continuous distributions with bounded support.
• Application of the probabilistic structures uncovered to computational lower bounds, derandomization, and network theory.

Conclusion

The paper establishes sharp asymptotics for the probability that an $O(\sqrt{\log n})$2 Bernoulli random matrix has corank at least $O(\sqrt{\log n})$3 in the crucial regime where $O(\sqrt{\log n})$4 grows as $O(\sqrt{\log n})$5, demonstrating that $O(\sqrt{\log n})$6 gives the leading order. The techniques combine intricate probabilistic combinatorics with advanced structural analysis of the kernel, and delineate the limits of current methods, setting a program for further study in the combinatorics of random matrices and concentration phenomena.