The invariant subspace problem and Rosenblum operators II

Abstract: Let $\mathcal{H}$ be the separable, infinite-dimensional complex Hilbert space. In the first paper of this series, we introduced, by means of the Rosenblum operators, the shift representation operators $$K_x=\sum_{n=0}^{{\infty}T^nx\otimes} e_n$$, which relate an operator $T$ with $r(T)&lt;1$ to the unilateral shift $S$ and reveal its hidden analytic structure. In this paper we replace the pair $(H<sup>2,S)$ by the Sobolev disk algebra $R(\mathbb{D})$ and the multiplication operator $M_z$, whose canonical left inverse $B$ plays the role of $S<sup>*$, and study the two-parameter family $$K_{x,y}=\sum_{n=0}^{{\infty}T^nx\otimes} B<sup>{*n}y$$. We prove that $T\in B(R(\mathbb{D}))$ with $r(T)&lt;1$ is intransitive whenever there exist a nonzero vector $g$ and a nontrivial $M_z$-invariant manifold $\mathcal{M}$ such that $f(T)g$ has a zero in the closed unit disk for every $f\in\mathcal{M}$; the proof explicitly constructs a nontrivial invariant subspace of $T$. We further introduce the ideal property and show that every intransitive operator on $\mathcal{H}$ is unitarily equivalent to an operator on $R(\mathbb{D})$ with this property, so that the Invariant Subspace Problem for arbitrary operators reduces to a problem about a concrete function algebra. Applications include a characterization of intransitivity through the rationality of the generating function $$\sum_{n=0}^{{\infty}\langle} T^{nξ,η\rangle} z<sup>n$$ and new results on Pearcy's problem. Our main application is Halmos's third problem, which has remained open for more than fifty years: we reduce it to invertible operators with conjugate bi-geometric form, and we prove that $T<sup>{-1}$ is intransitive for every invertible $T$ admitting a nontrivial projection $P$ with $PTP=TP$ and $PT(I-P)$ of rank one. Finally, we exhibit tridiagonal operators that are intransitive and admit infinite decreasing chains of invariant subspaces.