ColumnKeeper: Column-Centric Systems Overview

Updated 4 July 2026

ColumnKeeper is a family name for diverse column-centric management strategies applied in systems ranging from high-energy physics storage to DRAM security.
In storage systems like Ceph S3, ColumnKeeper enables selective access, independent column updates, and efficient parallel I/O while significantly reducing metadata overhead.
ColumnKeeper also spans dynamic runtime optimizations, incremental formal concept updates, fair subset selection, and semantic typing, showcasing its versatile application across computing domains.

to=arxiv_search 在天天中彩票json {"query":"all:ColumnKeeper","max_results":10,"sort_by":"relevance"} to=arxiv_search 天天种彩票json {"query":"(Smith et al., 2023) OR (Kakolyris et al., 21 Jun 2026) OR (Kriegel, 2019) OR (Kloibhofer et al., 2023) OR (Alkowaileet et al., 2021) OR (Floratou et al., 2011) OR (Matakos et al., 2023) OR (Khurana et al., 2020) OR (0909.1758)","max_results":20,"sort_by":"relevance"} ColumnKeeper is a research label applied, in the supplied literature, to several technically distinct column-centric artifacts rather than to a single canonical system. It is used for a Ceph S3 object data store for high-energy physics, for automated and optimized columnar arrays in a dynamic language runtime, for incremental maintenance of formal concept diagrams under attribute-column insertion and removal, and for a DRAM mitigation suite against the ColumnDisturb vulnerability. Related design syntheses further use ColumnKeeper as the target architecture for Hadoop-compatible columnar storage, schemaless LSM-based document stores, fair column subset selection, and semantic annotation of table columns (Smith et al., 2023, Kloibhofer et al., 2023, Kriegel, 2019, Kakolyris et al., 21 Jun 2026, Floratou et al., 2011, Alkowaileet et al., 2021, Matakos et al., 2023, Khurana et al., 2020).

1. Scope and nomenclature

Across these uses, the recurring abstraction is the decomposition of structured state into separately managed columns. The meaning of “column,” however, is domain-dependent: an event data product in HEP, a property array in a runtime, an attribute column in formal concept analysis, or an odd/even physical column class in DRAM subarrays. The supplied record therefore treats ColumnKeeper as a family name for column-oriented management strategies rather than as a single implementation lineage (Smith et al., 2023, Kloibhofer et al., 2023, Kriegel, 2019, Kakolyris et al., 21 Jun 2026).

Domain	Artifact	Column unit
HEP storage	Ceph S3 column-object event store	Event data product
Dynamic compilation	Automated columnar arrays	Object field
Formal concept analysis	Incremental update algorithm	Attribute column
DRAM security	ColumnDisturb mitigation	Odd/even disturbed columns
Distributed data systems	Columnar storage blueprints	Table/document path column

This multiplicity matters analytically. In some cases ColumnKeeper is a concrete mechanism with measured throughput, storage, and overhead data; in others it is a formally specified update algorithm; and in still others it is a design target onto which adjacent columnar techniques are mapped. A plausible implication is that any encyclopedia treatment must distinguish the operational semantics of each variant instead of collapsing them into a unified architecture.

2. ColumnKeeper as a Ceph S3 object data store for HEP

In the HEP setting, ColumnKeeper is treated as a column-object event data store in which each event data product is serialized independently and written to S3 objects in stripes, with a single index object, or an external metadata service, tying together the dataset structure. The stated goals are to obviate data tiers, enable selective column access, reduce metadata overhead, and maximize parallelism. Traditional file-based formats such as ROOT TTrees in AOD/MiniAOD force many products into the same file or TTree with tightly coupled QoS and lifecycle, which leads to broad reprocessing and duplication across tiers. The column-object design instead decouples products into independently managed objects, allows targeted updates such as changing only electrons, and aligns with object-store-based parallel I/O (Smith et al., 2023).

The data model is event-ordered. Each product becomes a column stream, written into multiple stripes, each covering a contiguous range of events. Stripe size is chosen on write; when the compressed buffer reaches a target such as 128–512 KiB, the stripe is finalized. Event batch size is a fixed constant, and stripe event counts are chosen so that the stripe’s event count evenly divides the batch size, which keeps metadata volume growing approximately like $(N_{\text{products}}) + (M_{\text{events}})$ rather than $N_{\text{products}} \times M_{\text{events}}$ . Very small products can be greedily grouped into product groups, while remaining independently addressable through S3 byte-range requests.

The evaluated storage back end is Ceph RGW with S3 access via libs3, asynchronous requests, and exponential backoff for transient network errors. Tested bucket configurations include EC4+2 with bucket index, EC4+2 without bucket index, and a triple-replicated bucket. In EC4+2, objects are chunked into 16 KiB pieces split into 4 data blocks and 2 parity blocks, while bucket index entries contribute about 300 bytes of metadata per object in a triply replicated NVMe pool. If the bucket index is disabled, listing is unavailable and lifecycle tracking must move to an external index service.

Selective access is central to the design. Clients resolve the dataset index, request only needed products, and use byte-range GETs when grouped stripes are involved. Parallelism comes from many independent GETs and PUTs to distinct object keys, scheduled in the prototype through an Intel TBB thread pool. Asynchronous S3 requests decouple network I/O from decompression and deserialization. The paper notes that when the S3 server is not saturated, I/O latency is hidden under CPU tasks, whereas saturation introduces stalls that reduce CPU efficiency.

The reported measurements emphasize both size efficiency and scale-out behavior. MiniAOD input is 55.7 kB/event with LZMA. The object format reaches 70–71 kB/event with ZSTD and 61.8 kB/event with LZMA; small-object granularity overhead drops from 6.5% to 1.4% with larger stripes and grouping. Single-client read-only and read-write tests show good scaling with negligible differences across bucket configurations. In multi-client conversion, about 350–400 workers saturate one RGW node at about 6300 events/s and about 450 MB/s to an EC4+2 bucket, writing 4.5 TB into 7.4 million objects. Against the mock tiered-storage scenario in Table 1, updating only slimmedElectrons reduces the total from 117.1 kB/event to 57 kB/event under the object model. The paper also notes that the absence of TTree struct-of-array splitting limits compression efficiency relative to MiniAOD.

Operationally, stripes are treated as immutable after publication. This suggests a versioned-write model: updates create new stripes or versions of affected products instead of mutating existing objects. The paper explicitly identifies future work in integrating object I/O into CMSSW, leveraging RNTuple’s object backend, and developing a robust metadata/index service to track object lifecycle.

3. ColumnKeeper in dynamic runtimes and classical column-store theory

In the dynamic-language-runtime setting, prior work referred to as ColumnKeeper automatically transforms hot arrays of uniform objects into columnar arrays in GraalVM JavaScript. When an array is eligible, the runtime creates one property array per field, copies values, and replaces array elements with proxy objects carrying arrRef and arrIdx, so that field loads are redirected to property arrays. The successor work introduces control flow duplication in the GraalVM Compiler: loops that may access columnar arrays are duplicated into a fast path specialized to the columnar state and a generic slow path that handles arbitrary state, including in-loop transformation. The compiler recognizes intrinsics such as <isColumnar>(arr), <transitive>(arr, cond), <transform>(arr), and <restore>(arr), then removes redundant checks, hoists property-array loads, and exposes contiguous indexed accesses for vectorization (Kloibhofer et al., 2023).

The same work adds multi-level storage transformation for nested structures. For nested objects, a single-level columnar transform leaves reference-valued fields pointing into scattered heap objects, so nested dereferences remain non-local. Multi-level transformation therefore creates property arrays not only for top-level fields but also for nested paths such as ccInfo.pan and ccInfo.service. Nested proxies carry arrRef, arrIdx, and path, allowing reads like c.ccInfo.pan to become indexed accesses into clients._propArrays["ccInfo.pan"] [i]. For JavaScript Date objects, this produces contiguous timestamp arrays and yields up to about $2\times$ speedups in date-centric microbenchmarks.

The performance results illustrate the underlying columnar rationale. Small workloads already show speedups of about $2\times$ on aggregate and writeProperty with 100 iterations and 10k elements; larger arrays and iteration counts reach about $7$– $10\times$ speedups; q6 in the JavaScript TPC-H port reaches nearly $5\times$ ; and cache behavior improves sharply, with cache-misses reduced from 2,575,530,996 to 50,406,360, L1-dcache-load-misses from 3,142,811,326 to 137,072,211, and LLC-load-misses from 520,871,505 to 530,809. The data block attributes these gains to locality, loop-invariant code motion, and SIMD-friendly regularity.

These mechanisms closely parallel classical column-store ideas. The database literature describes column stores as reading only the needed attributes, exploiting homogeneous-column compression such as dictionary, bitmap, and especially run-length encoding, processing over compressed data, and deferring row reconstruction via late materialization. “Teaching an Old Elephant New Tricks” shows that many of these benefits can be simulated in a row-store through materialized views and c-tables that store runs as tuples $(f, v, c)$ with clustered indexes on $f$ and covering indexes on $v$ ; queries are then rewritten with band joins and aggregates such as $N_{\text{products}} \times M_{\text{events}}$ 0 or $N_{\text{products}} \times M_{\text{events}}$ 1 rather than reconstructing rows early (0909.1758).

The connection is structural rather than terminological. In both the runtime and relational cases, the critical properties are positional alignment, selective access, compressed or contiguous storage, and late reconstruction. The runtime system realizes them through JIT specialization, proxy indirection removal, and property arrays; the relational simulation realizes them through c-tables, band joins, and materialized views. A plausible implication is that ColumnKeeper, in this sense, denotes an execution strategy as much as a storage layout.

4. ColumnKeeper-oriented storage architectures for distributed and schemaless systems

The supplied technical guides also use ColumnKeeper as a target architecture for distributed columnar storage. In Hadoop/MapReduce, the recommended design replaces naive text input with binary storage and then with a true columnar format in which each split-directory contains one file per column plus a schema file. Column co-location is enforced by a ColumnPlacementPolicy, so that all column files for a split-directory are placed on the same set of nodes and can be scheduled data-locally. A ColumnInputFormat reads only requested columns, while lazy record construction and skip-list column files permit fast skipping of unneeded records. The reported effects are substantial: SequenceFile yields about a $N_{\text{products}} \times M_{\text{events}}$ 2 scan speedup over text; CIF gives order-of-magnitude improvements; CIF-SL and CIF-DCSL add about a $N_{\text{products}} \times M_{\text{events}}$ 3 speedup beyond columnar-only techniques; and the crawl benchmark reports map-phase speedups of about $N_{\text{products}} \times M_{\text{events}}$ 4 for CIF, $N_{\text{products}} \times M_{\text{events}}$ 5 for CIF-SL, and $N_{\text{products}} \times M_{\text{events}}$ 6 for CIF-DCSL, with total job speedup about $N_{\text{products}} \times M_{\text{events}}$ 7 (Floratou et al., 2011).

Several details are specific to complex column types. Arrays, maps, and nested records incur high Java deserialization costs, so the design stores complex types as single columns, avoids eager deserialization, and uses Dictionary Compressed Skip Lists for map-typed columns. Skip blocks at granularities $N_{\text{products}} \times M_{\text{events}}$ 8 enable efficient forward navigation, while curPos and per-column lastPos support on-demand field materialization. The design is explicitly read-mostly, and the paper notes that wide scans over most columns reduce its advantage.

A second storage-oriented use arises in schemaless, LSM-based document stores. Here ColumnKeeper is presented as a columnar engine built on Dremel-like encoding but extended for document-store heterogeneity. Arrays are encoded by delimiter-based definition levels rather than repetition levels; union types are “objectified” into sibling leaf columns; and tombstones are encoded through primary-key definition levels with $N_{\text{products}} \times M_{\text{events}}$ 9 for insert or upsert and $2\times$ 0 for anti-matter. Two physical layouts are proposed: APAX, where B+-tree leaf pages contain per-column minipages, and AMAX, where a logical mega leaf spans multiple physical pages and allows selective reading of only the referenced columns plus Page 0. Flush and merge events in the LSM tree perform schema inference and columnization, while vertical merge first reconciles primary keys and then merges one column at a time (Alkowaileet et al., 2021).

The reported empirical results are strongly workload-dependent. On the sensors dataset, Open and VB are 7.2× and 4.8× larger than APAX, and 8.5× and 5.6× larger than AMAX. On tweet_1, AMAX achieves query runtimes such as 0.6 s versus 48.4 s for Open and 26.1 s for VB on one benchmark, whereas APAX suffers from the many small minipages induced by 933 columns. On wos, AMAX reaches 0.83 s versus 103.1 s for Open and 62.5 s for VB. Under update-heavy workloads with secondary indexes, however, APAX and AMAX are 24% and 35% slower than Open on ingestion because old-value retrieval requires linear decode and scan on the primary index. These results sharply distinguish scan-optimized column layouts from update-centric row layouts.

5. ColumnKeeper as an incremental FCA update algorithm

In formal concept analysis, ColumnKeeper denotes an incremental algorithm for maintaining a concept lattice and its attribute-additive concept diagram when a single attribute column is inserted or removed. The base structure is a formal context $2\times$ 1 with derivation operators

$2\times$ 2

and formal concepts $2\times$ 3 satisfying $2\times$ 4 and $2\times$ 5. The concept lattice is ordered by extent inclusion, and an attribute-additive diagram associates reduced object and attribute labels as well as seed vectors for irreducible attributes (Kriegel, 2019).

For insertion of a new attribute column $2\times$ 6, the updated context is the apposition

$2\times$ 7

with $2\times$ 8. The algorithm classifies old concepts into old, varying, and generating concepts. In the updated context, concepts are partitioned into old, varied, and generated concepts. The key structural fact is the existence of three bijections:

$2\times$ 9

with corresponding inverses on the updated lattice. Old concepts persist unchanged, varying concepts become varied by appending $2\times$ 0 to the intent, and generating concepts produce genuinely new concepts.

Neighborhood updates are local. Covers inside the old block and inside the varying block are preserved. Every generated concept $2\times$ 1 is covered by its generator $2\times$ 2. New-to-new covers are characterized by intervals in the generator substructure containing no other generator, and varied-to-new covers are characterized by intervals in the combined varying-plus-generating substructure containing no intermediate generator or varying concept. This localism is one of the algorithm’s central advantages: it avoids recomputation of unaffected parts of the Hasse diagram.

Labeling and reducibility are updated incrementally as well. If $2\times$ 3 is irredundant, its attribute concept is placed at the new concept generated by the greatest generator $2\times$ 4, so that $2\times$ 5. Object labels move only from generators to their corresponding new concepts when the object concept changes. Irreducibility changes are governed by structural conditions on unique upper neighbors and generator superconcepts. Seed vectors are then added or removed accordingly, and node positions are recomputed only for nodes whose intents or irreducibility status changed.

The complexity summary is explicit. If $2\times$ 6, with $2\times$ 7 generators and $2\times$ 8 varyings, classification takes $2\times$ 9 subset checks plus closure tests; concept construction is $7$0; and total insertion time is

$7$1

where $7$2 and $7$3 are the relevant cover-edge counts in the induced substructures. Compared with recomputation from scratch, the savings come from preserving unchanged blocks, limiting closure computations, and maintaining layout stability.

6. ColumnKeeper as a DRAM ColumnDisturb mitigation family

In DRAM security, ColumnKeeper is the first set of mitigations proposed for ColumnDisturb, a read-disturbance phenomenon that targets columns rather than rows and affects all cells across three consecutive DRAM subarrays because of open-bitline sharing. The paper introduces two variants: CK-D, a deterministic mechanism, and CK-P, a probabilistic one. The threat model assumes an attacker can orchestrate high-rate activations within a bank so that victim cells sharing disturbed bitlines accumulate enough exposure to exceed the ColumnDisturb threshold $7$4 (Kakolyris et al., 21 Jun 2026).

CK-D exploits the odd/even asymmetry of the open-bitline architecture. Each subarray has two counters, CT-E and CT-O, tracking exposure of even and odd columns, plus a Row Pointer Table entry identifying the next row to refresh round-robin. On an ACT to subarray $7$5, the controller increments both counters in $7$6 and increments the relevant half-column counters in $7$7 and $7$8. When

$7$9

reaches the preventive-refresh threshold

$10\times$ 0

the controller refreshes one row in subarray $10\times$ 1, advances the row pointer modulo $10\times$ 2, and resets both counters. With $10\times$ 3 and $10\times$ 4, the paper gives $10\times$ 5; with $10\times$ 6, $10\times$ 7; and with $10\times$ 8, $10\times$ 9.

CK-P eliminates counters. On each ACT to subarray $5\times$ 0, the controller refreshes one row in each of $5\times$ 1, $5\times$ 2, and $5\times$ 3 with independent probability $5\times$ 4. Its guarantee is expressed probabilistically: if $5\times$ 5 risky activations occur before failure and $5\times$ 6 is the number of protective refreshes, then the failure probability for one subarray before $5\times$ 7 refreshes have occurred is

$5\times$ 8

For a target annual failure probability $5\times$ 9 and $(f, v, c)$ 0 rows, the paper gives $(f, v, c)$ 1 for $(f, v, c)$ 2, $(f, v, c)$ 3 for $(f, v, c)$ 4, and $(f, v, c)$ 5 for $(f, v, c)$ 6.

The reported overheads are low at current and near-future thresholds. At the experimentally demonstrated threshold of 1M, CK-D and CK-P incur average single-core performance overheads of 0.15% and 0.36%, respectively. At 128K, these rise to 1.70% and 2.73%. Required area overheads are 0.1 mm² for CK-D and 0.03 mm² for CK-P; controller-side state is about 7.5 KB for CK-D and about 2.5 KB for CK-P in the evaluated system. With subarray-level parallelism, average overheads at 128K fall to 0.87% for CK-D and 1.20% for CK-P. The paper also reports that low thresholds such as 16K remain manageable with smaller subarrays or SALP.

The broader significance is that ColumnKeeper here is not a data layout but a physically grounded mitigation policy. The “column” is a disturbed physical bitline class spanning multiple subarrays, and the key design problem is to translate that geometry into refresh scheduling with either deterministic or probabilistic guarantees.

Two further syntheses in the supplied record use ColumnKeeper as a host architecture for higher-level column reasoning. The first concerns fair column subset selection. Given a matrix whose rows are partitioned into two groups, the task is to choose a column subset minimizing the maximum group-relative reconstruction error

$(f, v, c)$ 7

The paper shows that selecting a minimum-size fair subset is NP-hard even with two groups, that in some cases nearly $(f, v, c)$ 8 columns are information-theoretically necessary, and that deterministic leverage-score coverage can be extended to a fair setting. A two-phase greedy algorithm yields a subset of size at most $(f, v, c)$ 9, while practical heuristics based on fair L-RRQR and fair H-RRQR support the exact-budget case (Matakos et al., 2023).

The second extension concerns semantic annotation of tabular columns. The $f$ 0 system maps a column to a real-world concept using maximum-likelihood estimation over heterogeneous data sources. For a column vector $f$ 1 and concept $f$ 2, it scores

$f$ 3

and in practice uses the log form

$f$ 4

The system combines inverted entity-to-concept counts, numerical interval trees, composite pattern trees for mixed columns, concept co-occurrence, tuple validation with default $f$ 5, and belief sharing across synonyms, hierarchies, and Word2Vec similarities with threshold 0.4. It reports average accuracies of 62.9% at top-1, 74.6% at top-2, and 79.1% at top-3 across nine datasets; for numeric-only columns, top-1 is 41.2% (Khurana et al., 2020).

These two directions move beyond storage and execution into representation quality and semantics. Fair subset selection asks which columns should be retained under a group-sensitive reconstruction objective, whereas semantic typing asks what a column means. In both cases, the supplied record treats ColumnKeeper as a system that could absorb such capabilities without abandoning its basic column-wise decomposition. A plausible implication is that the name has evolved into a conceptual umbrella for column management across storage, execution, inference, and security.