The Pan-STARRS1 Database and Data Products (1612.05243v3)

Abstract: This paper describes the organization of the database and the catalog data products from the Pan-STARRS1 $3\pi$ Steradian Survey. The catalog data products are available in the form of an SQL-based relational database from MAST, the Mikulski Archive for Space Telescopes at STScI. The database is described in detail, including the construction of the database, the provenance of the data, the schema, and how the database tables are related. Examples of queries for a range of science goals are included. The catalog data products are available in the form of an SQL-based relational database from MAST, the Mikulski Archive for Space Telescopes at STScI.

• The paper presents the robust design and infrastructure of the PS1 database, efficiently organizing data from the expansive 3π survey.
• It details a SQL-based relational system and multi-stage processing pipelines that ensure precise calibration and timely data access.
• The study addresses the challenges of processing over 10 billion objects, setting a scalable benchmark for future astronomical surveys.

An Essay on "The Pan-STARRS1 Database and Data Products"

The paper "The Pan-STARRS1 Database and Data Products" by H. A. Flewelling et al. provides an exhaustive description of the Pan-STARRS1 (PS1) data management infrastructure, specifically focusing on the organization and dissemination of the data products derived from the PS1 $3\pi$ Steradian Survey. This survey has significantly contributed to the field of astronomy by producing a comprehensive mapping of the sky north of -30 degrees, incorporating multi-epoch imaging in five filters (grizy). This document is the sixth in a series of technical treatises detailing various facets of the Pan-STARRS1 Project, aimed at elucidating the technological and methodological advances involved in handling such voluminous data.

The PS1 survey, conducted using the Giga Pixel Camera 1 (GPC1) atop the Pan-STARRS1 1.8-meter telescope, provides an unprecedented resource for the astronomical community. The paper meticulously details the structure of the PS1 data products, emphasizing the use of a SQL-based relational database system accessible via the Mikulski Archive for Space Telescopes (MAST) at Space Telescope Science Institute (STScI). The design and implementation of this database leverage the experience gained from the Sloan Digital Sky Survey (SDSS), although adapted to meet the greater data complexity and larger sky coverage inherent to the PS1 survey.

Key components of the database architecture include the Published Science Products Subsystem (PSPS), which functions as a pivotal mechanism for data storage and retrieval, utilizing distributed partition views for managing large-volume data efficiently. The paper highlights the collaborative effort with database experts from The Johns Hopkins University to refine and scale the infrastructure.

The paper clearly outlines the stratified process underlying the data management, from raw image acquisition to the compilation of a structured, user-accessible catalog. This includes detailed descriptions of the imaging stages—chip, camera, stack, and diff processing stages—each contributing to the enrichment of the final data products. A significant aspect discussed is the forced photometry stage, wherein positions derived from stacked images guide subsequent photometric analyses on individual night-sky exposures.

Furthermore, the document explains the intricacies of the Desktop Virtual Observatory (DVO), a proprietary database system used for storing and calibrating the initially processed data. The DVO serves as an intermediary, ensuring consistent calibration before data ingestion into the PSPS. Following this, the paper explains the role of IppToPsps, a transformation layer that bridges DVO-generated data with the PSPS's schema, ensuring seamless data ingest and accessibility.

The paper completes its detailed exposition with a focus on practical aspects such as user access interfaces, querying capabilities, and metadata organization. The database schema is delineated into four main categories: Fundamental Data Products, Derived Data Products, Observational Metadata, and System Metadata. Each category builds on a comprehensive schema delineated in the rest of the paper, making extensive use of object and detection tables, thereby aiding complex astronomical queries.

While the paper does not delve deeply into the numerical analysis, it makes explicit mention of the massive scale and scope of the data handled, noting an overall object count upwards of 10 billion, underscoring the staggering computational and storage challenges surmounted in this project. The implications of such a voluminous and richly detailed data set span various domains of astronomy, paving the way for advancements in areas ranging from transient object analysis to the characterization of cosmological structures.

In conclusion, the paper "The Pan-STARRS1 Database and Data Products" stands as a masterful blueprint for large-scale data product handling in modern astronomy. It elucidates methodological rigor and technical sophistication in managing and distributing a survey of monumental scope. The work not only highlights the technical achievements of the Pan-STARRS1 infrastructure but also sets a benchmark for future astronomical surveys and the management of even larger data sets anticipated from forthcoming observatories like the Vera C. Rubin Observatory.