Skip to content

Large-scale Structure (LSS) Catalogs

Overview

Version 2.0 release notes

This VAC includes the large-scale structure catalogs for the DESI Early Data Release (EDR), including all intermediate files and the final, clustering-ready catalogs. Catalogs are available for all DESI extra-galactic tracer types, including from dark-time observations of Luminous Red Galaxies (LRG), Emission Line Galaxies (ELG), and Quasars (QSO), and the Bright Galaxy Sample (BGS) from bright-time observations.

Data Access

Data URL: https://data.desi.lbl.gov/public/edr/vac/edr/lss/v2.0/

NERSC access: 

/global/cfs/cdirs/desi/public/edr/vac/edr/lss/v2.0/

Documentation

Directory structure

There are four separate directories:

  1. LSScats: Most users will want to go directly to this directory, which contains the clustering-ready catalogs in the subdirectoryclustering, for each of the LSS target categoriess: BGS, ELG, LRG and QSO (with given subcategories and separated into the North and South photometric regions) and also the random catalogs associated with the target data. In the subdirectory full in the LSScats directory you can also find the catalogs containing information on all targets identified as reachable by the DESI fiber assignment software, with one entry for each. The files are split by target type and whether of not vetos for angular positions have been applied. Finally, in the LSScats directory we share the computed number density of each target as a function of redshift.

  2. altmtl: Here we include all of the “merged target lists” from 128 realizations (for each of dark- and bright-time) of the SV3 fiber assigment, produced by randomly shuffling the subpriorities that determine which target will get assigned. These products are used to compute the bitweights for pairwise inverse probability (PIP) weights and completeness quantities.

  3. potential_assigments: Here we include all of the potential assigments of reachable targets as determined by the DESI fiber assignment software, merged with the initial photometric targets, both for randoms and data.

  4. inputs_wspec: This directory contains intermediate files, including the joining of the potential assigment information with information from the observations, statistics on tiles, a concatenation of the emission line information, and a list of fibers determined to be greater than 3\(\sigma\) outliers in terms of their redshift success.

Data processing

All catalogs are created using the DESI LSS python repository on GitHub.

A brief description of the process to go from potential assigments to the clustering-ready catalog is the following:

  • Potential assigments are determined for the data for each DESI tile by the DESI fiber assign software previous to observations. For the randoms, they are determined afterwards, making sure to match the settings used for the data. We compute the merged files, concatenated across all tiles, found in potential_assigments.

  • The spectroscopic reduction and redshift extraction pipeline was run on all observed tiles. The information on the redshift extraction and details of the spectroscopic observation of each fiber on each tile is joined to the potential assignment information and used to create the products in inputs_wspec.

  • Combining the result of these two directories, we end up with the data in the full directory in LSScats, which reduces the information down to the unique set of targets that were reacheable by a fiber, whether observed or not.

  • In parallel, we run the altmtl pipeline that produces completeness estimates and PIP weights for the data.

  • Reading from the altmtl results, and the full catalogs, we produce the clustering-ready catalogs under the LSScats/clustering directory, ready for clustering statistics. At this stage we also compute the n(z), including completeness weights such that our estimated n(z) is that of a (observationally) complete sample.

DESI version

Version 2.0 uses the following DESI code tags:

Data model

The data model for the LSS catalogs can be found in this directory of the DESI data model. Here we explain the directory structure of the LSS EDR catalogs, and detail the content of all files, including descriptions of all columns and recommendations about their use.

Contact

Contact Ashley Ross for questions regarding these catalogs.