Skip to content

Data Access

Web access

Public DESI data releases: https://data.desi.lbl.gov/public

Latest data release (DR1): https://data.desi.lbl.gov/public/dr1/

This is best suited for browsing the directory structure and downloading individual files, including both spectra and catalog files. To download large amounts of data, please use Globus described below.

See Data Releases for information about individual data releases and their directory structure.

See Data Organization for details on how the data is organized across directories.

File download

For users who want to download a small sample of DESI data for testing or exploration, we provide the desi_get_dr_subset script. This tool allows you to download a focused subset of DESI data by:

  • Searching for targets near a specific sky position (RA/Dec)
  • Finding the healpix with the most targets in that region
  • Downloading all data files associated with that healpix
  • Optionally downloading the corresponding tile data

The script handles different data releases appropriately, automatically adjusting paths and defaults based on the data release. For example, it uses different default coordinates depending on the data release - for EDR it targets the Rosette 1 field (RA=179.6, Dec=0.0), while for DR1 it uses coordinates that retrieve healpix 23040 (RA=56.0, Dec=-9.0).

You can find this script in the DESI data repository: https://github.com/desihub/desida/blob/main/bin/desi_get_dr_subset

Basic usage: 

desi_get_dr_subset --dr dr1 --ra 56.0 --dec -9.0 --radius 0.1 --base-dir ./tiny_dr1

Key options:

  • --dr: Data release to use (edr, dr1)
  • --ra, --dec: Sky coordinates to search around
  • --radius: Search radius in degrees
  • --base-dir: Directory where files will be downloaded
  • --no-tiles: Download only healpix data, skip tile data

Note

Even a “small” sample of DESI data is pretty big – the default options will download 40 GB of data.

Database access

DESI catalogs also exist in a database format. People with NERSC accounts (e.g. through DESI, DES, LSST-DESC, CMB-S4…) can access the public portion of the databases at NERSC without requiring DESI membership. For convenience, a copy of the public databases is also hosted by the NOIRLab Astro Data Lab science platform, including both EDR and DR1. Please see Databases for connection information and further details about databases containing DESI data.

Spectroscopic Data as a Service

The SPectra Analysis and Retrievable Catalog Lab (SPARCL) service is provided by NOIRLab. This API provides functions for search and retrieval of multiple DESI spectra with the full flux versus wavelength. Although only the HEALPixel coadds are served at present, this provides a convenient way to retrieve spectra that could otherwise be spread among many separate files.

Interactive Sky Viewer

An interactive sky viewer shows the DESI Legacy Surveys DR9 imaging overlaid with locations of DESI tiles and spectra. Users can interactively pan and zoom, or pre-select specific RA,dec locations and zoom levels with URLs of the form

https://www.legacysurvey.org/viewer/desi-dr1/?ra=218.1973&dec=2.5435&zoom=15

Clicking through on an individual target brings up an interactive spectrum browser. Users can view any DESI science TARGETID from an DR1 catalog using URLs of the form

https://www.legacysurvey.org/viewer/desi-spectrum/dr1/targetid39627848784286649 https://www.legacysurvey.org/viewer/desi-spectrum/dr1/targetid39627848784285507

Bulk download with Globus

Globus provides an interface suited for bulk downloads of data. You specify what to download and the destination, and then Globus coordinates a parallel transfer including error checking and retries. It emails you an alert when the transfer has successfully completed. If your home institution does not provide a Globus Server, you can use Globus Connect Personal to setup an endpoint on your personal device.

All public DESI data are available from the DESI Public Data Globus collection. Each DESI Data Release is a separate subdirectory under that endpoint. This method requires a free Globus account but does not require a NERSC account or DESI membership.

Warning

Searching Globus collections for “DESI” returns a large number of collections unrelated to the DESI Survey, e.g. from people with “desi” in their name. Search for “DESI Public Data” or use the direct link to find the correct DESI Public Data collection.

Warning

Globus usually does not follow or copy symbolic links (“symlinks”), which occur in the DR1 target/ and spectro/redux/iron/exposures/ subdirectories. If you need to do bulk transfers of these directories, see the separate instructions for handling symlinks.

Amazon Web Services

As part of their Open Data program, Amazon Web Services (AWS) hosts both the DESI Early Data Release (under /edr) and Data Release 1 (under /dr1) in the desidata S3 bucket for on-demand querying. The web endpoint is located at desidata.s3.amazonaws.com/index.html. Amazon supports multiple ways to programmatically access the data for free, documented here.

If you are specifically interested in scientific data analysis, we recommend running our pre-built Docker image, which bundles a cloud-synced filesystem for DESI data access with a Jupyter server and DESI Python packages, making it compatible with existing code at NERSC with minimal modifications.

Public data releases at NERSC will be exactly mirrored by /<release> directories in the bucket, with the following exceptions: * All individual /<release>/target directories have been merged into the /target top-level directory. * All individual /<release>/spectro/data directories have been merged into the /raw_spectro_data top-level directory.

All other symlinked files/directories have been converted into real files/directories.

If you have a NERSC account…

If you have a NERSC account, e.g. through DESI, DES, LSST-DESC, or other DOE-sponsored projects, all of the public DESI data are available on disk at 

/global/cfs/cdirs/desi/public/

These are exactly the same files visible at https://data.desi.lbl.gov/public.

This directory is also visible from https://jupyter.nersc.gov for Jupyter notebook access to the DESI data.

DESI Software at NERSC

DESI provides access to its software environment at NERSC. To enable the DESI environment, and access Data Release 1, you can run these two commands: 

source /global/common/software/desi/desi_environment.sh 24.11
module swap desitree/dr1
24.11 is one of the DESI software releases. Other releases are described below. We also support tcsh: just replace desi_environment.sh with desi_environment.csh.

There are currently three software releases available for general use:

  • 24.11: recent, stable release; recommended.
  • 22.5: older release used to generate EDR/fuji.
  • 23.1: older release used to generate DR1/iron.
  • main: based on git checkouts of DESI software packages; most up-to-date, but not guaranteed to be stable.

DESI Collaborators with NERSC accounts can also use the NERSC DTN endpoint to access DESI Data.

Note

DESI data are released under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Use of DESI data requires including the citation and acknowledgment text given on the Data License and Acknowledgments page.