Skip to content

Known Issues with Data Release 1

Definitions of all data columns referenced below (e.g. ZWARN) can be found here.

After the Iron production was finished, several problems were found in the coaddition of target metadata, impacting the FIBERMAP tables in coadd*.fits and redrock*.fits files, which are further propagated into the summary redshift catalogs in spectro/redux/iron/zcatalog/v0/. Note that the underlying spectra, coadded spectra, and redshift fits are not impacted by the following issues, only the metadata about those spectra are impacted. These issues have been fixed in all summary catalogs in spectro/redux/iron/zcatalog/v1/, including both tile- and HEALPix-based versions, and will be correct in future data releases. They have not been fixed in the ~200k individual coadd and redrock files.

The following issues in the FIBERMAP metadata are resolved in the v1 catalogs, but not in v0 or the individual files:

  • 0.03% of targets incorrectly have COADD_FIBERSTATUS==0 even though all of their data are masked. These targets will correctly have ZWARN bit 9 (NODATA) set in the redshift fits. Quality cuts based solely upon COADD_FIBERSTATUS without also cutting on ZWARN will have some contamination. This amounts to roughly 8,500 HEALPix-based entries in the individual files. Those requiring COADD_FIBERSTATUS should get that information from the appropriate v1 summary catalog by matching on TARGETID.

  • MEAN_FIBER_{RA,DEC}, STD_FIBER_{RA,DEC}, MEAN_DELTA_{X,Y}, RMS_DELTA_{X,Y}, and MEAN_PSF_TO_FIBER_SPECFLUX incorrectly included all exposures, not just those that passed the quality cuts and were used in the coadd. This issue impacts approximately 0.7% of targets. This amounts to roughly 200,000 HEALPix-based entries in the individual files. Those requiring these quantities should get that information from the appropriate v1 summary catalog by matching on TARGETID.

  • STD_FIBER_RA, the standard deviation of contributing FIBER_RA values, is incorrect for ~17% of the targets. This amounts to roughly 4.75 million HEALPix-based entries in the individual files. It only impacts objects that were observed on more than one night, and all FIBER_RA values are correct. Those requiring STD_FIBER_RA should get that information from the appropriate v1 summary catalog by matching on TARGETID.

  • Targets that were originally observed under one survey/program and then were later reselected as secondary targets are missing their secondary target bits in the coadded DESI_TARGET, SCND_TARGET, SV1_DESI_TARGET, SV1_MWS_TARGET, and SV1_SCND_TARGET bit masks. This issue impacts ~0.0006% of targets. This amounts to less than 200 HEALPix-based entries in the individual files. Those requiring these quantities should get that information from the appropriate v1 summary catalog by matching on TARGETID.

Other known issues that have not been resolved for DR1 include:

  • For both EDR and DR1 the coadd resolution matrices are suboptimal due to mis-weighting of the individual resolution matrices. However, these effects only affect spectra that were coadded from more than one exposure and are small, with ~1.7% of pixels having resolution matrix elements incorrect by more than 0.01, and ~0.05% of pixels incorrect by more than 0.1. This is generally insignificant but can impact detailed studies of spectra. This is now fixed in the latest version of desispec, which can be used with the Iron spectra files to generate new coadd files with the correct resolution matrices.

  • There was a bug in inverse variance estimation during flux calibration of all spectra, which caused the variance of all spectra to be underestimated. The effects are negligible at low signal-to-noise (S/N) per pixel, but become significant at around S/N ~20-30. At a S/N per pixel of ~100, the underestimation of the uncertainties can be as much as a factor of 5.

  • Radial velocity systematics in DR1 main survey backup program data have been identified with a scatter of 2-3 km/s, which occasionally reach values up to 20 km/s, due to limitations in the wavelength calibration during Guadalupe and Iron processing.

  • We did not properly account for a change in the string representation of Boolean values between Gaia DR2 and EDR3. Due to this oversight, the Gaia DUPLICATED_SOURCE column is always False for DESI targets that derived their values from EDR3. The implication of this is that no source in DESI targeted using Gaia EDR3 would have been masked on DUPLICATED_SOURCE. Almost 2 million sources in the Iron HEALPix-based catalog were selected using Gaia EDR3. Approximately 2.8% of sources in Gaia EDR3 have the DUPLICATED_SOURCE column set, so roughly ~56k thousand of the ~2 million dark-time sources in DR1 may have been unintentionally targeted.

  • The QuasarNet afterburner provides Z_NEW computed from Redrock with only the quasar templates using a uniform prior of ±0.05 about its coarse estimated Z_QN. When the redshift prior overlaps with z>1.4, only the high redshift quasar templates were used. This causes a slight excess of quasars with Z_NEW≈ 1.4 (the high redshift template lower bound) and an increased ZWARN rate for quasars with 1.2 < z < 1.8. This is propagated to the Z column in the LSS catalogs, impacting ~0.5% of the quasars in the corresponding redshift range.

  • Redshifts for Lyα quasars are systematically underestimated with increasing redshift owing to incorrect modeling of the Lyman series optical depth in Redrock, also present for EDR. The average bias is modest at z ≈ 2, but exceeds 100 km/s at z>3. This will be corrected in future releases, but remains present in Iron. The DR1 ZLyA VAC provides corrected redshifts for Lyα quasars.

  • A calibration problem on petal/spectrograph 9 on night 20211212 led to many incorrect redshifts for fibers 4500–4999 on the 17 tiles observed on that night (8500 spectra, ~0.03% of Iron). % 8500 = 17 tiles * 500 spectra (one problematic petal) DR1 includes a supplementary release spectro/redux/reproc_20211212_iron/, produced using the same code tags as Iron but with a better choice of calibration data. All 17 tiles were reprocessed and provided as tile-based redshifts. This includes tile 7733 which was re-observed on 20211215 and thus included in the cumulative coadds for 20211215. All other impacted tiles were last observed on 20211212. HEALPix-based coadds and summary catalogs were not produced for this mini production. The reprocessed data were used for the LSS catalogs, but were not used to replace values in Iron itself.

  • FIBER_{RA,DEC} = FIBER_{X,Y} = 0.0 for broken fibers (whose locations cannot be measured) and for fibers whose positions were not measured by the Fiber View Camera on individual exposures. These cases have FIBERSTATUS bit 8 MISSINGPOSITION set.

  • In early main survey data, some non-target BAD_SKY locations near bright objects received the same TARGETID despite having different (RA,Dec) locations, and vice-a-versa (i.e., different TARGETID for the same (RA,Dec) on different overlapping tiles). This issue only impacts non-science, non-calibration spectra when a positioner was unable to reach a good target.

  • DESINAME, which gives an identifier for DESI objects based on sky location truncated to a precision of 0.0001 (1E-4) arcsseconds, is incorrect in the catalogs and individual files for -0.1 < TARGET_DEC < -0.0001. Correct DESINAME values can be generated using the modern version of desiutil by calling the function desiutil.names.radec_to_desiname with the object’s TARGET_RA and TARGET_DEC. This issue led to 121,132 incorrect names in the HEALPix-based catalog; or 0.4% of Iron.

  • Some stuck positioners from Survey Validation data with TARGETID<0 appear in the tile-based redshift catalogs, but not in the HEALPix-based redshift catalogs.

  • Tile 21917 (main survey bright program) has valid FIBER_{X,Y} values, but is missing the focal plane (x,y) -> (RA,Dec) transform in the raw data and thus has FIBER_{RA,DEC} = 0.0. In this case, the TARGET_RA and TARGET_DEC values record the intended target position and the FIBER_{X,Y} and DELTA_{X,Y} values remain valid and were used for quality cuts to set FIBERSTATUS bits 9 BADPOSITION and 10 POORPOSITION if the fiber was away from the intended target.

Finally, the following known issues were present in the previous data release (EDR). They remain issues in DR1 and are included here for completeness:

  • Redrock templates do not include Active Galactic Nucleus (AGN)-like galaxies with a mixture of broad and narrow lines. As a result, these types of galaxies are often fit equally well (or equally poorly) with either GALAXY or QSO templates at the same redshift, which can also trigger ZWARN bit 2 (value 2²=4) for LOW_DELTACHI2 since the χ² difference between the two fits is small, indicating an ambiguous answer.

  • There are cases where Redrock is overconfident and reports ZWARN==0, i.e., no known problems, even though the fit is incorrect. This can include unphysical fits due to the over-flexibility of PCA template linear combinations. This is particularly true for sky fibers which have a higher fraction of ZWARN==0 than would be expected from purely random fluctuations. Users should be especially cautious in any search for serendipitous targets in nominally blank sky fibers.

  • The Redrock galaxy fits extend to redshift z=1.7, though the range 1.6 < z < 1.63 is only constrained by the [O II] doublet in the midst of significant sky background and 1.63 < z < 1.7 has no major emission line coverage. Thus 1.6 < z < 1.7 is particularly susceptible to unphysical fits. This was the motivation for the LSS catalogs to only consider galaxies with z<1.6.

  • For most of the tiles in Target Selection Validation (SV1) the proper-motion corrections were applied in Fiberassign when the tile was designed (which can differ from when a tile was observed). A consequence is that the (TARGET_RA, TARGET_DEC, and REF_EPOCH) values are altered to have a REF_EPOCH of the date that the tile was designed, which makes them differ from the input photometric column values. The information is correct and consistent with the photometry, however.