SPARCL Release Notes

Authentication-B

Date of release: September 25, 2024

Added

• DESI-DR1 personality

Fixed

• Token expiration
• Prevent session caching

Authentication-A

Date of release: July 9, 2024

Added

• User authentication using NOIRLab's SSO
• Authorization such that specific data sets can be accessed only by authenticated users

Changed

• Data set name SDSS-DR17 to SDSS-DR17-test (for auth testing purposes)

<2023-06-29 Thu>

Added

• Support restricted (non-public) Data Sets. For such Data Sets a user can only get data (spectra and/or metadata) if their user-id has been granted read access to the Data Set and they are logged in.

Beta-4

Date of release: May 21, 2023

Added

• [client] Align spectra across records (by wavelength)
• Add links to Survey team websites for Dataset provenance and file info
• Add link to how-to Notebook on homepage

Fixed

• User Manual and About buttons on homepage
• count=Y now works on find

Changed

• Update fields descriptions and units on SPARCL website

Beta-3

Date of release: April 24, 2023

Added

• Error message from client.retrieve() when trying to retrieve more than 24,000 records at once
• Acknowledgment page to SPARCL homepage
• More descriptive SPARCL homepage

Deprecated

• Removed dataset_list from client.find()

Fixed

• SDSS logo not showing up on SPARCL homepage

Changed

• Changed id to sparcl_id
• SPARCL User Manual to include DESI-EDR information

Beta-2

Date of release: November 30, 2022

Added

• Link to SPARCL User Manual
• Limit maximum connect_timeout (3.1 seconds) and read_timeout (90 minutes) that can be used in the SparclClient() constructor.
• List all fields associated with a data set when clicking on the data set on homepage.

Deprecated

• LOCATION field in DESI-EDR
• PLATEID field in SDSS-DR16 and BOSS-DR16
• TARGETID AUX field in DESI-EDR

Changed

• Changed Data Set name "DESI-edr" to "DESI-EDR".

MVP

Date of release: September 27, 2022

Added

• Message of The Day (MOTD) for SPARCL homepage
• Footer to SPARCL homepage with AURA, CSDC, NSF, NOIRLab logos
• Release Notes to SPARCL homepage

Fixed

• Updated docstrings
• When ids from different data sets are passed to client.retrieve() and only one data set is stated in dataset_list, it now returns just the records corresponding to the stated data set and gives a warning about the others.

Changed

• Allow default outfields for client.find()
• .clear() now also resets .info and .records

Beta-1

Date of release: August 16, 2022

Added

• Units and descriptions for each field in http://astrosparcl.datalab.noirlab.edu:8000/sparc/sfc/

Deprecated

• client.orig_field()
• client.client_field()

Fixed

• Error when no INCLUDE fields
• Incorrect return for find() when specifying only one value

Changed

• Refine UI/UX for Found and Results classes

Alpha-5

Date of release: June 28, 2022

Added

• Data Discovery is accessible via client.find()
• Retrieve by id (uuid) using client.retrieve()
• "limit" parameter to find and retrieve to limit the number of records returned
• SPARCL homepage has links to Fields and Categoricals
• More informative server connection error message

Deprecated

• Specid is no longer required to be unique
• "rtype" from client args and docs
• Provenance tables and attributes

Changed

• Retrieve by specid using client.retrieve_by_specid()

Alpha-4

Date of release: April 6, 2022

Added

• Data Discovery, accessible via client.find()
• Retrieval by UUID, client.uuid_retrieve(), in addition to by specid

Fixed

• Incorrect Exception raised in client.uuid_retrieve
• Exception not raised when invalid field name passed to client.find()

Changed

• The results returned by client are now put into a Results object

Alpha-3

Date of release: December 7, 2021

Added

• Ability to convert records to data types: json, numpy, pandas, spectrum1d. Strictly Client-side functionality
• Miscellaneous client convenience functions: fields_available(records), record_examples(records), get_metadata(records), rename_fields(rename_dict, records), client.get_field_names(structure), client.orig_field(structure, client_name), client.client_field(structure, orig_name), client.sample_specids(samples=5, structure=None, random=True), client.version(self), client.clientversion(self), client.missing_specids(specid_list, countOnly=False, verbose=False), client.retrieve(...), client.sample_records(count, structure=None), client.normalize_field_names(recs)
• Swagger API page to SPARCL homepage

Fixed

• Updated docstrings
• Sample retrieved with web service not found by .retrieve()
• Client gives bogus error message when Server/Client versions mismatch
• BadPath error when trying to view the record structure for DESI-denali using .show_record_structure()

Changed

• Allow client to specify sid_list without dr for case where there is no ambiguity
• "sid" to "specid" in every user-visible place
• .retrieve() "include" parameter to accept a list of field names instead of a dictionary

Alpha-2

Date of release: August 6, 2021

Added

• Given list of SIDs return ones that are NOT in the database
• Allowing users to ask for selected portions of stored spectra info in the face of DR variability
• A SPARCL homepage that shows a summary view of records ingested into the SPARC database
• "include" parameter to retrieve specific columns and specify an alias for these columns
• Generalized extraction selection

Deprecated

• "columns" parameter is .retrieve() call
• SDSS specific extraction code

Fixed

• .sample_sids() always returns SDSS-DR16, no matter what "dr" parameter is set to
• "samples" parameter in .sample_sids() accepts scientific notation

Changed

• The default is to retrieve all core columns and aux fields stored in 'spectra'. The core columns are: 'specid', 'filename', 'dirpath', 'ra_center', 'dec_center', 'plateid', 'mjd', 'fiberid', 'data_release_id', 'updated', 'spectra'. The content and structure of ‘spectra’ is different for each ingested data release
• The .retrieve() function returns a list of a record's data or metadata, rather than a dict object with "status" and "records"
• Accessing record data is less complicated and less bracket-y
• "structure" instead of "dr" to identify currently ingested data sets
• Warning message when trying to retrieve SIDs not stored in the SPARC database (previously had to access "status" to see this message)

Alpha-1

Date of release: July 6, 2021

Fixed

• Specobjids do not return content when passed to retrieve().