Downloading data¶

Every file on an observation's Data page has its own download link, and that is the right tool for a frame or two. For a whole observation, or any slice of it, Skynet packages the files into zip archives for you: pick what you want, start the download, and come back for the links when the archives are ready.

Packaged downloads need a Skynet account. On a public observation, visitors who are not signed in can still download files one at a time.

Starting a download¶

Open the observation and choose Download data… from the header's more-actions menu, or press Download on the Data page. The Data page button starts from whatever filters the grid is showing, so a good habit is to filter until the grid shows exactly the files you want, then download that.

The dialog lets you narrow the selection:

What: results only (the file that fulfilled each sample: a processed frame for a single-frame task, the stack or mosaic for a multi-frame one), originals (the frames as captured), processed (every pipeline product: calibrated frames, stacks, mosaics), or originals and processed together. Calibration frames (bias, dark, flat) and diagnostics (pointing tests, rejected retakes) are separate switches. The Download results button on the Results page opens the dialog with results only pre-selected.
Roles: restrict to specific roles, or leave empty for all of them.
When: a shutter-time window. Enter times in your local zone; the selection is made on when each exposure started. For an occultation, this is how you grab just the event window.
Filename: a case-insensitive glob such as *-B-* or *-60s-*.
Archive size: how large each archive may grow (1, 2 or 4 GB; 2 GB by default).

The footer shows how many files match, how big they are, and how many archives they will be packaged into. Files that have not finished transferring from the telescope are not counted; they can be downloaded once they arrive.

While it packages¶

Your downloads appear in the Downloads strip at the top of the Data page, with a progress bar per export that updates while you keep the page open. You can cancel one while it is being packaged. A packaged download of a few hundred frames is usually ready within minutes; the strip shows when it is.

You do not have to keep the page open. When the archives are ready, or if packaging fails, Skynet sends you a notification that opens the Data page: in the bell at the top of the site and, by default, by email as well. These arrive under My observations in your notification preferences, so turning email off there turns it off for downloads too. Nobody else who can see the observation is told about your download.

The archives¶

An export is split into archives of at most the size you chose. Each archive is a complete, ordinary zip that opens on its own; files never straddle archives, and the archives are filled in exposure order, so part-01.zip holds the earliest frames. Inside each archive:

results/, originals/, processed/, calibration/, diagnostics/: the files, under their Skynet file names. A result's deliverable goes under results/ even when it is also a processed frame; processed/ holds the other pipeline products.
MANIFEST.csv: every file in the whole export, with the archive it is in, its role, processing flags, exposure start and duration, size, and a SHA-256 checksum for the files in that archive.
observation.json: the observation's spec and the filters that built the export.
README.txt: how the archives relate and when the links expire.

FITS files are packaged exactly as Skynet stores them, which for science frames means tile-compressed FITS. Astropy, DS9, PixInsight and most current tools read these directly. The zip itself does not compress further, so packaging is quick and the archive size is a reliable guide to the download size.

The archives are kept for seven days after they are ready, then removed. The export stays in the strip with what it contained, and Regenerate rebuilds it from the same filters (re-evaluated, so files added or removed since are reflected). Delete now removes the archives early. The archives count toward your storage usage while they exist.

For many archives, Copy download list puts every link on the clipboard, one per line, for a download manager or wget -i.

Scripted access¶

The export is the right tool for a one-off bulk pull. A script that follows an observation as it runs is better served by the per-file API, which exists already:

GET /v1/observations/{id}/assets?createdAfter=<last seen>&sortBy=created_on lists the files that landed since the last poll.
POST /v1/files/data/presign with up to 200 file ids returns short-lived direct links to the bytes.

Exports are available to scripts too: POST /v1/observations/{id}/exports with the same filters the dialog uses, GET …/exports/{uid} to poll, then GET /v1/files/{fileId}/data?download=1 for each part. See the developer docs for authentication and token scopes (observations:read to list and poll, observations:write to create).