Release Notes: v3.3.0¶
Highlights¶
projark: configurable staging location with a free-space preflight¶
Large deposits could fill /scratch/$USER: the tarball was created, but split failed with No space left on device. Splitting writes the chunks while the full tar still exists, so peak usage is about 2× the tar size. Until this release nothing checked free space, and HPC-DME collections were created before the failure.
projark deposit -f /data/CCBR/projects/CCBR-1234 -p 1234 --staging-dir /data/$USER/staging
export PROJARK_STAGING_DIR=/data/$USER/staging # default for deposit and retrieve
projark retrieve -p 1234 --staging-dir /data/$USER/staging
--staging-dir DIR(orPROJARK_STAGING_DIR) forprojark depositandprojark retrieve. The default is still/scratch/$USER, and files go in<DIR>/CCBR-<projectnumber>/<datatype>/. Forretrieve,--staging-dirand-f/--folderare mutually exclusive.- Free-space preflight:
projark depositestimates the tar and.filelistsizes and aborts before creating any HPC-DME collection if the staging filesystem lacks space. It requires about 2× the tar size when the tar will be split (else 1×), plus the.filelistand a margin.projark retrievechecks free space before downloading, including merged tars with--unsplit. - Safer staging:
depositrefuses a staging folder that is, contains, or is inside the source folder.- Only staging folders carrying the
.projark_stagingmarker are ever deleted (on rerun or cleanup). A marked folder left by a previous run counts as reclaimable space. retrievenever writes the marker, and refuses to download into a marked staging folder or any folder inside one.
See projark deposit, projark retrieve, Operational Guidance and Troubleshooting. (#72, #73, @kopardev)
Behavior Changes¶
- An existing unmarked
/scratch/$USER/CCBR-<projectnumber>folder (for example, left by an older version with-k) now makesprojark depositabort instead of being silently deleted. Check its contents and remove it manually. - The free-space check cannot see per-user or per-group quotas (e.g.
/data/$USER,/data/CCBR). Usecheckquotabefore large deposits.
Documentation¶
- Refreshed the
parkit --helpoutput inREADME.mdand listedlsin the parkit overview. (#71, @kopardev) - Fixed the docs home page showing a stale "In This Version (
v3.1.0)" in every versioned build, and added the v3.2.0 release notes. (#74, @kopardev)