jobby

jobby

Display job information for past SLURM job IDs.

ABOUT

jobby is a command-line utility that collects and displays job metadata from SLURM-managed clusters. It supports retrieving job IDs from direct input, Snakemake logs, or Nextflow logs, and presents job status and resource usage in a standardized output format.

Why? jobby aims to simplify and unify the job-querying process by abstracting away cluster-specific tools and normalizing output into common formats. jobby will ensure consistent reporting from multiple CCBR Snakemake and Nextflow pipelines when embedded in onsuccess/onerror/oncomplete blocks.

FEATURES

  • Parses SLURM job IDs from CLI args, .nextflow.log, and snakemake.log.
  • Queries SLURM using sacct to gather job information such as state, runtime, CPU/memory usage, etc.
  • Converts time fields to seconds, memory fields to GB, and calculates CPU efficiency.
  • Supports multiple output formats: Markdown (default), TSV, JSON, and YAML.
  • Optionally include job log files and their contents for failed jobs (–outerr), or also for all jobs with –include-completed. These columns are never included when the output format is markdown.

USAGE

jobby <jobid1> [jobid2 ...] [--tsv|--json|--yaml]
jobby <jobid1>,<jobid2> [--tsv|--json|--yaml]
jobby snakemake.log [--tsv|--json|--yaml] [--outerr] [--include-completed]
jobby .nextflow.log [--tsv|--json|--yaml] [--outerr] [--include-completed]

DEPENDENCIES

  • Python 3.7+
  • pandas (required)
  • numpy (required)
  • PyYAML (optional, required only for –yaml output)

Examples

jobby 12345678 12345679
jobby snakemake.log --json
jobby .nextflow.log --yaml
jobby 12345678,12345679 --tsv
jobby .nextflow.log --outerr
jobby .nextflow.log --outerr --include-completed

Functions

Name Description
extract_jobids_from_file Extract SLURM job IDs from a Snakemake or Nextflow log file.
format_df Format the DataFrame for output based on the requested format.
jobby Processes a list of job IDs or a file containing job IDs to retrieve job information.
parse_mem_to_gb Convert SLURM memory strings like ‘4000M’, ‘4G’, ‘102400K’ to GB as float.
parse_time_to_seconds Convert SLURM time formats like ‘1-02:03:04’, ‘02:03:04’, ‘37:55.869’, or ‘55.869’ to seconds.
records_to_df Convert a list of job records to a pandas DataFrame.

extract_jobids_from_file

jobby.extract_jobids_from_file(filepath)

Extract SLURM job IDs from a Snakemake or Nextflow log file.

format_df

jobby.format_df(df, output_format)

Format the DataFrame for output based on the requested format.

jobby

jobby.jobby(
    args,
    include_out_err=False,
    include_completed=False,
    completed_state='COMPLETED',
    success_exit_code=0,
)

Processes a list of job IDs or a file containing job IDs to retrieve job information.

Parameters

Name Type Description Default
args list A list of job IDs or a single-element list containing a file path with job IDs. required
include_out_err bool Whether to include output and error file information. Defaults to False. False
include_completed bool Whether to include completed jobs in the results. Defaults to False. False
completed_state str The state string that indicates a job is completed. Defaults to “COMPLETED”. 'COMPLETED'
success_exit_code int The exit code that indicates a job was successful. Defaults to 0. 0

Returns

Name Type Description
dict A list of job records as dictionaries, or an empty dictionary if no jobs are found.

Raises

Name Type Description
TypeError If ‘args’ is not a list.

parse_mem_to_gb

jobby.parse_mem_to_gb(mem_str)

Convert SLURM memory strings like ‘4000M’, ‘4G’, ‘102400K’ to GB as float.

parse_time_to_seconds

jobby.parse_time_to_seconds(t)

Convert SLURM time formats like ‘1-02:03:04’, ‘02:03:04’, ‘37:55.869’, or ‘55.869’ to seconds.

records_to_df

jobby.records_to_df(records)

Convert a list of job records to a pandas DataFrame.