jobby

jobby

Display job information for past SLURM job IDs.

ABOUT

jobby is a command-line utility that collects and displays job metadata from SLURM-managed clusters. It supports retrieving job IDs from direct input, Snakemake logs, or Nextflow logs, and presents job status and resource usage in a standardized output format.

Why? jobby aims to simplify and unify the job-querying process by abstracting away cluster-specific tools and normalizing output into common formats. jobby will ensure consistent reporting from multiple CCBR Snakemake and Nextflow pipelines when embedded in onsuccess/onerror/oncomplete blocks.

FEATURES

  • Parses SLURM job IDs from CLI args, .nextflow.log, and snakemake.log.
  • Queries SLURM using sacct to gather job information such as state, runtime, CPU/memory usage, etc.
  • Converts time fields to seconds, memory fields to GB, and calculates CPU efficiency.
  • Supports multiple output formats: Markdown (default), TSV, JSON, and YAML.
  • Optionally include job log files and their contents for failed jobs (–outerr), or also for all jobs with –include-completed. These columns are never included when the output format is markdown.

USAGE

jobby <jobid1> [jobid2 ...] [--tsv|--json|--yaml]
jobby <jobid1>,<jobid2> [--tsv|--json|--yaml]
jobby snakemake.log [--tsv|--json|--yaml] [--outerr] [--include-completed]
jobby .nextflow.log [--tsv|--json|--yaml] [--outerr] [--include-completed]

DEPENDENCIES

  • Python 3.7+
  • pandas (required)
  • numpy (required)
  • PyYAML (optional, required only for –yaml output)

Examples

jobby 12345678 12345679
jobby snakemake.log --json
jobby .nextflow.log --yaml
jobby 12345678,12345679 --tsv
jobby .nextflow.log --outerr
jobby .nextflow.log --outerr --include-completed

Functions

Name Description
extract_jobids_from_file Extract SLURM job IDs from a Snakemake or Nextflow log file.
format_df Format the DataFrame for output based on the requested format.
jobby Processes a list of job IDs or a file containing job IDs to retrieve job information.
parse_mem_to_gb Convert SLURM memory strings like ‘4000M’, ‘4G’, ‘102400K’ to GB as float.
parse_time_to_seconds Convert SLURM time formats like ‘1-02:03:04’, ‘02:03:04’, ‘37:55.869’, or ‘55.869’ to seconds.
read_slurm_log Reads the contents of a SLURM log file, escapes all special characters using JSON encoding,
records_to_df Convert a list of job records to a pandas DataFrame.

extract_jobids_from_file

jobby.extract_jobids_from_file(filepath)

Extract SLURM job IDs from a Snakemake or Nextflow log file.

format_df

jobby.format_df(df, output_format)

Format the DataFrame for output based on the requested format.

jobby

jobby.jobby(
    args,
    include_out_err=False,
    include_completed=False,
    completed_state='COMPLETED',
    success_exit_code=0,
)

Processes a list of job IDs or a file containing job IDs to retrieve job information.

Parameters

Name Type Description Default
args list A list of job IDs or a single-element list containing a file path with job IDs. required
include_out_err bool Whether to include output and error file information. Defaults to False. False
include_completed bool Whether to include completed jobs in the results. Defaults to False. False
completed_state str The state string that indicates a job is completed. Defaults to “COMPLETED”. 'COMPLETED'
success_exit_code int The exit code that indicates a job was successful. Defaults to 0. 0

Returns

Name Type Description
dict A list of job records as dictionaries, or an empty dictionary if no jobs are found.

Raises

Name Type Description
TypeError If ‘args’ is not a list.

parse_mem_to_gb

jobby.parse_mem_to_gb(mem_str)

Convert SLURM memory strings like ‘4000M’, ‘4G’, ‘102400K’ to GB as float.

parse_time_to_seconds

jobby.parse_time_to_seconds(t)

Convert SLURM time formats like ‘1-02:03:04’, ‘02:03:04’, ‘37:55.869’, or ‘55.869’ to seconds.

read_slurm_log

jobby.read_slurm_log(filepath)

Reads the contents of a SLURM log file, escapes all special characters using JSON encoding, and returns the escaped string without the outer quotes.

Parameters

Name Type Description Default
filepath str Path to the SLURM log file. required

Returns

Name Type Description
str The contents of the file with special characters JSON-escaped and outer quotes removed.

records_to_df

jobby.records_to_df(records)

Convert a list of job records to a pandas DataFrame.