pipeline.hpc

pipeline.hpc

Classes for working with different HPC clusters.

Use get_hpc to retrieve an HPC Cluster instance, which contains default attributes for supported clusters.

Classes

Name Description
Biowulf The Biowulf cluster – child of Cluster
Cluster Base class for an HPC cluster - evaluates to None
FRCE The FRCE cluster – child of Cluster

Biowulf

pipeline.hpc.Biowulf()

The Biowulf cluster – child of Cluster

Attributes

Name Type Description
name str The name of the cluster.
modules dict A dictionary mapping module names to their corresponding commands.
singularity_sif_dir str The directory path for Singularity SIF files.
env_vars str A string representing the environment variables to be set on the cluster.

Cluster

pipeline.hpc.Cluster()

Base class for an HPC cluster - evaluates to None

Attributes

Name Type Description
name str The name of the cluster.
modules dict A dictionary containing the modules installed on the cluster. The keys are the module names and the values are the corresponding versions.
singularity_sif_dir str The directory where Singularity SIF files are stored.
env_vars str A string representing the environment variables to be set on the cluster.

FRCE

pipeline.hpc.FRCE()

The FRCE cluster – child of Cluster

Attributes

Name Type Description
name str The name of the cluster.
modules dict A dictionary mapping module names to their corresponding commands.
singularity_sif_dir str The directory path for Singularity SIF files.
env_vars str A string representing the environment variables to be set on the cluster.

Functions

Name Description
get_hpc Returns an instance of the High-Performance Computing (HPC) cluster based on the specified HPC name.
get_hpcname Get the HPC name using scontrol
is_loaded Check whether a module is loaded
list_modules Get the list of loaded modules using module list
parse_modules Parse the output of module list to extract module names and versions
scontrol_show Run scontrol show config and parse the output as a dictionary

get_hpc

pipeline.hpc.get_hpc(debug=False)

Returns an instance of the High-Performance Computing (HPC) cluster based on the specified HPC name.

If the HPC is not known or supported, an instance of the base Cluster class is returned.

Parameters

Name Type Description Default
debug bool If True, uses debug as the HPC name. Defaults to False. False

Returns

Name Type Description
cluster Cluster An instance of the HPC cluster.

See Also

~ccbr_tools.pipeline.hpc.Cluster.create_hpc: The base class for HPC clusters.

Examples

>>> get_hpc()
>>> get_hpc(debug=True)

get_hpcname

pipeline.hpc.get_hpcname()

Get the HPC name using scontrol

Returns

Name Type Description
hpcname str The HPC name (biowulf, frce, or an empty string)

is_loaded

pipeline.hpc.is_loaded(module='ccbrpipeliner')

Check whether a module is loaded

Parameters

Name Type Description Default
module str The name of the module to check (default: “ccbrpipeliner”) 'ccbrpipeliner'

Returns

Name Type Description
is_loaded bool True if the module is loaded, False otherwise

list_modules

pipeline.hpc.list_modules()

Get the list of loaded modules using module list

Returns

Name Type Description
loaded_modules str The output of module list

parse_modules

pipeline.hpc.parse_modules(ml_output)

Parse the output of module list to extract module names and versions Args: ml_output (str): The output of module list Returns: modules (dict): A dictionary containing module names and their versions Example: >>> ml_output = “1) module_name/version” >>> parse_modules(ml_output) {‘module_name’: ‘version’} >>> parse_modules(list_modules()) {‘module_name’: ‘version’, …}

scontrol_show

pipeline.hpc.scontrol_show()

Run scontrol show config and parse the output as a dictionary

Returns

Name Type Description
scontrol_dict dict dictionary containing the output of scontrol show config