pipeline.hpc
pipeline.hpc
Classes for working with different HPC clusters.
Use get_hpc
to retrieve an HPC Cluster instance, which contains default attributes for supported clusters.
Classes
Name | Description |
---|---|
Biowulf | The Biowulf cluster – child of Cluster |
Cluster | Base class for an HPC cluster - evaluates to None |
FRCE | The FRCE cluster – child of Cluster |
Biowulf
pipeline.hpc.Biowulf()
The Biowulf cluster – child of Cluster
Attributes
Name | Type | Description |
---|---|---|
name | str | The name of the cluster. |
modules | dict | A dictionary mapping module names to their corresponding commands. |
singularity_sif_dir | str | The directory path for Singularity SIF files. |
env_vars | str | A string representing the environment variables to be set on the cluster. |
Cluster
pipeline.hpc.Cluster()
Base class for an HPC cluster - evaluates to None
Attributes
Name | Type | Description |
---|---|---|
name | str | The name of the cluster. |
modules | dict | A dictionary containing the modules installed on the cluster. The keys are the module names and the values are the corresponding versions. |
singularity_sif_dir | str | The directory where Singularity SIF files are stored. |
env_vars | str | A string representing the environment variables to be set on the cluster. |
FRCE
pipeline.hpc.FRCE()
The FRCE cluster – child of Cluster
Attributes
Name | Type | Description |
---|---|---|
name | str | The name of the cluster. |
modules | dict | A dictionary mapping module names to their corresponding commands. |
singularity_sif_dir | str | The directory path for Singularity SIF files. |
env_vars | str | A string representing the environment variables to be set on the cluster. |
Functions
Name | Description |
---|---|
get_hpc | Returns an instance of the High-Performance Computing (HPC) cluster based on the specified HPC name. |
get_hpcname | Get the HPC name using scontrol |
is_loaded | Check whether a module is loaded |
list_modules | Get the list of loaded modules using module list |
parse_modules | Parse the output of module list to extract module names and versions |
scontrol_show | Run scontrol show config and parse the output as a dictionary |
get_hpc
=False) pipeline.hpc.get_hpc(debug
Returns an instance of the High-Performance Computing (HPC) cluster based on the specified HPC name.
If the HPC is not known or supported, an instance of the base Cluster
class is returned.
Parameters
Name | Type | Description | Default |
---|---|---|---|
debug | bool | If True, uses debug as the HPC name. Defaults to False. |
False |
Returns
Name | Type | Description |
---|---|---|
cluster | Cluster | An instance of the HPC cluster. |
See Also
~ccbr_tools.pipeline.hpc.Cluster.create_hpc
: The base class for HPC clusters.
Examples
>>> get_hpc()
>>> get_hpc(debug=True)
get_hpcname
pipeline.hpc.get_hpcname()
Get the HPC name using scontrol
Returns
Name | Type | Description |
---|---|---|
hpcname | str | The HPC name (biowulf, frce, or an empty string) |
is_loaded
='ccbrpipeliner') pipeline.hpc.is_loaded(module
Check whether a module is loaded
Parameters
Name | Type | Description | Default |
---|---|---|---|
module | str | The name of the module to check (default: “ccbrpipeliner”) | 'ccbrpipeliner' |
Returns
Name | Type | Description |
---|---|---|
is_loaded | bool | True if the module is loaded, False otherwise |
list_modules
pipeline.hpc.list_modules()
Get the list of loaded modules using module list
Returns
Name | Type | Description |
---|---|---|
loaded_modules | str | The output of module list |
parse_modules
pipeline.hpc.parse_modules(ml_output)
Parse the output of module list
to extract module names and versions Args: ml_output (str): The output of module list
Returns: modules (dict): A dictionary containing module names and their versions Example: >>> ml_output = “1) module_name/version” >>> parse_modules(ml_output) {‘module_name’: ‘version’} >>> parse_modules(list_modules()) {‘module_name’: ‘version’, …}
scontrol_show
pipeline.hpc.scontrol_show()
Run scontrol show config
and parse the output as a dictionary
Returns
Name | Type | Description |
---|---|---|
scontrol_dict | dict | dictionary containing the output of scontrol show config |