pipeline.hpc
pipeline.hpc
Classes for working with different HPC clusters.
Use get_hpc to retrieve an HPC Cluster instance, which contains default attributes for supported clusters.
Classes
| Name | Description |
|---|---|
| Biowulf | The Biowulf cluster – child of Cluster |
| Cluster | Base class for an HPC cluster - evaluates to None |
| FRCE | The FRCE cluster – child of Cluster |
Biowulf
pipeline.hpc.Biowulf()The Biowulf cluster – child of Cluster
Attributes
| Name | Type | Description |
|---|---|---|
| name | str | The name of the cluster. |
| modules | dict | A dictionary mapping module names to their corresponding commands. |
| singularity_sif_dir | str | The directory path for Singularity SIF files. |
| env_vars | str | A string representing the environment variables to be set on the cluster. |
Cluster
pipeline.hpc.Cluster()Base class for an HPC cluster - evaluates to None
Attributes
| Name | Type | Description |
|---|---|---|
| name | str | The name of the cluster. |
| modules | dict | A dictionary containing the modules installed on the cluster. The keys are the module names and the values are the corresponding versions. |
| singularity_sif_dir | str | The directory where Singularity SIF files are stored. |
| env_vars | str | A string representing the environment variables to be set on the cluster. |
FRCE
pipeline.hpc.FRCE()The FRCE cluster – child of Cluster
Attributes
| Name | Type | Description |
|---|---|---|
| name | str | The name of the cluster. |
| modules | dict | A dictionary mapping module names to their corresponding commands. |
| singularity_sif_dir | str | The directory path for Singularity SIF files. |
| env_vars | str | A string representing the environment variables to be set on the cluster. |
Functions
| Name | Description |
|---|---|
| get_hpc | Returns an instance of the High-Performance Computing (HPC) cluster based on the specified HPC name. |
| get_hpcname | Get the HPC name using scontrol |
| is_loaded | Check whether a module is loaded |
| list_modules | Get the list of loaded modules using module list |
| parse_modules | Parse the output of module list to extract module names and versions |
| scontrol_show | Run scontrol show config and parse the output as a dictionary |
get_hpc
pipeline.hpc.get_hpc(debug=False)Returns an instance of the High-Performance Computing (HPC) cluster based on the specified HPC name.
If the HPC is not known or supported, an instance of the base Cluster class is returned.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| debug | bool | If True, uses debug as the HPC name. Defaults to False. |
False |
Returns
| Name | Type | Description |
|---|---|---|
| cluster | Cluster | An instance of the HPC cluster. |
See Also
~ccbr_tools.pipeline.hpc.Cluster.create_hpc: The base class for HPC clusters.
Examples
>>> get_hpc()
>>> get_hpc(debug=True)get_hpcname
pipeline.hpc.get_hpcname()Get the HPC name using scontrol
Returns
| Name | Type | Description |
|---|---|---|
| hpcname | str | The HPC name (biowulf, frce, or an empty string) |
is_loaded
pipeline.hpc.is_loaded(module='ccbrpipeliner')Check whether a module is loaded
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| module | str | The name of the module to check (default: “ccbrpipeliner”) | 'ccbrpipeliner' |
Returns
| Name | Type | Description |
|---|---|---|
| is_loaded | bool | True if the module is loaded, False otherwise |
list_modules
pipeline.hpc.list_modules()Get the list of loaded modules using module list
Returns
| Name | Type | Description |
|---|---|---|
| loaded_modules | str | The output of module list |
parse_modules
pipeline.hpc.parse_modules(ml_output)Parse the output of module list to extract module names and versions Args: ml_output (str): The output of module list Returns: modules (dict): A dictionary containing module names and their versions Example: >>> ml_output = “1) module_name/version” >>> parse_modules(ml_output) {‘module_name’: ‘version’} >>> parse_modules(list_modules()) {‘module_name’: ‘version’, …}
scontrol_show
pipeline.hpc.scontrol_show()Run scontrol show config and parse the output as a dictionary
Returns
| Name | Type | Description |
|---|---|---|
| scontrol_dict | dict | dictionary containing the output of scontrol show config |