nf-core/configs: Mahuika HPC Configuration
nf-core pipelines have been successfully run on the Mahuika HPC.
To run an nf-core pipeline on Mahuika, run the pipeline with -profile mahuika. This will download and launch the mahuika.config which has been pre-configured with a setup suitable for Mahuika. Using this profile, a Docker image containing all of the required software will be downloaded, and converted to an Apptainer image before execution of the pipeline.
Access to Mahuika
Please be aware that you will need to have a user account and be a member of an active project on Mahuika in order to use this infrastructure. See documentation for details regarding creating an account and applying for a project on Mahuika.
Launch an nf-core pipeline on Mahuika
Prerequisites
Before running the pipeline you will need to load Nextflow, which is a globally installed module on Mahuika. You can do this by running the commands below:
#See what versions are available
module avail Nextflow
#Load one of the versions
module load Nextflow/<version>Execution command
module load Nextflow/<version>
nextflow run <nf-core_pipeline>/main.nf \
-profile mahuika \
<additional flags>Specifying a partition
By default the mahuika profile will use both the Genoa and Milan partitions.
To specify a partition you can add its name as an additional profile.
For example, to run exclusively on the Genoa partition -profile mahuika,genoa.
Additional information
More information about running Nextflow on Mahuika can be found on the Nextflow page in the documentation.
Config file
// Mahuika nf-core configuration profile
params {
config_profile_description = 'Mahuika HPC profile provided by nf-core/configs'
config_profile_contact = 'Jennifer Reeve (@jen-reeve)'
config_profile_url = 'https://docs.nesi.org.nz'
max_cpus = 166
max_memory = 1024.GB
max_time = 504.h
}
env {
APPTAINER_TMPDIR="/home/${System.getenv('USER')}/.apptainer/tmp"
APPTAINER_CACHEDIR="/home/${System.getenv('USER')}/.apptainer/cache"
}
process {
resourceLimits = [
cpus: 166,
memory: 1.TB,
time: 504.h
]
cache = 'lenient'
stageInMode = 'symlink'
executor = 'slurm'
queue = 'genoa,milan'
errorStrategy = { task.exitStatus in ((130..145) + 104 + 175) ? 'retry' : 'finish' }
maxRetries = 2
withLabel: 'process_gpu' {
clusterOptions = '--gpus-per-node 1'
containerOptions = '--nv'
}
}
executor {
queueSize = 999
submitRateLimit = '60 min'
}
// Enable use of Apptainer to run containers
apptainer {
enabled = true
pullTimeout = '2h'
autoMounts = true
}
cleanup = true
profiles {
debug {
cleanup = false
}
genoa {
process {
resourceLimits = [
cpus: 166,
memory: 1.TB,
time: 504.h
]
cache = 'lenient'
stageInMode = 'symlink'
executor = 'slurm'
queue = 'genoa'
errorStrategy = { task.exitStatus in ((130..145) + 104 + 175) ? 'retry' : 'finish' }
maxRetries = 2
withLabel: 'process_gpu' {
clusterOptions = '--gpus-per-node 1'
containerOptions = '--nv'
}
}
params {
config_profile_description = 'Mahuika HPC genoa partition profile provided by nf-core/configs'
config_profile_contact = 'Jennifer Reeve (@jen-reeve)'
config_profile_url = 'https://docs.nesi.org.nz'
max_cpus = 166
max_memory = 1432.GB
max_time = 504.h
}
}
milan {
process {
resourceLimits = [
cpus: 166,
memory: 1.TB,
time: 504.h
]
cache = 'lenient'
stageInMode = 'symlink'
executor = 'slurm'
queue = 'milan'
errorStrategy = { task.exitStatus in ((130..145) + 104 + 175) ? 'retry' : 'finish' }
maxRetries = 2
withLabel: 'process_gpu' {
clusterOptions = '--gpus-per-node 1'
containerOptions = '--nv'
}
}
params {
config_profile_description = 'Mahuika HPC milan partition profile provided by nf-core/configs'
config_profile_contact = 'Jennifer Reeve (@jen-reeve)'
config_profile_url = 'https://docs.nesi.org.nz'
max_cpus = 126
max_memory = 1024.GB
max_time = 504.h
}
}
}