Hyper-threading/Multi-threaded Jobs

Applications that support the use of Hyper-threading and Multithreading benefit from the abundance of cores available in modern CPUs. With multiple cores, the performance of the application can be improved significantly in comparison with single-core application. The differences between Hyper-threading and Multithreading are discussed below.

What’s Multithreading?

Multi-threading allows a program or an application to use multiple cores for processing. For example, consider one of Discovery’s (discovery-g[7]) two Intel CPUs (Intel Xeon Gold 5120 2.2G) with a total of 28 physical cores. Each of its cores has 2 threads which makes it a total of 56 threads.

What’s Hyper-threading?

Hyper-threading basically is a proprietary Intel technology which applies the idea of multi-threading but allows a single physical core to split into virtual and logical cores. Therefore, the operating system would see the single core as a dual core.

Hyper-threading is considered effective when programs are suitably optimized for muli-threading.

In as much as Hyper-threading can be beneficial for some programs, if your program isn’t tuned to support hyper-threading, it can degrade performance. It’s recommended that you run a small test job so as to figure out what works best for you.

Hyper-threading & Multiprocessing on Discovery

By default, Slurm schedules Multithreaded jobs using hyper-threads (Virtual or logical cores). Hyper-threading has been enabled across the Discovery cluster and this means that a CPU is equal to a hyper-thread.

For example the command below helps you to decide if your nodes have more than one thread per core (Focus on the value of ThreadsPerCore).

scontrol show discovery-g7

Below is a list of basic Slurm directives that are used to enable the use of multiprocessing and hyper-threading on Discovery.

Directives Value (example) Description

--cpus-per-task

2

Number of CPUs required per task

--ntasks-per-node

4

Number of tasks to call on each node

--ntasks-per-core

2

Number of tasks to call on each core

--hint

multithread or nomultithread

Use or don’t use extra threads with In-Core Multithreading

To enable/disable hyper-threading for your program, add either of the directives specified below to your Slurm script.

#SBATCH --hint=multithread   ## Allow the use of in-core multi-threading
#SBATCH --hint=nomultithread ## Avoid the use of in-core multi-threading

Scheduling Multithread Jobs

Multithreaded Job

The example below advices Slurm to assign multiple threads from each core to your job.

...
#SBATCH --ntasks=4
#SBATCH --mem=10000G
#SBATCH --hint=multithread   ## Allow the use of in-core multi-threading
...

Non Multithreaded Job

The example below advices Slurm to assign one thread from each core to your job.

...
#SBATCH --ntasks=4
#SBATCH --mem=10000G
#SBATCH --hint=nomultithread   ## Avoid the use of in-core multi-threading
...