Alex Rivera | Logout

How do CUDA blocks/warps/threads map onto CUDA cores?

Asked 2012-05-05T09:58:02.750
184

I have been using CUDA for a few weeks, but I have some doubts about the allocation of blocks/warps/thread. I am studying the architecture from a didactic point of view (university project), so reaching peak performance is not my concern.

First of all, I would like to understand if I got these facts straight:

  1. The programmer writes a kernel, and organize its execution in a grid of thread blocks.

  2. Each block is assigned to a Streaming Multiprocessor (SM). Once assigned it cannot migrate to another SM.

  3. Each SM splits its own blocks into Warps (currently with a maximum size of 32 threads). All the threads in a warp executes concurrently on the resources of the SM.

  4. The actual execution of a thread is performed by the CUDA Cores contained in the SM. There is no specific mapping between threads and cores.

  5. If a warp contains 20 thread, but currently there are only 16 cores available, the warp will not run.

  6. On the other hand if a block contains 48 threads, it will be split into 2 warps and they will execute in parallel provided that enough memory is available.

  7. If a thread starts on a core, then it is stalled for memory access or for a long floating point operation, its execution could resume on a different core.

Are they correct?

Now, I have a GeForce 560 Ti so according to the specifications it is equipped with 8 SM, each containing 48 CUDA cores (384 cores in total).

My goal is to make sure that every core of the architecture executes the SAME instructions. Assuming that my code will not require more register than the ones available in each SM, I imagined different approaches:

  1. I create 8 blocks of 48 threads each, so that each SM has 1 block to execute. In this case will the 48 threads execute in parallel in the SM (exploiting all the 48 cores available for them)?

  2. Is t

cuda gpgpu nvidia warp-scheduler
Edit
Report

1 Answer

11

"E. If a warp contains 20 threads, but currently there are only 16 cores available, the warp will not run."

is incorrect. You are confusing cores in their usual sense (also used in CPUs) - the number of "multiprocessors" in a GPU, with cores in nVIDIA marketing speak ("our card has thousands of CUDA cores").

Cuda core (so answer) is a hardware concept and thread is a software concept. Even with only 16 cores available, you can still run 32 threads. However, you may need 2 clock cycles to run them with only 16 hardware cores.

The CUDA core count represents the total number of single precision floating point or integer thread instructions that can be executed per cycle

warp scheduler is responsible to find cores to run instructions (so answer)

A warp is a logical assembly of 32 threads of execution. To execute a single instruction from a single warp, the warp scheduler must usually schedule 32 execution units (or "cores", although the definition of a "core" is somewhat loose).

A warp itself can only be scheduled on a SM (multiprocessor, or streaming multiprocessor), and can run up to 32 threads at the same time (depending on cores in SM); it cannot use more than a SM.

The number "48 warps" is the maximum number of active warps (warps which may be chosen to be scheduled for work in the next cycle, at any given cycle) per multiprocessor, on NVIDIA GPUs with Compute Capability 2.x; and this number corresponds to 1536 = 48 x 32 threads.

Answer based on this webinar

answered 2012-10-25T12:02:40.567

Your Answer