Pascal GTX 1080 Async Compute Explored

Last week, a report came out suggesting that Pascal may include improved asynchronous compute support. However, Nvidia also claimed asynchronous compute support with Maxwell but that proved to be less than optimal solution. From the leak of the GTX 1080 slide deck, we’re now able to glean a few more details about what Nvidia has meant about async compute support and how it has been improved with Pascal.

Async compute basically means working on a graphics workload at the same time as a compute one, making the most of the GPU’s resources. This can work on a number of levels, either at the GPU level or SM/CU level. Maxwell only worked at a GPU level, assigning each SM to either a graphics or compute task. Scheduling was done in the shader and in order add/switch tasks, the previous task had to finish or be preempted/stopped. Furthermore, Maxwell only had static partitioning, so graphics and compute tasks scheduling at the same time had to both finish and weren’t able to dynamically reallocate resources if one task finished first. This led to GCN leading when it came to async compute.

Pascal brings a number of changes. First off, the promised improved preemption has come. Pascal will be able to offer more fine-grained control, entering a new task between pixels or instructions. This will allow for better and likely faster preemption. The next change is dynamic load balancing. This allows Pascal to reallocate resources dedicated to either graphics or compute on the fly. This GPU level means that once a graphics/compute task is finished, the idle SMs can now be added to those working on another graphics/compute task, speeding up completion. This should allow for much better async compute performance compared to Maxwell.

Even with all of these additions, Pascal still won’t quite match GCN. GCN is able to run async compute at the SM/CU level, meaning each SM/CU can work on both graphics and compute at the same time, allowing even better efficiency. Nonetheless, Pascal is finally offering a solution with hardware scheduled async compute, bringing Nvidia closer to AMD. Either way, with both Nvidia and AMD working on async compute, developers are more likely to take notice and make sure our GPUs are fully utilized.