Batching is the practice of grouping multiple discrete operations, requests or data items so that they are processed together in a single pass rather than individually. It amortises fixed per-operation overheads such as draw calls, network round trips or kernel launches across many items, improving throughput at the cost of added latency for the items that wait to be grouped. Batching appears throughout computing, from GPU rendering (combining draw calls to reduce state changes) to machine learning inference (grouping requests to maximise accelerator utilisation).
In Plain Terms
- Gathering lots of small jobs and running them together in one go rather than one at a time. It is far more efficient overall — like posting a stack of letters in a single trip — though each item may wait a little longer while its group fills up.
Content
- Batching is the practice of grouping multiple discrete operations, requests or data items so that they are processed together in a single pass rather than individually. It amortises fixed per-operation overheads such as draw calls, network round trips or kernel launches across many items, improving throughput at the cost of added latency for the items that wait to be grouped. Batching appears throughout computing, from GPU rendering (combining draw calls to reduce state changes) to machine learning inference (grouping requests to maximise accelerator utilisation).