Simple predator-prey type models have brought much insight into the dynamics of both nonspecific and antigen-specific immune responses. However, until now most attention has been focused on examining how the dynamics of interactions between the parasite and the immune system depends on the nature of the function describing the rate of activation or proliferation of immune cells in response to the parasite. In this paper we focus on the term describing the killing of the parasite by cell-mediated immune responses. This term has previously been assumed to be a simple mass-action term dependent solely on the product of the densities of the parasite and the immune cells and does not take into account a handling time (which we define as the time of interaction between an immune cell and its target, during which the immune cell cannot interact with and/or destroy additional targets). We show how the handling time (i) can be incorporated into simple models of nonspecific and specific immunity and (ii) how it affects the dynamics of both nonspecific and antigen-specific immune responses, and in particular the ability of the immune response to control the infection.