It's not a matter of having trouble responding to the interrupts. (You'd be in big trouble if you were getting more interrupts than you could process). The problem is that the ratio of time spent in the kernel versus time running your program is increased in the case of USB input devices, which has a deleterious effect on responsivity of the program if the program (such as a game) is aggressive enough with CPU usage to contend with the kernel for CPU time.
Score:2