Can you try removing the gpm package (and rebooting if necessary)? This might also require changing the mouse device in the XFree86 configuration to/dev/input/mice or something similar.
GPM has always given me sporadic problems, and if you're not using the mouse on the Linux console, you don't need it at all. I think most distributions (even GUI-oriented ones) install it by default anyway for some reason.