This problem hasn't occurred to me because I always use the mouse for camera movement. Maybe that's a workaround for you, too?