Why does a gunman get a jerk on firing a bullet?

This is because when the bullet is fired from the gun, it moves forwards and exerts a force on the gun; an equal amount of force is exerted on the bullet by the gun. This is because according to third law of motion, when a force is applied on a body, the body exerts an equal and opposite force on another body.