Why are elemental dopants for Silicon or Germanium usually chosen from group XIII or group XV?

When a pure semiconductor is mixed with a small amount of impurities i.e. elements having number of valence electrons slightly different, the concentration of charge carries changes dramatically. So to increase the number of a particular type of charge carriers the Silicon and Germanium crystals that are elements of group XIV, are doped with a small amount of the elements of groups XIII or XV.