Charge doping versus impurity scattering in chemically substituted iron pnictides