For mild street engines, dyno testing has proven that smaller-diameter primary pipes (such as 1-5/8 inch for a mild 350 Chevy) tend to build more torque at more useful low and midrange engine speeds. Larger-diameter headers tend to kill low-end torque in favor of higher-rpm power increases....
It's all what you're shooting for ....