If you saw the spectra for two speakers and one was obviously smoother than the other, wouldn't you want to go with the visibly smoother pair even if you didn't think you could hear a difference? I know that I would, and do.
Remember that there is no way to A/B/X something like this unless you created a speaker that could instantly shift to two different drivers mounted in the two positions. So sometimes I have to trust my eyes, rather than my ears, to optimize what's there.
You can optimize a car for the 1/4 mile, but unless you have timing equipment to make the most of it, your butt isn't going to feel a couple 10ths of a second difference. Same thing.