1. Driving the amp into clipping!
2. Running at high power levels above the wattage rating of the speaker for a long period of time.
As long as you never get into ether condition 1 or 2 your OK!...