Yes they need to stay there for long, at least as long as you do not have another protection system, and are confident about the whole system (amp DC, human errors on filtering or cabling, turn on/off noises of amps and other equipements, etc).
You need such caps for the tweeter and also for any compression driver tou will use (but you would need a higher value caps for these, because they will be cut lower in frequency).
For the voltage 100V would be more than enough, but bigger than that would not hurt either.