For me? OFF while listening
They can do no good on while in use
(that is why they made them so easy to remove and re-install)
Grilles are primarily for some small degree of protection and an aesthetic afterthought and I doubt that the engineers had them in mind when designing the system as for sound
That's not to say that MARKETING didn't have them in mind when it came to selling them though!
:
Some are better than others (less intrusive)
Anything you put in front of a driver will change the sound, whether that's a problem for you (a negative), or you can hear it, only you can decide
Fine for protection and keeping dust and dirt away from things when you're not playing music (if you enjoy their looks)
Besides, who need grilles when you can have this!:
http://www.ebay.com/itm/JBL-4343-Lac...-/261338179724