Indeed - I made a fiberglass screen "shield" around the 2 woofers to keep the fill from getting into the speaker proper.
Actually, I switched from polyfill to a measured amount of raw lambswool, based on feedback from my audio guru,
who explained he'd had significantly better luck with lambswool.