Put the other 3 back in just far enough to stay put in the T-nuts if you want to go that route
Never been a fan or had to use the drop the cabinet approach, not good for the cabinets, not good for the drivers
A strong tapered pick with a 90 degree bend on the end is the most aggressive tool I have ever had to use on any JBL with a stuck woofer (inserted through the frame's screw hole, NOT between the frame and the baffle), that or wooden shims, small blocks of wood or dowels (which can be carefully used between the frame and the baffle) and a mallet or small hammer (to hit the wood blocks, NOT the drivers )
I also save the dense plastic knock outs from the handles on gallon milk jugs too for these sorts of jobs, an ideal none destructive wedge material
Does take patience sometimes
Good luck
Joe