I'd go for it ! But keep in mind that the 604's are point source and they aren't real good off -axis.They were designed as studio monitors which for all intense of purposes they are or should be at ear level or pointing at your ears.The problem is the small sectoral horn on the 604.I have a pair in 612C cabinets and when they are placed at the right height,they sound Fabulous.Nothing sounds better than the LF and HF coming from the same center point, hence (Point Source Speaker) or Duplex.
Now for your other question you can make your cabinet as big as you want.Bigger is always better WRT the bass response "No Such Thing As A Too Big Of A Cabinet" as long as it is properly damped ! Cabinets are only for the LF anyway.The bigger the cabinet the better the bass output/response.Now if you could build some really strong cabinets and put them (604's)on the bottom with the 19's on top you will like the combo much better.But bring a few friends they aren't lite.
Here's a link for a Cabinet volume calculator from "Home Theater Shack"
http://www.hometheatershack.com/foru...me-calculator/
The 604 should always be at or pointing to your ears and Never above or below them ! Again the 604's small horn is terrible off -axis.
BTW: I have a pair of Cornwall's at the moment on top of my 19's and i have used the 612's with a 416 and a 1005/288 tar filled combo & tried the 604-8G's on top of the 19's too, and enjoyed them immensely.The Horns need some paint,but they sound good.The 604 wasn't as good so high above my ears though.When your space is limited, do what you have to, to achieve your goals.
Kind Regards ~ AB