Just go to your favorite liquor store and try and find some that are moderately priced. Prices vary, so it's hard to give you any definite answers. For me, SN Harvest Ales are only $4.99 for 24 oz. bottles.
As for the caps, you should be fine with most bombers. The only place you may run into a fit-up issue is if you are using bottles that come with Belgian-style cork enclosures. These are typically (but not always) made to accept the larger crown cap, and not the kind you typically buy from your LHBS.