I don't think that's too late. I planted rhizomes last year in April and they did very well indeed - especially for first year plantings. And you're in a slightly warmer zone than I am.
Either way, I'd go ahead and do it, even later if need be. It's an investment, and most advise is to plan on the first year providing little if any yield anyway...
you could consider getting a head-start by starting them off in pots:
order your rhizomes now. keep in fridge for a few weeks, then plant them in pots a few weeks before you're ready to put them in the ground. by the time planting day rolls around, you'll have plants that have already sprouted.