Thats exactly what we're doing. But getting the cold brew right is the key. Some beans are high in acid, others way too bitter. So getting the cold brew right is the key. Beans also have oils that contribute to the bitterness depending on the roast depth.
The fun thing about meads, like beer, you can experiment until you find the right combo for your liking.