For beers like rye pale ales and rye IPAs, I like to go with 20% rye. It's enough to be noticeable, but not overwhelming.
Special B has these distinct "toffee" notes that make it stand out. A little of it goes a very long way. I have a friend who has a brewery in Wisconsin, and he uses it in his brown ale. It's definitely noticeable, in a nice way, but it some beers it would be over the top. I would go with no more than 5% of it, assuming the rest of the crystal malt in total is under 12% or so.
I've never enjoyed a rye stout, although I love other rye beers. Either rye malt or flaked rye could be used.