No you haven't ruined it. It just won't attenuate as much as if you used brewer's yeast. It might taste "bready," as well. It will still be drinkable, but you may or may not like the finished product.
The basic brewing guys did a video on switching yeasts.
September 28, 2007 - Trading Places: Beer and Bread Yeast
James makes a Simcoe Ale with bread yeast, and Steve makes bread with beer yeast.
Flash or Streaming mp-4