I did the same thing with my first batch, which wasn't all that long ago.
Without a chiller, I found that it cools WAY faster if you put the few gallons you boiled in an ice bath BEFORE you add the cold water that you have chilled in a freezer. My first batch I went the other way. The 2-gallons were right on the verge of freezing, in fact there were ice chunks dropping in with the water. Then I put the bucket in the sink and gave it an ice bath. It took hours! I skimped on the ice a bit too. Won't make that mistake again-
A few batches later I put the cooking pot into the ice bath right off the burner, stirred it a bit and swirled the ice water a few times, had a brew. Poured the cold water in the bucket then added the wort and I was in business.
Still not as cool as a wort chiller but if you are doing the ice bath chill like me, order of operation makes a difference.
For what it's worth, that first batch tasted fine anyhow. And I made so many other very stupid mistakes along the way that I forgot about the slow chill.