As it turns out it wasn't a complete disaster. I had to dig out a couple of pots and drain into those after mashing, clean out my main pot then dump in and continue on. I had a spare burner so I used that to keep one pot of wort hot while I cleaned quick. I have a false bottom in my kettle so there was actually little grain in my wort.
If it makes you feel batter, you're not the only one to forget the bag for BIAB. I once started a batch and minutes before I was set to dough in I realized my bag was being used to hold strawberries for wine in a fermenting bucket.