I think that would be a great idea.
Personally, I use Warminster Floor-Malted Maris Otter and Biscuit malt in my Pumpkin Ale, it's no where near brown on the color scale, but it does have a wonderful, full, toast/bread malt character to it.
You could use an ESB base, and only use the 60 minute hops. Maybe something like:
8-10 lb Maris Otter
1/2 lb English Medium Crystal
1/4 lb English Xtra Dark Crystal
1.5 oz Fuggles/EKG
Wyeast 1968, ferment around 64-66ish to subdue some of the esters
And then add your pumpkin/spice on top of that.