You should have no problem mashing a no-sparge 3 gal batch in a 10 gal cooler. The biggest difficulty will be to collect all the wort (the bigger the cooler, the greater the loss of wort to the mash tun, generally, which can become significant on small batches).
A couple of suggestions to avoid heat loss:
- preheat the cooler REALLY well before mashing
- you can fabricate a circular piece of insulation from rigid foam that fits inside your cooler just above the mash to reduce the dead air space above (preheat that too!)
- mash pretty thin to maximize the volume in the tun