If you are doing natural carbonation, yes, you will need to leave it at room temperatures until it carbs. If you are just letting it condition until you force carbonate, you are fine as is. After 3 weeks in primary, all you will be doing with a pale ale is maturing and clearing which will be fine in your kegerator.
Primary: Belgian Blonde (for cask)
Drinking: Robust Porter
Up Next: Christmas Ale, SNPA, Simcoe IPA