Astringency differs from bitterness by having a puckering quality, like sucking on a tea bag. It is dry, kind of powdery and is often the result of steeping grains too long or when the pH of the mash exceeds the range of 5.2 - 5.6. Oversparging the mash or using water that is too hot are common causes for exceeding the mash pH range. It can also be caused by over-hopping during either the bittering or finishing stages. Bacterial infections can also cause astringency, i.e. vinegar tones from aceto bacteria.
The brown scum that forms during fermentation and clings to the side of the fermentor is intensely bitter and if it is stirred back into the beer it will cause very astringent tastes. The scum should be removed from the beer, either by letting it cling undisturbed to the sides of an oversize fermentor, or by skimming it off the krausen, or blowing off the krausen itself from a 5 gallon carboy. I have never had any problems by simply letting it cling to the sides of the fermentor.
That would be something to look at as the cause.
Kegged: Aramis Pale Ale
Drinking: All of the above :)