I suspect you are talking about what is commonly called "grainy". It is common and generally the result of crushing problems. If the husks get shredded during the crush, they'll release these flavors. Sometimes backing off on the crush will help, sometimes it just takes a good cleaning of the mill. Once the flavor is in the brew, a long clearing at low temperatures can help. Filtering also works.