Diacetyl is a chemical compound that yeast make during fermentation. It has a flavor and aroma of movie theater buttered popcorn. Diacetyl is added to fake butter flavoring.
Yeast will take up diacetyl at the end of fermentation. A diacetyl rest is simply leaving the beer alone for 3 days after fermentation has stopped so that the yeast continue to metabolize. Typically on the homebrew scale, you can raise the beer's temperature 5-10 degrees to speed the process and ensure it is completed. This is more critical with lager strains than ale strains, because of the low ferment temp.
A typical procedure is to raise the temperature to 60F for lagers or 70F for ales once the fermentation is about 75% done, or when the gravity drops to below 1.020. Checking gravity daily, the diacetyl rest is usually complete when the same gravity is measured for 3 consecutive days.
You can test for diacetyl by pulling a sample, microwaving it for about 30-60 seconds to get the temp up to 140F, then cooling it back down. What this will do is produce diacetyl from any of the precursor compounds found in the unfinished beer.
Your heated sample should taste no different than a regular sample. If it does, your beer needs a couple more days at least.
One caveat here is that some people, in fact many people, are not sensitive to diacetyl. It's possible that you won't perceive a strong off flavor. It has also been described as producing a "slick" or oily mouthfeel.