Actually, they don't even need to investigate fertilizer / etc. See
Wickard v. Filburn from 1942.
The case revolved around a farmer growing crops on his own land for his own family's consumption. The Supreme Court ruled that was interstate commerce because if he didn't grow his own wheat, he might buy wheat in interstate commerce, and therefore his activity affected interstate commerce.
This was extended specifically to marijuana in
Gonzales v Raich in 2005. It stated that even an
*illegal* good like marijuana is considered interstate commerce, although that commerce shouldn't <according to the federal gov't> exist. So even growing your own pot for your own [medical] use is subject to federal regulation based on the idea that there is an interstate illicit market.
So yeah, if the feds want to prosecute, they're pretty much going to get away with it, due to the supremacy clause. It doesn't matter AT ALL to the feds that the state declared it to be legal.