I pick wild fruits all fall- chokecherries, blueberries, blackberries, raspberries, apples, crabapples, wild grapes, etc. And in the spring, I use dandelions and rhubarb for wine. There are other wild fruits and flowers that can easily be used for wine as well- serviceberries, gooseberries, elderberries, etc. And if a neighbor has an apple tree, you can ask for the ones that fall on the ground.
It really depends on where you live, and how hard you're willing to work for it.