Yeast should go in the fridge. If it's dry yeast, that's not much of an issue, ESPECIALLY not in a one or two week time frame. Grains probably go in the fridge too but, again, not a big deal.
Yeast NEVER goes in the freezer unless it's been preserved with glycol; otherwise the cells will burst when the water inside freezes. I'd avoid putting the grains in the freezer too; obviously burst grain cells aren't going to have the same effect as dead yeast will, but I can't imagine it wouldn't have an effect - even a small one - on the final product. Maybe even just making it a little more hazy.
Fruit you'd treat like normal fruit: freeze if it's frozen, refrigerate if you need to keep it more than a few days, otherwise you can leave it on the table in a bowl. Fruit extract usually doesn't even need to be refrigerated.
Hops, on the other hand, can stale quickly if they aren't properly packaged and stored. Pellet hops last longer than whole hop flowers but, regardless, they both go in the freezer if you plan to wait more than a day or two to brew.