Is it soluable? Do you really want to ferment milkfat, if it is even possible? Personally, I think it will be rank, and likely for failure. But, obviously, I have never tried, and tend to be conservative, so YMMV.
Why dont you break down and analyze the flavors you like from the fudge, and get them through regular beer ingredients? Is it the pumpkin? Use pumpkin. Is a toffee like sweetness? Use english medium crystal. Is it creaminess? Add oats or other malts for mouthfeel. Is it the milk flavor? Add lactose. Is there spices? Identify them and use those. I think thats a far safer approach.