The important thing for calculating bitterness is to know the AA of the hops. If you know that, you can adjust quantities to match. But of course you don't know it for homegrown hops.
If in doubt, an easy solution is to use bought (and therefore known AA) hops for bittering, and keep the homegrown ones for flavor/aroma where the AA is less relevant.
Or, if you really have a lot, it might be worth getting them analyzed. I've never done that, but hear it is possible although expensive.
Or you can just guess. I'm currently fermenting a Kolsch that I brewed with hops from my neighbours yard, of unknown variety. I figured they smelled pretty good and just went with the flow. I judged the bittering addition by a mixture of eye and taste: put some in, boiled for 10 minutes, tasted the wort, compared against my memory of other brews with known IBU that I previously tasted after 10 minutes, decided there was nowhere near enough bitterness going on, and ended up adding about 4x my original guess. Too early to say how it turned out, but my hydrometer sample tasted promising...