You're using wyeast right?
Their suggested pitch rate for a 1.080 wort is >18 million cells per mL
http://www.wyeastlab.com/hb_pitchrates.cfm
According to the wyeast pitch rate calculator (
http://www.wyeastlab.com/hb_pitchrate.cfm) a 0.674 gallon (2.55 liters) starter with one smack pack and without a stir plate, when added to 5.13 gallons of wort, will contribute 12.63 million cells per mL.
Now, assuming you decant and only pitch, say, 100 mL of starter instead of 2.55 liters, adjust the total volume of wort added to reflect the actual final volume. So you say 4.4824 gallons added wort instead of 5.13 to arrive at the correct final volume of 5.1564 (5.13 gallons plus 100 mL). This is what your pitch rate will be. According to wyeast your pitch rate is 14.21 million cells per mL. Not the full 18 that is needed. Honestly at this point I wouldn't sweat it too much. Just pitch what you've got. There's a pretty good chance that it will still ferment out just fine.