It may be possible to get close with pellets, but whole hops are going to be pretty close to impossible to measure that way. There's too much variability in compression of the whole cones. You'll be glad you got a scale.
I suppose it would be possible to build a simple beam-balance if you had something of a known weight to balance against. But I'd buy the scale. You'll be glad you did.
EDIT: Joey just said that but with better info. Too slow on the keyboard!