Yes, it's fine to use just the plum juice. If you're a little short some water is no problem. Several experienced people use the pure juice. You will need to check your SG to see how much sugar to add -- you'll probably need some. Also, it's probably better to just ferment the juice alone w/o the pulp and skins which can add a lot of acid.
On the other hand, many recipes including Jack Keller's ones use water. I've done these in the past. This year, I'll do some of each. Good luck with your wine!