I just went thought this exact exercise.
Make sure your boil volume is set correctly. In the "Recipe View", click on "Choose" next to the greyed out "Equipment" field. Then click on "New" and enter your custom brew kettle size if you don't see something listed that will work. You should see your IBU change instantly.
Doing it this way will also calculate how much water you should start with given a 9% evaporation rate for your boil time. That 9% is customizable too, but I guess that's about right?? That's the default number that the beersmith people used.
And just to let you know, I don't think the free trial saved my recipes that I was working up yesterday...at least I coudn't find it when quickly looking. So make sure you copy the recipe into notepad or something just in case the free trial dosen't allow you to save.
Did you pick this up at How do you Brew? I just picked up the English Pale Ale. I really wanted the IPA, but they were out of it. I'm guess you're the guy responsible for that???