A few things. First, you do not need both special roast and victory. If you are indeed using a good quality UK Maris Otter I would drop the victory entirely and use the special roast at no more than 5% - maybe 0.25lbs. Special roast can have a bready-tartness in higher amounts that some equate to a sourdough bread type of flavor.
Also, you need more dark malts here. As is, you got more of a nut brown ale than a brown porter. I try to incorporate 'brown malt' into all my b.porters as it gives both those toasty flavors and some light chocolate/coffee type flavors that add complexity. As for the crystal, 120L is good, though I would probably go with something a bit lighter like 60 or 80L to get more toffee flavors than raisin/prune. Pale chocolate is a great malt and 0.5lbs is fine. However, I might back it up with a quarter pound of chocolate malt to add some of those darker roast flavors.
Also, yeast choice is a big part of this style as much of the flavor and malt profile will come from the yeast. I would recommend staying away from s-04, notty, and the whitbread strains with this type of beer, as they are really too dry and neutral when fermented properly. Yeasts like wy1968/wlp002 and wy1318 are great choices for this type of beer.