Well, it depends on what is meant by "expensive.". What I have cost chicken feed compared to some people in this forum, with their RIMS, HERMS, and whatnot. Compared to a few other hobbies I've had, the capital investment is pretty cheap.
I look at it from the standpoint of what I'm doing. I'm spending a minimum of $25 - 30 on ingredients to brew a 5 gallon batch, and I want to make the best beer possible.
While most of the homebrewing "authorities" I've read say that great beer can be made with simple equipment, I wanted to buy equipment that was dedicated to that purpose.
My original brewing kit was Midwest's Intermediate kit, bought two years ago for $80. My brewpot is a 30qt. SS unit, bought on sale for $40. I found it would help my process if I added some accessories not in the original kit. I found that costs for that stuff mounts up, especially when I made the switch to AG, converted a cooler for an MLT, got a grain mill, propane burner, more primary and secondary fermenters, etc.
So- right now my best estimate is that I've got around $700 invested in non-consumable equipment for this pastime, currently amortized over about 50 x 5 gallon batches of beer, and that cost per gallon (less than $3, only about 10% of my ingredient cost per batch) gets lower for every batch I brew. I'm happy with the beer I'm brewing, so I feel that's very acceptable.