The last brown I brewed was good right out of the fermenter, but I could taste the separate ingredients. Nothing was melded together yet. I let it sit another 2 weeks, and at that point the beer had an off-taste. The beer was still good, but right at the end of the taste profile, it was just a bit off.
I let it sit for 4 more weeks, and the beer had finally matured to where all of the tastes complimented each other and worked in harmony.
I think you just have a young beer. My last brown took 8 weeks from brew day to mature, and even after that, it got better week to week. Give yours some time.