The actual HONEY in honey doesn't add any sweetness to beer. The honey itself is a simple sugar that ferments out 100%, but often the flower/nector source that made the honey will provide some residual sweetness.
What honey DOES do while it is fermenting out completely is lower your FG, dry out your beer, and increase your ABV. All of this must be done in moderation, or else it can seriously ruin a batch.
Next time, the best way to really get HONEY flavor in a beer is to use HONEY MALT, which has nothing to do with actual honey besides the fact that it tastes like it while retaining the characteristics of a malt, making it much easier to deal with in the brewing process.
Also, FYI, if you got an ale yeast instead of a kolsch yeast, you aren't making a kolsch anyway. The kolsch style must be lagered. With the ale yeast, you are making closer to a blonde. I'd stop messing with it, it'll be fine.