Yes cold crashing will help. What yeast are you using? It sounds like you are using a low flocculating yeast, which would explain why there is always more coming out. It's just hasn't finished settling yet, but cold crashing will help with that. If some gets left behind while you dry hop, it shouldn't hurt anything.