That happens to me all the time. I decant the wort from my starter and what's left is a yeast slurry caked to the bottom. I just pour in plain water and swish the container around. The pouring in of water on top of it breaks up some, and the swishing around breaks up the rest.
Sometimes it doesn't get it all, so I pitch what did break loose, then repeat until all the yeast is gone.
There might be a better way, but that works for me.