Definitely start with some patience. It can take harvested yeast a little while longer to get started. I harvested some Pacman once and it showed no activity at all for over 48 hours, then all of a sudden near 72 hours just took off - the starter went from totally clear to very cloudy with a tall krausen. According to many folks on here, some starters may show very little signs of activity at all (but that's never been my experience). If it goes on too long with no visible signs, it might be a good idea to start giving it a sniff test to see if any nasties took hold. I had a failed attempt on a bottle of Rochefort, and after a few days it did get pretty nasty. Even if your starter does look healthy, it would not be a bad idea to smell it. Don't be looking for a perfect beer like aroma as it will be oxidized. But if it's gone bad it will probably not smell great.