Just leave it in the primary. It'll be OK. If you can keep the towel wet while you're gone by placing the whole thing in a container of water, that will help to keep the temps down while you're away.
That california ale yeast is a clean fermenting yeast, I wouldn't worry about the clove and banana. It ain't gonna be there.
Secondary #2: Empty
Bottle Conditioning: Oatmeal Stout
Drinking from Keg: Ordinary Bitter, Kolsch
Drinking bottled: Brown Autumn Wee Heavy
Peaches and Cream Weizen
"This is grain, which any fool can eat, but for which the Lord intended a more divine means of consumption... Beer!"
-Robin Hood, Prince of Thieves, Friar Tuck.
Next up: Hefe Weizen