Depends on the person. Ranging from an allergic reaction in some to a complete inability to sense even abnormally high concentrations in others. Most find sulfites to be more noticeable than sorbate in the same concentration IMO, but sorbate is generally used in higher dosages.
My threshold for sulfites is around 100ppm of free SO2, and find it gives a slightly metalic, sulfa taste. Never had too much sorbate identified as such but I assume it would be similar.
Never heard of anyone using sorbates without sulfites so can't comment on that part. But properly used, 99% of the population cannot taste them in a product.