Plastic taste after bottling

So this is my first post on this forum after reading through it for many hours before and after brewing my first batch (two weeks ago) I've searched the forum and some posters have hinted at this, but I wanted to have a definitive answer.

The brew is a basic light ale kit that came from morebeer and was included in my starter brewkit. Everything went more or less as planned. Sanitation was maintained and the recipe was followed. Aside from having a hard time achieving a boil on the stove-top and probably steeping the crystal too long(color ended up too dark) and possibly burning some extract(kind of a caramel taste) brewing was fine. Cooled the wort and pitched the yeast about the right temp and fermentation went fine. Two weeks in primary and the hydrometer readings matched(went from 1.04 to 1.01 dead on for the recipe). I tasted the wort at this point on bottling day and enjoyed half a glass. The beer tasted completely fine with no off taste. I boiled the priming sugar and let reach room temp and then poured into
Bottling bucket. Siphoning got off on the wrong foot with a little air in the tube, but quickly resolved. Bottled without any problems. After one week (I know it's early!!) I chilled one and then tasted it. Half a second of good beer, then horrible plastic taste that lingers till near the finish. Could this be from the vinyl racking tube included in the kit? It was rinsed/sanitized along with everything else and the beer tasted fine before it passed through it. Will the taste ever diminish with aging?

Did you use a plastic (PET) carboy, plastic bottling bucket or glass carboy?

I prefer glass for all things. Just for reference, if you did use PET, get a bottle of water from anywhere, let it sit in the sun a day or 2 and taste the water.

I used a plastic Carboy for fermentation and a plastic bottling bucket, but like I said the beer tasted fine after moving from the plastic fermenter. It's only after traveling thru the vinyl tubing and sitting in bottles for a week that the flavor arose. Oh btw carbonation was pretty decent for having only been bottled for a week. If it wasn't for the plastic taste I would consider the beer just fine!

What do you sanitize with? Not bleach I hope.

Star-San for sanitizing. The only thing I can think of is that I didn't actually "clean" the tubing since it was new, but I rinsed with hot water for a while before sanitizing

I had this same problem on few batches and wasn't able to nail the exactly reason for the off flavor. Few things come to my mind, oxidation during bottling, off flavor produced by the yeast during natural carbonation, or the off flavor was already there but you could not detect it due to other stronger flavors on the green beer. Once the beer mellows, the off flavor takes over.
My advice is just let it sit for a couple of weeks. I had batches that tasted like that and turned out ok with time.