Sweats are like the wearable equivalent of one of those memory foam pillows...
If you drool on them they start to smell like stale spit and your shoulder gets all soggy, er, I mean, once you put your mark on them, they become the go-to choice, just based on sheer comfortability (I mean crap, have you used one of those things?)
Now, if you got a well cut, well fitting sweat piece, that's a whole different story. You've entered the realm of comfort & stuntability, a realm only few travel.
Throw a button up under a sweatshirt, and it's real easy to look like a P.E. teacher who just needed to feel like a "young professional."
With the right fit and pairings tho, you'd look like his english pedagogical counterpart. ::