(After the actor Bradley Cooper mentions that he left "Alias" because he wasn't happy with his part)
That really is admirable. Because after about Season Five on [Buffy], or the end of Season Four, I was that guy. Joss [Whedon] actually said Xander was done--that there was no more. I was just kind of relegated to the background. It was one of those things that, where I was at in my life, the money was more important than my pride or taking care of myself. [Leaving], especially as the show's just starting to take off...
People who watched "Buffy" had differing reactions to the increasingly decreased role of Xander as the show went on. Some people didn't much care, or at least felt that he'd been de-emphasized in favor of other, darker characters who could carry more grown-up storylines. And some of us thought that Xander was the best character on the show and had been screwed over in favor of various boring romantic storylines and Fonzie-like vampires.
In my experience, people who started watching "Buffy" in the first or second season usually, not always but usually, share my frustration with the de-Xanderization of the show. It was a symptom of a number of things that choked off and diminished "Buffy" as it went on: the increased emphasis on the soapy romance element, the increasingly absolute division of all episodes into "serious" and "comic relief" moments (instead of being funny and serious at the same time, as the best episodes usually were), and a glut of dull new characters.
The show would probably have developed differently if Xander had been given the big storyline in the fourth season; I don't know that it's ever been officially confirmed, but there's a general consensus that the story about "The Initiative," and a character who gets recruited by that shadowy organization, was supposed to be about Xander. Instead it was given to a new character, Riley, introduced as a new love interest for Buffy; Xander therefore spent most of the fourth season without anything to do except make wisecracks and have a dull romantic storyline of his own, and this continued for the duration of the series.
"Buffy" was probably done in -- creatively, at least -- by the "'shipper" mentality, the idea that the most important thing about a show is pairing off each character with a romantic partner and focusing as much attention as possible on the question of who would get together with who. By far the worst thing about "Buffy" in its first (and best) three seasons was the Buffy/Angel romance; it may have been a godsend to fanfic writers, back before slash became the fanfic style of choice, but it mostly produced goopy, treacly, mopey, dopey episodes. (Except for the "Angel goes evil" storyline in the second season, because that was an appropriately "Buffy"-esque combination of angst and humor.) Boils and Blinding Torment has a funny redux of one of those interminable Buffy/Angel scenes:
Buffy: A date! You're asking me on a date!
Angel: Hello? No, I am after all a vampire. I'm merely being confusing and obtuse.
Buffy: Since I'm been all lovey-dovey already in this conversation, I guess it's time for BitchyBuffy. I like to mix things up by going back and forth. So get the hell away from me.
Angel: Fine! After all, we have coffee, next thing you know, it'll be sex, and who knows, you could cause me to lose my soul or something.
Buffy: Great. Well, you are such a terrible kisser than when you kiss me I wish I were dead. Watch out or I might stab you and send you to hell.
So Angel leaves to headline his own spinoff and actually be interesting, given that he now has the opportunity to do something other than teenybopper romance (well, until that show gets taken over by who-loves-who plots, but I digress), and what does "Buffy" do? Concentrate on the interesting stories about friendship and fitting in and using monster stories as serio-comic metaphors for the everyday problems and decisions that young people face? No: introduce more boring love interests, male (Riley) and female (Tara), and reduce all the characters' lives to the question of who they are or are not sleeping with at the moment. One of the refreshing things about early "Buffy" was that it was one of the few high school shows that acknowledged that there are, in fact, other aspects of a teenager's life than who he or she is romantically involved with: there's also the pressure to conform (Xander in "The Pack"); the pressure of trying to combine school, extracurricular activities, and having some fun on the side ("Reptile Boy," but, more generally, Buffy's whole life is about trying to combine school with a crushing after-school training program); trying to change your image to be cooler (Willow in "Doppelgangerland"), and so on, all realistic teenage problems that happen to be played out as monster stories. Once a show spends too much time on romance, it's cut itself off from the really interesting stories.
Xander was hit the hardest because he had literally nothing to do on the show except be another supporting character's romantic partner, and it's a long way to fall from being one of the funniest and most interesting characters on TV for the first three years.
Incidentally, I wish someone would release a better DVD of the original "Buffy" movie. I know the Whedon fanatics despise it because it Wasn't True To Joss's Vision, but vision or no vision, the way it took a nutty concept and played it almost (but not quite) straight was something of a revelation at the time, and it anticipated the strengths of the series by finding the reality in its subject: it's ridiculous, but underneath the ridiculousness is a real story about a shallow girl who realizes that there's more to life than what she's been getting by with. Lots of memorable lines, too ("Kill him a lot!"). I'd a lot rather see the movie again than most of the episodes from the last two or even three seasons of the "Buffy" series, which are considerably less true to the original comic/dramatic vision of the series than the movie ever was.