It's way too late to be responding to the Onion A.V. Club's article "Is Improvisation Ruining Film Comedy?", but what the heck. The discussion, which ran last month, makes some good points about the inclusion of improvised or semi-improvised moments in movie comedy today. But one thing I find interesting about today's improv movie moments is that a lot of them seem to serve the opposite purpose from "traditional" movie improvisation.
As I've emphasized before, improv was very common in movies in the silent era and even into the sound era; there were directors like Leo McCarey and Greg La Cava who simply never got used to the idea that a movie should have a tight, finalized script, and who instead would write up a bunch of scenes and create other scenes from scratch on the set. What they were trying to do, in encouraging actors to improvise, was to create a sense of naturalness that you normally didn't get in the artificial, heightened world of studio movies. The theory -- to the extent that they consciously had a theory -- was that if you create a loose atmosphere and encourage the actors to improvise, you'll wind up with characters who act like actual human beings, rather than actors. (That's probably one of the things Jean Renoir was referring to when he said that McCarey "understands people" better than any other director: because the acting in his movies is so natural and unforced, the people onscreen really do seem like people.)
The improv in your typical Will Ferrell or Ben Stiller movie usually has a different purpose. These are artificial characters in deliberately artificial comedies, and instead of making the acting more natural, the improv serves the purpose of making everything less natural: the actors are riffing on the material to make it crazier, to come up with bits of business that are larger than life.
Of course, part of that may just be the change in acting styles over the years. In Leo McCarey's time, acting was very artificial and non-naturalistic; improv was a useful tool for achieving more naturalism. Today, most actors are trained to have a certain amount of naturalism. Directors may have to encourage the actors to riff and improvise in order to get them to be less naturalistic. (The improv in a movie like Knocked Up does help to increase the naturalism of the scene, but even without it, the actors still wouldn't be talking as fast or gesturing as formally as actors did seventy years ago.)
And, of course, there's plenty of improvisation that doesn't quite fall into clear categories. Like this scene from McCarey's The Bells of St. Mary's, where he gathered together a bunch of kids, told them to do a Nativity play, and filmed the results. The result is a classic example of improvisation in film, but it doesn't fall neatly into my "larger than life" vs. "naturalistic" improvisation categories; it's sort of both at the same time.