Anyone who thinks the Internet has made the public dialogue appreciably worse has likely never had the infernal job of reading letters to the editor. It was the first thing I was ever tasked to do in journalism, for the same reason sports teams make rookies carry the luggage: because it's heavy, and because it's a quick way to break people. The smoldering wreckage of public discourse has always been out there; the letters to the editor page has long been a mask of sanity over it, assembled from bits and pieces into a simulacrum of reasoned debate.

In 1983, the Chicago Tribune tried something different and quite radical. Its endorsement of Harold Washington during the racially charged mayoral contest stirred up a torrent of vile response, and rather than use the letters page as a reflection of what should be, it turned it into a reflection of what was reality.

Leanita McClain, then a Tribune columnist, explained the decision in her famous Washington Post essay "How Chicago Taught Me to Hate Whites."

The Chicago Tribune endorsed Harold Washington in a long and eloquent Sunday editorial. It was intended to persuade the bigots. It would have caused any sensible person at least to think. It failed. The mail and calls beseiged the staff. The middle range of letters had the words "LIES" and "NIGGER LOVERS" scratched across the editorial.

Hoping to shame these people, make them look at themselves, the newspaper printed a full page of these rantings. But when the mirror was presented to them, the bigots reveled before it. The page only gave them aid and comfort in knowing their numbers. That is what is wrong with this town; being a racist is as respectable and expected as going to church.

The page itself was not as intense a shaming as that implies; no racial epithets are to be found. But the tone is familiar enough to anyone who's waded through a comment thread on a web page.

The 'World's Greatest Newspaper' has degenerated into another populist rag, with no integrity whatsoever…. I will not allow self-serving, socialist propaganda in my home.

I wonder what Robert McCormick would say if he knew what you have done to his cherished Tribune. He probably would have you horsewhipped."

The Tribune has deteriorated from an excellent, conservative newspaper to a liberal rag, as evidenced by the endorsement of the convicted income tax evader, racist, black candidate for the mayor of Chicago, Washington.

It is Washington and his supporters who have been behind the race issue. He made race an issue—not the white people.

This is not bad data to have, necessarily. When Rick Perlstein was writing Nixonland, he made use of a box of vicious letters on housing policy to then-senator Paul Douglas in charting the fracturing of the New Deal coalition: "I quote many of them in a section of Nixonland of which I'm most proud, the one with the most original research and historical insights: the one on how 'open housing' opened up the conservative backlash that inaugurated the Republican dominance of the politics of our own generation."

But that has to be weighed against what McClain observed, and for which there's at least a bit of hard evidence: that opening up news pages to a flood of abuse and hostility not only doesn't aid the mission of a newspaper, it actively harms it.

The Sun-Times just decided this to be the case, and made the decision to cut off their comment sections completely while they design a better system—better for commenters, and better for the people who have to monitor the comments. This led to more pushback than I'd expect, like from Robert Feder: "But a big-city daily’s digital news site that summarily shuts off dialogue with its readers strikes me as wrongheaded and backward-thinking to the extreme. It’s not the first time the Sun-Times has turned a deaf ear to its readers."

Feder suggests just leaving it running and moderating it until the new system is designed. But for a major paper, that comes at considerable expense. Take the Boston Globe, for instance:

At Boston.com, the website of The Boston Globe, a team of moderators – or “mods” – monitor the comments. Actually, with just one or two mods on per shift, and an average of more than 6,000 comments posted every day, on every corner of the site, the mods could never hope to monitor all the simultaneous chatter. Instead, they focus on evaluating the “abuse reports” that commenters file against one another. For Steve Morgan, a veteran editor who coordinates the monitoring, the color of trouble is red. The crimson message at the top of his computer screen keeps a running total of the abuse reports that are awaiting action. Some complaints don’t ultimately turn up abuse – coarse language, ad hominem attacks, and the like – but rather just a political stance that the person doing the complaining doesn’t care for. So a mod needs to evaluate each complaint and decide either to remove the comment or let it stand.

Or the New York Times, which, in 2010, had a team of eight mods:

Brisbane writes that despite having a team dedicated to moderating conversation on NYTimes.com, not every story is open to comments, and others are closed “early” due to the overwhelming volume of reader responses.

If you don't want to hire mods, and you don't want to subject your staff to the deadening task, you can contract out comment moderation, though it doesn't come cheap: thousands of dollars per month to clean out the Augean stables of the web.

Moderators (or “mods,” as some call them) can earn anywhere from $40,000 to $80,000 annually, but need to be prepared for daily exposure to humanity at its vilest. Extreme racism and bigotry, images of pedophilia, and even personal threats are all too common. Littleton, who has even had her home address and phone number posted by disgruntled commenters, makes sure new recruits undergo extensive background checks. “You need good common sense, and you need a really thick skin,” she says.

The strain can take its toll. Although nasty comments make up less than 10% of what appears online, according to Littleton, the bad apples are what moderators are paid to deal with. A significant number of new hires with ICUC last less than two weeks. To cope, moderators work on sites in short shifts, flipping between forums prone to maliciousness (news stories about Israel, say) and something more joyful (LEGO fan pages).

The other component, besides manpower, is a system that encourages good comments and discourages bad ones—involving sophisticated, complex UX decisions that media companies like Gawker have been working on and usually failing at for years. (Using Facebook comments does seem to discourage abuse, but it doesn't seem to encourage thoughtful dialogue, either, and can drive away potential commenters who don't want their comments turning up in their Facebook news feeds.)

The Sun-Times opened up the debate to Reddit, getting some thoughtful feedback on how to build such a system. It'll give you a sense of the decision-making involved, designing a simple and coherent framework that encourages the best and discourages the worst angels of our nature.

For high-traffic, general interest publications targeted to the widest audience possible, it's a tremendous amount of effort and expense. Talk is cheap, but dialogue comes at a considerable cost.