When I spoke with Sara Heller about her study of Chicago's One Summer Plus, which found that the jobs-plus-mentoring program reduced violent crime among the treatment group by almost half, she mentioned that she was grateful the city allowed her to turn it into a randomized control study. As she pointed out, the study could have found that the program didn't do anything. It could have found that it made things worse; well-intentioned educational programs have actually done that. But you want to know if a program isn't having its intended effect; you definitely want to know if it's having the opposite effect.

I thought about that when I read the Trib's new report on Chicago's red-light cameras, for which they commissioned a report from two academics with ties to the well-known Texas Transportation Institute. It's pretty damning on the city's presentation of the data:

"The city's study is worthless, making no adjustments for any potential bias," said Joseph Hummer, professor and chairman of civil and environmental engineering at Wayne State in Detroit. He also noted that, for some sites, the city used 2005 data in the "before" section of its analysis even though the cameras had been installed there in 2003 and 2004.

Scheinfeld said in an interview last week that the city's safety claims are based on "more basic," nonscientific comparisons of crash statistics. She said the state transportation department statistics are "still the best information that we have, and we acknowledge they have made that change in their reporting methodology, and it would have some impact on those numbers, but we don't know exactly what impact."

Instead, the Trib-commissioned study, produced at substantial cost, had pretty typical findings for a red-light camera study: angle crashes went down, rear-end crashes went up. Within the study is a review of prior research, which generally goes in the same direction.

According to the study itself: "a non-significant increase of 5% in the total number of injury crashes, a statistically significant reduction of 15% in angle and turning-injury crashes, and a statistically significant increase of 22% in rear-end collisions were noted." In raw numbers, that's a reduction of 76 angle and turning-injury crashes at 90 intersections over three years, and an increase of 54 rear-end crashes.

But it's worth delving into the study a bit.

The authors broke it down further for intersections with four or more injury crashes per year vs. those with less than four. Their results? At intersections with four or more injury crashes, there were 82.4 fewer angle/turning crashes than expected (a 24 percent decrease), and 30.8 more rear-end crashes (a 20 percent increase).

At intersections with fewer than four crashes per year, there was an increase in both types of crashes—a statistically significant 19 percent combined, but not statistically significant for each type of crash. So that's not good. And suggestive of prior research that red-light cameras are more appropriate for problem intersections.

And overall, the numbers don't look great at first glance. But the difference between angle/turning crashes and injury crashes is important. Angle/turning injury crashes tend to be worse. In 2005, the Federal Highway Administration did an economic analysis of red-light cameras, finding a similar trend as the Trib-commissioned study: right-angle injury crashes declined by 16 percent, rear-end injury crashes increased by 24 percent.

They compared these trends to the estimated cost of those crashes: $64,500 for a right-angle injury crash, $53,700 for a rear-end injury crash. The conclusion? Red-light cameras saved $39,000-$50,000 per site per year:

This statistically defendable study found crash effects that were consistent in direction with those found in many previous studies, although the positive effects were somewhat lower that those reported in many sources. The conflicting direction effects for rear end and right-angle crashes justified the conduct of the economic effects analysis to assess the extent to which the increase in rear end crashes negates the benefits for right-angle crashes.

[snip]

The modest benefit per site is an average over all sites. As the analysis of factors showed, this benefit can be increased through careful selection of the sites to be treated (e.g., sites with a high ratio of right-angle to rear end crashes as compared to other potential treatment sites) and program design (e.g., high publicity, signing at both intersections and jurisdiction limits).

And that was reflected by the experts the Trib talked to. Rather than tossing the idea of red-light cameras altogether, they emphasized the importance of placement at intersections with angle-crash problems. It's a conclusion mirrored by Richard Ashton in The Police Chief, who also proposes an interesting idea: that violations should accrue as drivers-license points rather than fines to divorce the intention of public safety from the prospect of increased revenues.

More important, they should be placed with rigor—in the context of the money involved with the red-light camera program, a mere $14,000 to ensure it works as designed is well worth it.