Virginia’s junk criminal science

Originally published in the Washington Post

Sean Kennedy Dec 23, 2016

The most cited but misused statistic in criminal justice policy is recidivism.

Enter Virginia Gov. Terry McAuliffe’s (D) press conference last week purporting to show the commonwealth with the lowest recidivism rate in the country at 24.3 percent of prisoners returning to prison within three years.

The problem is the term “recidivism” has become a catch-all for whatever the purposes of the respective jurisdiction, elected official or academic are. As a result, it has been rendered meaningless.

The U.S. Department of Justice National Institute of Justice offers this definition: Recidivism is measured by criminal acts that resulted in rearrest, reconviction or return to prison with or without a new sentence during a three-year period following the prisoner’s release.

Justice’s landmark 2014 study of recidivism only measured the re-arrest rate but did it over three time frames (one-year, three-year and five-year rates). Moreover, that study only sampled state prisoners in 30 jurisdictions, including the most populous and diverse states such as Texas, California, Florida and New York but excluding many smaller, more homogeneous states such as Vermont and Wyoming.

States don’t have a mandate or an incentive to collect quality data and so commit far worse analysis and produce data that can be only called junk data science.

Virginia’s Department of Corrections released data last week that illustrates the imperfect nature of these non-comparable data sets. Although the public relations team in the governor’s office was likely elated by the headlines, any serious observer should be unnerved by the commonwealth’s data release.

Politicians are using and abusing these statistics for political points without any regard for data integrity and comparability. McAuliffe said at the report’s unveiling, “Since the beginning of my administration, we have aggressively pursued policies and initiatives that rehabilitate incarcerated individuals, so they can develop the tools and skills to be successful.”

Except the data begins in fiscal 2012 and ends in fiscal 2015 and McAuliffe took office in January 2014. According to simple math, the governor is eager to take credit for something his policies only partially made possible, if the data is even accurate.

A closer reading of the report’s data reveals that Virginia officials are guilty of “fuzzy math” in their pursuit of positive press, as the Associated Press noted, “The report doesn’t count as recidivists people who have been re-arrested on a new felony but whose cases haven’t gone to trial. It also doesn’t deem people recidivists unless their new crime is a felony and they got more than a year to serve.”

On face, the Virginia data set is suspicious. Largely homogeneous, low-crime Utah has a “recidivism” rate more than 270 percent higher than Virginia per the governor’s commissioned report. New York and California are almost double Virginia’s rate. Texas, Illinois, Maryland and Georgia are absent because “data was not available or the recidivism definition was different.”

McAuliffe’s data does not even jibe with publicly available data sets from the states the report purports to analyze “uniformly.” Barring the commonwealth having access to raw data sets of individual cases for every state, it is unlikely this analysis was produced in a true apples-to-apples manner.

Take California, for example. The Golden State’s most recent report was release in August 2016 based on fiscal 2010 to fiscal 2011 prisoner releases. Per the California Department of Corrections and Rehabilitation report that Virginia cites, California re-incarcerates 44.6 percent of offenders within 3 years.

Except California doesn’t define return to prison (re-incarceration), let alone recidivism, the same way as McAuliffe’s report contends. Instead, California’s data analysts write that the three-year return to prison rate constitutes, an “individual convicted of a felony and incarcerated in a CDCR adult institution who was released to parole, discharged after being paroled, or directly discharged during Fiscal Year (FY) 2010‐11 and subsequently returned to State prison within three years of their release date.”

A key footnote in the California report also states, “This may include individuals who returned to prison pending revocation, but whose cases are ‘continued on parole’ or dismissed.”

It doesn’t exclude felons serving less than one year or pre-trial imprisonment, as Virginia does.

The result of Virginia’s analysis is a Swiss-cheese of actual recidivism data since states do not measure the same thing over the same time.

States should be required to collect uniform data on offenders, report it publicly and make it available to academics and independent analysts. The data should be collected over even longer time frames (five-year or even 10-year periods), as well as at regular intervals (every six months), include multiple categories of criminal justice system interactions (re-arrest, parole and probation violation, re-incarceration), new and previous offense types and across jurisdictions (because crime does not stop at state lines).

But, at the very least, politicians should stop holding press conferences based on junk social science.

Sean Kennedy is a visiting fellow at the Maryland Public Policy Institute.