Home > data science, guest post, journalism, modeling > Guest post: The dangers of evidence-based sentencing

Guest post: The dangers of evidence-based sentencing

October 21, 2014

This is a guest post by Luis Daniel, a research fellow at The GovLab at NYU where he works on issues dealing with tech and policy. He tweets @luisdaniel12. Crossposted at the GovLab.

What is Evidence-based Sentencing?

For several decades, parole and probation departments have been using research-backed assessments to determine the best supervision and treatment strategies for offenders to try and reduce the risk of recidivism. In recent years, state and county justice systems have started to apply these risk and needs assessment tools (RNA’s) to other parts of the criminal process.

Of particular concern is the use of automated tools to determine imprisonment terms. This relatively new practice of applying RNA information into the sentencing process is known as evidence-based sentencing (EBS).

What the Models Do

The different parameters used to determine risk vary by state, and most EBS tools use information that has been central to sentencing schemes for many years such as an offender’s criminal history. However, an increasing amount of states have been utilizing static factors such as gender, age, marital status, education level, employment history, and other demographic information to determine risk and inform sentencing. Especially alarming is the fact that the majority of these risk assessment tools do not take an offender’s particular case into account.

This practice has drawn sharp criticism from Attorney General Eric Holder who says “using static factors from a criminal’s background could perpetuate racial bias in a system that already delivers 20% longer sentences for young black men than for other offenders.” In the annual letter to the US Sentencing Commission, the Attorney General’s Office states that “utilizing such tools for determining prison sentences to be served will have a disparate and adverse impact on offenders from poor communities already struggling with social ills.” Other concerns cite the probable unconstitutionality of using group-based characteristics in risk assessments.

Where the Models Are Used

It is difficult to precisely quantify how many states and counties currently implement these instruments, although at least 20 states have implemented some form of EBS. Some of the states or states with counties that have implemented some sort of EBS (any type of sentencing: parole, imprisonment, etc) are: Pennsylvania, Tennessee, Vermont, Kentucky, Virginia, Arizona, Colorado, California, Idaho, Indiana, Missouri, Nebraska, Ohio, Oregon, Texas, and Wisconsin.

The Role of Race, Education, and Friendship

Overwhelmingly states do not include race in the risk assessments since there seems to be a general consensus that doing so would be unconstitutional. However, even though these tools do not take race into consideration directly, many of the variables used such as economic status, education level, and employment correlate with race. African-Americans and Hispanics are already disproportionately incarcerated and determining sentences based on these variables might cause further racial disparities.

The very socioeconomic characteristics such as income and education level used in risk assessments are the characteristics that are already strong predictors of whether someone will go to prison. For example, high school dropouts are 47 times more likely to be incarcerated than people in their similar age group who received a four-year college degree. It is reasonable to suspect that courts that include education level as a risk predictor will further exacerbate these disparities.

Some states, such as Texas, take into account peer relations and considers associating with other offenders as a “salient problem”. Considering that Texas is in 4th place in the rate of people under some sort of correctional control (parole, probation, etc) and that the rate is 1 in 11 for black males in the United States it is likely that this metric would disproportionately affect African-Americans.

Sonja Starr’s paper

Even so, in some cases, socioeconomic and demographic variables receive significant weight. In her forthcoming paper in the Stanford Law Review, Sonja Starr provides a telling example of how these factors are used in presentence reports. From her paper:

For instance, in Missouri, pre-sentence reports include a score for each defendant on a scale from -8 to 7, where “4-7 is rated ‘good,’ 2-3 is ‘above average,’ 0-1 is ‘average’, -1 to -2 is ‘below average,’ and -3 to -8 is ‘poor.’ Unlike most instruments in use, Missouri’s does not include gender. However, an unemployed high school dropout will score three points worse than an employed high school graduate—potentially making the difference between “good” and “average,” or between “average” and “poor.” Likewise, a defendant under age 22 will score three points worse than a defendant over 45. By comparison, having previously served time in prison is worth one point; having four or more prior misdemeanor convictions that resulted in jail time adds one point (three or fewer adds none); having previously had parole or probation revoked is worth one point; and a prison escape is worth one point. Meanwhile, current crime type and severity receive no weight.

Starr argues that such simple point systems may “linearize” a variable’s effect. In the underlying regression models used to calculate risk, some of the variable’s effects do not translate linearly into changes in probability of recidivism, but they are treated as such by the model.

Another criticism Starr makes is that they often make predictions on an individual based on averages of a group. Starr says these predictions can predict with reasonable precision the average recidivism rate for all offenders who share the same characteristics as the defendant, but that does not make it necessarily useful for individual predictions.

The Future of EBS Tools

The Model Penal Code is currently in the process of being revised and is set to include these risk assessment tools in the sentencing process. According to Starr, this is a serious development because it reflects the increased support of these practices and because of the Model Penal Code’s great influence in guiding penal codes in other states. Attorney General Eric Holder has already spoken against the practice, but it will be interesting to see whether his successor will continue this campaign.

Even if EBS can accurately measure risk of recidivism (which is uncertain according to Starr), does that mean that a greater prison sentence will result in less future offenses after the offender is released? EBS does not seek to answer this question. Further, if knowing there is a harsh penalty for a particular crime is a deterrent to commit said crime, wouldn’t adding more uncertainty to sentencing (EBS tools are not always transparent and sometimes proprietary) effectively remove this deterrent?

Even though many questions remain unanswered and while several people have been critical of the practice, it seems like there is great support for the use of these instruments. They are especially easy to support when they are overwhelmingly regarded as progressive and scientific, something Starr refutes. While there is certainly a place for data analytics and actuarial methods in the criminal justice system, it is important that such research be applied with the appropriate caution. Or perhaps not at all. Even if the tools had full statistical support, the risk of further exacerbating an already disparate criminal justice system should be enough to halt this practice.

Both Starr and Holder believe there is a strong case to be made that the risk prediction instruments now in use are unconstitutional. But EBS has strong advocates, so it’s a difficult subject. Ultimately, evidence-based sentencing is used to determine a person’s sentencing not based on what the person has done, but who that person is.

  1. October 21, 2014 at 7:57 am

    So EBS may be arguably bad, but what better alternative are Holder, et. al., proposing?


    • October 22, 2014 at 10:48 am

      I don’t know if I have the right sensitivity to the right words to properly interpret, but it seems to me that DoJ has a preference for sentencing guidelines that are based on two factors:
      (1) the crime committed
      (2) past criminal history
      both of which are “proven” as convictions through the trial process (or guilty pleas).

      This was the framework established in the 70s/80s as part of “truth-in-sentencing.”

      They seem rather supportive of model-based approaches for determining treatment and, I guess, parole decisions.


  2. Dennis
    October 21, 2014 at 9:07 am

    This is the topic that you wrote about before, and I think you would be interested in the article “Grooming Students for A Lifetime of Surveillance” http://modelviewculture.com/pieces/grooming-students-for-a-lifetime-of-surveillance HT http://www.schneier.com


  3. October 21, 2014 at 9:16 am

    EBS is yet another ploy to hide the largest cause of recidivism which are the corrections officers in the prisons. They are abusive, dehumanizing, demoralizing and in many documented cases rapists and criminals themselves. After having your humanity destroyed by ‘the (wo)man’ then released into society what results do we expect to see. Until the corrections officers are trained to respect a persons humanity and trained to reduce recidivism and not increase it expect all these ‘scientific’ methods to do exactly nothing.

    But this is a feature not a bug because the corporate prison complex wants and needs to be over 100% occupancy.


    • October 21, 2014 at 10:11 am

      Can you blame the first-time rapist or murderer on abusive guards or the corporate prison complex? Should rapists and murderers just be free to roam to commit more crimes rather than being incarcerated?

      I’m not defending the current system – which is very easy to criticize – but can someone come up with alternatives?


      • October 21, 2014 at 10:12 am

        Abe, the argument is not about whether anyone should ever go to jail. It’s about how we determine the recidivism risk.

        On Tue, Oct 21, 2014 at 10:11 AM, mathbabe wrote:



        • October 21, 2014 at 10:56 am

          Cathy, fair enough. Is there an alternative way to calculate recidivism risk? Will that alternate method make it safer for law-abiding citizens? Will it be fairer to criminals?


        • Nathanael
          October 24, 2014 at 1:04 pm

          Serious studies have shown that certain crimes have high risks of recidivism (rape) and others have very low risk of recidivism (first-degree murder).

          Of course, everything is skewed by the fact that over half of the prisoners in the US are in prison for things which shouldn’t be crimes at all, like selling marijuana. And also skewed by the extremely high rate of frame-ups and false convictions thanks to police/DA corruption.

          Frankly, I think the sentencing rule should ask whether the criminal abused a position of power. The more that is true, the longer the criminal needs to stay locked in a box. So G W Bush and Dick Cheney need to be imprisoned for life — if that’s enough, they might need to be executed — while ordinary petty criminals shouldn’t generallly be imprisoned at all.


  4. murray
    October 21, 2014 at 1:16 pm

    Is it generally believed to be okay to sentence different poor people to different sentences than rich people in the USA? I realize it happens everywhere, but it seems weird that you bring up all these points about how it’s wrong to target poor people with harsher sentences because those poor people are predominately black and latino… without mentioning “oh yeah, by the way, we’re officially on the books giving poor people harsher sentences than rich people, even if they don’t have better lawyers. What’s up with that?”


    • October 22, 2014 at 6:51 am

      No, it’s not okay to sentence people to longer prison terms because they are poor. At least I don’t think it is. Honestly all this stuff is a question of society’s notion of fairness, and different societies obviously think differently.

      I think the focus on race is historical, stemming from a long and disgusting history of racist policies that are officially “color blind”. Also, race is a protected class but being poor isn’t, so it’s also a question of legal recourse.


  5. MikeM
    October 21, 2014 at 10:26 pm

    Thirty years ago I wrote a book, Recidivism; those of you who are interested in the topic can find it on the web. Let’s face it, recidivism is useful for some things but not others — specifically, for evaluating *programs*, not *people*. An apt metaphor: knowing the wind velocity (an aggregate property) tells you next to nothing about the trajectory of individual molecules of air. Knowing the recidivism rate of a group of releasees may tell you something about their subsequent behavior, as a group, but not enough to forecast individual trajectories.


  6. October 22, 2014 at 11:06 am

    I found the sections on punishment in the Illustrated Guide to Criminal Law to be useful background: http://lawcomic.net/guide/?page_id=5. The five objectives of criminal punishment/sentencing:
    – Rehabilitation
    – Deterrence
    – Removal
    – Retribution
    – Retaliation


  7. mike
    October 24, 2014 at 7:57 am

    Pre-sentence data such as described given to judges has gone on for decades in pre-sentence investigation reports/reviews, and the judges have used them completely subjectively based on their “gut” to determine idiosyncratic sentences based on their own beliefs and prejudices. What MO does with its system actually provides more systematic and fairer information not just for judges but for those who would audit and hold accountable. It’s interesting that no one in this post, including its author, discusses the MO sentencing commission effort to use risk assessment to reduce prison assignments in sentencing, an effort that got shot down by the state DAs who prefer their own “guts” to that silly old data analysis. In fact, when you look at sentence effectiveness in terms of risk levels and later recidivism after release of offenders in the aggregate, you will find support for both alternative sentencing than prison and for less harsh sentencing for low and moderate risk offenders. IOW, you can end up making a case for less prison overall and for lower sentences when used IN THE NAME OF PUBLIC SAFETY, the last refuge of the “gut” advocates.

    The actual problem is indeed that the risk assessments should not be used for individual cases on the front end, a practice that will undermine its own credibility with the failure of “low” risk offenders showing up on the front pages of state newspapers (and, very unfortunately, politicizing assessment’s valid uses for treatment purposes as “gut” advocates begin to develop their own dueling assessments that get the harsher results they want). The proper use of the assessments for sentencing policy would be to inform policymakers about the likely impact of particular sentences for particular types of offenders, not individual ones. At the very least, opponents of our overincarceration could use the data to generate questions for our “gut” DAs and judges about why they are deciding on sentences demonstrated to produce less public safety because the recidivism rates associated with them are higher than the lower sentences forgone.

    So concern on this topic, while somewhat valid, should be better informed and less knee-jerk, and should particularly avoid citing our intellectually challenged Attorney General who manages to raise his uninformed voice on risk assessment but has nary a word to say about the federal sentencing guidelines which are the epitome of what he claims to oppose. Opponents will find themselves on the same side as the “gut” professionals [sic] who have given us our imprisonment embarrassment and will be the beneficiaries of the uninformed shutdown of the use of these assessment tools.


    • joe
      October 24, 2014 at 10:24 am

      Our sentencing and corrections policies have lurched from the
      “rehabilitation ideal,” which predominated through the early 1970s,
      to the retribution-minded “just-deserts” model, which has
      predominated over the last thirty years. We have essentially gone from
      the extreme of trying to rehabilitate everyone to the extreme of trying
      to rehabilitate no one.



  1. No trackbacks yet.
Comments are closed.
%d bloggers like this: