"We need to ask," said Holder, "whether [the war on drugs] has been fully effective and usher in a new approach". As the war has ravaged black communities it is symbolic America's first black attorney general, whose boss is the first black president, should challenge drug warrior mindsets.
The National Association for the Advancement of Coloured People - mindful that the incarceration rate for African-Americans is six times the national average, with 40 per cent of black males in their 20s and early 30s locked up, according to the Pew Centre - has long campaigned for an end to mandatory minimum sentences and applauded Holder's move.
Holder's decision, revealed to the American Bar Association, was freighted with caveats: violent offenders with links to gangs or drug cartels were exempt. But it was a step towards confronting glaring realities: US prisons are full to bursting point, three-strikes laws have often exacerbated the problem, and more Americans tolerate the use of marijuana, classified by the US as a Schedule 1 banned substance with heroin and LSD.
Significantly, Holder's move did not spark a political or media backlash.
The upsurge in prison numbers began with harsh penalties imposed in the aftermath of the 1980s urban crack cocaine epidemic and associated rising felony rates. Politicians vied with one another to be tough on crime.
"The federal statutes can be draconian," says James Felman, vice chair of the ABA's Criminal Justice Section, expressing his own views. "You can have someone who got probation for two minor drug possessions in a state court get life without parole for their first offence in a federal court. That's a huge part of what Holder's announcement is about."
One of the most egregious US laws was the Anti-Drug Abuse Act 1986. It levelled the same five-term term whether you were caught with 5g of crack cocaine or 500g of powder cocaine. Crack plagued black communities, while powder was popular with whites. In 2010 the law was changed so that the ratio was no longer 100 to 1 but 18 to 1. "I believe the reduction in the penalty for crack cocaine is the first time in our history where the US has ever lowered the penalty for a crime," says Felman.
The federal move follows state efforts over the past decade to tackle the soaring costs of incarceration, a process accelerated by the 2008 recession.
"The states felt the economic pinch sooner," says Felman. "Corrections are a larger percentage of smaller state budgets." Washington is now feeling the pinch. Without reform US prison costs will jump 30 per cent by 2020.
Since 2007, led by Texas, about half the states have altered sentencing laws and sought alternatives to incarceration. In Texas crime and recidivism are down and US$2 billion in prison spending has been saved.
"There's more public support for treatment as an alternative to prison," says Marc Mauer, executive director of The Sentencing Project, which advocates reform.
"At state level many local counties have drug courts to divert people to treatment. I think what's held it back at federal level have been the mandatory penalties. Where judges have their hands tied."
As is often the case, California stands out. In July the state had 124,363 inmates, down from 170,588 in 2007. The US Supreme Court insists this total be reduced to 111,000 by year's end. California is trying "realignment", shifting some prisoners to county or out-of-state jails.
Meanwhile, a voter-driven law, Proposition 36, passed in November, modified the state's three-strikes law - castigated by the New York Times as "based largely on fear, ignorance and vengeance" - to focus on violent crime.
Perhaps the greatest impact of Holder's move, suggests Mauer, may be in the political realm, where lawmakers are pondering tentative reform. Bipartisan reform bills to give judges more sentencing leeway have appeared in the Senate, while the House of Representatives is looking to trim "over criminalisation", slashing the number of US drug statutes.
Last November voters in Colorado and Washington state approved initiatives that legalised marijuana, defying Washington, which has shown no signs of wishing to intervene.
"The mood of the country has shifted," says Felman. "It's a tough situation for the Attorney General, as marijuana possession is still a federal offence. On the other hand some states have made it legal. I think they [Justice Department] want to have some respect for the will of the people. My guess is he won't prosecute unless it involves a big-time organisation."
In addition, 20 states, plus the District of Columbia, allow medical marijuana. Dana Rohrabacher, a Republican congressman from California's ultra-conservative Orange County, is an unlikely advocate.
"I'm in Florida and medical marijuana is a big issue being debated," says Felman. "Five years ago it wouldn't have been viable. More and more states are discussing whether to decriminalise or legalise marijuana."
Elsewhere, Uruguay broke ranks with the US-led war on drugs this month when the lower house voted to legalise the production, sale and distribution of marijuana for adults. Other Latin American states, including Mexico and Brazil, have raised the issue of drug legalisation.
Mauer stresses Holder's move still amounts to merely "tinkering around the edges". Even if all drug offenders were released from prison, the US "would still lead the world in incarceration by far". The train on penal and drug reform may be leaving the station, as the war on drugs eases slightly, but America is just beginning the debate on what it plans to do next.
