The End of Human Accountability

Eric C. Williams

Managing Director @ Detroit Justice Center | Founder @ Eric C. Williams, PLLC

February 23, 2024

In late January, I took advantage of the public comment period at a meeting of the Detroit Board of Police Commissioners (BPOC) to address the Detroit Police Department’s (DPD) use of AI augmented surveillance, specifically, the Project Green Light (PGL) surveillance program. This program, deployed in 2016, has consumed tens of millions of public dollars, failed to demonstrate any impact on public safety, and presents a serious threat to civil liberties.  Despite these shortcomings, the Board of Police Commissioners (BPOC), the public body charged with police oversight, has consistently failed to, well, provide oversight of police surveillance. While change may be on the horizon, I worry it’s too little, too late.

Shortly after my public comments, I was invited to sit in on a February meeting of the BPOC’s Public Policy Subcommittee and share the concerns of Detroit’s civil liberties community. Following the meeting, Commissioners Linda Bernard and Willie Burton committed to examining and evaluating PGL’s cost, efficacy, and civil liberties implications. More importantly, the Commissioners were open to looking at the DPD surveillance as a whole. This may not sound groundbreaking, but it is. To understand why, it’s necessary to take a quick look back at PGL, its exponential growth, and its role in Detroit’s intrusive surveillance infrastructure. And then I’ll tell you why I’m worried it won’t be enough.

First, PGL is unusual in that it is a public-private surveillance program. According to program’s website:

Project Green Light Detroit is the first public-private-community partnership of its kind, blending a mix of real-time crime-fighting and community policing aimed at improving neighborhood safety, promoting the revitalization and growth of local businesses, and strengthening DPD’s efforts to deter, identify, and solve crime.

Participating PGL businesses (program “partners”) pay for the purchase and installation of HD cameras (a minimum of 4 per location), lighting, and signage. They also pay for subscription services to transmit video to the Detroit’s Real Time Crime Centers and store the video (about $140 a month). The estimated total cost to partners is between $4,000 and $6,000 dollars.

In return for joining the program, DPD, at its discretion, may:

• Monitor the Entity’s cameras, including, but not limited to, during emergencies and other exigent circumstances. However, the MOU does not oblige DPD to monitor the Entity’s cameras at any time.

• Convene a meeting of a designated DPD representative, City personnel, and community members to discuss public safety issues concerning the Entity and its surrounding neighborhood.

• Coordinate visits that may encompass, but are not limited to, entering the Entity, signing in at the Entity, patrolling parking lots and other parts of the Entity’s property, engaging loiterers, and working with Entity employees for the purpose of furthering law enforcement efforts.

You can read the entire agreement here.

Aside from the surveillance, PGL partners don’t seem to be getting much above what one would expect from the police. And even the surveillance component is at DPD’s discretion.

Based on the agreement between partners and DPD, PGL seems to be only a modestly obnoxious surveillance program. Yes, there were grumblings that some business owners felt pressured into participating. Yes, there were concerns about police surveillance, but since the cameras were on private property, there wasn’t much that could be done about it. Plus, DPD insisted that the City of Detroit wasn’t paying anything for the program, so what the heck. Finally, with only a dozen or so participating businesses, how big a threat was it really?

That all changed in 2019, when the proposed renewal of a software license used for the program brought to the public’s attention that the system possessed facial recognition capabilities. It also quickly came to light that DPD didn’t have a single internal policy in place to govern the use or punish the misuse of facial recognition data. Detroiters were not happy. Nope, not one little bit. The Board of Police Commissioners delayed a vote to approve the software contract…then delayed again, giving DPD time to make its case publicly and develop a skeletal policy. Protests continued and eventually a more comprehensive, but still flawed, set of policies and procedures was adopted and the contract was approved.

Those policies are still in place and now there are more than 1,000 PGL participants with more than 5,000 cameras. However, the rapid expansion of PGL owes more to DPD propaganda than to actual effectiveness. It is gospel among senior Detroit police officials that PGL is effective. Citizens and partners are frequently trotted out by DPD to provide anecdotal evidence of PGL’s impact. It is a common talking point of Detroit officials that all of this “success” is free of charge to the City of Detroit.

Neither of these things is true.

PGL partners pay for the lights, signs, and cameras but the rest of the infrastructure is paid for and maintained by the City. The Real Time Crime Centers. The monitors. The people to watch the monitors. The software on the back end. In total, more than $20 million in public dollars have been spent on Project Greenlight since 2016. And that’s what we know about. Because PGL isn’t a budget line item, no one outside DPD knows how much is being spent on it. Not City Council. Not the Board of Police Commissioners. And certainly not the public. What we do know is that no matter the costs, Detroiters are being over charged.

PGL isn’t a bargain at any price because THERE IS NO EVIDENCE IT WORKS. I repeat, there is no evidence it works. There is, however, evidence it does not. To date, there have been two studies of PGL’s impact. In 2020 the Michigan State University School of Criminal Justice found PGL had “limited impact” on trends in crime. In 2022, the research arm of the U.S. Department of Justice, the National Institute of Justice, rated PGL as having “no effect” on crime. In short, there is no evidence PGL makes communities safer and two substantive studies showing PGL does not make communities safer. Despite this, the program continues to grow and siphon public dollars from other services. (I’m pretty sure if you asked Detroiters how they would spend $20 million to make Detroit safer, surveillance would not be at the top of the list.

[1] Despite PGL’s expansion and the steady support of Detroit officials, public concern over PGL never really went away.

[2] Finally, the bubbling discontent, paired with a false arrest based on a faulty facial recognition identification, prompted Detroit City Council to take action.

[3] Led by Mary Sheffield, then Council President pro tem and now Council President, Detroit City Council passed the clumsily named Civilian Input Over Government Surveillance Ordinance (CIOGS), a watered down version of the model Civilian Control Over Government Surveillance ordinance, promulgated by the American Civil Liberties Union, the Electronic Frontier Foundation, and other civil liberties groups. CIOGS doesn’t govern surveillance, but it does impose heightened disclosure and notice requirements for any procurement requests for surveillance technology. Unfortunately, CIOGS’ shortcomings are magnified by the DPD’s willingness to ignore it.

[4] What CIOGS does get right is applicability to all surveillance technology purchases. PGL is only one of many surveillance programs operating in Detroit. Others include:

• gunshot detection system (formerly ShotSpotter, now SoundThinking)

• drones

• cell-site simulators (also known as IMSI catchers or Stingray)

• license plate readers

• smart traffic cameras

These surveillance tools, as well as video from private cameras, feed data to Detroit’s Real Time Crime Centers (RTCC). The primary RTCC occupies 9,000 square feet in Detroit Police Headquarters. It initially cost $8 million, funded by municipal bonds, and was expanded in 2019 at a cost of $4 million. The expansion included the creation of two smaller satellite centers.

I want to pause here to point out that the main RTCC is big. Really big. And there is a LOT of information flowing into it. In addition to video feeds from almost 5, 000 PGL cameras, there is the data from ShotSpotter, more than 100 cameras on smart traffic cameras, feeds from an unknown number of private cameras[5], and data from license plate readers. I should also mention that the database used by Michigan law enforcement (SNAP) contains mugshots, surveillance images, and all Michigan driver license and personal ID photos as well as identifying information. (Michigan places limits on how this database can be used, but local agencies are free to create their own databases with different limitations.) You get the point: there is a lot of information available to DPD and other law enforcement agencies.

Oh wait, did I forget to mention that all of DPD’s surveillance tools are available to state and federal law enforcement agencies upon request? Yeah, that’s really a thing. ICE. FBI. ATF. All the Alphabet Boys. Homeland Security.

In the past, this is where I talked about concerns with facial recognition accuracy (melanated people beware) and law enforcement abuse (COINTEL PRO, Ghetto Informant Program, surveillance of Muslims after 9/11). In a city with large Black, Arab, and Muslim communities and a long tradition of civil rights and labor activism, these are important issues. However, there is another issue that isn’t getting nearly enough attention: Human accountability.

Once you look at the number of individual programs and the amount of data captured, it’s obvious that AI is going to play an increasing role in managing and analyzing all that data. It’s what AI is best at, taking massive amounts of information, finding patterns, and making it make sense for humans. And once you recognize the variety of surveillance programs, their distinct capabilities, and the inconsistent policies on the usage, you realize meaningful human oversight is impossible. Right now, DPD insists that trained human beings make the final call when it comes to arrests, but given the amount of data involved, it is only a matter of time before that changes.

Welcome to the end of human accountability.

This is more than a little frightening. Accountability (i.e., consequences), whether civil, criminal, or social, are what keep our society both in check and moving forward. It’s what holds civilians, politicians, corporations, and law enforcement legally responsible for harms they cause. It’s the mechanism that prompts change when the system doesn’t work. But what happens when there is no one to hold accountable? Computer error is an unsatisfying explanation for your credit card being declined. Imagine that being the explanation for being arrested. Worse yet, imagine it’s not an error; it’s simply the way the algorithm was meant to work. And there is no redress, no one to make you whole, and thus no incentive to fix the problem.

Which brings me back to the willingness of the Detroit Board of Police Commissioners to examine the totality of Detroit’s surveillance infrastructure. Right now, the many surveillance programs already in place have minimal oversight. The overall infrastructure has none at all. The first step in ensuring accountability is to take stock of the surveillance environment that has grown in Detroit like a weed. From there, we can begin to identify the threats and design policies that protect us from AI errors…and impose accountability for them on real live human beings. If we don't? Well, good luck suing AI in court. 

[1] I feel compelled to note that I don’t have a beef with DPD. Chief White wants the same thing I do – a safer Detroit. We just strongly disagree on how to get there.

[2] Here is a good summary of the issue, starting at the 9:40 mark.

[3] Since then, there have 2 additional documented cases of false arrest linked to facial recognition.

[4] Disclosure: I am currently counsel, through Detroit Justice Center, along with The Sugar Law Center for Economic & Social Justice, and Jack Schulz Law for plaintiffs suing DPD and Detroit officials over their failure to comply with CIOGS’ notice and disclosure requirements in acquiring the gunshot detection technology known as ShotSpotter.

[5] A number of large companies operate video surveillance networks in Detroit's downtown and Midtown, including Bedrock, DTE Energy Co., Blue Cross Blue Shield of Michigan, and Ilitch Holdings Inc. All share video with DPD when they deem it necessary.