BALTIMORE COUNTY, Md. — After an AI weapon detection system at a BCPS school mistook a Black student’s bag of chips for a gun, many people, including the boy’s family, claimed it was racial bias.
WATCH: State officials say AI system didn’t discriminate based on color when it mistook students’ bag of chips for a gun
BCPS AI cleared of bias after bag of chips triggers gun alert
But a report from the state’s Inspector General of Education says that’s not the case.
An AI weapon detection system used by Baltimore County Public Schools, called Omnilert, mistook 16-year-old Kenwood High School student Taki Allen’s crumpled bag of chips for a gun.
At that moment, Allen didn’t know what to do.
“The police came, like eight police cars, and then they all came out with guns pointed at me talking about getting me on the ground. I was putting my hands up saying, ‘What’s happening?’ He told me to get on my knees and he arrested me and handcuffed me,” Allen said.
The Maryland Office of the Inspector General of Education investigated the incident.
A recently published report states that the AI system flagged the position of Taki’s hand while he was holding an object as a potential weapon threat.
The office investigated claims that the system discriminated against students of color, but found no evidence of bias.
However, Taki Allen’s grandfather, Lamont Davis, still believes that the AI system is designed to automatically detect black people.
“Let’s say a white woman with a dog walks there on a Sunday morning and sits down, the AI system won’t give a false positive. But I think if a black person walks there and walks their dog on the property, it will give a false positive,” Davis said.
Although the OIGE found no bias, it nevertheless highlighted flaws in the situation.
The report states that the principal at Kenwood High School did not see the cancellation notice from other staff members and contacted police about an already canceled threat, revealing flaws in the system.
And once law enforcement gets involved, the report says BCPS loses control of alert notifications, potentially creating dangerous situations.
OIGE recommends that the BCPS School Safety Department review its protocol regarding the number of individuals who receive alert notifications, and also calls on BCPS to conduct biannual training for school executive staff and members of the Baltimore County Police Department on the use and established notification protocols when a threat is verified using Omnilert.
In an earlier statement, BCPS said the district is committed to taking additional steps to ensure all staff fully understand and follow established protocols to prevent such incidents from recurring.
