Thousands of American schools are turning to AI-powered surveillance technology for 24/7 monitoring of student accounts and school-issued devices like laptops and tablets.

The goal is to keep children safe, especially amid a mental health crisis and the threat of school shootings. Machine-learning algorithms detect potential indicators of problems like bullying, self-harm or suicide and then alert school officials.

But these tools raise serious questions about privacy and security. In fact, when The Seattle Times and The Associated Press partnered to investigate school surveillance, reporters inadvertently received access to almost 3,500 sensitive, unredacted student documents through a records request. The documents were stored without a password or firewall, and anyone with the link could read them.

Here are key takeaways from the investigation.

The privacy and security risks became apparent when Seattle Times and AP reporters submitted a public records request to Vancouver Public Schools in Washington, seeking information about the kind of content flagged by the monitoring tool Gaggle. Used by around 1,500 districts, Gaggle is one of many different companies offering surveillance services, including GoGuardian and Securly.

Gaggle saved screenshots of digital activity that set off each alert. School officials accidentally provided the reporters with links to them, not realizing they weren’t protected by a password. Students in these documents opened up about the most intimate aspects of their personal lives, including suicide attempts.

After learning about the records inadvertently released to reporters, Gaggle updated its system. Now, after 72 hours, only those logged into a Gaggle account can view the screenshots. Gaggle said this feature was already in the works but had not yet been rolled out to every customer.

The company says the links must be accessible without a login during those 72 hours so the school’s emergency contacts — who often receive these alerts late at night on their phones — can respond quickly.

The long-term effects of surveillance technology on safety are unclear. No independent studies have shown it measurably lowers student suicide rates or reduces violence. A 2023 RAND report found only “scant evidence” of either benefits or risks from artificial intelligence surveillance.

“If you don’t have the right number of mental health counselors, issuing more alerts is not actually going to improve suicide prevention,” said report co-author Benjamin Boudreaux, an AI ethics researcher.

Experts warn having privacy to express feelings is important to healthy child development. But proponents of digital monitoring point out school computers are not the appropriate setting for this kind of unlimited self-exploration.

Surveillance software poses unique risks to LGBTQ+ students, advocates warn. In the records released by Vancouver schools, at least six students were potentially outed to school officials after writing about being gay, transgender or struggling with gender dysphoria.

When Durham Public Schools in North Carolina piloted Gaggle, an LGBTQ+ advocate reported a Gaggle alert about self-harm had led to a student being outed to their family. Another student brought concerns about losing trust with teachers. The board voted to stop using the technology, finding it wasn’t worth the risk of eroding relationships with adults.

Parents interviewed for this article said their child’s school either did not disclose it used surveillance software or buried the disclosure in long technology use forms. Even when families are aware of surveillance, schools may refuse to let them opt out.

“Imagine growing up in a world where everything you’ve ever said on a computer is monitored by the government,” said Tim Reiland, who unsuccessfully lobbied his kids’ school district in Owasso, Oklahoma, to let his children opt out of Gaggle. “And you just have to accept it and move on. What kind of adults are we creating?”

____

The Associated Press’ education coverage receives financial support from multiple private foundations. AP is solely responsible for all content. Find AP’s standards for working with philanthropies, a list of supporters and funded coverage areas at AP.org.