• 🥳 📚 Internal bigtech company documents are publicly accessible and searchable. The lead law firm in the social media addiction MDL (against Meta, TikTok, YouTube, and Snap) has published a publicly accessible archive of internal company documents from the litigation. A multidistrict litigation is basically a way of pooling together a large number of similar lawsuits into one court proceeding — in this case nearly 3,000 individual “actions” in the Northern District of California.

• Apple and Google are profiting off of nudify apps, and the San Francisco’s City Attorney said no. The apps (marketed as harmless face-swapping tools) generate nonconsensual nude images… and some of these were rated appropriate for children. The interesting thing here is that the City Attorney is not arguing about content moderation or liability — he’s arguing that Apple and Google are profiting from apps that create non-consensual intimate imagery. The problem here is that these are reactive consequences, and for kids and parents… the damage is already done. So while hooray and thank you SF, the bigger question is: Why are apps like this legal, and why are they able to target kids?

• Dr. posted this week about how the volume and frequency of messages kids are receiving disrupts key social experiences. She talks about a child who got 42 messages during a two-hour ballgame, along with 1,452 Snapchat messages in a single day. She says “This is not socializing. Socializing has a start time and a finish time. This many messages in one day is deprivation.” She ends by suggesting that parents teach messaging etiquette before giving kids access to messaging apps. And yes, we can say who has time for this? but we can at least model it in our own lives.

• Meta AI may already have access to your entire camera roll — every photo, every screenshot, including ones you never shared. ParentsTogether shared a quick tutorial on how to check and shut it off in under 60 seconds. Make sure your people know about this.

• Surveillance is officially a luxury. Madison Square Garden (MSG) famously has a sprawling surveillance system that can monitor guests down to the second. But apparently that can be turned off if you’re fancy enough because some lucky few apparently deserve privacy, while the rest of us need to just shut up about it already. Basically MSG turned off its facial recognition system for Taylor Swift and Travis Kelce’s rehearsal dinner, because of a “ privacy request.” For the record… this has never happened before, not even for events hosted by NBCUniversal or the NYPD. The question for us to consider here: should the wealthy get to opt out of surveillance systems the rest of us are subjected to by default?

• Meta is testing an AI app that generates personalized bedtime stories for children. Parents upload a photo of a toy or person, choose a world, and select values like “kindness” or “courage.” Meta confirmed the pilot is available only to users over 18. This is the same company being sued in thousands of cases for knowingly addicting children to its platforms… and now they are building AI bedtime story apps for the kids jumping into their platform pipeline. I wish I could say I’m surprised, but … shrug. And while I don’t agree that it’s “reasonable” I enjoyed 9to5Mac’s take: “StoryKit sounds like a perfectly reasonable app that would be well-received if it were from almost anyone but Meta.”

• A New York school district bought a humanoid robot teacher, and planned to deploy it in 11th and 12th grade classes this fall. I honestly don’t even know where to begin with this one. The pilot is paused while they “work through enhanced student data privacy agreements” but you have to hear this. So first: the robot will have access to student learning data, and will apparently identify students by number… Yes, apparently “personalized learning” involves students being treated as if they are literal objects… which is less surprising when you know the history of the company making the humanoids: Realbotix. This company is a mashup between two companies that were formerly working on a combination of crypto and sex dolls. I am not joking. Realbotix was formerly known as Tokens.com — a company that helped customers use cryptocurrency to rent “digital land” in the Metaverse. In April 2024, the company acquired Simulcra, the parent company behind RealDoll, known for hyperrealistic sex dolls and later sex robots. I just… WHAT. PS: this district is part of a native reservation, and immediately had me thinking about the brutal history of testing on native populations.

• crisply lays out what a post-literate society is (and warns it’s coming). She says it’s less about people not being able to read, and more about a world in which text is no longer the primary medium for knowledge and ideas. A world in which text is replaced by short-form video, audio, and AI-generated answers… a world in which cognitive friction and sustained attention whither away. And when you consider the changes happening with search, it’s not hard to imagine a very near-term future in which our entire world view is pre-packed and delivered to us by an algorithm that someone else has programmed.

Sources & mentions this week: WIRED, New York Focus, Motley Rice, ParentsTogether, The Rundown AI, , Dr. , Futurism

Brain Snacks

In short: I absolutely loved it. Like the kids say: 10/10. They nailed it… on so many levels. It presents a complex topic with nuance, care, and even love. I cried… several times.

For those who haven’t seen it yet, the basic plot is this:

The little girl that we met in Toy Story 4 (Bonnie) is shy, and having a hard time connecting with other kids. Her parents want to help, so they buy the toy that all her peers are using (a tablet) hoping that it will help her connect with other kids and make friends. Very quickly, Bonnie gets sucked into a platform that demands and reinforces time on the device, as opposed to what she is more personally interested in: unstructured creative play with her toys. But she essentially gets peer-pressured into using this thing that she actually doesn’t like. The main turning point comes from the most drama-filled corner of devices: a group chat.

There are so many reasons to love this movie, but here’s what especially stood out to me:

1/ The parents and toys are all trying to help, but they take different approaches.

(… and while well-intentioned, the parents’ approach makes everything worse.)

This was instantly fascinating to me, and is a tension that the movie captures perfectly in the first few scenes. The toys and the parents are both worried about the same thing, and desperately want to help. While the parents succumb to group-think and social pressure, and take the path of least resistance (they just buy the tablet), the toys seek to understand the root of the problem, and discover that the tablet is what’s stopping kids from doing what Bonnie wants to do most: unstructured imaginative play.

The tragic irony that the movie sets up beautifully from the first few scenes, is that everyone cares about Bonnie, and wants her to connect with other kids and build relationships. But the parent’s hyper-focus on what other kids are doing leads them to overlook what their daughter actually wants and needs — which is other kids who share her interests, and a space in which to pursue them.

2/ The entire plot was about connection and relationships — and what devices do to both of those things.

So much of what we hear about kids and tech is dominated by content and contacts: what kids are consuming, and who they are interacting with. What we hear far less about is the less tangible, less measurable impact of technology: how it shapes our experiences, relationships with others, and ultimately with ourselves.

Throughout the movie, we see devices replace (not facilitate) relationships, and even erode Bonnie’s own sense of self. I found myself wishing I could see the Inside Out version of Bonnie’s brain throughout the movie — get on it, Disney / Pixar!

As the toys discover when Jessie investigates the neighborhood, devices have become a wall between people, preventing them from interacting in the most basic of human ways. From house to house, Jessie sees kids and parents all consumed by whatever is happening on the screen in front of them… even when they are sitting right next to each other. And then we watch as the device (and all the people pushing it) leads Bonnie to abandon herself in pursuit of someone else’s idea of friendship. (BTW Snapchat is extremely guilty of this.)

3/ The FOMO is real…

The “fear of missing out” that children experience is hard to quantify, but is something that parents around the world have seen in their homes regularly. When I spoke to parents about the movie, this is the part that seemed to resonate most universally.

This experience was on full display when Bonnie wakes up and Lily Pad has no batteries. She completely freaks out, and unravels in a panic. Her dad scrambles to get it working again.

This tracks with research about the effect of Snapchat on kids, and how “streaks” create an obsessive relationship with the app, rooted in a fear of missing out. For example: studies in Austria and Belgium found that teens share their Snapchat passwords so others can maintain their streaks when they cannot. The FOMO situation related to social media is a global phenomenon, though from what I see in my homeland (Greece) it’s less intense… possibly because the kids have so many unstructured in-person alternatives.

4/ Childhood is being shortened

This I found extra fascinating, and extra painful. In several moments, the movie points out that childhood is being cut short. In one part, kids tease Bonnie for “still playing with toys.” In another moment, Jessie is in Blaze’s room, and as she looks around exasperated, she shouts something along the lines of: This doesn’t look like a kids room!!! Where are the toys?! Where is the mess?! Why is it so clean?

Preach, Jessie.

Childhood happens once, and it flies by. Grass-stained clothes, dinosaurs set up for fights for weeks on end, stuffed animal classrooms, piles of cool sticks and rocks their found stuffed into jars (and sometimes random parts of their closet…).

We should be protecting it all — not turning them into mini adults.

I’m not advocating for chaos and disorder. We should still teach them to put their things away, and/but… there’s some degree of mess that comes along with a brain that is learning and exploring the world through play. (FWIW, the line I draw is for toys that are set up for a particular purpose… not just random toys and clothes thrown on the ground because they can’t bother to put it away.)

5/ The drama comes from a group chat.

The drama and heartache is not coming from some creep on social media… (though yes, those exist.) It’s coming from social dynamics among peer groups, in a group chat.

Every single parent I have spoken to has a group chat story. Every. Single. One.

The public narrative is very focused on social media as the enemy. I don’t disagree that it’s awful, but it’s also an incomplete story. And if we actually want to address any problems, we have to be clear-eyed about where kids are actually spending time with their peers, and what is happening.

Unfortunately for us, researchers follow grant money, and social media is the hot ticket right now — not chat groups. So even though people working with kids every day can see that online bullying is worse when school is in session, and group chats are a huge source of conflict… the quantitative data does not seem to be there (yet).

But as I’ve said a thousand times: parents need to make decisions right now, and there has to be room for observable reality… like: group chats are clearly a significant source of drama and heartache.

6/ The “old school” technology highlights that every generation of technology has promised convenience and personalization.

I love this one. The old school tech is a cute throw-back that highlights two critical points:

1. Every generation of technology has been sold as a convenience, and promises to “solve” a problem that technology cannot actually solve, and

2. In the world of technology and consumer-facing products, “connection“ is something to be achieved. It’s a box to check, a thing to obtain… It is not connection. It is “a connection.“

In one scene, the Lily Pad argues with Jessie… saying that I know her, I want to help, and the kids are HERE! These are the arguments parents everywhere have been hearing for years: this is where the kids are, these products understand your kids better than you do and can help because personalized blah blah blah. YOU NEED ME…. Jessie sees right through it, and claps back that those are not real relationships.

We (especially in the United States) are so focused on optimization and measurable outcomes, that we often forget that the things we need most — and the very things that make us human… that make us feel alive, worth something, and loved — are not measurable.

Digital products create the illusion of connection by making it visible and measurable, while eroding the very skills and experiences that lead to real and lasting connection. Our obsession with metrics and optimization is killing our (and our kids) souls. AI is taking this to the next level.

7/ The role of the parents.

This one is hard. The very uncomfortable reality that they surface in the movie is that parents play a huge role in all of this. Yes, it’s a collective problem, and I am fully aware of the complexity. But the movie very tactfully demonstrates how parents factor into things. It is the PARENTS who….

1. … use technology constantly, and they are frequently holding a cellphone or on their computer.

2. … accept the notion that this is just what friendship looks like now, and follow the crowd (without considering that perhaps this is not the right crowd for their kid…)

3. … buy the tablet and just hand it to their (very) young kid with zero direction or guidance.

4. … allow their kid to use the tablet completely unattended.

5. … leave the tablet in the kids’ room overnight.

6. … allow kids at a sleepover to bring their tablets and spend their entire sleepover on them.

I am sympathetic to the fact that parents have been largely in the dark about digital products, including what they are, how they work, and what the impact on kids can be. It’s easy to assume that the products our kids have include some sort of limitations or guardrails, just as is true with so many other products we buy. But we now know enough to be certain that this is not true, and that the danger zone is in ANY internet-connected device and algorithmically-driven experiences.

I have three young kids ranging between Bonnie and Blaze’s ages. I am acutely aware of the realities and challenges of being a working parent with young kids, and understand the draw of just handing a screen to a kid because how the hell else am I supposed to get everything done and have just 10 @#$%ing seconds during which 3 kids are not simultaneously asking me for something.

And/but… it’s time we accept the fact that we are part of the problem. Not me or you specifically. But all of us. We did this to them, collectively. (And so did our policymakers.)

It’s not THEIR fault they grow up thinking this phone has magical properties, or that it’s THE symbol of adulthood. We carry around these products as if our lives depend on them.

It’s not THEIR fault that kids don’t play outside.

It’s not THEIR fault that the only opportunity they have to play is in highly structured activities with constant adult supervision (and often direction).

It’s not THEIR fault that some parents monetize their children online, and create entire subcultures of kids content that mirrors adult content (like Get Ready With Me routines, and home organization products).

It’s not THEIR fault that their parents don’t know that the products they are using are literally designed to exploit their weaknesses in order to absorb every ounce of their attention at all times.

And it’s definitely not THEIR fault that our policy-makers have allowed and even enabled all of this to happen, in the name of innovation.

I could go on but you get the picture. It is time we WAKE UP, and take control of our own relationship to devices, because our kids are watching us. If we behave as if the phone is the most important thing in the room, they will believe that it is.

7/ The fix comes from a moment of trust.

The devastated little girl shows her mom what happened on the device. The mom listens, and responds with love and care.

Build that trust now, because this moment is coming for all of us.

Final thoughts…

I found the movie really refreshing, especially considering the overly simplistic takes that we see all over the media portraying tech as either the entire problem, or the entire solution.

Somehow, the crew behind Toy Story 5 understands what is ultimately an incredibly complex and multi-faceted problem, and communicates it in a way that is clear, relatable, and even uplifting. They focus on relationships, connection, and unstructured imaginative play — because THAT is what adds up to the beating heart of childhood.

• This buried iPhone feature turns your smart phone into a “dumb phone.”

• Meta created an AI feature that allowed any user to tag a public Instagram account and ask the chatbot to create new A.I. photos based on it… The consent for this was on by default for public accounts. It quickly (and very predictably) went to a dark place, and Meta pulled down the new feature. While I’m glad the media was quick to cover it (and that Meta reacted) the bigger problem is that companies assume they can do stuff like this. And why wouldn’t they? Tech companies have been allowed to operate as if the rules of our world do not apply to them. They paused for now, but as we speak, they are building more AI features and integrations for WhatsApp, Facebook and Messenger, and I am sure video is next.

• In a move that will pretty instantly be weaponized (especially against women and children) Meta apparently embedded face-recognition technology for its smart glasses into Meta’s AI app. While it’s not fully operational yet, they’ve essentially put all the pieces in place for what would ultimately be a global facial-recognition system based out of a consumer product. And this is apparently not new… it’s a restart of a facial recognition system that they shut down several years ago due to privacy concerns. BTW I am FAR more worried about this than social media.

• On a related note… It turns out people who have Meta glasses are paying people on Facebook marketplace to remove the light that indicates they are recording… so the people around them won’t have any indication they are being recorded. (Maybe because the backlash is so strong that owners are afraid to wear them in public. Good.)

• Henrico County, Virginia (home to 37 data centers) sent its schools an email asking employees to “conserve electricity.” The county expects a 25% rise in electricity costs next year, adding an estimated $5 million to its budget. Employees were advised to close the blinds and turn off their computers (yes, seriously) while 17 more data centers are being built. It remains unclear to me why schools are footing the bill for AI infrastructure expansion.

• wrote a fantastic essay about how the biggest threat AI poses to students is not what it does to their writing, but what it does to reading. He argues that when students offload reading to AI summaries, they never build the foundational vocabulary, background knowledge, and cognitive patience that make it possible to read anything complex, including AI output itself. The students who use AI most effectively, he found, are the ones who already read and write well. In other words… It’s widening the gap, not closing it (as supporters argue it will).

• Waymo called the cops on two 15-year-olds. The teens were allegedly drinking and shooting toy guns, which triggered a “safety response.” This is a very slippery slope, and all I will say right now is this: automated systems have a long and well-documented history of errors (see examples here and here) and when lives and reputations are on the line, allowing a robot on wheels to preemptively call the cops is asking for trouble and tragedy. Remember: these systems reflect the beliefs and assumptions of the people that create them, and the context that they were “trained” in.

• AI clinical documentation tools are hallucinating nonexistent medical issues in patient records. A special report out of a Canadian audit warned that AI medical scribes were “not evaluated adequately,” and may present “fabricated information.” These tools claim to reduce administrative burden, but they are creating far more of a problem… and clinicians are essentially being forced to use them.

• AI isn’t exactly going as planned… CEOs of countless companies are not seeing a return on ROI. Mark Zuckerberg, admitted that AI isn’t working out the way he imagined, Google’s Gemini ran a coffee shop and was kind of a disaster, and software developers are saying that AI is rotting their brains. I hope those advocating for AI in schools are taking note…

• Meta’s AI data center leaked Legionella bacteria into a town’s irrigation water. The cooling systems used in large-scale data centers require enormous amounts of water and create conditions where Legionella (the bacteria that causes Legionnaires’ disease) can thrive. If Legionnaire’s sounds familiar, it’s because there’s an outbreak in Manhattan right now. I am not saying data centers lead to Legionnaire’s disease… but. BUT. This is probably worth considering as people grapple with the impact of data centers on neighborhoods and communities. (Because apparently cooling systems are a common source of the bacteria.)

• I wanted to end with a quote from my incredible friend & colleague who recently wrote about how to ban surveillance pricing — the pretty universally opposed practice of companies using your data to price gauge you. This quote in her recent article is a perfectly stated reminder that what these companies create and do is a policy choice. And it absolutely applies beyond surveillance pricing:

“In a political climate where voters are looking to elected officials to stand up to abuse and to deliver results, policymakers must remain vigilant. Anything less is a failure to govern at a time when working people are becoming live test subjects, absorbing all of the economic and social risks of transformative technologies.”

Sources & mentions this week: 404 Media, NPR, Futurism, Forbes, Euronews, , , , The Capitol Forum, NPR, ACLU, BBC, , WIRED

Brain Snacks

I am aware that they do a lot on “child safety” and that they are a platform that researchers and journalists think (or thought) highly of. I respect that. At the same time, I’m not here to debate the intentions or track record of platform safety. My interest is in helping parents grapple with the impact of the products their kids are using, and providing clarity and perspective that might help them make informed decisions.

I (fairly) got a lot of flak for getting a detail wrong about a feature on the platform, which I quickly corrected. But what was fascinating to me about that moment was that my point about the impact of the platform on something as central to childhood as friendship, was largely dismissed as outlandish. The push-back I got was largely about how great Snapchat was and how hard they were trying, and all of the great things they are doing to keep kids “safe” on their platform. Ok. But safety was not my point. My point was how the platform was affecting the very idea and experience of friendship, which is CENTRAL to childhood, and the creation of identity.

Here we are, 2 years later… and this very topic is now a common point of discussion. It was even a central part of the new Toy Story movie.

So, I wanted to dust off this old post, because I think it holds up well.

(Reposted from a now-dormant newsletter in February 2024.)

As I passively observed the Superbowl ads this year, one in particular grabbed my attention: Snapchat.

There is a *lot* to unpack in that ad, which came on the heels of Snap breaking ranks with its trade group NetChoice to oppose the Kids Online Safety Act (which has its own issues as lays out here), but suffice it to say: they really really don’t want to be considered a “social media” company.

Do not believe them.

Any time I interact with teenagers I ask about their friends, and inevitably Snapchat becomes the center of the conversation. In a recent conversation with a dear friend’s 13-year old son, he lamented the pressure to use “that stupid app.” This is where I learned about “streaks” and I — and his parents — had to collect our jaws off of the floor. (I know I’m not breaking any news here, but it’s far from common knowledge and I think it is worth highlighting here for our policy friends, and possibly some parents.)

In short: “Streaks” are a feature that essentially defines who your best friends are based on how frequently you interact on the app.

Put differently: Snapchat measures friendship by a combination of the frequency, recency, and length of photos and videos sent between two people. They then rank those friendships and attach emojis and other labels like ‘Best Friends Forever’ (BFF) and “Super BFF.” And as if that’s not enough, you can see where your friends are at all times — where they are, together, without you.

Pause for a moment and consider your own adolescence. Imagine a list of your friends, from “best” on down, and now imagine the order of that list was determined by a 3rd party with some convoluted metrics of friendship. That’s what is happening on Snapchat.

A study recently conducted in Belgium, where 91% of the youth between 12 and 18 own a Snapchat account, found that “students viewed breaking a Snapchat streak as a personal rejection. Snapchat streaks, thus, serve as a way for teenagers to evaluate the strength of their interpersonal connections.”

Studies in Austria and Belgium found that teens share their Snapchat passwords so others can maintain their streaks when they cannot.

So much of the policy conversation centers on content moderation, but what about features like this? Features not only designed to addict users, but features that arguably warp the definitions and perceptions of something as critical to young people as *friendship*?

In 2019 this feature was targeted in a bill aiming to restrict awards given to users for higher engagement. Today, Snapchat is actively working to distance itself from other BigTech companies. It’s worth noting here that perhaps instead of legislation that is about surveillance and parental approval (like KOSA), we need products that at a minimum are not turning the most fundamental parts of childhood into a game.

What is a streak?

Streaks are a measure of activity with another Snapchat user. When you send or receive Snaps (photos or videos) from a person at least once for 3 consecutive days, a ‘Snapstreak’ begins and a fire emoji (🔥) appears next to the contact’s name, along with a number next to the fire emoji counting the number of days of an active “streak.” If you maintain your streak for 100 days, you get a 💯 emoji, which is a coveted achievement… apparently even among teenagers across borders.

To keep a streak going, users have to exchange snaps (photos or videos) every single day — it is designed to keeps users hooked.

“Snapchat Streaks is one of the crucial aspects of Snapchat, and it keeps users hooked to this ephemeral messaging and social media platform.” -Beebom.com

The number next to the fire emoji keeps increasing to denote the number of days of an active Snapstreak. To keep a streak going, the app requires users to send and receive photos and videos to friends at least once within a 24-hour window. If they don’t, the streak is lost.

Explainers coach new users on how to game the system, with advice like this:

“The easiest way to increase Snapstreak is to send photo Snaps to the other person. If you are running out of ideas to send pictures of, you could just send a blank Snap to keep the streak going. Most long-term Snapstreaks are usually blank unless both participants are socially active and outgoing.” -Beebom.com

… and videos like this that show people how to artificially inflate their “snap score.”

Parents are told the risks are in the content of the disappearing messages, sexting, connecting with unknown adults, or even the location feature that shows where you are on a map. I read countless how-to guides for both users and parents, and none of them cover the streaks feature.

What is Snapchat?

Snapchat came out in 2011 and quickly exploded in popularity, reaching over 383 million daily active users worldwide this year. It was originally about secure, person-to-person communication, touting its disappearing messages as a feature in a social media environment with a permanent fingerprint. It has morphed into an app that is marketed as a place to be goofy with your friends and includes features to live chat with friends, send text messages, create videos, or create avatars.

Get in the weeds

• Check out this research out of Belgium that explores the impact that gamified interactions have on young people.

• Check out this research out of Austria that explores how the gamification of social media affects interaction and relationships.

• Read this piece by danah boyd, unpacking KOSA and how the legislation is not positioned (or designed) to help kids.

I’ve noticed a few patterns in the comments that I’ve read (which is only a small fraction of the 7,118 comments so far). Since there’s so much interest in the topic, I want to expand on what I mean, why it matters, and why I think we need to be paying attention to this right now.

The tl;dr

Google has been adding AI features to its search box for some time now, and on May 19th announced that they are going all-in on AI, changing the interface from the search box we know, to something that seems to operate more like a chatbot.

Sundar Pichai called it the biggest upgrade to Google Search in over 25 years, which is a nice way of saying that they are essentially phasing out the product that 90% of all search traffic globally runs through. This is not hyperbole. If you look at what they are saying (and doing) their message is clear: Search as you know it is over. Instead of helping you search for information, we will just provide you what we decide the answer is.

Basically, Search (the product that students and schools globally use as their default research tool) is becoming an always-on AI agent. It is going from a search engine to an “answer” engine.

The search engine itself may still be working much like before, but what has changed is YOUR INTERACTION with it, and your experience of finding what you are looking for. How?

Search is a way to find what you are looking for on the World Wed Web (aka. the internet.) Search engines do this by (among other things) indexing the internet — this is essentially a way of organizing information in order to find it easily. Think libraries. How do you find things in a library? Literally, an index. Someone organized and categorized the information in a way people can explore and find relevant resources.

Ok, so a search engine categorizes / organizes information online and then helps you find stuff that is relevant to what you are interested in.

So now, while Google Search may still technically be operating much the same way under the hood, it is no longer finding and sharing relevant links for you to explore. Instead, it’s finding, analyzing, and synthesizing information… and packaging it up for you as a baked answer. Yes, the blue links are still technically there, but they are no longer the primary response, and are increasingly buried.

To be clear here: I am not talking about “AI summaries,” which is AI appended to search. These changes ultimately amount to search being rebuilt entirely around AI.

As Tech Crunch put it:

This shift means that “searching the web” will increasingly be performed by AI agents rather than humans. Instead, people will focus more on acting on the information those agents provide instead of manually clicking links.

Consider for a moment what it means when the “people” in this sentence are students.

While Google says AI is not the default experience, the user interface encourages users to ask follow-up questions instead of scrolling down to the links, and it’s abundantly clear that this is just the beginning. Even the business and marketing world is now asking itself how it can shift from “search engine optimization” to “answer engine optimization”

I understand that some people might see the changes and think “Great! this is super helpful in all the following ways, and there will be some disruption. Shrug.” Ok… But the collateral damage to kids and in schools will be huge. You think edtech is an issue now? Just wait.

I’m not going to sit here and argue about the direction of search. What I want to do is ring a giant alarm bell for parents and educators, because the decisions Google is making right now have a direct impact on kids and classrooms all over the world.

*By the way, complicating this is that the entire internet seems to be devolving into garbage, but I’ll have to tackle that in a separate post.

Why does this matter?

1. Because whether we like it or not, Google has become invisible educational infrastructure.

Google Classroom is used by between 80-90% of school districts in the United States, and over 80% of the top 100 universities. They have hundreds of millions of users globally, in over 230 countries and territories. Google has become the operating system that our kids use first, and the Google search engine is their front door to the internet — especially when it comes to doing research and homework of various kinds.

All of this makes their choices as a company very relevant to us and our kids, pretty immediately.

2. The search engine results page (SERP) we know has already begun transforming into something personalized to the individual.

This deepens the “filter bubble,” and every problem stemming from it.

You don’t need longitudinal studies to understand that a filter bubble is completely at odds with education.

Why am I focused on Google?

Yes, there are other search engines... but Google matters because:

1. Google dominates search, period. About 90% of global search runs through Google (and I think that number is higher on mobile) which is an enormous amount of power. To put it more bluntly: they control the traffic (deciding what people see and in what order) which also puts anyone who wants that traffic at their mercy. This is at the heart of how they’ve built their empire.

2. It is baked into school systems globally. Whether we like it or not, Google has become invisible educational infrastructure. As I said before: Google Classroom is used by between 80-90% of school districts in the United States, over 80% of the top 100 universities, and they have somewhere between 150-170 million users globally, in over 230 countries & territories.

3. It is our kids’ front door to the internet. Because of this dependency, our kids use Google for everything by default… doing homework, writing reports, creating presentations, and doing research of all kinds. “Digital literacy” — the thing we like to say we’re teaching kids — is largely built around the ability to navigate the internet. And navigating the internet has, for a generation plus of students, meant using Google.

So yes, *YOU* can switch. ( has some great advice on this) but schools can’t just flip a switch, because Google is basically load-bearing infrastructure for classrooms ranging from kindergarten to graduate school.

Ok, so what actually changed?

On May 19th, Google announced the biggest changes to Google search in 25 years.

Let’s unpack the changes directly from Google’s official announcement:

1. Default model in AI mode = souped up for agents and coding.

“Starting today, we’re upgrading Search with Gemini 3.5 Flash — our newest Flash model delivering sustained frontier performance for agents and coding — as the new default model in AI Mode for everyone globally.”

The new default model in AI mode is Gemini 3.5 Flash, which is souped up for agents and coding. Ok, fine. They go on to say…

2. It is pro-actively personalized and predictive.

(So the more / longer you have used Google, the more it knows about you…)

“Designed to anticipate your intent, it also helps you formulate your question with AI-powered suggestions that go beyond autocomplete.”

It is “designed to anticipate your intent” … You know how Google already has a bunch of autocomplete suggestions when you type something in the search box? Now it goes further than trying to complete your sentence, and actually tries to help you “formulate your question.”

(By the way, as any educator will tell you, formulating your question effectively is central to learning.)

3. The interface is now essentially a chatbot.

“We’re also making it even simpler to continue the conversation with Search. You can easily ask a follow-up question right from an AI Overview, and flow into a conversational back and forth with AI Mode.”

They are describing a chatbot. So you start with a question that might look like what you’d normally type into a search bar, and then “flow into a conversation…”

I think everyone is largely familiar with the issues and risks that come with chatbots…

• Sycophancy

• Filter-bubble on steroids

• Deepening isolation

Ultimately it is a machine programmed to keep you engaged as long as possible at all costs. How this would be considered for a learning environment for even a nanosecond is beyond me.

(Not to mention the rapidly growing list of cases in which a kid started using a chatbot for homework, and the “conversation” took a dark turn ending in tragedy.)

4. “Information agents” (the prototype for what will become “search agents”) will be operating in the background 24/7.

“We’re entering the era of Search agents, where you can easily create, customize and manage multiple AI agents for your many tasks, right in Search. We’re starting with information agents. Operating in the background, 24/7, these agents intelligently reason across information to find exactly what you need at exactly the right moment.”

(Starts this summer for some users.)

You may have heard of “agents” in terms of AI, but if not… they are basically an automated worker bee that you can point at a specific task, and it does what you tell it, while you do other stuff. What they are saying here is that their goal is for the search bar to be a place where you can launch and manage MULTIPLE AI agents for whatever is important to you, and that the first step in that direction (this summer) is what they call “information agents” that will send you an “... intelligent, synthesized update.”

The difference is something like this: imagine libraries suddenly stopped letting you browse the stacks. Instead, a librarian meets you at the door, asks what you want, and hands you a summary or recommendations that they’ve prepared. You never touch the actual books. That’s what’s happening.

“We’re bringing the power of Google Antigravity and the agentic coding capabilities of Gemini 3.5 Flash right into Search. Search can build the ideal response, in the right format for your question — completely on the fly.”

This means the search bar will also be able to code for you. Sounds kinda like vibe-coding to me, but I know “agentic coding” is technically different. But basically: you can type instructions into the search bar and it will make things for you.

And here’s the cherry on top…

“Finally, for AI to be most helpful, it shouldn’t just know the world’s information, it should understand your context, too. So today, we’re expanding Personal Intelligence in AI Mode to more people in nearly 200 countries and territories across 98 languages — no subscription required. You can securely connect apps like Gmail and Google Photos, and soon Google Calendar.”

Translation: we need all your data so we can make this product useful. We won’t even make you pay for the privilege of letting us hoover up your emails, photos, and entire life schedule. So seriously, give us everything and we promise search will be sorta useful maybe.

I mean. Read the room guys.

“But Emily! We need to teach kids to use these tools.”

Yes, we do. But there’s a *huge* difference between learning how to use technology and using a technology to learn. One builds skills and judgment. The other outsources them.

Also: AI is not a tool. It is the entire process.

What I’m talking about here is the difference between saying you want to learn how to cook, and then practicing meal after meal with various tools, techniques, and ingredients… versus getting a Grub Hub delivery. That is the difference, and you can send me all the hate mail you want… it’s true.

Is it convenient? Sure. Is it learning? Absolutely not.

There were already debates about the value of technology in the classroom. The argument was that they were a tool for learning. How can Google argue that their products are tools for learning and the production of knowledge, when they have turned into tools for consuming pre-packaged knowledge? It is a mediocrity machine at best.

A primary objective of education is learning how to learn. Instead, Google seems to be teaching them how to consume what someone else has crafted in the most mediocre way possible. I would really like to know who is interested in this definition of education.

“AI has always been part of how Google works.”

Technically true. And also beside the point.

I am not talking about the mechanics of what’s running under the hood. The search engine itself may not have changed that much. In fact, I imagine what’s happening is that the product we know is becoming less visible but is still “searching” the internet in a similar way. BUT… what I’m talking about is OUR experience. What is the touchpoint we engage with, and how does it impact our relationship with information.

For a long time, Google’s north star was helping you find what you were looking for. The goal was to get you somewhere useful as efficiently as possible.

That goal has clearly changed. The product is no longer trying to connect you to information. It’s trying to give you a packaged version of their interpretation of information.

“Why would Google do this? Isn’t search their whole business?”

Honestly I don’t know enough about Google to give you a real answer. But I can give you my opinion:

I think that they believe AI is the future of how people interact with the internet, and they need to create demand for it before competitors do. So they’re transforming the product people already rely on, because it’s the path of least resistance. In other words: they decided what the future should look like, and are dragging their customers into that future.

It’s a business strategy, and whether or not it’s a good one for users — especially young ones — is not their concern. Remember: the Google Suite is already integrated into schools across the globe. They are banking on entropy, and the lack of both knowledge and resources to change course.

What now?

So the question this all raises is… what now? What options do schools have? How should schools react to this?

My honest (and very unsatisfying answer) is… I’m really not sure yet.

But what I can say for sure right now is:

• If you’re a parent: start asking if your school is aware of the changes to search, and how they plan to adjust this year. If you don’t know what to say, send this to them as a start.

• If you are advocating to keep AI out of schools: reframe your argument upstream of AI… to the products the school is using. The truth is that AI is already integrated into products schools are using, and “keeping AI out” might not be as simple a proposition as you might think. This is far more complicated than opposing a product, because Google is essentially woven into the workflow of schools. The decision is not a simple one, and will likely require investment of both time and money.

(And if you’re an educator or decision-maker in a school, and you’d like to talk, DM me. I’d really like to think through this with people who are actually in classrooms.)

More on the changes to Google Search

• Mashable crisply lays out how Google Search is changing forever, and what that means for navigating the internet.

• NPR’s coverage of Google’s recent announcement.

• This interesting look at what happens to the internet if no one clicks search links.

• Time’s coverage of the changes and implications for industry.

• This list of Google algorithm updates sorted by release dates, for anyone who wants to take a nose-dive into that rabbit hole.

• Check out ’s theory of “Google Zero,” aka the moment when Google Search stops sending traffic outside of its search engine. He wrote this two years ago, and I haven’t stopped thinking about how he describes how “the entire web is Google’s platform.” He talks about this largely from a web traffic and market perspective, but boy are the implications for the education massive.

Brain Snacks

Home Screen is a multimedia project grappling with how technology is shaping our lives, relationships, and kids. Subscribe to stay in the loop.

Subscribe now

I’ll start with the tech-related news taking up the most space in my brain this week: big changes to Google search.

Google announced that “Google Search is AI Search.”

Sundar Pichai called it the biggest upgrade to Google Search in over 25 years, which is a nice way of saying that they are essentially phasing out the product that 90% of all search traffic globally runs through. This is not hyperbole. If you look at what they are saying (and doing) their message is clear: Search as you know it is over. Instead of helping you search for information, we will just provide you what we decide the answer is.

Basically, search (the product that students and schools globally use as their default research tool) is becoming an always-on AI agent. It is going from a search engine to an “answer” engine. This has been in motion for some time, but now they’ve said it all out loud. The implications of this are massive, rapid, and far-reaching. But my most acute concern is the impact on schools and education.

I put out a video on this a week ago, and it went gangbusters viral (over 100k likes in 2 days). Since it apparently struck a nerve, I put out a second one articulating why I thought this was such a huge problem for schools. My basic take:

• This isn’t AI added onto search — it’s search being completely remade around AI. The search engine we know may still be “finding” stuff online, but instead of surfacing that for you, it will now analyze and package it for you to consume fully processed. Put differently: it’s no longer giving you a list of links to peruse through… it’s doing the analysis and synthesis for you, and handing you its own version of an answer.

• Yes, there are other search engines, but Google creates a seismic impact because: (1) Google dominates search globally, with about 90% of global search traffic, (2) it is baked into school systems around the world, and (3) it is our kids' front door to the internet.

• Yes, they could use other search engines, but schools don’t have much of a choice because of the degree to which Google is integrated into both their curricula and their operations.

• Schools do not seem aware of the degree to which this will impact them, and need to quickly grapple with the implications of these changes on not just their students and curriculum — but on their operations as institutions.

A number of people in the comments of my Instagram post have been asking for “solutions“ and unfortunately, I think the very honest answer is: no one really knows. Anyone who says they know how to “solve” this problem is either ignorant of the technology, ignorant of the school system, lying — or some combination of all three. This will require collective work, action, and comparing of notes.

Making all of this more complicated: the entire web is changing. Over half of web traffic is now bots, and a rapidly increasing majority of content all over the web is AI-generated. (Experts predict that ~90% of content online will be AI-generated this year.)

I will be writing more about this in the coming days and weeks, but in the meantime… If you’re an educator or work in a school, I’d love to talk. DM me here. And if you just want to stay in the loop — subscribe.

Subscribe now

In other news….

• Platforms must remove flagged deepfake nudes within 48 hours. — The Take It Down Act (requiring platforms to remove flagged deepfake nudes) deadline was May 19. Indicator reviewed the removal flows for 16 different platforms to see if they actually complied with the law, and basically: most built something, but Apple did not. Indicator’s piece is also a practical guide for anyone who needs to file a takedown request and doesn’t know where to start. They have made this post public, and you should absolutely bookmark it.

• OpenAI is being sued for “medical advice” that led to the death of a 19-year-old college student. Basically: ChatGPT offered a sophomore at UC Merced personalized tips on how to consume drugs and maximize his high. On May 31, 2025, ChatGPT coached him to mix kratom and Xanax, suggested dosages, and when the student told it he felt nauseous, instead of telling him that the combination could kill him and to seek medical attention, it recommended he go to a “dark, quiet room.” He died that night. What really caught my attention: the student started using ChatGPT during his senior year of high school for help with homework and computer troubleshooting. Apparently, as his trust with the chatbot deepened, he started asking for advice on drug use. This immediately reminds me of Adam Raine’s case, and how a chatbot went from helping with homework, to coaching him to suicide.

• Meta silently added facial recognition code to millions of phones — without telling anyone. Wired analyzed Meta’s AI companion app (the one that pairs with Ray-Ban smart glasses) and found a fully built but not-yet-enabled facial recognition system embedded in it. If activated, it would alert the wearer when it recognizes someone nearby, and raises immediate concerns about how quickly using smart glasses could be used for biometric surveillance. Many organizations have also warned that this technology would be a dream for stalkers, abusers, and anyone who wants to identify people (kids?) in public.

• Researchers want preschool teachers to wear cameras — to train AI. A document obtained by 404 Media and given to parents at a preschool read: “With your permission, your child’s lead teacher may wear a small teacher-worn camera that captures the teacher’s approximate first-person perspective, and/or we may place a fixed video camera in the classroom.” The purpose: to generate training data for AI models. The children in the classroom were not the subjects, technically. But they would be in every frame. And of course, even if you don’t consent, if other parents do… your kid’s image and data still gets hoovered up.

• Software developers say AI is rotting their brains. A peak quote from a 404 Media survey of developers is: “It’s making me dumber for sure.” They describe losing the ability to problem-solve without reaching for AI first, forgetting syntax they used to know, and feeling less confident in their own judgment. These are professionals who had achieved a high skill level, and are feeling it atrophy. Now consider what this means for kids who are supposed to be developing those capabilities in the first place.

• And just for funsies… not only is Apple putting cameras IN ITS AIRPODS, you can also buy a Dyson air purifier with a camera in it! According to Dyson, it just roams around the room, processes images on-device and deletes them immediately, and you can turn it off in the app. Why exactly we’d need cameras on devices that we use to listen to podcasts, or filter our air… is beyond me. Though Wired has some ideas.

Sources & mentions this week: 404 Media, Indicator, Futurism, WIRED, The Verge, Bloomberg

Brain Snacks

Also, how crazy is this… I’m referenced in the new book Muskism by & ! (If you’re wondering how my experience with technology in government is relevant to kids + technology: I’ve spent most of my career helping people make sense of technology, and navigate effective implementation in their institutions… and as it turns out, technology is also the “spinal cord” of schools and apparently also social life 🤷🏻‍♀️)

You can find the full original interview here, but I’ve pasted excerpts in-line below along with my own commentary. I promise, it’s worth the read (less for my commentary, and more for his thoughts).

Also, a couple other things about Professor Weizenbaum:

• He created a chatbot called ELIZA, and observed people using it. He was shocked to realize that many people poured their hearts out to the program, and even attributed human-like feelings to it. Many were convinced ELIZA was intelligent and understood them. Sound familiar?

• He helped design the first computer banking system in the United States, for Bank of America.

• In his book Computer Power and Human Reason, he makes the distinction between deciding and choosing, asserting that a decision is essentially a calculation (which can be programmed), but that choice is a judgement—not a calculation.

• He argued that the computer preserves that status quo, and is an obstacle to progress and innovation.

Ok, excerpts from the Weizenbaum interview below:

“… a solution looking for a problem.” There are certainly things it has been, and can be, helpful with… but it is ultimately a tool that we look at and think… ok, how can I / we use this?

He goes on to say:

This is exactly right. When considering how/where to use a new product, the first question should always be about goals:

1. What is OUR goal, as an institution?

2. What problems do we have / what is obstructing our goal?

3. Which of these can this product potentially help with?

4. What would the goal of this product be here in this institution?

I dug into this a few weeks ago in this post on the difference between learning to use technology VS using technology to learn.

Ok so much to unpack here. But 3 things really stand out to me:

1. He essentially says that before schools start considering innovative techniques, must first be delivering on three key priorities: (1) mastery of language, (2) connection to culture/society, and (3) fundamental skills for observing, understanding, and navigating the world.

2. He explicitly says “… we ought not to use entire generations of schoolchildren as experimental subjects.”

3. He asserts that what is being taught about computers is either wrong, or easy to learn on your own.

I agree with all of the above.

Again, he’s focused here on mastery of language as the foundation of everything else.

He goes on to incisively critique the school system:

Remember: this interview was in 1985, and he thought it was clear that schools were failing to meet their basic goals, and that kids were far from mastering language. Also: It is WILD that 800/1000 MIT freshman failed whatever their writing test was at that time. IN 1985. (Do I even want to know how that would do today?)

Well, the problems in education have certainly been consistent… Also, I absolutely love that this computer scientist sounds like a sociologist.

And here, he absolutely nails it. “It is much nicer, it is much more comfortable, to have some device, say the computer, with which to flood the schools, and then to sit back and say, “You see, we are doing something about it, we are helping,” than to confront ugly social realities.”

Technology allows us to create the illusion of action and problem-solving, while allowing us to essentially mask the problem.

And kudos to this interviewer (who I presume was a student at the time) for asking the right follow-up questions…

AGAIN. We can only address problems when we bring them to the surface. If we avoid or cover-up reality with short-term superficial fixes, we make it harder to tackle (or even see) the real problems. (Problems which, by the way are going nowhere and will absolutely resurface, likely even more complex than before.)

This is worth re-reading. What he’s saying here is that digital technology created the illusion of innovation and progress, while in reality protecting and even strengthening the dysfunctional status-quo.

This is a huge point, and of course leads me to wonder… what does the protected status quo look like in education? Healthcare?

Notice how much he uses the word “fun” to describe the task of figuring out complex problems. Many of my colleagues at the USDS have echoed this sentiment. This is how smart, curious, and well-meaning people who end up building something novel miss the bigger picture and risks. Weizenbaum spent the later part of his career raising flags and critiquing the very technologies he helped create.

He credits his understanding and beliefs on this topic to WRITING. “when I started to write about these things, sometimes just more or less for myself, or in letters to others, the realities I am talking about became clear to me.”

He describes the critical role that the PROCESS of programming and writing play in thinking and discovery: “in the act of programming you discover new ideas, and most particularly you discover that there are deep holes in your knowledge that you have to fill before you go on. That happens with writing too.”

Writing IS thinking. Creating is thinking. This is why generative AI and LLMs are so problematic for the human mind, especially in an educational setting — and even more so among young people who are developing the ability to think.

They go on to talk about how computers are essentially “a military instrument,” and Weizenbaum then lays out why technology is not neutral, and computers are not “mere tools.”

Re-read that last line: “… we’re not living in an abstract society, we’re living in the society in which we in fact live.”

This is exactly why I strongly believe that the potential of technology breaks down in its implementation. It may be true that a new technology has the potential to do all sorts of things that are good for humanity. And I actually do believe in that potential. That said, we also know that inevitably potential is undercut by business interests.

I’m now officially diving into a rabbit hole of this man’s publications and talks… but wanted to make sure you all saw this. Because none of what we are wrestling with right now is new, and all of it was foreseeable.

The question is: how can we enable ourselves and our communities to play an active role in our — and our children’s — future?

• Canvas (basically a core operating system for classrooms) was hacked this week, throwing schools into chaos in the last stretch of the year. One researcher called it “the biggest student data privacy disaster in history.” This highlights the risk of building a single point of failure into the center of education, and raises questions about the degree to which education is mediated (and dependent on) profit-driven technology. You can read my thoughts here.

• A widely-cited study claiming ChatGPT improves student learning has been retracted due to “discrepancies in the meta-analysis.”

• New research from the Rithm Project surveyed 2,383 young people about how AI is shaping their relationships, and the biggest predictor of high-risk AI use isn’t screen time… it’s loneliness. Teens who feel they can’t be real with the people around them are the most likely to turn to AI for emotional support. Over half of those using AI characters for companionship report feeling they have no one else. All the more reason to nurture friendships, independence, confidence, and relationships in which kids can really be themselves.

• Amazon has been using AI pricing algorithms to raise prices across the entire internet. The FTC’s antitrust case alleges Amazon’s systems monitor competitors’ algorithms in real time and manipulate them into raising prices too.

• Flock (the surveillance company that sells camera networks to police departments) accessed cameras inside a children’s gymnastics room, a playground, a school, a Jewish community center, and a pool as a sales demo to other police departments. A resident discovered this by pulling Flock’s access logs via a public records request, and Flock confirmed it. And in the final gut punch… city council was informed, but renewed the contract anyway… 🤷🏻‍♀️

• Speaking of surveillance… Meta cannot resist the siren call of surveilling its own employees. They recently told U.S. staff that the company is installing software (called the Model Capability Initiative) on their computers to record everything they do — and there is no opt out. lays it out here in his classic incisive tone: “I’ve been watching this man run the same playbook for twenty years. He monetized our friendships, then shoved fake AI friends in our feeds when the real ones dried up. He patented our ghosts so he could keep us posting after we die. He has shown contempt for every user, every family, every regulator in his path. Now he’s turning that same machine on his own people.” He goes on to remind us that AI didn’t make these decisions — people did. We (our companies, governments, organizations, schools…) can make different choices.

• And in good / hopeful news… after 12 years as a tech columnist at the Washington Post and WSJ, is helping launch the new Youth AI Safety Institute at Common Sense Media, to test the AI products shaping childhood. And while I wish we were on a timeline that didn’t require any of this, Geoffrey shaping this new institute is a good thing. I’m excited to see where it goes.

• And since tomorrow is Mother’s Day… this beautiful ad/video is circulating again and it gets me every time ❤️ Hits harder right now as I am thinking about how to coach/guide kids to navigate the kinds of things we talk about here.

Sources & mentions this week: 404 Media, The Rithm Project, Spark & Stitch Institute, The Indicator, Washington Monthly, Nature/Retraction Watch, , ,

As I read the coverage, I thought of a piece I wrote over a year ago (as DOGE rampaged through the government) highlighting that technology is the spinal cord of government. While my focus in that piece was on how technology is a dependency of public policy, what I articulated is true everywhere… just replace the word “government” with “education” or “health care.”

Technology (and implementation more broadly) has long been dismissed as an afterthought by policy experts both inside and outside of the government. Over decades, the US government systematically outsourced technology to the private sector through multi-billion dollar contracts. Today, government employees largely do not design or build products or systems, they “manage” implementation of systems developed by contractors or consultants.

In this formulation, technology is subordinate to the policy work, when the truth is that policy is inextricably entangled with technology. Separating policy from the technology it depends on has been a root cause of much of the dysfunction we have grappled with across government for decades.

Technology is not an extra thing that you add onto government programs and services—it IS the service. It’s not an extra thing that you add into the institution—it is the spinal cord of the institution. Sort of like how cars are no longer mechanical, they are now computers wrapped in metal. People working in tech understand this implicitly.

I stand by every word, and want to clearly say: Whether we like it or not, this is true for everything. Every institution, every sector… everything is mediated by technology, whether we see it or not.

I, and many of my colleagues, have spent the better part of the past 20 years trying to convince lawmakers, academia, and the media of exactly this when it comes to government. Our institutions continue to treat technology as something that is subordinate to their primary mission, when the mission is entirely dependent on (often invisible) technology. That this point has been largely dismissed as hyperbolic or extreme would be an understatement.

But as with all of my work here, I want to focus on how this is most directly relevant to us as parents — and our kids’ schools.

Schools have outsourced their agency.

Over the past several decades, schools have done exactly what every other institution has done: they have systematically outsourced their operational backbone to technology products and the companies that build them. The list of platforms that now mediate the basic functions of education — attendance, grades, assignments, communication — is almost comically long. As I’ve written before, most schools are running 10–15 different EdTech products that all promise efficiency, but in reality create confusion, frustration, and complicate what was once much simpler.

The trade-off is giving up control and information/data in exchange for efficiency. Schools traded control of their operations for cost savings, scale, and the recurring promise that technology would finally (this time!) transform learning. Well, it hasn’t. And now, when a single platform gets compromised, the entire operation of thousands of schools grinds to a halt.

The Canvas hack made the invisible dependency visible. Don’t look away.

Here are the questions I want everyone in a position of authority in schools and school systems to sit with:

Don’t say “efficiency” is what you are getting, because efficiency is not the goal of education. The goal of education is LEARNING, and (as I’ve argued before) learning is inherently inefficient — that’s what makes it stick. Schools are not supply chains to be optimized, the things that make teachers great are not scaleable, and the things that make learning meaningful are not automatable. Stop pretending like they are.

If we actually cared about education, we’d be asking questions that illuminate uncomfortable realities, but as MIT Professor Joseph Weizenbaum (inventor of the first widely-known chatbot) said in an interview back in 1985:

It is much nicer, it is much more comfortable, to have some device, say the computer, with which to flood the schools, and then to sit back and say, “You see, we are doing something about it, we are helping,” than to confront ugly social realities.

So again: do schools recognize the degree to which they’ve outsourced the backbone of their mission? And if so, what exactly are they getting in return? Because right now, what I see is institutions trading agency and control for the promise of operational convenience, only to discover that it may just be a Faustian bargain.

The Devil tempts Faust (Poster for the play ‘Faust’/Wikimedia Commons)

Subscribe now

• I’ll start with just one thing from the deluge of manosphere content that seemed to be everywhere I went this week… if you haven’t seen Inside the Manosphere yet, see it. It’s a deep-dive into the ecosystem of “alpha male” culture that is shaping how boys view both themselves and girls. Even if you don’t have boys, watch it… because it will eventually affect your girls. Why? Kids don’t need to be engaging directly with these influencers, because it’s seeping out of the manosphere and into social circles IRL. Also have a look at this spoken word piece by Sambrownex.

• On a related note: WIRED and Indicator found deepfake sexual abuse incidents at roughly 90 schools across 28 countries since 2023, with more than 600 identified victims. Boys are downloading social media photos of girls they know and using “nudify” apps to generate fake nude images in seconds. While schools and law enforcement struggle to figure out what to do, Apple and Google’s app stores are actively steering users toward nudify apps through search autocomplete, ads, and recommended results. Oh, and in the case of one major nudify app, about 90% of traffic comes from Meta platforms… I know, shocking.

• 🎉 In good news Maryland just became the first US state to pass a law banning surveillance pricing — the practice of using your personal data to charge you more than someone else for the same item. The law takes effect October 1 and covers grocery stores and delivery platforms.

• Roblox (the world’s biggest social gaming platform for kids, with 151 million daily active users, including 1/2 of U.S. kids under 16) will be launching kids accounts in June, globally: “Roblox Kids” for ages 5-8, and “Roblox Select” for ages 9-15. The chat function will be off by default for kids under 9, requiring a parent to individually approve every person their child can chat with. Steps in the right direction, but only after lawsuits in multiple states... On a related note, their AI-powered (facial recognition) age verification has been a bit of a mess.

• The CEO of the prediction (aka gambling) market Kalshi announced that parents can submit their ID to check whether their kids signed up, and are launching AI facial recognition to age gate the product. Aside from now having an excuse to gather personal data from non-users, it also raises the question: why does a sports gambling platform have so many children on it?

• The internet’s primary tool for preserving history and creating a record of the web is under serious threat. (But not intentionally?) Basically… major websites (including the NYT, USA Today, and Reddit) are trying to stop bots from scraping their content and using archived information to train AI. Makes sense. BUT the collateral damage is that the Wayback Machine — an archive of the internet over time, that journalists, researchers, and lawyers rely on to prove what was said, when, and by whom — is being suffocated.

• Thousands of men are buying hacking and surveillance tools on Telegram to stalk wives / girlfriends / former partners. Researchers analyzed nearly 2.8 million messages across 16 Italian and Spanish Telegram communities and found a thriving marketplace for digital abuse, including stalkerware marketed as “parental controls,” and over 82,000 abusive images shared alongside the listings.

• Google, Microsoft, and Meta are tracking you even if you opt out... According to a forensic audit, 55% of audited sites set ad cookies despite opt-out requests. All three companies disputed the findings, because of course they do. 🥂 Poetic twist: the audit was led by a former head of Google’s own cookie policy team.

• A new lawsuit claims that Meta deliberately profits from scam ads, which based on everything we’ve seen from Meta would be 0% surprising. Internal documents cited in the complaint show Meta projected 10% of its 2024 revenue (~ $16 billion) would come from scams and banned goods. AND… not only did they NOT block high-risk advertisers, Meta charged them a premium. The lawsuit also alleges that they ignored or incorrectly rejected 96% of valid user fraud reports each week. Again… this would be 0% surprising.

Sources & mentions this week: WIRED, Gadget Review, 404 Media, TechCrunch, Tech Transparency Project, Tech Policy Press, webXray/California Privacy Audit, Good News Network, Tech Justice, Wayback Machine, Gizmodo, Tech Crunch, The Guardian

Over the years, the promises of technology in the classroom have been recycled with each new wave of technology:

• “I believe that the motion picture is destined to revolutionize our educational system and that in a few years it will supplant largely, if not entirely, the use of textbooks. I should say that on the average we get about two percent efficiency out of schoolbooks as they are written today. The education of the future, as I see it, will be conducted through the medium of the motion picture… where it should be possible to obtain one hundred percent efficiency.” — Thomas Edison, 1922

• “Today, we really are on the verge of a watershed moment, when a new generation of Web-based tools could help assess the learning of each student while delivering personalized instruction that builds on specific strengths and addresses individual weaknesses. Unfortunately, if the nation—the education community, in particular—fails to embrace this new technology, we risk squandering an extraordinary opportunity to create a renaissance in teaching and learning.” — Education Week, 2012

• In 2015 the Chan Zuckerberg Initiative launched with an explicit promise to develop software “that understands how you learn best and where you need to focus.” They even publicly declared that they could take an average student to the 98th percentile. After pouring well over $100 million into “personalized learning” initiatives, there’s little to nothing to show for it.

Sound familiar? Every generation of EdTech promises the same thing — personalization at scale, equity through access, the end of the one-size-fits-all classroom — and frames it as finally (!) technologically achievable. While the mechanism has changed over time (radio, TV, the internet, computers, tablets, MOOCs, AI) the promises are nearly identical from generation to generation.

What we see over the decades is a transactionalization and commodification of education, wrapped in the language of “personalization.” Nearly 30 years into the internet era, we should all know by now that “personalized” means that you and your data are now the commodity.

Rupert Murdoch once described K-12 education as “... a $500 billion sector in the U.S. alone that is waiting desperately to be transformed by big breakthroughs that extend the reach of great teaching.” To investors and the tech industry, education was an untapped market waiting to be monetized. So… who exactly was “waiting desperately”...?

Today, almost 90% of schools provide children with internet-connected devices for the purposes of “learning,” and many schools use up to 10-15 different EdTech products / apps that parents, students and teachers then spend hours trying to navigate. (Stop and think about how many apps you need to access to engage with your kids learning.)

To parents, students, and educators the promise is “personalization.” To the market the promise is efficiency. But learning is an inherently arduous, inefficient, and repetitive process — that’s what makes learning stick. There’s no hack. The goal of technology in education (and I would argue in most spaces) should not be “efficiency,” it should be effectiveness. EdTech advocates will argue that effectiveness IS the goal, but we’ve seen enough to know that even if it is in theory, that is not at all what is happening in practice.

We’ve seen this movie before.

The promises of technology as a transformative force in the classroom have not only failed to materialize, they also seem to be counter-productive. Could they be helpful? Sure. Are there places that are doing it well? Probably. But for the most part what we’ve seen is technology being deployed as a replacement for human capital — a replacement that in practice becomes an additional burden that teachers and staff have to carry, while degrading the experience of students. (Next time you’re around a teacher, ask them how much of their time they spend triaging IT issues or screen drama. )

The truth is that no matter what the POTENTIAL of a technology is, potential breaks down in implementation.

Unlike generative AI, the digital products that are currently in classrooms (laptops, ipads, and the internet) were known and understood when they were brought in. Maybe not by all teachers, but the people who built and managed the technology understood how and why it worked. (They were what scholars refer to as “adoption technologies.”) They entered the picture slowly, and their performance was fairly consistent, reliable, and predictable, with challenges largely stemming from keeping kids on task, and preventing them from accessing particular kinds of content. (Which by all measures has proven nearly impossible to do, considering the fact that kids are regularly playing games, accessing social media, watching YouTube, and even porn on their school-issued devices.)

Generative AI is completely different. It has arrived at break-neck speed, it is neither consistent nor reliable, and even the people who create it don’t fully understand how it works. Even when it DOES turn out to be useful, there are countless conditions that are necessary to achieve that utility. Using it effectively is complex even for grown adults who are proficient with technology, and even then there are debates about utility and limitations.

Generative AI also presents new and more complex risks that are less visible than playing video games and watching YouTube, and are (in my view) far more worrisome. I’m talking about issues like cognitive offloading, the reduction in critical thinking, and even a dulling of engagement that is visible even in Ivy-league seminars.

So why exactly are we rushing to integrate this into education? I regret to inform you that I don’t have a definitive answer. I suspect it’s some combination of false marketing, magical thinking, and transactional decisions that prioritize operational “efficiency” over the very naturally inefficient process of learning. But what I can offer is a suggestion:

We need to separate learning how to use technology, from using technology to learn.

These are two totally different things, and they require different approaches and different conditions for success.

Teaching kids how to use new technology makes total sense… it’s a thing in the world and they can and should understand that it exists, and how to use it effectively. Sure. We can have a debate about when that is educationally appropriate (like kids don’t get calculators until they know basic arithmetic) but it is a reasonable thing to propose. To do this, you need people who understand the technology teaching them about it, and providing opportunities to use it in order to accomplish something.

However, using technology as a vehicle for learning is an entirely different proposition. I will not argue there’s no way it’s useful — but I will say that there are no “hacks” to learning. Maybe there are places in the workforce where it would be helpful to shave off some time or repetitive tasks…. but not in education because REPETITION IS HOW HUMANS LEARN. Learning is an inherently tedious and laborious process. Sometimes it’s interesting and exciting, but much of the time it’s frustrating or F*CKING BORING… especially in grade school, when you’d rather be playing with your friends.

Even in the best of circumstances, AI turns learning into a hollow transaction, reinforcing a society-wide obsession with outputs.

The bottom line is that teaching through technology, especially at lower grade levels, is essentially an outsourcing exercise. It’s about replacing human capital… Either because they need to reduce costs, or because there’s a shortage of the necessary humans and they need an alternative. Either way, it’s not in the classroom because it’s helping kids learn.

But Emily, we need “digital literacy!”

If by “digital literacy” you mean learning to use computers and the internet to find and navigate information, create artifacts, and understand how the technology works… ok. This makes sense. But using technology to teach kids foundational knowledge (like reading, writing, and math) is NOT “digital literacy” — it is an illusion of learning that stunts actual literacy.

At the risk of being annoyingly repetitive here… Being digitally literate means knowing how to USE technology to accomplish something. It does not mean digital products teach you how to read, write, and do math. It’s like saying you can teach arithmetic using calculators. Are calculators bad? No. Can they be helpful? Of course. Can a 2nd grader learn basic arithmetic through a calculator? Of course not. You do calculations with a calculator in high school BECAUSE YOU F*CKING LEARNED THE BASICS FIRST, and can now do higher level math because the concepts exist in your brain.

Kids learn by doing. This is not my opinion, it is the commonly held position of experts on child psychology and development. They are little scientists, and they learn by EXPERIENCING the world… struggling, failing, and exploring. Giving them what (some) adults imagine is a “hack” to learning, is stripping them of the very experiences that build knowledge, competence, and confidence.

But Emily! If we don’t do the AI our kids will get left behind!

Since this is an achievement-oriented question… who do you think will get into and graduate from MIT? The kid who can use AI, or the one who can design, build, and think outside of the box? Kids who become dependent on these products will live their lives in the box “AI” creates for them. Dependency strips them of agency. It strips them of the ability to produce their own thoughts, arrive at their own conclusions, and develop their own views and voice. The notion of being “left behind” is just parents buying the hype. Don’t buy it.

Justin Reich, Professor of Digital Media at MIT and author of this book said that he has…

“never encountered an example of a school system – a country, state or municipality – that rapidly adopted a new digital technology and saw durable benefits for their students. The first districts to encourage students to bring mobile phones to class did not better prepare youth for the future than schools that took a more cautious approach. There is no evidence that the first countries to connect their classrooms to the internet stand apart in economic growth, educational attainment or citizen well-being.

Read that again. “There is no evidence that the first countries to connect their classrooms to the internet stand apart…”

All of that said, unfortunately we are not the ones making these decisions. In many cases, schools are being pressured to integrate AI into the classroom for any number of reasons. So what can we do?

1. Draw a distinction between helping kids learn how to USE a technology, and using technology to learn.

2. Remind school officials that they have already been sold this promise before, and they are currently picking up the pieces. What exactly makes them think things will go differently this time? Especially considering the fact that unlike laptops, ipads, and the internet, generative AI is neither consistent nor reliable, and even the people who create it don’t fully understand how it works.

3. Ask WHAT IS THE GOAL? Is it closing operational gaps? What does success look like? What are indicators of success, and how will you track them?

I appreciate guides like this one by MAMA, but what is the goal? and what are indicators of success? should precede any other conversation. Skipping these questions accepts that generative AI in the classroom is a foregone conclusion — it should not be.

This is the second time around that we’ve been sold a tall tale about the potential of technology to transform education! and personalizing learning for all kids! and democratizing learning! We are picking up the pieces now…. We fell for it once, let’s not fall for it again.

More on tech in the classroom…

• This absolutely brilliant piece by .

• This video of explaining the impact of tech in the classroom.

• A look at the total lack of evidence of i-Ready — one of the most widely used ‘adaptive tutoring’ tools in U.S. classrooms. Also by

• The recent NY Times article about “Chromebook Remorse.”

• This article by NY Times diving into the relationship between tech and test scores.

• A thoughtful look at what we can learn from past edtech failures.

• ’s essay (based on a talk) about the impact of technology on childhood and education.

• This helpful article by that explains the difference between “adoption” technologies and “arrival” technologies.

• This illustrated guide to resisting “AI is inevitable” in education.

• My rant on Instagram 🎉

Brain Snacks

And, since we are basically talking about the “enshittification” of education, this brilliant campaign by the Norwegian government is worth a watch:

Cited & mentioned here: Chalkbeat, Kondrad Lawson, MAMA, , The Conversation, The New York Times, , Jonathan Reich, , Futurism, EdWeek.

• The man who invented the infinite scroll (Aza Raskin) sat down with Planet Money’s The Indicator to talk about what platforms should actually do to give people their attention back. Spoiler: he wants to kill what he created.

• The Indicator also reported that wholesale electricity prices have increased 267% in areas near data centers over the past five years. They talk about the local communities and people being affected… and pushing back.

• AI content scanners are being used to supercharge book banning. A tool called Blockade uses AI to scan books for content that could trigger parental objection, and is being used to accelerate the volume and speed of book challenges.

• In my first weekly roundup I flagged that a company is turning Zoom meetings into a podcast, but there’s an update. 404 Media kept digging and has found that this also includes anonymous recovery programs… recorded and posted publicly.

• AI is rapidly flattening and homogenizing discussion and thought, even in Ivy-league seminar classes. As one Yale student put it, “everyone now kind of sounds the same.” This absolutely tracks with my own experience and observations in class over the past year. It is also validated by a new study on the homogenizing effect of LLMs on human expression and thought. The implications of this are massive, as what we are talking about is essentially the manufacturing of a singular version of “truth” at a scale that even Orwell couldn’t imagine — but would recognize instantly.

• New UCLA research finds that AI is removing the very thing that builds real learning. They conducted large-scale experiments involving fractions and reading comprehension, and found (among other things) that “after just ∼10 minutes of AI-assisted problem-solving, people who lost access to the AI performed worse and gave up more frequently than those who never used it.” The finding cuts to the heart of what education is actually for: not just not producing correct answers, but building the capacity to arrive at them. They flag that because current AI systems are “optimized only for short-term helpfulness” they “risk eroding the very human capabilities they are meant to support.” Now consider what this means for your kids’ education. Full paper here. *Thank you to Erika Hall for flagging this

• Google’s AI Overviews are providing misinformation on a massive scale. An analysis conducted by AI startup Oumi found that AI Overviews are accurate about 91% of the time. That might sound tolerable, but the analysis highlights that Google processes roughly five trillion searches a year, meaning that it provides tens of millions of wrong answers every hour, and hundreds of thousands every minute. Google (of course) called the analysis flawed, but even Google’s own internal testing found that Gemini 3 produced incorrect information 28% of the time. In case it needs emphasis: Google holds over 90% of the global search engine market (and that includes schools / education) and AI Overview is the sitting at the top of every search result. *Futurism highlighted that studies have also found that people are blindly trusting AI, with one report finding that “only 8 percent of users actually double checked an AI’s answer,” and another finding that “users still listened to AI when it gave them the wrong answer nearly 80 percent of the time.”

Sources & mentions this week: Futurism, 404, Erika Hall, NPR’s Planet Money

Let’s go with the swimming analogy. Kids can be taught to swim. It is a learnable skill, and once you learn it, staying safe is about making safe/smart decisions. You can send them to the best swim schools, with the best coaches. You can supervise, and they can demonstrate competence. But learning to swim only partially helps, because what they will actually be jumping into is not really a swimming pool…

Imagine a pool that detects when you are tired and then spins up whirlpools and currents designed specifically for your body, with the sole purpose of keeping you in the pool forever. The pool is designed to suck you to the bottom and keep you there.

That is what these products do. They are designed in a manner that tailors your experience specifically to your interests and behavior in order to maximize “engagement” (aka attention). The degree to which they succeed on each person may vary, but there should no longer be any doubt about this — it’s literally in their own documentation. Unfortunately for us, technology in classrooms also operates this way. Just look at the promises of tech in education. They revolve around “personalized” learning. “Personalized” education is education that is tailored to you (sound familiar?) and is basically shorthand for we need all your data, and in order to get that data we need you to use this thing as much as possible. As says: “EdTech is just BigTech in a sweater vest.”

Parents can and should be engaged, aware, and guide their kids as they learn to use various products. At the same time, there is only so much that parents can do to keep their children safe on platforms that are quite literally designed to suck them in. While so many parents celebrate the growing backlash against social media, the same exact pattern is unfolding in the very AI products that public officials are celebrating (and even encouraging) as the next frontier of education.

I hope this helps.

• “Chromebook remorse” is spreading (Thank you for flagging this story.) While the pressure grows to integrate generative AI into classrooms, schools all over the country are dialing back the use of “edtech.” A middle school in Kansas decided to stop using Chromebooks, and schools in North Carolina, Virginia, Maryland, and Michigan are actively re-evaluating classroom use of screens. As I said in Tech Policy Press a few years ago: “past is often prologue — at least in terms of how “disruptive” technology will play out in real-world settings. If we want a preview of what’s coming, all we need to do is look at the recent past. After several cycles of “disruptive” technology, I think we can safely say that the potential of new technology often breaks down in its implementation”

• Speaking of schools… a large multidistrict case will be heard in a jury trial starting this June. Districts are arguing that social media companies have undermined education and created a crisis that has forced schools to divert resources from their educational mission to address issues resulting from addictive products. Claims include: negligence, failure to warn about the social media’s addictive features, and public nuisance. I’m surprised there’s not more coverage about this, especially considering the implications of the recent social media trials in LA and New Mexico.

• AI slop has flooded YouTube Kids (and the internet more broadly), putting bizarre and even disturbing short-form videos in front of kids. You’ve probably seen it before… the weird trance-like animation that has zero plot, and just changes from scene to scene. (My son has discovered whatever the f*ck this genre of video is…) There are tons of these accounts, and to put the volume into perspective: one of the most popular accounts was pumping out 50 videos PER DAY. Top AI slop channels targeting children have earned over $4.25 million, so YouTube has very little incentive to intervene. Ps: about 21-33% of videos on regular ole’ YouTube may be AI slop.

• Ronan Farrow & Andrew Marantz published an investigative piece about Sam Altman in the New Yorker. This quote in particular stopped me dead in my tracks: “He’s unconstrained by truth. He has two traits you almost never see together in one person. The first is a strong desire to please people, to be liked in any given interaction. The second is almost a sociopathic lack of concern for the consequences that may come from deceiving someone.” This could quite literally describe the products coming out of his company. There is a long track record of products mirroring their creators — wrote about this a couple of years ago. These are the people shaping our world, our lives, and the lives of our kids.

• Microsoft Copilot is apparently “for entertainment purposes only” according to… Microsoft’s Terms of Service. The TOS goes on to say “It can make mistakes, and it may not work as intended. Don’t rely on Copilot for important advice. Use Copilot at your own risk." Well thanks, Microsoft. Noted. (ps: This applies to all chatbots and Microsoft was just caught in a moment of honesty that they are apparently now correcting. I’m sorry… “updating…”)

• The Take It Down Act has a key deadline hitting next month. (This is a May 2025 law making it illegal to publish sexually explicit images of a person— including AI-generated deepfakes — without their consent.) Platforms have until May 19, 2026 to have removal processes in place for nonconsensual intimate imagery, including deepfakes. Because this involves requests to remove images, the burden falls on victims or organizations advocating for them. It is not a stretch to say that will involve a massive documentation burden that will hit schools hard. (Which, call me crazy, seems to validate the claims in the multidistrict case I talk about above.)

• California’s community colleges are paying millions for AI chatbots that students say give wrong answers, can’t correctly name their own college’s president, and leave students so frustrated that they turn to unofficial social media channels. Worth bearing in mind as schools are pressured to adopt and integrate these products into schools at a massive expense to taxpayers, teachers, and students. This underlines issues raised in a recent open letter to Georgetown students. Also reminds me of a post I wrote about chatbots a few years ago, and this gem from Cyd Harrell in 2021:

*update: added New Yorker piece about Sam Altman

Sources & mentions this week: The Markup, , , Fortune, NYTimes, Gadget Review, , Tech Policy Press, Courthouse News, EdTech Week, Georgetown Center on Privacy & Technology, and The New Yorker.

• This was not a good week for Meta (or “social media” more broadly) in the courts. By now you’ve probably heard about the big cases in New Mexico and LA, but there was another one in Delaware that hasn’t gotten as much coverage but has pretty major implications: a Delaware court ruled that Meta’s insurers don’t have to cover the costs of defending the thousands of lawsuits piling up against them. Why? Because what Meta is being accused of is the result of deliberate design decisions — not an accident. This has major financial implications because Meta may now need to pay for legal expenses… which are quickly mounting.

• Wikipedia’s editors officially banned AI-generated content from the platform. The reason is pretty simple: AI makes stuff up, and Wikipedia is built on verified human knowledge. Thank you, Wikipedia.

• In yet another move that no one asked for… Google has been using AI to replace the actual headlines of news articles in search results. Google calls it a “small experiment,” but they have said that before their small experiment turns into a permanent feature. **This is directly relevant to our kids, who are learning to do research online, and / or using Google Classroom.

• Speaking of the classroom… the First Lady walked into the White House with a robot and asked us to imagine it replacing teachers. This as we grapple with (and clean up the damage from) the last wave of “transformative” classroom tech. The Atlantic ran a glorious satirical piece imagining what Plato School actually looks like, reminding us why satire is uniquely qualified for confronting uncomfortable realities.

• RIP Metaverse. Remember when Meta spent somewhere upwards of $80 billion building a virtual world that was going to change everything and then renamed the entire company to triple down on that future? Well… it’s dead. Please remember this the next time someone tells you some new sci-fi fantasy BS is going to change everything.

• OpenAI shut down Sora (its heavily hyped AI video generator) just six months after launch. Turns out it was burning through $1 million per day with fewer and fewer actual users. Disney had signed a $1 billion deal to license its characters to the platform, yet they found out it was being shut down less than an hour before the public announcement and the deal died with it. Again… remember this for whatever the next round of hype is. (Also… if this is how they treat business partners that give them $1billion, imagine how they treat everyone else…)

• A company is turning your Zoom calls into podcasts. Yes, exactly what it sounds like… A company called WebinarTV has been scanning the internet for Zoom meeting links, joining calls, and turning them into AI-generated podcasts. This includes calls that are not recorded, and is apparently totally legal.

• The Executive Director of Georgetown’s Center on Privacy & Technology wrote a remarkable open letter to students responding to the university’s push to integrate AI into activities across the campus. It’s an honest and clear-eyed description of what we actually give up when we hand education over to AI. It’s worth your time… please read it.

If you have suggestions for what to include next week, please DM me here on Substack.

In California, jurors decided that Meta and YouTube were negligent in the design and operation of their platforms, and have to pay $6 million in damages. In New Mexico, a jury found that Meta “knowingly harmed children’s mental health and concealed what it knew about child sexual exploitation on its social media platforms,” and was ordered to pay $375 million in damages. The jury agreed with prosecutors that Meta “engaged in ‘unconscionable’ trade practices that unfairly took advantage of the vulnerabilities of and inexperience of children.”

I especially enjoyed reading the specific charges in the LA case, which I have now read like 50 times. It’s hard to believe, but wow.

As Nicki Petrossi said: “They knew it was dangerous and they did it anyway. We knew that… and now everybody knows that.” AMEN, Nicki.

The lead lawyer for the plaintiff called it “the engineering of addiction” and focused on 4 features of the platforms:

1. Infinite scroll: Exactly what it sounds like… the feed with no stopping point. It can keep going on forever.

2. Algorithmic recommendations: Content that the platform suggests for you, based entirely on the aim of keeping you watching / scrolling.

3. Autoplay: Content that keeps appearing without a prompt. Your video ends, and the next one that the platform selected for you starts automatically, without you touching anything.

4. Constant notifications: Pings of various kinds that are unprompted by you. The platform is essentially calling for your attention… “Ping ping! Come back! Ping ping! Look at this!”

These are right on, and I am cautiously optimistic. Yet while I am thrilled about the outcome here, I am seeing a reaction that makes me a bit nervous we may win a battle but lose the war.

I’m seeing post after post proclaiming “it’s their fault!” and while my first instinct is F*CK YEA, it’s really not entirely true. Companies did what companies do. These companies behaved in a completely predictable way, and that is why we have public institutions — to push back and provide some sort of constraint. And yet… public officials have allowed and even enabled their behavior. Those who have managed to make progress at that federal level (like at the FTC and CFBP) have been gutted.

As I said last year, the uncomfortable reality (and much harder problem to solve) is that in many ways, we allowed this to happen, and even enabled it. And when I say “we” I don’t mean you or me… I mean the collective WE: the people who design and create the products, investors that finance and incentivize them, the media that covers and hypes them up, parents who use them as babysitters because in a pinch an ipad is really helpful and how the hell else are we supposed to get everything done, and government officials who have given a hall pass to an entire industry to develop addictive garbage technology that atrophies our brains, divides our communities, and warps our relationships with ourselves and each other.

These two are the first of what will be an avalanche of cases, so there’s lots to follow in the weeks and months ahead, and I hope we continue to celebrate. At the same time, let’s remember that the behavior of these companies and the design of the products are not unique to social media—it is persistent across the tech industry. Everything we are worried about with social media is happening RIGHT NOW with AI, at a seismically larger and faster scale. And yet… AI is being integrated into products that we already use (often with no opt-out), and forced into hospitals, schools, and anywhere else that there’s a large user-base with a trove of data.

There’s a lot we don’t know about AI, but there’s also a lot we DO know. We are currently picking up the pieces from what big tech has gotten away with over the past 20 years. Let’s not make the same mistake again.

More on this topic…

• A solid overview on the California trial, by NPR

• Detailed coverage of the trial by the Heat Initiative + Scrolling2Death, along with an emotional reading of the final verdict.

• Big Tech Should Pay for What It Has Done to Us

• A reference guide for how existing laws apply to AI chatbots.

Share Home Screen

Brain Snacks

The mass surveillance piece is getting most of the attention, but the second point seems to be getting lost, probably because it’s about operations and reliability and is less exciting. Taking a position against mass surveillance is ultimately a values argument, and I’m grateful they are making it. But their point about fully autonomous weapons is about RELIABILITY — and that is a point that is acutely and immediately relevant to our lives and our kids right now. Let me explain.

Anthropic’s CEO explained their position on fully autonomous weapons like this:

Some uses are also simply outside the bounds of what today’s technology can safely and reliably do. [...] without proper oversight, fully autonomous weapons cannot be relied upon to exercise the critical judgment that our highly trained, professional troops exhibit every day. They need to be deployed with proper guardrails, which don’t exist today.

Read that again... The CEO of one of the world’s leading AI companies had the spine to say the truth, publicly: LLMs are still emerging, they are not reliable enough for this context and use, and they need to be deployed with “proper guardrails.”

This is a very VERY important point, especially for those of us who are wrestling with how LLMs are being used in our lives, our kids’ classrooms, and our workplaces.

Anthropic has drawn their lines at mass surveillance and fully autonomous weapons. Good, thanks. But ultimately the issues they are flagging apply to some degree everywhere else too. LLMs are an emergent technology. It is not reliable, and it is not consistent. Even the people who create it cannot fully explain how and why it does what it does. This is true everywhere, not just in a military setting.

While the stakes may be different, they are also high in all sorts of other environments: hospitals, emergency response, classrooms and education, caretaking of all kinds… And yet, leaders everywhere have decided to integrate what is ultimately *emergent and experimental technology* into products, systems, and environments that we and our kids engage with every day. To be clear: while I wish companies cared about this, I do not expect them to… because they are companies. As I said 2 years ago:

Of course, it is not surprising to find companies chasing the market. What is baffling is that people and organizations working in the public trust are scrambling to integrate — not just experiment with, integrate — technology that is, in the words of [Sam Altman], “a barely useful cell phone” into some of our most foundational and critical systems: education, the military, caretaking, and healthcare.

The basic technology is the same — what differs is how we think about or understand “...what today’s technology can safely and reliably do,” and our tolerance for associated risks. How do we define safe? How do we define reliable? The answers to these questions vary depending on context, and we cannot expect for companies to think through this for us. These are questions that some combination of public officials, and leaders considering adoption and integration of LLMs need to consider with the seriousness it deserves. So here’s my question:

If AI is not reliable enough for fully autonomous military action, who determines what is reliable enough for clinical settings? For educational settings? For operating rooms? For classrooms?

I have long had concerns about rushing to integrate LLMs into high-stakes environments for years. Two years ago I wrote a piece about how the potential of technology breaks down in its implementation. In it, I said:

So much of how a technology (or anything) works in the world is affected by a combination of its design, human behavior, and the environment that it is embedded in. That includes how the technology will work in the environment, and how humans will use it. But something we often overlook is the capacity of institutions and organizations to effectively implement and manage the technology.

I went on to quote Nancy Levenson (Professor of Aeronautics and Astronautics at MIT who specializes in software safety) who once said that “... the problem is that we are attempting to build systems that are beyond our ability to intellectually manage.” I closed the piece by saying that “we are adopting technology beyond our ability to implement safely or even effectively.”

This is precisely what Anthropic just told us. LLMs need “proper oversight” and must be “deployed with proper guardrails, which don’t exist today.” This is not only true for Anthropic’s technology… it’s true for all LLMs. And while the stakes may be different, this is just as true for combat as it is for the classroom.

It cannot and should not be the job of companies to be our moral compass, and our public institutions are… not very helpful at the moment. We cannot wait around for others to advocate for our kids. If we are concerned about LLMs being integrated into the products and systems that we and our kids use, we must ask:

1. What is the goal / intended outcome of integrating LLMs? What problem is it solving, and FOR WHOM?

2. Why is an LLM an improvement on the current state? If it is a bandaid for a deeper issue, then the LLM will only mask (and even amplify) the problem it seeks to solve.

3. What can LLMs safely and reliably do here / in this context?

4. Who determines what is safe and reliable “enough” here?

**And remember: Anthropic had the spine to say this out loud. But their statement is not an indication of moral authority—it’s just a clear-eyed awareness of risk and the limitations of its technology. And there is a sea of competitors with products who are silent… and their products are not as good.

ICYMI

More on this issue, along with a few other things I’m tracking this week….

• wrote a fantastic piece on how the Pentagon vs Anthropic fight threatens to make long-standing concerns about AI a reality.

• I posted about this on Instagram earlier this week.

• My now 2-year old thoughts on the rush to integrate what is ultimately emergent and experimental technology, in ’s Tech Policy Press.

• I feel like ’s Resist & Unsubscribe is everywhere, and I am here for it. If you scroll down, they have made it super simple to take action.

• Meta wants to own your digital zombie. Yes, Meta now owns a patent for keeping dead users accounts alive, to continue posting. breaks this down incisively in , saying that “Meta identifies death as a content-gap problem. The severity they’re measuring is the impact on… the feed. Not on the grieving family. Not on the deceased person’s dignity. On the feed.” You can see my reaction to this story here.

Brain Snacks

There’s something about how things are now that seems like it’s kind of the worst it’s ever been in a lot of ways, and I do feel like my feelings and opinions about all of this are shifting a little bit. And I’m still resolving a lot of that. - Damon Beres

I found myself thinking about a post I had recently seen last month on LinkedIn by Dylan Orchard, a PhD candidate at King’s College. I shared his quote with Damon, and keep thinking about it so I want to pull it out and focus on it for a moment. Watch/listen and I’ll tell you why after.

Dylan says: “This isn’t a tech issue, it’s not even a regulatory issue - it’s a large portion of our ruling political and financial classes quietly accepting the necessity of CSAM, encouraging suicide and self harm, deepfake porn and a laundry list of other abhorrent things as the price of doing business.”

Damon calls it a “mask-off moment.” He says that

“the sad thing is that… for those of us who live in this world and kind of obsess about tech news… like there’s been an element of truth to that statement for a long time. But that’s a good point… that’s a really big point. But the new thing is that it is an open sewer.”

So while there are elements that are consistent, what is different is (1) the scale, (2) the pace, and (3) the now very public and normalized acceptance of all of this.

The Epstein files loom heavy in the background here. I hear this quote and instantly think of how painfully consistent it is with the nauseating scale of abuse and impunity that are coming through the Epstein files.

Dylan goes on in his quote to say: “The technology may be the catalyst for this push but the nascent culture behind it is something else entirely.”

What the Epstein files are showing us is that in some ways, this “culture” is not nascent… in fact, it’s so entrenched in the highest levels of our society that it has literally ended up *codified* into the software that our world now runs on. Literally. This also comes through the Meta trial in LA right now… ( btw is doing a fantastic job covering all the tech trials.)

I’m not quite sure what to do with this, but for now I just want to highlight that the relationship between power and the technology we use is MASSIVE, and it is shaping our lives and communities in very real ways.

Every hand in the room went up.

“Now keep your hand up if you ended up watching it.”

Not a single hand went down. Whispers and giggles scattered across the room.

We were 45 minutes into what was supposed to be a talk about my career, and I had completely abandoned my outline. Instead, I was standing in front of a room full of third graders talking about algorithms, why they see what they see on their screens, and how YouTube decides what to show them next.

And they totally got it.

Let me back up: Last week I spoke to my kid’s third grade class for career day. I was supposed to talk about my career path, but what I really wanted to do was talk with all of them about technology. Not about the future, but about how what is on their devices right now actually works, and what their own experiences are.

To my delight we spent the next hour on three main topics: (1) algorithmic mediation, (2) business models & incentives, and (3) data. Of course, highly over-simplified for an 8-year old audience, but (to my great surprise) the basic points landed.

Below is my somewhat dramatized synopsis of how it went.

Subscribe now

1. Algorithmic Mediation (or… who is making decisions here?)

I started with a question: Do you like it when your parents tell you what to read, or who your friends should be, or what clothes you should like?

• Class: No!

• Me: Ok. Me neither! Does anyone know what an algorithm is?

A few hands went up. The basic answer that surfaced was some version of “code.”

• Me: Yes, in a way. Algorithms are basically rules for computers. Where do they come from?

• Class: The computer?

• Me: They are created by people, and they are created to accomplish something specific — they have a goal. Who here knows what YouTube is?

Every single hand went up.

• Me: What is YouTube?

• Class: Funny videos! YouTubers! Lego videos! Dinosaurs!

• Me: Who makes those videos?

• Class: … people!

• Me: Yes. All kinds of people, anywhere in the world can put anything they want on YouTube. How does what you see on YouTube show up on your screen?

• Class: [silence]

• Me: An algorithm decides. A group of people decided how the algorithm should decide what you see next, and those are the rules YouTube follows. So… did YOU decide what you see?

One little boy pipes up: But I find the videos myself…!

Ok, now we are cooking.

• Me: Sort of… how did you find it?

• Kid: I searched.

• Me: And then what happened?

• Kid: I saw more stuff like that.

• Me: How did YouTube know you wanted more of that?

• Kid: Oh! The algorithm!

Yes, small person. The algorithm.

I go on: And what does the algorithm want you to do?

Silence for a bit, and then a hand goes up: To keep looking at videos?

• Me: EXACTLY! To keep looking at videos. And how does it do that?

• Kid: … it… gives me more videos?

• Me: Yep. It gives you more of what it thinks that you want. And it can do this because it watches what you do — on YouTube and other places on the internet — and then based on its rules, it guesses what you want to see. It guesses what will keep you looking at videos. This is also what happens on TikTok and anything else powered by an algorithm.

A room of about 30 kids was totally silent. I asked: How many of you use a tablet or phone?

All the hands went up. Do any of you ever have a hard time putting it down and walking away?

Heads nodded. I ask: Now why is that?

A hand went up. The algorithm doesn’t want us to walk away?

I honestly almost cried. Correct. The algorithm doesn’t want you to walk away.

At this point I was ready to pivot to business incentives, but before I did, I recapped the discussion with what accidentally turned into a somewhat comical call-response:

• What is an algorithm? Rules for computers!

• Where do they come from? They are made by people!

• What are they created to do? To keep us using it!

I continued…. So remember: you said that you don’t like it when your parents tell you what to do, what to read, what to wear, who you should be friends with… but that’s exactly what algorithms are doing—it’s just invisible. What you see is mostly decided by the algorithm, but did you know that we are not even allowed to know how the algorithm works?

The ENTIRE class erupts. WHAT?! That’s not fair! Why is that allowed!?

They got it immediately. I responded: I know… Why?... Well… they do it because people who could stop them, let them. And then I moved on because honestly, I had no idea how to have that conversation.

Onto the business model…

2. Business Models + Incentives (or… why is addiction the goal?)

I called back to an earlier moment: Ok, so they want you to keep watching videos. Why?

No hands. I said: Basically the more time you spend on their app or site, the more money they make. And the goal of any company is to make as much money as possible.

I asked them about their own use. What do they spend time on their devices doing? They were VERY excited to share their answers: YouTube! Minecraft! Funny videos! Roblox! Amazon! AI picture apps!

We talked about why those things are designed to keep you coming back. Why games have daily rewards. Why videos autoplay. Why you get notifications. Why games ask to buy things to use in the game.

Have any of you played with a claw machine? Heads nod. Do you ever win? Lots of chatter, and the basic conclusion is not really but sometimes.

I tell them: Right. That game is designed to make you THINK you can win… “just one more try mom!! PLEASE!!!” (they laugh) so you stay there and keep putting money in and keep asking mom or dad for more money that the claw machine gets to keep, while you get frustrated and mad (more laughs). Games and apps powered by an algorithm are just like this.

Heads were nodding. Ok, I think they got it.

Now, on to one of my favorite topics… and I really didn’t know how this one would land, but somehow it did…

3. Data (or… who is actually in charge of your stuff?)

This one I knew would be tricky. So I started very concrete: Do you take pictures on your devices?

Every hand went up.

• Me: Who is in charge of those pictures?

• Class: Me! My mom! My dad! My sister!

• Me: Really? Where do those pictures live?

• Class: On my ipad, so they’re mine.

I paused. How many of you have heard of the cloud?

About half the class raised their hands. (Wait seriously?!) Ok, who can tell me what the cloud is?

The answers came in. Surprisingly, they were not wrong… so I validated, and then I said: the cloud is really just somebody else’s computer. So if you have an ipad full of pictures, where do your pictures live?

Silence… then a hand: on Apple’s computer?

YES. So who is in charge of your photos?

I could see them processing this… then another hand: … Apple?

YES. They are still your photos, but are you really in charge of them?

I tried an analogy they’d understand: If you let your friend borrow some of your toys and they take them home, they are still technically yours, but who is *in charge* of your toys?

• Class: My friend.

• Me: Correct. Your friend is now in charge and they can play with them as much as they want. Can they give your toys to other people?

• Class: ...Yes.

• Me: What if you tell them they cannot give them to other people? What if they promise they won’t let other people play with your toys?

• Class: [thinking…]

Right. So... who is in charge of your photos?

Hands shot up. The lightbulbs were going off. THAT’S NOT FAIR!! THEY ARE MINE! WHY CAN THEY DO THAT?

Excellent question. This didn’t compute with their sense of fairness, and they really wanted to know WHY they were allowed to do this. One kid suggested we just “don’t use Apple.”

• Me: That’s an interesting idea… what would you use?

• Kid: I don’t know!

• Me: what do you use to take photos?

• Class: My mom’s phone, my tablet…

• Me: Right. So let’s make a list of different tablets and phones.

I made a list on the board as they call out device names. Next to each I put the name of the company. Google, Apple, Samsung, Amazon… This conversation was FASCINATING. They were floored that there were only a couple of options.

I hadn’t planned to talk about monopolies, but that’s what ended up happening — and they totally got it.

But then just for funsies, because I was already floored by how much they could engage… I really really wanted to know what they knew about generative AI.

4. Generative AI

I asked: who knows what a chatbot is? Nothing. Ok. Who knows what ChatGPT is?

Almost every hand went up. A tool! A robot! (lots of laughs for some reason)

• Me: ChatGPT is a type of chatbot… A chatbot is a thing you type into and it responds back to you. It’s not quite a tool — a tool is something that humans can control and use in a specific way. This is a bit different. Is it like a calculator?

• Class: Yes!!

• Me: Is a calculator ever wrong?

• Class: …no.

• Me: If everyone in this class types 20 + 20 into a calculator, what will the answer be?

• Class: 40!

• Me: Right. Every single one of you would get 40. But that’s not how a chatbot works. If everyone in this class typed exactly the same question into a ChatGPT, you would each get different answers.

The class erupted. I continued: Who knows what autocomplete is? A few hands went up.

• Kid: it helps me type a text on my mom’s phone.

• Me: Yes… it corrects misspelled words or fills in words… Is it always right?

• Kid: … sometimes.

• Me: Right. Do you know how it works?

I went on to explain that autocomplete is basically trying to guess what they want to say based on patterns it has “learned.” And then I said: That’s kind of how chatbots work. They are trying to guess what you want. How do chatbots guess? Silence

• Me: I’ll give you a hint: it’s similar to how YouTube decides what videos to put on your screen.

• Class: AN ALGORITHM!!!!!

• Me: Yes, exactly. So… what does ChatGPT want?

• Class: For you to use it all the time.

• Me: Correct. And how does it do that?

• Class: By giving you stuff you want.

• Me: Correct. Does the stuff you want need to be correct?

From here we talked about how chatbots are programmed to give you what you WANT, and not necessarily what’s correct or good for you.

• Me: Who likes candy? What’s your favorite?

• Class: Reeses! Skittles! Nerds! Twix!

• Me: Should you eat them every day?

• Class: laughs.

• Me: Who stops you from eating candy all day?

• Class (grumpily): Mom… dad… my dentist haha!

• Me: Why don’t they let you eat candy all the time?

• Class: … … … because… it’s not good for us. It ruins our teeth. It’s too much sugar.

Exactly.

• Me: Right. It’s not good for you, and the people who care about you will sometimes have to stop you from doing what you WANT because sometimes what we want is not good for us. What does a chatbot want?

• Class: … for us to use it a lot.

• Me: Why?

• Class: to make money.

• Me: What makes a chatbot work?

• Class: … an algorithm?

• Me: EXACTLY. And what is an algorithm?

• Class: rules for computers!

• Me: Where do they come from?

• Class: People!

• Me: What is their goal?

• Class: For us to keep using it!

A+ small people. I closed by saying something to the effect of:

Many of these things we use on screens can be useful and interesting — but WE need to be in control of them, and WE need to be making decisions for ourselves… So when you use your ipad, or play Minecraft or whatever… remember that the technology you use doesn’t care about you. The company that makes that technology doesn’t care about you either. They have made a thing that is useful in some ways, and their job is to make money… YOUR job is to make your brain as strong as possible, so you can decide for yourself how and when and for what you use all of this stuff… and for now that might mean reading books and doing your homework and listening to your parents when they say it’s time to turn off Minecraft. (laughs)

3rd graders are totally capable of engaging on these issues.

I was surprised that I didn’t have to convince them that something is off. They already feel it. They just don’t have the language for it yet.

They know that it feels bad when they can’t stop playing a game. They know that it feels weird when an app seems to know what they want before they do. They know that it doesn’t feel right that their stuff is actually not on their device and that they don’t know how it’s being used.

And their questions (the ones they kept coming back to) were exactly the right ones: Why don’t we have a choice? Why is it allowed to be this way? Who decided this was okay?

I didn’t have perfect answers, but what I did tell them was: These questions that you are asking… these are the questions that matter. Keep asking them.

Our public officials need this same lesson.

I didn’t talk to the kids about social media or specific platforms. We talked about algorithms, business models, and data—foundational problems that drive the behavior of digital products ranging from Instagram and TikTok to chatbots.

I worry that while the degree to which social media and digital platforms have warped childhood is starting to sink for people, everything that we’ve seen on social media—everything we are worried about—is happening all over AI. Just at a seismically larger and faster scale.

We need to stop focusing on specific products and platforms and focus on the roots of the problem:

• Algorithmic mediation that invisibly shapes our experiences

• Business models that feed off of “engagement” (aka addiction)

• Data that is extracted, amassed, and weaponized against users—jacking up prices, targeting kids with predatory content, and keeping you scrolling.

TikTok or Instagram or Roblox or whatever… the products are not THE problem. They are the SYMPTOM of the problem(s). If 8-year olds can get it, surely our policymakers can.

I’m really excited to share this interview because Damon is at a different stage of parenting than my other guests. He has a 4-year-old son, which means he’s not yet wrestling with phones, social media, and AI chatbots in the same way that parents of older kids are. Instead, his focus right now is on something more foundational: building muscles and love for creativity, curiosity, dialogue, and critical thinking before screens enter the picture.

We met online during the pandemic, and I’ve always appreciated his voice and perspective. Damon has increasingly been writing about how frictionless technologies are affecting our ability to think and engage with the world and each other, and as you’ll hear in this conversation, the stakes feel higher than ever.

A few things that were referenced in our interview: Damon’s articles The Age of Anti-Social Media, and @Grok, Did Venezuela Deserve It?, and the LinkedIn post that I quoted.

💡“Aha” moments

• Chatbots aren’t designed to help you learn or find information — they’re designed to generate mediocre guesses that sound convincing. I can’t stop thinking about the implications here for kids who are learning how to do research, and find / analyze information. As my own kids do their first research projects, I watch them use Google, and they do not scroll past the AI overview unless I prompt them to and actively guide them through a discovery process. Without active guidance, these products are training them to accept whatever the algorithm serves them first.

• The story of the internet has been about removing friction, and maximizing efficiency and convenience. We are at peak “efficiency brain” where as Damon says… “it is really easy now just to be in an endless conversation with yourself and your own mind.”

• Generative AI didn’t come from nowhere… it is the latest chapter in the internet’s “convenience is king” ethos. It follows algorithmic social media, which in Damon’s words is essentially an “… automated process of discerning user taste and serving them content accordingly.”

• Face-to-face conversations have friction (interruptions, disagreement, unexpected questions, nonverbal cues) and that friction is essential for childhood development. Digital technology is eliminating that friction at an increasing scale.

• We’re living through a “mask off moment” where business and political elites have quietly accepted that CSAM, deepfake porn, and other horrors are just “the price of doing business.”

• Platforms that once had some content moderation, some sense of a line and/or consequences, are abandoning even their minimal guardrails.

✨Highlights

“There’s something about how things feel now that seems like it’s kind of the worst it’s ever been in a lot of ways. And I do feel like my feelings and opinions about all this are shifting a little bit. And I’m still resolving a lot of that.” - Damon Beres

“This has been kind of a light bulb moment for me. Not in the sense that this kind of AI generated sex abuse material or AI generated harmful stuff is new [...] I’ve known all this stuff and I know how children—how all people, including children—can be victimized by this technology and by bad people who want to do bad things. There is something about the massive social contagion effect of this technology existing on X that has been really disturbing and new-feeling to me. And suddenly I’ve had this feeling of: oh, things can change quite rapidly [...] These massive platforms that at least nominally used to have content moderation and seemed like they used to care about the safety of their users, in the most extreme sense anyway… just may not anymore. And this has given me the feeling of, wow, maybe I should scrub everything about my child that’s ever existed online, private or not, because it is so easy to do the most heinous things with media now, with AI out there.” - Damon Beres

[regarding a LinkedIn post that really resonated with me]

“... [the author] says, ‘this isn’t a tech issue. It’s not even a regulatory issue. It’s a large portion of our ruling political and financial classes quietly accepting the necessity of CSAM, encouraging suicide and self harm, deep, deep fake porn, and a laundry list of other abhorrent things as the price of doing business.’ And even as I read it now, my hair is standing on end because I think for me, that’s exactly it… I didn’t have words for it. And then I read this and I was like, this is why it feels different. It’s this thing.” - Emily Tavoulareas

“... it’s a mask off moment. And I think that this person is right to articulate it in that way. Perhaps it’s been true for a long time, but there was always this kind of veil of respectability […] there’s just this kind of feeling, I think, especially in this post-generative AI moment of kind of anything goes and the platforms really can’t control the technology fully. And so, a lot is being shrugged off right now, and it feels like kind of a dangerous moment for that reason. It’s different.” - Damon Beres

“I think to many people, it feels exactly the way you’re describing… it is this open sewer. It’s one that we no longer have an option about. It’s being placed in our homes, in our devices, in our kids classrooms… there’s no opt out. And whether you’re interacting with Grok or not is almost irrelevant. The sewer is porous. That whole ecosystem is so porous that if that public sewer is full enough, it’s going to spill into everything else.” - Emily Tavoulareas

“I do think that it’s not necessarily as simple as this product is just evil, right? Because sure, we wouldn’t say Photoshop is evil. I do think, however, what is clearly the case here is that there is an issue of scale that is just... profoundly different now. It is so easy to make so much of this material at such a high quality that I don’t really buy that this is just a pen thing… I do think that there is something very new here.” - Damon Beres

“I think that the reduction of friction and the results of that, oftentimes problematic results, is kind of the story of the internet. [...] my sort of top-lined view on this is that the internet, speaking in very general terms, has been a pretty long story of making certain functions and certain behaviors easier and easier and easier. And I really think that there is sort of a lineage here. You look at generative AI, it didn’t come from nothing. It is situated within a context where we have algorithmic social media, which is an AI-driven technology of a different sort, but is fundamentally this sort of automated process of discerning user taste and serving them content accordingly.

Algorithmic media emerged from earlier forms of social media where people connected to each other and built up these stores of data in a way that felt much more convenient than keeping an address book and business cards and calling people on the phone and sending emails and stuff. early social media emerged from an iteration of the internet where you had, you know, file sharing, Napster, LimeWire, BitTorrent, all that stuff, where there is this idea that, like, everything could be placed on the internet for free access and that the sort of convenience is king ethos like really sank in in those early years, right? So when it comes to friction, I think that the major thing is that all of these technologies have made it very easy to act without thinking critically.” - Damon Beres

“When things become very easy, people want to take that convenience. And I think that this has profound effects on our ability to think, but also our ability to relate to one another and to understand other people. Because the phase that we’re in right now, with generative AI and chatbots in particular, is that it is really easy now just to be in an endless conversation with yourself and your own mind. There is no friction there anymore because you will type to a machine that parrots human-like language to you in a very compelling way and affirms your ideas and your thoughts. There have been so many cases reported in the media about “AI psychosis” (people getting really sick and sometimes ending their own lives following interactions with these bots) and that is sort of the ultimate friction reduction where there is no longer even a barrier between your mind and your mouth… you’re kind of in this constant cycle with yourself and that’s very worrisome, I think.” - Damon Beres

“... one of the elements of my book will be, and one of the elements in the Atlantic article I wrote was about children growing up in this environment and in this paradigm. And something as simple as having a face-to-face vocal conversation with a person involves a lot of friction in a certain way. Someone can disagree with you, someone interrupts or asks an unexpected question, someone has a facial unspoken cue that kind of stops you that you respond to. Experts that I spoke to for that Atlantic article were very clear that that kind of interaction is key for early childhood development. And I would argue that it’s probably key for anyone at any point to have interactions like that, but especially for a developing mind. That is how a person learns to exist in the world and learns how to socialize and learns how to be.” - Damon Beres

“… my major interest in exploring this topic in a bigger way in the book, is sort of that we have allowed society to be steered to a great extent by digital technology over the past, I would kind of put it at like past 20 years or so [...] I think that what I’m concerned about is that our society slips into just this kind of passive state of allowing all of this to control and steer our experiences, which is itself maybe what speaks to a lack of friction in some way. Like, I think like a lot of this feels really good to people… it feels soothing to use these products in many ways.” - Damon Beres

“And I think that people need to, I don’t want to say people should resist it because I think that there are applications of the technology that I think are very good… but I do think that people must be thoughtful and aware, and we have to be able to communicate in real terms with one another on a like societal basis to help set the terms (legally or otherwise) of engagement with this technology and what is permissible and what’s not… just because we have seen the very bad things that happen when we don’t do that. And I think that this is a moment where we can both reflect on the recent past and establish some kind of architecture for a future that looks a little different.” - Damon Beres

“... the article that I was working on this past weekend about Venezuela was about how chatbots handled the breaking news situation and how they handled delivering information about that event… and they were really bad. And they were bad for multiple different reasons which I don’t need to get into, but the upshot is that these are not really tools that are designed for knowledge discovery or to make you smarter. They are not necessarily tools that are designed to reflect reality. What a large language model is designed to do is to deliver human-like language according to training data and according to the way that the model has been set up and programmed.” - Damon Beres

“... what I think will happen, absent an intervention, is that a lot of people are going to exist in a kind of mediocre state of accepting questionable material that has been served to them by chatbots. They will be less media literate. They will be less able to socialize with other people. They will be less capable of critical thinking. I think that there is every reason to expect that those things will be true as the technology becomes more widespread and as it becomes not just more widespread, but kind of the only option.” - Damon Beres

“I think that the lesson from the past 20 years has just been that people get lulled into patterns by these tools that, after all, are acting on their minds. You know, this is not a tool that is tactile or has something to do with your literal embodied experience of the world. It’s the material that is entering your brain and influencing your thoughts. And again, you know… sometimes in very good ways. You find a great new perspective on social media. If someone says something really smart, you find community… so many good things. I’m not so… whatever… that I would say otherwise, but I do think in an aggregate sense, you see these problems. And I think the lesson is that we have to take a very active and critical role in thinking for ourselves and establishing what is acceptable.” - Damon Beres