I was recently honored by the Electronic Frontier Foundation. Alongside Oakland Privacy and William Gibson, I received a 2019 Barlow/Pioneer Award. I was asked to give a speech. As I reflected on what got me to this place, I realized I needed to reckon with how I have benefited from men whose actions have helped uphold a patriarchal system that has hurt so many people. I needed to face my past in order to find a way to create space to move forward.
This is the speech I gave in accepting the award. I hope sharing it can help others who are struggling to make sense of current events. And those who want to make the tech industry to do better.
I cannot begin to express how honored I am to receive this award. My awe of the Electronic Frontier Foundation dates back to my teenage years. EFF has always inspired me to think deeply about what values should shape the internet. And so I want to talk about values tonight, and what happens when those values are lost, or violated, as we have seen recently in our industry and institutions.
But before I begin, I would like to ask you to join me in a moment of silence out of respect to all of those who have been raped, trafficked, harassed, and abused. For those of you who have been there, take this moment to breathe. For those who haven’t, take a moment to reflect on how the work that you do has enabled the harm of others, even when you never meant to.
The story of how I got to be standing here is rife with pain and I need to expose part of my story in order to make visible why we need to have a Great Reckoning in the tech industry. This award may be about me, but it’s also not. It should be about all of the women and other minorities who have been excluded from tech by people who thought they were helping.
The first blog post I ever wrote was about my own sexual assault. It was 1997 and my audience was two people. I didn’t even know what I was doing would be called blogging. Years later, when many more people started reading my blog, I erased many of those early blog posts because I didn’t want strangers to have to respond to those vulnerable posts. I obfuscated my history to make others more comfortable.
I was at the MIT Media Lab from 1999–2002. At the incoming student orientation dinner, an older faculty member sat down next to me. He looked at me and asked if love existed. I raised my eyebrow as he talked about how love was a mirage, but that sex and pleasure were real. That was my introduction to Marvin Minsky and to my new institutional home.
My time at the Media Lab was full of contradictions. I have so many positive memories of people and conversations. I can close my eyes and flash back to laughter and late night conversations. But my time there was also excruciating. I couldn’t afford my rent and did some things that still bother me in order to make it all work. I grew numb to the worst parts of the Demo or Die culture. I witnessed so much harassment, so much bullying that it all started to feel normal. Senior leaders told me that “students need to learn their place” and that “we don’t pay you to read, we don’t pay you to think, we pay you to do.” The final straw for me was when I was pressured to work with the Department of Defense to track terrorists in 2002.
After leaving the Lab, I channeled my energy into V-Day, an organization best known for producing “The Vagina Monologues,” but whose daily work is focused on ending violence against women and girls. I found solace in helping build online networks of feminists who were trying to help combat sexual assault and a culture of abuse. To this day, I work on issues like trafficking and combating the distribution of images depicting the commercial sexual abuse of minors on social media.
By 2003, I was in San Francisco, where I started meeting tech luminaries, people I had admired so deeply from afar. One told me that I was “kinda smart for a chick.” Others propositioned me. But some were really kind and supportive. Joi Ito became a dear friend and mentor. He was that guy who made sure I got home OK. He was also that guy who took being called-in seriously, changing his behavior in profound ways when I challenged him to reflect on the cost of his actions. That made me deeply respect him.
I also met John Perry Barlow around the same time. We became good friends and spent lots of time together. Here was another tech luminary who had my back when I needed him to. A few years later, he asked me to forgive a friend of his, a friend whose sexual predation I had witnessed first hand. He told me it was in the past and he wanted everyone to get along. I refused, unable to convey to him just how much his ask hurt me. Our relationship frayed and we only talked a few times in the last few years of his life.
So here we are… I’m receiving this award, named after Barlow less than a week after Joi resigned from an institution that nearly destroyed me after he socialized with and took money from a known pedophile. Let me be clear — this is deeply destabilizing for me. I am here today in-no-small-part because I benefited from the generosity of men who tolerated and, in effect, enabled unethical, immoral, and criminal men. And because of that privilege, I managed to keep moving forward even as the collateral damage of patriarchy stifled the voices of so many others around me. I am angry and sad, horrified and disturbed because I know all too well that this world is not meritocratic. I am also complicit in helping uphold these systems.
What’s happening at the Media Lab right now is emblematic of a broader set of issues plaguing the tech industry and society more generally. Tech prides itself in being better than other sectors. But often it’s not. As an employee of Google in 2004, I watched my male colleagues ogle women coming to the cafeteria in our building from the second floor, making lewd comments. When I first visited TheFacebook in Palo Alto, I was greeted by a hyper-sexualized mural and a knowing look from the admin, one of the only women around. So many small moments seared into my brain, building up to a story of normalized misogyny. Fast forward fifteen years and there are countless stories of executive misconduct and purposeful suppression of the voices of women and sooooo many others whose bodies and experiences exclude them from the powerful elite. These are the toxic logics that have infested the tech industry. And, as an industry obsessed with scale, these are the toxic logics that the tech industry has amplified and normalized. The human costs of these logics continue to grow. Why are we tolerating sexual predators and sexual harassers in our industry? That’s not what inclusion means.
I am here today because I learned how to survive and thrive in a man’s world, to use my tongue wisely, watch my back, and dodge bullets. I am being honored because I figured out how to remove a few bricks in those fortified walls so that others could look in. But this isn’t enough.
I am grateful to EFF for this honor, but there are so many underrepresented and under-acknowledged voices out there trying to be heard who have been silenced. And they need to be here tonight and they need to be at tech’s tables. Around the world, they are asking for those in Silicon Valley to take their moral responsibilities seriously. They are asking everyone in the tech sector to take stock of their own complicity in what is unfolding and actively invite others in.
And so, if my recognition means anything, I need it to be a call to arms. We need to all stand up together and challenge the status quo. The tech industry must start to face The Great Reckoning head-on. My experiences are all-too common for women and other marginalized peoples in tech. And it it also all too common for well-meaning guys to do shitty things that make it worse for those that they believe they’re trying to support.
If change is going to happen, values and ethics need to have a seat in the boardroom. Corporate governance goes beyond protecting the interests of capitalism. Change also means that the ideas and concerns of all people need to be a part of the design phase and the auditing of systems, even if this slows down the process. We need to bring back and reinvigorate the profession of quality assurance so that products are not launched without systematic consideration of the harms that might occur. Call it security or call it safety, but it requires focusing on inclusion. After all, whether we like it or not, the tech industry is now in the business of global governance.
“Move fast and break things” is an abomination if your goal is to create a healthy society. Taking short-cuts may be financially profitable in the short-term, but the cost to society is too great to be justified. In a healthy society, we accommodate differently abled people through accessibility standards, not because it’s financially prudent but because it’s the right thing to do. In a healthy society, we make certain that the vulnerable amongst us are not harassed into silence because that is not the value behind free speech. In a healthy society, we strategically design to increase social cohesion because binaries are machine logic not human logic.
The Great Reckoning is in front of us. How we respond to the calls for justice will shape the future of technology and society. We must hold accountable all who perpetuate, amplify, and enable hate, harm, and cruelty. But accountability without transformation is simply spectacle. We owe it to ourselves and to all of those who have been hurt to focus on the root of the problem. We also owe it to them to actively seek to not build certain technologies because the human cost is too great.
My ask of you is to honor me and my story by stepping back and reckoning with your own contributions to the current state of affairs. No one in tech — not you, not me — is an innocent bystander. We have all enabled this current state of affairs in one way or another. Thus, it is our responsibility to take action. How can you personally amplify underrepresented voices? How can you intentionally take time to listen to those who have been injured and understand their perspective? How can you personally stand up to injustice so that structural inequities aren’t further calcified? The goal shouldn’t be to avoid being evil; it should be to actively do good. But it’s not enough to say that we’re going to do good; we need to collectively define — and hold each other to — shared values and standards.
People can change. Institutions can change. But doing so requires all who harmed — and all who benefited from harm — to come forward, admit their mistakes, and actively take steps to change the power dynamics. It requires everyone to hold each other accountable, but also to aim for reconciliation not simply retribution. So as we leave here tonight, let’s stop designing the technologies envisioned in dystopian novels. We need to heed the warnings of artists, not race head-on into their nightmares. Let’s focus on hearing the voices and experiences of those who have been harmed because of the technologies that made this industry so powerful. And let’s collaborate with and design alongside those communities to fix these wrongs, to build just and empowering technologies rather than those that reify the status quo.
Many of us are aghast to learn that a pedophile had this much influence in tech, science, and academia, but so many more people face the personal and professional harm of exclusion, the emotional burden of never-ending subtle misogyny, the exhaustion from dodging daggers, and the nagging feeling that you’re going crazy as you try to get through each day. Let’s change the norms. Please help me.
we’re all taught how to justify history as it passes by
and it’s your world that comes crashing down
when the big boys decide to throw their weight around
but he said just roll with it baby make it your career
keep the home fires burning till america is in the clear
i think my body is as restless as my mind
and i’m not gonna roll with it this time
no, i’m not gonna roll with it this time
— Ani Difranco
(This post was originally posted on Medium.)
For the second time in a week, my phone buzzed with a New York Times alert, notifying me that another celebrity had died by suicide. My heart sank. I tuned into the Crisis Text Line Slack channel to see how many people were waiting for a counselor’s help. Volunteer crisis counselors were pouring in, but the queue kept growing.
Celebrity suicides trigger people who are already on edge to wonder whether or not they too should seek death. Since the Werther effect study, in 1974, countless studies have conclusively and repeatedly shown that how the news media reports on suicide matters. The World Health Organization has adetailed set of recommendations for journalists and news media organizations on how to responsibly report on suicide so as to not trigger copycats. Yet in the past few years, few news organizations have bothered to abide by them, even as recent data shows that the reporting on Robin Williams’ death triggered an additional 10 percent increase in suicide and a 32 percent increase in people copying his method of death. The recommendations aren’t hard to follow — they focus on how to convey important information without adding to the problem.
Crisis counselors at the Crisis Text Line are on the front lines. As a board member, I’m in awe of their commitment and their willingness to help those who desperately need support and can’t find it anywhere else. But it pains me to watch as elite media amplifiers make counselors’ lives more difficult under the guise of reporting the news or entertaining the public.
Through data, we can see the pain triggered by 13 Reasons Why and the New York Times. We see how salacious reporting on method prompts people to consider that pathway of self-injury. Our volunteer counselors are desperately trying to keep people alive and get them help, while for-profit companies reap in dollars and clicks. If we’re lucky, the outlets triggering unstable people write off their guilt by providing a link to our services, with no consideration of how much pain they’ve caused or the costs we must endure.
I want to believe in journalism. But my faith is waning.
I want to believe in journalism. I want to believe in the idealized mandate of the fourth estate. I want to trust that editors and journalists are doing their best to responsibly inform the public and help create a more perfect union.But my faith is waning.
Many Americans — especially conservative Americans — do not trust contemporary news organizations. This “crisis” is well-trod territory, but the focus on fact-checking, media literacy, and business models tends to obscure three features of the contemporary information landscape that I think are poorly understood:
Let me begin by apologizing for the heady article, but the issues that we’re grappling with are too heady for a hot take. Please read this to challenge me, debate me, offer data to show that I’m wrong. I think we’ve got an ugly fight in front of us, and I think we need to get more sophisticated about our thinking, especially in a world where foreign policy is being boiled down to 140 characters.
I was a teenager when I showed up at a church wearing jeans and a T-shirt to see my friend perform in her choir. The pastor told me that I was not welcomebecause this was a house of God, and we must dress in a manner that honors Him. Not good at following rules, I responded flatly, “God made me naked. Should I strip now?” Needless to say, I did not get to see my friend sing.
Faith is an anchor for many people in the United States, but the norms that surround religious institutions are man-made, designed to help people make sense of the world in which we operate. Many religions encourage interrogation and questioning, but only within a well-established framework.Children learn those boundaries, just as they learn what is acceptable insecular society. They learn that talking about race is taboo and that questioning the existence of God may leave them ostracized.
Like many teenagers before and after me, I was obsessed with taboos and forbidden knowledge. I sought out the music Tipper Gore hated, read the books my school banned, and tried to get answers to any question that made adults gasp. Anonymously, I spent late nights engaged in conversations on Usenet, determined to push boundaries and make sense of adult hypocrisy.
Following a template learned in Model UN, I took on strong positions in order to debate and learn. Having already lost faith in the religious leaders in my community, I saw no reason to respect the dogma of any institution. And because I made a hobby out of proving teachers wrong, I had little patience for the so-called experts in my hometown. I was intellectually ravenous, but utterly impatient with, if not outright cruel to the adults around me. I rebelled against hierarchy and was determined to carve my own path at any cost.
I have an amazing amount of empathy for those who do not trust the institutions that elders have told them they must respect. Rage against the machine. We don’t need no education, no thought control. I’m also fully aware that you don’t garner trust in institutions through coercion or rational discussion. Instead, trust often emerges from extreme situations.
Many people have a moment where they wake up and feel like the world doesn’t really work like they once thought or like they were once told. That moment of cognitive reckoning is overwhelming. It can be triggered by any number of things — a breakup, a death, depression, a humiliating experience.Everything comes undone, and you feel like you’re in the middle of a tornado, unable to find the ground. This is the basis of countless literary classics, the crux of humanity. But it’s also a pivotal feature in how a society comes together to function.
Everyone needs solid ground, so that when your world has just been destabilized, what comes next matters. Who is the friend that picks you up and helps you put together the pieces? What institution — or its representatives — steps in to help you organize your thinking? What information do you grab onto in order to make sense of your experiences?
Contemporary propaganda isn’t about convincing someone to believe something, but convincing them to doubt what they think they know.
Countless organizations and movements exist to pick you up during your personal tornado and provide structure and a framework. Take a look at how Alcoholics Anonymous works. Other institutions and social bodies know how to trigger that instability and then help you find ground. Check out the dynamics underpinning military basic training. Organizations, movements, and institutions that can manipulate psychological tendencies toward a sociological end have significant power. Religious organizations, social movements, and educational institutions all play this role, whether or not they want to understand themselves as doing so.
Because there is power in defining a framework for people, there is good reason to be wary of any body that pulls people in when they are most vulnerable. Of course, that power is not inherently malevolent. There is fundamental goodness in providing structures to help those who are hurting make sense of the world around them. Where there be dragons is when these processes are weaponized, when these processes are designed to produce societal hatred alongside personal stability. After all, one of the fastest ways to bond people and help them find purpose is to offer up an enemy.
And here’s where we’re in a sticky spot right now. Many large institutions — government, the church, educational institutions, news organizations — are brazenly asserting their moral authority without grappling with their own shit.They’re ignoring those among them who are using hate as a tool, and they’re ignoring their own best practices and ethics, all to help feed a bottom line. Each of these institutions justifies itself by blaming someone or something to explain why they’re not actually that powerful, why they’re actually the victim. And so they’re all poised to be weaponized in a cultural war rooted in how we stabilize American insecurity.And if we’re completely honest with ourselves, what we’re really up against is how we collectively come to terms with a dying empire. But that’s a longer tangent.
Any teacher knows that it only takes a few students to completely disrupt a classroom. Forest fires spark easily under certain conditions, and the ripple effects are huge. As a child, when I raged against everyone and everything, it was my mother who held me into the night. When I was a teenager chatting my nights away on Usenet, the two people who most memorably picked me up and helped me find stable ground were a deployed soldier and a transgender woman, both of whom held me as I asked insane questions. They absorbed the impact and showed me a different way of thinking. They taught me the power of strangers counseling someone in crisis. As a college freshman, when I was spinning out of control, a computer science professor kept me solid and taught me how profoundly important a true mentor could be. Everyone needs someone to hold them when their world spins, whether that person be a friend, family, mentor, or stranger.
Fifteen years ago, when parents and the news media were panicking about online bullying, I saw a different risk. I saw countless kids crying out online in pain only to be ignored by those who preferred to prevent teachers from engaging with students online or to create laws punishing online bullies. We saw the suicides triggered as youth tried to make “It Gets Better” videos to find community, only to be further harassed at school. We saw teens studying the acts of Columbine shooters, seeking out community among those with hateful agendas and relishing the power of lashing out at those they perceived to be benefiting at their expense. But it all just seemed like a peculiar online phenomenon, proof that the internet was cruel. Too few of us tried to hold those youth who were unquestionably in pain.
Teens who are coming of age today are already ripe for instability. Their parents are stressed; even if they have jobs, nothing feels certain or stable. There doesn’t seem to be a path toward economic stability that doesn’t involve college, but there doesn’t seem to be a path toward college that doesn’t involve mind-bending debt. Opioids seem like a reasonable way to numb the pain in far too many communities. School doesn’t seem like a safe place, so teenagers look around and whisper among friends about who they believe to be the most likely shooter in their community. As Stephanie Georgopulos notes, the idea that any institution can offer security seems like a farce.
When I look around at who’s “holding” these youth, I can’t help but notice the presence of people with a hateful agenda. And they terrify me, in no small part because I remember an earlier incarnation.
In 1995, when I was trying to make sense of my sexuality, I turned to various online forums and asked a lot of idiotic questions. I was adopted by the aforementioned transgender woman and numerous other folks who heard me out, gave me pointers, and helped me think through what I felt. In 2001, when I tried to figure out what the next generation did, I realized thatstruggling youth were more likely to encounter a Christian gay “conversion therapy” group than a supportive queer peer. Queer folks were sick of being attacked by anti-LGBT groups, and so they had created safe spaces on private mailing lists that were hard for lost queer youth to find. And so it was that in their darkest hours, these youth were getting picked up by those with a hurtful agenda.
Teens who are trying to make sense of social issues aren’t finding progressive activists. They’re finding the so-called alt-right.
Fast-forward 15 years, and teens who are trying to make sense of social issues aren’t finding progressive activists willing to pick them up. They’re finding the so-called alt-right. I can’t tell you how many youth we’ve seen asking questions like I asked being rejected by people identifying with progressive social movements, only to find camaraderie among hate groups. What’s most striking is how many people with extreme ideas are willing to spend time engaging with folks who are in the tornado.
Spend time reading the comments below the YouTube videos of youth struggling to make sense of the world around them. You’ll quickly find comments by people who spend time in the manosphere or subscribe to white supremacist thinking. They are diving in and talking to these youth, offering a framework to make sense of the world, one rooted in deeply hateful ideas.These self-fashioned self-help actors are grooming people to see that their pain and confusion isn’t their fault, but the fault of feminists, immigrants, people of color. They’re helping them believe that the institutions they already distrust — the news media, Hollywood, government, school, even the church — are actually working to oppress them.
Most people who encounter these ideas won’t embrace them, but some will. Still, even those who don’t will never let go of the doubt that has been instilled in the institutions around them. It just takes a spark.
So how do we collectively make sense of the world around us? There isn’t one universal way of thinking, but even the act of constructing knowledge is becoming polarized. Responding to the uproar in the news media over “alternative facts,” Cory Doctorow noted:
We’re not living through a crisis about what is true, we’re living through a crisis about how we know whether something is true. We’re not disagreeing about facts, we’re disagreeing about epistemology. The “establishment” version of epistemology is, “We use evidence to arrive at the truth, vetted by independent verification (but trust us when we tell you that it’s all been independently verified by people who were properly skeptical and not the bosom buddies of the people they were supposed to be fact-checking).”
The “alternative facts” epistemological method goes like this: “The ‘independent’ experts who were supposed to be verifying the ‘evidence-based’ truth were actually in bed with the people they were supposed to be fact-checking. In the end, it’s all a matter of faith, then: you either have faith that ‘their’ experts are being truthful, or you have faith that we are. Ask your gut, what version feels more truthful?”
Doctorow creates these oppositional positions to make a point and to highlight that there is a war over epistemology, or the way in which we produce knowledge.
The reality is much messier, because what’s at stake isn’t simply about resolving two competing worldviews. Rather, what’s at stake is how there is no universal way of knowing, and we have reached a stage in our political climate where there is more power in seeding doubt, destabilizing knowledge, and encouraging others to distrust other systems of knowledge production.
Contemporary propaganda isn’t about convincing someone to believe something, but convincing them to doubt what they think they know. Andonce people’s assumptions have come undone, who is going to pick them up and help them create a coherent worldview?
Deeply committed to democratic governance, George Washington believed that a representative government could only work if the public knew their representatives. As a result, our Constitution states that each member of the House should represent no more than 30,000 constituents. When we stopped adding additional representatives to the House in 1913 (frozen at 435), each member represented roughly 225,000 constituents. Today, the ratio of congresspeople to constituents is more than 700,000:1. Most people will never meet their representative, and few feel as though Washington truly represents their interests. The democracy that we have is representational only in ideal, not in practice.
As our Founding Fathers knew, it’s hard to trust an institution when it feels inaccessible and abstract. All around us, institutions are increasingly divorced from the community in which they operate, with often devastating costs.Thanks to new models of law enforcement, police officers don’t typically come from the community they serve. In many poor communities, teachers also don’t come from the community in which they teach. The volunteer U.S. military hardly draws from all communities, and those who don’t know a solider are less likely to trust or respect the military.
Journalism can only function as the fourth estate when it serves as a tool to voice the concerns of the people and to inform those people of the issues that matter. Throughout the 20th century, communities of color challenged mainstream media’s limitations and highlighted that few newsrooms represented the diverse backgrounds of their audiences. As such, we saw the rise of ethnic media and a challenge to newsrooms to be smarter about their coverage. But let’s be real — even as news organizations articulate a commitment to the concerns of everyone, newsrooms have done a dreadful job of becoming more representative. Over the past decade, we’ve seen racial justice activists challenge newsrooms for their failure to cover Ferguson, Standing Rock, and other stories that affect communities of color.
Meanwhile, local journalism has nearly died. The success of local journalismdidn’t just matter because those media outlets reported the news, but because it meant that many more people were likely to know journalists. It’s easier to trust an institution when it has a human face that you know and respect. Andas fewer and fewer people know journalists, they trust the institution less and less. Meanwhile, the rise of social media, blogging, and new forms of talk radio has meant that countless individuals have stepped in to cover issues not being covered by mainstream news, often using a style and voice that is quite unlike that deployed by mainstream news media.
We’ve also seen the rise of celebrity news hosts. These hosts help push the boundaries of parasocial interactions, allowing the audience to feel deep affinity toward these individuals, as though they are true friends. Tabloid papers have long capitalized on people’s desire to feel close to celebrities by helping people feel like they know the royal family or the Kardashians. Talking heads capitalize on this, in no small part by how they communicate with their audiences. So, when people watch Rachel Maddow or listen to Alex Jones, they feel more connected to the message than they would when reading a news article. They begin to trust these people as though they are neighbors. They feel real.
No amount of drop-in journalism will make up for the loss of journalists within the fabric of local communities.
People want to be informed, but who they trust to inform them is rooted in social networks, not institutions. The trust of institutions stems from trust in people. The loss of the local paper means a loss of trusted journalists and a connection to the practices of the newsroom. As always, people turn to their social networks to get information, but what flows through those social networks is less and less likely to be mainstream news. But here’s where you also get an epistemological divide.
As Francesca Tripodi points out, many conservative Christians have developed a media literacy practice that emphasizes the “original” text rather than an intermediary. Tripodi points out that the same type of scriptural inference that Christians apply in Bible study is often also applied to reading the Constitution, tax reform bills, and Google results. This approach is radically different than the approach others take when they rely on intermediaries to interpret news for them.
As the institutional construction of news media becomes more and more proximately divorced from the vast majority of people in the United States, we can and should expect trust in news to decline. No amount of fact-checking will make up for a widespread feeling that coverage is biased. No amount of articulated ethical commitments will make up for the feeling that you are being fed clickbait headlines.
No amount of drop-in journalism will make up for the loss of journalists within the fabric of local communities. And while the population who believes that CNN and the New York Times are “fake news” are not demographically representative, the questionable tactics that news organizations use are bound to increase distrust among those who still have faith in them.
If you’re still with me at this point, you’re probably deeply invested in scholarship or journalism. And, unless you’re one of my friends, you’re probably bursting at the seams to tell me that the reason journalism is all screwed up is because the internet screwed news media’s business model. So I want to ask a favor: Quiet that voice in your head, take a deep breath, and let me offer an alternative perspective.
There are many types of capitalism. After all, the only thing that defines capitalism is the private control of industry (as opposed to government control). Most Americans have been socialized into believing that all forms of capitalism are inherently good (which, by the way, was a propaganda project). But few are encouraged to untangle the different types of capitalism and different dynamics that unfold depending on which structure is operating.
I grew up in mom-and-pop America, where many people dreamed of becoming small business owners. The model was simple: Go to the bank and get a loan to open a store or a company. Pay back that loan at a reasonable interest rate — knowing that the bank was making money — until eventually you owned the company outright. Build up assets, grow your company, and create something of value that you could pass on to your children.
In the 1980s, franchises became all the rage. Wannabe entrepreneurs saw a less risky path to owning their own business. Rather than having to figure it out alone, you could open a franchise with a known brand and a clear process for running the business. In return, you had to pay some overhead to the parent company. Sure, there were rules to follow and you could only buy supplies from known suppliers and you didn’t actually have full control, but it kinda felt like you did. Like being an Uber driver, it was the illusion of entrepreneurship that was so appealing. And most new franchise owners didn’t know any better, nor were they able to read the writing on the wall when the water all around them started boiling their froggy self. I watched my mother nearly drown, and the scars are still visible all over her body.
I will never forget the U.S. Savings & Loan crisis, not because I understood it, but because it was when I first realized that my Richard Scarry impression of how banks worked was way wrong. Only two decades later did I learn to seethe FIRE industries (Finance, Insurance, and Real Estate) as extractive ones.They aren’t there to help mom-and-pop companies build responsible businesses, but to extract value from their naiveté. Like today’s post-college youth are learning, loans aren’t there to help you be smart, but to bend your will.
It doesn’t take a quasi-documentary to realize thatMcDonald’s is not a fast-food franchise; it’s a real estate business that uses a franchise structure to extract capital from naive entrepreneurs. Go talk to a wannabe restaurant owner in New York City and ask them what it takes to start a business these days. You can’t even get a bank loan or lease in 2018 without significant investor backing, which means that the system isn’t set up for you to build a business and pay back the bank, pay a reasonable rent, and develop a valuable asset.You are simply a pawn in a financialized game between your investors, the real estate companies, the insurance companies, and the bank, all of which want to extract as much value from your effort as possible. You’re just another brick in the wall.
Now let’s look at the local news ecosystem. Starting in the 1980s, savvy investors realized that many local newspapers owned prime real estate in the center of key towns. These prized assets would make for great condos and office rentals. Throughout the country, local news shops started getting eaten up by private equity and hedge funds — or consolidated by organizations controlled by the same forces. Media conglomerates sold off their newsrooms as they felt increased pressure to increase profits quarter over quarter.
Building a sustainable news business was hard enough when the news had a wealthy patron who valued the goals of the enterprise. But the finance industry doesn’t care about sustaining the news business; it wants a return on investment. And the extractive financiers who targeted the news business weren’t looking to keep the news alive. They wanted to extract as much value from those business as possible. Taking a page out of McDonald’s, they forced the newsrooms to sell their real estate. Often, news organizations had to rent from new landlords who wanted obscene sums, often forcing them to move out of their buildings. News outlets were forced to reduce staff, reproduce more junk content, sell more ads, and find countless ways to cut costs. Of course the news suffered — the goal was to push news outlets into bankruptcy or sell, especially if the companies had pensions or other costs that couldn’t be excised.
Yes, the fragmentation of the advertising industry due to the internet hastened this process. And let’s also be clear that business models in the news business have never been clean. But no amount of innovative new business models will make up for the fact that you can’t sustain responsible journalism within a business structure that requires newsrooms to make more money quarter over quarter to appease investors. This does not mean that you can’t build a sustainable news business, but if the news is beholden to investors trying to extract value, it’s going to impossible. And if news companies have no assets to rely on (such as their now-sold real estate), they are fundamentally unstable and likely to engage in unhealthy business practices out of economic desperation.
Untangling our country from this current version of capitalism is going to be as difficult as curbing our addiction to fossil fuels. I’m not sure it can be done, but as long as we look at companies and blame their business models without looking at the infrastructure in which they are embedded, we won’t even begin taking the first steps. Fundamentally, both the New York Times and Facebook are public companies, beholden to investors and desperate to increase their market cap. Employees in both organizations believe themselves to be doing something important for society.
Of course, journalists don’t get paid well, while Facebook’s employees can easily threaten to walk out if the stock doesn’t keep rising, since they’re also investors. But we also need to recognize that the vast majority of Americans have a stake in the stock market. Pension plans, endowments, and retirement plans all depend on stocks going up — and those public companies depend on big investors investing in them. Financial managers don’t invest in news organizations that are happy to be stable break-even businesses. Heck, even Facebook is in deep trouble if it can’t continue to increase ROI, whether through attracting new customers (advertisers and users), increasing revenue per user, or diversifying its businesses. At some point, it too will get desperate, because no business can increase ROI forever.
ROI capitalism isn’t the only version of capitalism out there. We take it for granted and tacitly accept its weaknesses by creating binaries, as though the only alternative is Cold War Soviet Union–styled communism. We’re all frogs in an ocean that’s quickly getting warmer. Two degrees will affect a lot more than oceanfront properties.
In my mind, we have a hard road ahead of us if we actually want to rebuild trust in American society and its key institutions (which, TBH, I’m not sure is everyone’s goal). There are three key higher-order next steps, all of which are at the scale of the New Deal.
Fundamentally, we need to stop triggering one another because we’re facing our own perceived pain. This means we need to build large-scale cultural resilience. While we may be teaching our children “social-emotional learning”in the classroom, we also need to start taking responsibility at scale.Individually, we need to step back and empathize with others’ worldviews and reach out to support those who are struggling. But our institutions also have important work to do.
At the end of the day, if journalistic ethics means anything, newsrooms cannot justify creating spectacle out of their reporting on suicide or other topics just because they feel pressure to create clicks. They have the privilege of choosing what to amplify, and they should focus on what is beneficial. If they can’t operate by those values, they don’t deserve our trust. While I strongly believe that technology companies have a lot of important work to do to be socially beneficial, I hold news organizations to a higher standard because of their own articulated commitments and expectations that they serve as the fourth estate. And if they can’t operationalize ethical practices, I fear the society that must be knitted together to self-govern is bound to fragment even further.
Trust cannot be demanded. It’s only earned by being there at critical junctures when people are in crisis and need help. You don’t earn trust when things are going well; you earn trust by being a rock during a tornado. The winds are blowing really hard right now. Look around. Who is helping us find solid ground?
George Lincoln Rockwell, the head of the American Nazi party, had a simple media strategy in the 1960s. He wrote in his autobiography: “Only by forcing the Jews to spread our message with their facilities could we have any hope of success in counteracting their left-wing, racemixing propaganda!”
Campus by campus, from Harvard to Brown to Columbia, he would use the violence of his ideas and brawn of his followers to become headline news. To compel media coverage, Rockwell needed: “(1) A smashing, dramatic approach which could not be ignored, without exposing the most blatant press censorship, and (2) a super-tough, hard-core of young fighting men to enable such a dramatic presentation to the public.” He understood what other groups competing for media attention knew too well: a movement could only be successful if the media amplified their message.
Contemporary Jewish community groups challenged journalists to consider not covering white supremacists’ ideas. They called this strategy “quarantine”, and it involved working with community organizations to minimize public confrontations and provide local journalists with enough context to understand why the American Nazi party was not newsworthy.
In regions where quarantine was deployed successfully, violence remained minimal and Rockwell was unable to recruit new party members. The press in those areas was aware that amplification served the agenda of the American Nazi party, so informed journalists employed strategic silence to reduce public harm.
The Media Manipulation research initiative at the Data & Society institute is concerned precisely with the legacy of this battle in discourse and the way that modern extremists undermine journalists and set media agendas. Media has always had the ability to publish or amplify particular voices, perspectives and incidents. In choosing stories and voices they will or will not prioritize, editors weigh the benefits and costs of coverage against potential social consequences. In doing so, they help create broader societal values. We call this willingness to avoid amplifying extremist messages “strategic silence”.
Editors used to engage in strategic silence – set agendas, omit extremist ideas and manage voices – without knowing they were doing so. Yet the online context has enhanced extremists’ abilities to create controversies, prompting newsrooms to justify covering their spectacles. Because competition for audience is increasingly fierce and financially consequential, longstanding newsroom norms have come undone. We believe that journalists do not rebuild reputation through a race to the bottom. Rather, we think that it’s imperative that newsrooms actively take the high ground and re-embrace strategic silence in order to defy extremists’ platforms for spreading hate.
Strategic silence is not a new idea. The Ku Klux Klan of the 1920s considered media coverage their most effective recruitment tactic and accordingly cultivated friendly journalists. According to Felix Harcourt, thousands of readers joined the KKK after the New York World ran a three-week chronicle of the group in 1921. Catholic, Jewish and black presses of the 1920s consciously differed from Protestant-owned mainstream papers in their coverage of the Klan, conspicuously avoiding giving the group unnecessary attention. The black press called this use of editorial discretion in the public interest “dignified silence”, and limited their reporting to KKK follies, such as canceled parades, rejected donations and resignations. Some mainstream journalists also grew suspicious of the KKK’s attempts to bait them with camera-ready spectacles. Eventually coverage declined.
The KKK was so intent on getting the coverage they sought that they threatened violence and white boycotts of advertisers. Knowing they could bait coverage with violence, white vigilante groups of the 1960s staged cross burnings and engaged in high-profile murders and church bombings. Civil rights protesters countered white violence with black stillness, especially during lunch counter sit-ins. Journalists and editors had to make moral choices of which voices to privilege, and they chose those of peace and justice, championing stories of black resilience and shutting out white extremism. This was strategic silence in action, and it saved lives.
The emphasis of strategic silence must be placed on the strategic over the silencing. Every story requires a choice and the recent turn toward providing equal coverage to dangerous, antisocial opinions requires acknowledging the suffering that such reporting causes. Even attempts to cover extremism critically can result in the media disseminating the methods that hate groups aim to spread, such as when Virginia’s Westmoreland News reproduced in full a local KKK recruitment flier on its front page. Media outlets who cannot argue that their reporting benefits the goal of a just and ethical society must opt for silence.
Newsrooms must understand that even with the best of intentions, they can find themselves being used by extremists. By contrast, they must also understand they have the power to defy the goals of hate groups by optimizing for core American values of equality, respect and civil discourse. All Americans have the right to speak their minds, but not every person deserves to have their opinions amplified, particularly when their goals are to sow violence, hatred and chaos.
If telling stories didn’t change lives, journalists would never have started in their careers. We know that words matter and that coverage makes a difference. In this era of increasing violence and extremism, we appeal to editors to choose strategic silence over publishing stories that fuel the radicalization of their readers.
(Visit the original version at The Guardian to read the comments and help support their organization, as a sign of appreciation for their willingness to publish our work.)
The below original text was the basis for Data & Society Founder and President danah boyd’s March 2018 SXSW Edu keynote,“What Hath We Wrought?” — Ed.
Growing up, I took certain truths to be self evident. Democracy is good. War is bad. And of course, all men are created equal.
My mother was a teacher who encouraged me to question everything. But I quickly learned that some questions were taboo. Is democracy inherently good? Is the military ethical? Does God exist?
I loved pushing people’s buttons with these philosophical questions, but they weren’t nearly as existentially destabilizing as the moments in my life in which my experiences didn’t line up with frames that were sacred cows in my community. Police were revered, so my boss didn’t believe me when I told him that cops were forcing me to give them free food, which is why there was food missing. Pastors were moral authorities and so our pastor’s infidelities were not to be discussed, at least not among us youth. Forgiveness is a beautiful thing, but hypocrisy is destabilizing. Nothing can radicalize someone more than feeling like you’re being lied to. Or when the world order you’ve adopted comes crumbling down.
The funny thing about education is that we ask our students to challenge their assumptions. And that process can be enlightening.
The funny thing about education is that we ask our students to challenge their assumptions. And that process can be enlightening. I will never forget being a teenager and reading “The People’s History of the United States.” The idea that there could be multiple histories, multiple truths blew my mind.Realizing that history is written by the winners shook me to my core. This is the power of education. But the hole that opens up, that invites people to look for new explanations…that hole can be filled in deeply problematic ways.When we ask students to challenge their sacred cows but don’t give them a new framework through which to make sense of the world, others are often there to do it for us.
For the last year, I’ve been struggling with media literacy. I have a deep level of respect for the primary goal. As Renee Hobbs has written, media literacy is the “active inquiry and critical thinking about the messages we receive and create.” The field talks about the development of competencies or skills to help people analyze, evaluate, and even create media. Media literacy is imagined to be empowering, enabling individuals to have agency and giving them the tools to help create a democratic society. But fundamentally, it is a form of critical thinking that asks people to doubt what they see. And that makes me nervous.
Most media literacy proponents tell me that media literacy doesn’t exist in schools. And it’s true that the ideal version that they’re aiming for definitely doesn’t. But I spent a decade in and out of all sorts of schools in the US, where I quickly learned that a perverted version of media literacy does already exist.Students are asked to distinguish between CNN and Fox. Or to identify bias in a news story. When tech is involved, it often comes in the form of “don’t trust Wikipedia; use Google.” We might collectively dismiss these practices as not-media-literacy, but these activities are often couched in those terms.
I’m painfully aware of this, in part because media literacy is regularly proposed as the “solution” to the so-called “fake news” problem. I hear this from funders and journalists, social media companies and elected officials. My colleagues Monica Bulger and Patrick Davison just released a report on media literacy in light of “fake news” given the gaps in current conversations. I don’t know what version of media literacy they’re imagining but I’m pretty certain it’s not the CNN vs Fox News version. Yet, when I drill in, they often argue for the need to combat propaganda, to get students to ask where the money is coming from, to ask who is writing the stories for what purposes, to know how to fact-check, etcetera. And when I push them further, I often hear decidedly liberal narratives. They talk about the Mercers or about InfoWars or about the Russians. They mock “alternative facts.” While I identify as a progressive, I am deeply concerned by how people understand these different conservative phenomena and what they see media literacy as solving.
I get that many progressive communities are panicked about conservative media, but we live in a polarized society and I worry about how people judge those they don’t understand or respect. It also seems to me that the narrow version of media literacy that I hear as the “solution” is supposed to magically solve our political divide. It won’t. More importantly, as I’m watching social media and news media get weaponized, I’m deeply concerned that the well-intended interventions I hear people propose will backfire, because I’m fairly certain that the crass versions of critical thinking already have.
My talk today is intended to interrogate some of the foundations upon which educating people about the media landscape depends. Rather than coming at this from the idealized perspective, I am trying to come at this from the perspective of where good intentions might go awry, especially in a moment in which narrow versions of media literacy and critical thinking are being proposed as the solution to major socio-cultural issues. I want to examine the instability of our current media ecosystem to then return to the question of:what kind of media literacy should we be working towards? So let’s dig in.
In 2017, sociologist Francesca Tripodi was trying to understand how conservative communities made sense of the seemingly contradictory words coming out of the mouth of the US President. Along her path, she encountered people talking about making sense of The Word when referencing his speeches. She began accompanying people in her study to their bible study groups. Then it clicked. Trained on critically interrogating biblical texts, evangelical conservative communities were not taking Trump’s messages as literal text. They were interpreting their meanings using the sameepistemological framework as they approached the bible. Metaphors and constructs matter more than the precision of words.
Why do we value precision in language? I sat down for breakfast with Gillian Tett, a Financial Times journalist and anthropologist. She told me that when she first moved to the States from the UK, she was confounded by our inability to talk about class. She was trying to make sense of what distinguished class in America. In her mind, it wasn’t race. Or education. It came down to what construction of language was respected and valued by whom. People became elite by mastering the language marked as elite. Academics, journalists, corporate executives, traditional politicians: they all master the art of communication. I did too. I will never forget being accused of speaking like an elite by my high school classmates when I returned home after a semester of college. More importantly, although it’s taboo in America to be explicitly condescending towards people on the basis of race or education, there’s no social cost among elites to mock someone for an inability to master language.For using terms like “shithole.”
Linguistic and communications skills are not universally valued. Those who do not define themselves through this skill loathe hearing the never-ending parade of rich and powerful people suggesting that they’re stupid, backwards, and otherwise lesser. Embracing being anti-PC has become a source of pride, a tactic of resistance. Anger boils over as people who reject “the establishment” are happy to watch the elites quiver over their institutions being dismantled. This is why this is a culture war. Everyone believes they are part of the resistance.
But what’s at the root of this cultural war? Cory Doctorow got me thinkingwhen he wrote the following:
We’re not living through a crisis about what is true, we’re living through a crisis about how we know whether something is true. We’re not disagreeing about facts,we’re disagreeing about epistemology. The “establishment” version of epistemology is, “We use evidence to arrive at the truth, vetted by independent verification (but trust us when we tell you that it’s all been independently verified by people who were properly skeptical and not the bosom buddies of the people they were supposed to be fact-checking).”
The “alternative facts” epistemological method goes like this: “The ‘independent’ experts who were supposed to be verifying the ‘evidence-based’ truth were actually in bed with the people they were supposed to be fact-checking. In the end, it’s all a matter of faith, then: you either have faith that ‘their’ experts are being truthful, or you have faith that we are. Ask your gut, what version feels more truthful?”
Let’s be honest — most of us educators are deeply committed to a way of knowing that is rooted in evidence, reason, and fact. But who gets to decide what constitutes a fact? In philosophy circles, social constructivists challenge basic tenets like fact, truth, reason, and evidence. Yet, it doesn’t take a doctorate of philosophy to challenge the dominant way of constructing knowledge. Heck, 75 years ago, evidence suggesting black people were biologically inferior was regularly used to justify discrimination. And this was called science!
In many Native communities, experience trumps Western science as the key to knowledge. These communities have a different way of understanding topics like weather or climate or medicine. Experience is also used in activist circles as a way of seeking truth and challenging the status quo. Experience-based epistemologies also rely on evidence, but not the kind of evidence that would be recognized or accepted by those in Western scientific communities.
Those whose worldview is rooted in religious faith, particularly Abrahamic religions, draw on different types of information to construct knowledge. Resolving scientific knowledge and faith-based knowledge has never been easy; this tension has countless political and social ramifications. As a result, American society has long danced around this yawning gulf and tried to find solutions that can appease everyone. But you can’t resolve fundamental epistemological differences through compromise.
No matter what worldview or way of knowing someone holds dear, they always believe that they are engaging in critical thinking when developing a sense of what is right and wrong, true and false, honest and deceptive. But much of what they conclude may be more rooted in their way of knowing than any specific source of information.
If we’re not careful, “media literacy” and “critical thinking”will simply be deployed as an assertion of authority over epistemology.
Right now, the conversation around fact-checking has already devolved to suggest that there’s only one truth. And we have to recognize that there are plenty of students who are taught that there’s only one legitimate way of knowing, one accepted worldview. This is particularly dicey at the collegiate level, where us professors have been taught nothing about how to teach across epistemologies.
Personally, it took me a long time to recognize the limits of my teachers. Like many Americans in less-than-ideal classrooms, I was taught that history was a set of facts to be memorized. When I questioned those facts, I was sent to the principal’s office for disruption. Frustrated and confused, I thought that I was being force-fed information for someone else’s agenda. Now I can recognize that that teacher was simply exhausted, underpaid, and waiting for retirement. But it took me a long time to realize that there was value in history and that history is a powerful tool.
The political scientist Deen Freelon was trying to make sense of the role of critical thinking to address “fake news.” He ended up looking back at a fascinating campaign by Russian Today (known as RT). Their motto for a while was “question more.” They produced a series of advertisements as teasers for their channel. These advertisements were promptly banned in the US and UK, resulting in RT putting up additional ads about how they were banned and getting tremendous mainstream media coverage about being banned. What was so controversial? Here’s an example:
“Just how reliable is the evidence that suggests human activity impacts on climate change? The answer isn’t always clear-cut. And it’s only possible to make a balanced judgement if you are better informed. By challenging the accepted view, we reveal a side of the news that you wouldn’t normally see. Because we believe that the more you question, the more you know.”
If you don’t start from a place where you’re confident that climate change is real, this sounds quite reasonable. Why wouldn’t you want more information? Why shouldn’t you be engaged in critical thinking? Isn’t this what you’re encouraged to do at school? So why is asking this so taboo? And lest you think that this is a moment to be condescending towards climate deniers, let me offer another one of their ads.
“Is terror only committed by terrorists? The answer isn’t always clear-cut. And it’s only possible to make a balanced judgement if you are better informed. By challenging the accepted view, we reveal a side of the news that you wouldn’t normally see. Because we believe that the more you question, the more you know.”
Many progressive activists ask whether or not the US government commits terrorism in other countries. The ads all came down because they were too political, but RT got what they wanted: an effective ad campaign. They didn’t come across as conservative or liberal, but rather a media entity that was “censored” for asking questions. Furthermore, by covering the fact that they were banned, major news media legitimized their frame under the rubric of “free speech.” Under the assumption that everyone should have the right to know and to decide for themselves.
We live in a world now where we equate free speech with the right to be amplified. Does everyone have the right to be amplified? Social media gave us that infrastructure under the false imagination that if we were all gathered in one place, we’d find common ground and eliminate conflict. We’ve seen this logic before. After World War II, the world thought that connecting the globe through financial interdependence would prevent World War III. It’s not clear that this logic will hold.
For better and worse, by connecting the world through social media and allowing anyone to be amplified, information can spread at record speed.There is no true curation or editorial control. The onus is on the public to interpret what they see. To self-investigate. Since we live in a neoliberal society that prioritizes individual agency, we double down on media literacy as the “solution” to misinformation. It’s up to each of us as individuals to decide for ourselves whether or not what we’re getting is true.
Yet, if you talk with someone who has posted clear, unquestionable misinformation, more often than not, they know it’s bullshit. Or they don’t care whether or not it’s true. Why do they post it then? Because they’re making a statement. The people who posted this meme (figure 1) didn’t bother to fact check this claim. They didn’t care. What they wanted to signal loud and clear is that they hated Hillary Clinton. And that message was indeed heard loud and clear. As a result, they are very offended if you tell them that they’ve been duped by Russians into spreading propaganda. They don’t believe you for one second.
Misinformation is contextual. Most people believe that people they know are gullible to false information, but that they themselves are equipped to separate the wheat from the chaff. There’s widespread sentiment that we can fact check and moderate our way out of this conundrum. This will fail. Don’t forget that for many people in this country, both education and the media are seen as the enemy — two institutions who are trying to have power over how people think. Two institutions that are trying to assert authority over epistemology.
Growing up on Usenet, Godwin’s Law was more than an adage to me. I spent countless nights lured into conversation by the idea that someone was wrong on the internet. And I long ago lost count about how many of them ended up with someone invoking Hitler or the Holocaust. I might have even been to blame in some of these conversations.
Fast forward 15 years to the point when Nathan Poe wrote a poignant comment on an online forum dedicated to Christianity: “Without a winking smiley or other blatant display of humor, it is utterly impossible to parody a Creationist in such a way that someone won’t mistake for the genuine article.”Poe’s Law, as it became known, signals that it’s hard to tell the difference between an extreme view and a parody of an extreme view on the internet.
In their book, “The Ambivalent Internet,”media studies scholars Whitney Phillips and Ryan Milner highlight how a segment of society has become so well-versed at digital communications — memes, GIFs, videos, etc. — that they can use these tools of expression to fundamentally destabilize others’communication structures and worldviews. It’s hard to tell what’s real and what’s fiction, what’s cruel and what’s a joke. But that’s the point. That is howirony and ambiguity can be weaponized. And for some, the goal is simple:dismantle the very foundations of elite epistemological structures that are so deeply rooted in fact and evidence.
Many people, especially young people, turn to online communities to make sense of the world around them. They want to ask uncomfortable questions, interrogate assumptions, and poke holes at things they’ve heard. Welcome to youth. There are some questions that are unacceptable to ask in public and they’ve learned that. But in many online fora, no question or intellectual exploration is seen as unacceptable. To restrict the freedom of thought is to censor. And so all sorts of communities have popped up for people to explore questions of race and gender and other topics in the most extreme ways possible. And these communities have become slippery. Are those taking on such hateful views real? Or are they being ironic?
In the 1999 film The Matrix, Morpheus says to Neo: “You take the blue pill,the story ends. You wake up in your bed and believe whatever you want. You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes.” Most youth aren’t interested in having the wool pulled over their head, even if blind faith might be a very calming way of living. Restricted in mobility and stressed to holy hell, they want to have access to what’s inaccessible, know what’s taboo, and say what’s politically incorrect. So who wouldn’t want to take the red pill?
In some online communities, taking the red pill refers to the idea of waking up to how education and media are designed to deceive you into progressive propaganda. In these environments, visitors are asked to question more. They’re invited to rid themselves of their politically correct shackles. There’s an entire online university designed to undo accepted ideas about diversity, climate, and history. Some communities are even more extreme in their agenda. These are all meant to fill in the gaps for those who are opening to questioning what they’ve been taught.
In 2012, it was hard not to avoid the names Trayvon Martin and George Zimmerman, but that didn’t mean that most people understood the storyline.In South Carolina, a white teenager who wasn’t interested in the news felt like he needed to know what the fuss was all about. He decided to go to Wikipedia to understand more. He was left with the impression that Zimmerman was clearly in the right and disgusted that everyone was defending Martin. While reading up on this case, he ran across the term “black on white crime” on Wikipedia and decided to throw that term into Google where he encountered a deeply racist website inviting him to wake up to a reality that he had never considered. He took that red pill and dove deep into a worldview whose theory of power positioned white people as victims. Over a matter of years, he began to embrace those views, to be radicalized towards extreme thinking. On June 17, 2015, he sat down for an hour with a group of African-American church-goers in Charleston South Carolina before opening fire on them, killing 9 and injuring 1. His goal was simple: he wanted to start a race war.
It’s easy to say that this domestic terrorist was insane or irrational, but he began his exploration trying to critically interrogate the media coverage of a story he didn’t understand. That led him to online fora filled with people who have spent decades working to indoctrinate people into a deeply troubling, racist worldview. They draw on countless amounts of “evidence,” engage in deeply persuasive discursive practices, and have the mechanisms to challenge countless assumptions. The difference between what is deemed missionary work, education, and radicalization depends a lot on your worldview. And your understanding of power.
The majority of Americans do not trust the news media. There are many explanations for this — loss of local news, financial incentives, hard to distinguish between opinion and reporting, etc. But what does it mean to encourage people to be critical of the media’s narratives when they are already predisposed against the news media?
Perhaps you want to encourage people to think critically about how information is constructed, who is paying for it, and what is being left out. Yet, among those whose prior is to not trust a news media institution, among those who see CNN and The New York Times as “fake news,” they’re already there. They’re looking for flaws. It’s not hard to find them. After all, the news industry is made of people in institutions in a society. So when youth are encouraged to be critical of the news media, they come away thinking that the media is lying. Depending on someone’s prior, they may even take what they learn to be proof that the media is in on the conspiracy. That’s where things get very dicey.
Many of my digital media and learning colleagues encourage people to make media to help understand how information is produced. Realistically, many young people have learned these skills outside the classroom as they seek to represent themselves on Instagram, get their friends excited about a meme, or gain followers on YouTube. Many are quite skilled at using media, but to what end? Every day, I watch teenagers produce anti-Semitic and misogynistic content using the same tools that activists use to combat prejudice. It’s notable that many of those who are espousing extreme viewpoints are extraordinarily skilled at using media. Today’s neo-Nazis are a digital propaganda machine. Developing media making skills doesn’t guarantee that someone will use them for good. This is the hard part.
Most of my peers think that if more people are skilled and more people are asking hard questions, goodness will see the light. In talking about misunderstandings of the First Amendment, Nabiha Syed of Buzzfeedhighlights that the frame of the “marketplace of ideas” sounds great, but is extremely naive. Doubling down on investing in individuals as a solution to a systemic abuse of power is very American. But the best ideas don’t always surface to the top. Nervously, many of us tracking manipulation of media are starting to think that adversarial messages are far more likely to surface than well-intended ones.
This is not to say that we shouldn’t try to educate people. Or that producing critical thinkers is inherently a bad thing. I don’t want a world full of sheeple.But I also don’t want to naively assume what media literacy could do in responding to a culture war that is already underway. I want us to grapple with reality, not just the ideals that we imagine we could maybe one day build.
It’s one thing to talk about interrogating assumptions when a person can keep emotional distance from the object of study. It’s an entirely different thing to talk about these issues when the very act of asking questions is what’s being weaponized. This isn’t historical propaganda distributed through mass media. Or an exercise in understanding state power. This is about making sense of an information landscape where the very tools that people use to make sense of the world around them have been strategically perverted by other people who believe themselves to be resisting the same powerful actors that we normally seek to critique.
Take a look at the graph above. Can you guess what search term this is? This is the search query for “crisis actors.” This concept emerged as a conspiracy theory after Sandy Hook. Online communities worked hard to get this to land with the major news media after each shooting. With Parkland, they finally succeeded. Every major news outlet is now talking about crisis actors, as though it’s a real thing, or something to be debunked. When teenage witnesses of the mass shooting in Parkland speak to journalists these days, they have to now say that they are not crisis actors. They must negate a conspiracy theory that was created to dismiss them. A conspiracy theory that undermines their message from the get-go. And because of this, many people have turned to Google and Bing to ask what a crisis actor is. They quickly get to the Snopes page. Snopes provides a clear explanation of why this is a conspiracy. But you are now asked to not think of an elephant.
You may just dismiss this as craziness, but getting this narrative into the media was designed to help radicalize more people. Some number of people will keep researching, trying to understand what the fuss is all about. They’ll find online fora discussing the images of a brunette woman and ask themselves if it might be the same person. They will try to understand the fight between David Hogg and Infowars or question why Infowars is being restricted by YouTube. They may think this is censorship. Seeds of doubt will start to form. And they’ll ask whether or not any of the articulate people they see on TV might actually be crisis actors. That’s the power of weaponized narratives.
One of the main goals for those who are trying to manipulate media is to pervert the public’s thinking. It’s called gaslighting. Do you trust what is real?One of the best ways to gaslight the public is to troll the media. By getting the news media to be forced into negating frames, they can rely on the fact that people who distrust the media often respond by self-investigating. This is the power of the boomerang effect. And it has a history. After all, the CDC realized that the more news media negated the connection between autism and vaccination, the more the public believed there was something real there.
In 2016, I watched networks of online participants test this theory through an incident now known as Pizzagate. They worked hard to get the news media to negate the conspiracy theory, believing that this would prompt more people to try to research if there was something real there. They were effective. The news media covered the story to negate it. Lots of people decided to self-investigate. One guy even showed up with a gun.
The term “gaslighting” originates in the context of domestic violence. The term refers back to an 1944 movie called Gas Light where a woman is manipulated by her husband in a way that leaves her thinking she’s crazy. It’sa very effective technique of control. It makes someone submissive and disoriented, unable to respond to a relationship productively. While many anti-domestic violence activists argue that the first step is to understand that gaslighting exists, the “solution” is not to fight back against the person doing the gaslighting. Instead, it’s to get out. Furthermore, anti-domestic violence experts argue that recovery from gaslighting is a long and arduous process, requiring therapy. They recognize that once instilled, self-doubt is hard to overcome.
While we have many problems in our media landscape, the most dangerous is how it is being weaponized to gaslight people.
And unlike the domestic violence context, there is no “getting out” that is really possible in a media ecosystem. Sure, we can talk about going off the grid and opting out of social media and news media, but c’mon now.
In 2017, Netflix released a show called 13 Reasons Why. Before parents and educators had even heard of the darn show, millions of teenagers had watched it. For most viewers, it was a fascinating show. The storyline was enticing, the acting was phenomenal. But I’m on the board of Crisis Text Line, an amazing service where people around this country talk with trained counselors via text message when they’re in a crisis. Before the news media even began talking about the show, we started to see the impact. After all, the premise of the show is that a teen girl died by suicide and left behind 13 tapes explaining how people had bullied her to justify her decision.
At Crisis Text Line, we do active rescues every night. This means that we send emergency personnel to the homes of someone who is in the middle of a suicide attempt in an effort to save their lives. Sometimes, we succeed. Sometimes, we don’t. It’s heartbreaking work. As word of 13 Reasons Why got out and people started watching the show, our numbers went through the roof. We were drowning in young people referencing the show, signaling how it had given them a framework for ending their lives. We panicked. All hands on deck. As we got things under control, I got angry. What the hell was Netflix thinking?
Researchers know the data on suicide and media. The more the media normalizes suicide, the more suicide is put into people’s head as a possibility,the more people who are on the edge start to take it seriously and consider it for themselves. After early media effects research was published, journalists developed best practices to minimize their coverage of suicide. As Joan Donovan often discusses, this form of “strategic silence” was viable in earlier media landscapes; it’s a lot harder now. Today, journalists and media makers feel as though the fact that anyone could talk about suicide on the internet means that they should have a right to do so too.
We know that you can’t combat depression through rational discourse.Addressing depression is hard work. And I’m deeply concerned that we don’t have the foggiest clue how to approach the media landscape today. I’m confident that giving grounded people tools to think smarter can be effective.But I’m not convinced that we know how to educate people who do not share our epistemological frame. I’m not convinced that we know how to undo gaslighting. I’m not convinced that we understand how engaging people about the media intersects with those struggling with mental health issues.And I’m not convinced that we’ve even begun to think about the unintended consequences of our good — let alone naive — intentions.
In other words, I think that there are a lot of assumptions baked into how we approach educating people about sensitive issues and our current media crisis has made those painfully visible.
Oh, and by the way, the Netflix TV show ends by setting up Season 2 to start with a school shooting. WTF, Netflix?
So what role do educators play in grappling with the contemporary media landscape? What kind of media literacy makes sense? To be honest, I don’t know. But it’s unfair to end a talk like this without offering some path forward so I’m going to make an educated guess.
I believe that we need to develop antibodies to help people not be deceived.
That’s really tricky because most people like to follow their gut more than than their mind. No one wants to hear that they’re being tricked. Still, I thinkthere might be some value in helping people understand their own psychology.
Consider the power of nightly news and talk radio personalities. If you bring Sean Hannity, Rachel Maddow, or any other host into your home every night,you start to appreciate how they think. You may not agree with them, but youbuild a cognitive model of their words such that they have a coherent logic to them. They become real to you, even if they don’t know who you are. This is what scholars call “parasocial interaction.” And the funny thing about humanpsychology is that we trust people who we invest our energies into understanding. That’s why bridging difference requires humanizing people across viewpoints.
Empathy is a powerful emotion, one that most educators want to encourage.But when you start to empathize with worldviews that are toxic, it’s very hard to stay grounded. It requires deep cognitive strength. Scholars who spend a lot of time trying to understand dangerous worldviews work hard to keep their emotional distance. One very basic tactic is to separate the different signals. Just read the text rather than consume the multimedia presentation of that. Narrow the scope. Actively taking things out of context can be helpful for analysis precisely because it creates a cognitive disconnect. This is the opposite of how most people encourage everyday analysis of media, where the goal is to appreciate the context first. Of course, the trick here is wanting to keep that emotional distance. Most people aren’t looking for that.
I also believe that it’s important to help students truly appreciate epistemological differences. In other words, why do people from different worldviews interpret the same piece of content differently? Rather than thinking about the intention behind the production, let’s analyze the contradictions in the interpretation. This requires developing a strong sense of how others think and where the differences in perspective lie. From an educational point of view, this means building the capacity to truly hear and embrace someone else’s perspective and teaching people to understand another’s view while also holding their view firm. It’s hard work, an extension of empathy into a practice that is common among ethnographers. It’s also a skill that is honed in many debate clubs. The goal is to understand the multiple ways of making sense of the world and use that to interpret media.Of course, appreciating the view of someone who is deeply toxic isn’t always psychologically stabilizing.
Another thing I recommend is to help students see how they fill in gaps when the information presented to them is sparse and how hard it is to overcome priors. Conversations about confirmation bias are important here because it’s important to understand what information we accept and what information we reject. Selective attention is another tool, most famously shown to students through the “gorilla experiment.” If you aren’t familiar with this experiment, it involves showing a basketball video and focusing on counting passes made by people in one color shirt and then asking if they saw the gorilla. Many people do not. Inverting these cognitive science exercises,asking students to consider different fan fiction that fills in the gaps of a story with divergent explanations is another way to train someone to recognize how their brain fills in gaps.
What’s common about the different approaches I’m suggesting is that they are designed to be cognitive strengthening exercises, to help students recognize their own fault lines, not the fault lines of the media landscape around them. I can imagine that this too could be called media literacy and if you want to bend your definition that way, I’ll accept it. But the key is to realize the humanity in ourselves and in others. We cannot and should not assert authority over epistemology, but we can encourage our students to be more aware of how interpretation is socially constructed. And to understand how that can be manipulated. Of course, just because you know you’re being manipulated doesn’t mean that you can resist it. And that’s where my proposal starts to get shaky.
Let’s be honest — our information landscape is going to get more and more complex. Educators have a critical role to play in helping individuals and societies navigate what we encounter. But the path forward isn’t about doubling down on what constitutes a fact or teaching people to assess sources.Rebuilding trust in institutions and information intermediaries is important, but we can’t assume the answer is teaching students to rely on those signals.The first wave of media literacy was responding to propaganda in a mass media context. We live in a world of networks now. We need to understand how those networks are intertwined and how information that spreads through dyadic — even if asymmetric — encounters is understood and experienced differently than that which is produced and disseminated through mass media.
Above all, we need to recognize that information can, is, and will be weaponized in new ways. Today’s propagandist messages are no longer simply created by Madison Avenue or Edward Bernays-style State campaigns. For the last 15 years, a cohort of young people has learned how to hack the attention economy in an effort to have power and status in this new information ecosystem. These aren’t just any youth. They are young people who are disenfranchised, who feel as though the information they’re getting isn’t fulfilling, who struggle to feel powerful. They are trying to make sense of an unstable world and trying to respond to it in a way that is personally fulfilling.Most youth are engaged in invigorating activities. Others are doing the same things youth have always done. But there are youth out there who feel alienated and disenfranchised, who distrust the system and want to see it all come down. Sometimes, this frustration leads to productive ends. Often it does not. But until we start understanding their response to our media society, we will not be able to produce responsible interventions. So I would argue that we need to start developing a networked response to this networked landscape. And it starts by understanding different ways of constructing knowledge.
Special thanks to Monica Bulger, Mimi Ito, Whitney Phillips, Cathy Davidson, Sam Hinds Garcia, Frank Shaw, and Alondra Nelson for feedback.
Update (March 16, 2018): I crafted some responses to the most common criticisms I’ve received to date about this work here. (Also, the original version of this blog post was published on Medium.)
(This was originally posted on NewCo Shift.)
A friend of mine worked for an online dating company whose audience was predominantly hetero 30-somethings. At some point, they realized that a large number of the “female” accounts were actually bait for porn sites and 1–900 numbers. I don’t remember if users complained or if they found it themselves, but they concluded that they needed to get rid of these fake profiles. So they did.
And then their numbers started dropping. And dropping. And dropping.
Trying to understand why, researchers were sent in. What they learned was that hot men were attracted to the site because there were women that they felt were out of their league. Most of these hot men didn’t really aim for these ultra-hot women, because they felt like they would be inaccessible, but they were happy to talk with women who they saw as being one rung down (as in actual hot women). These hot women, meanwhile, were excited to have these hot men (who they saw as equals) on the site. These also felt that, since there were women hotter than them, that this was a site for them. When they removed the fakes, the hot men felt the site was no longer for them. They disappeared. And then so did the hot women. Etc. The weirdest part? They reintroduced decoy profiles (not as redirects to porn but as fake women who just didn’t respond) and slowly folks came back.
Why am I telling you this story? Fake accounts and bots on social media are not new. Yet, in the last couple of weeks, there’s been newfound hysteria around Twitter bots and fake accounts. I find it deeply problematic that folks are saying that having fake followers is inauthentic. This is like saying that makeup is inauthentic. What is really going on here?
From the earliest days of Friendster and MySpace, people liked to show how cool they were by how many friends they had. As Alice Marwick eloquentlydocumented, self-branding and performing status were the name of the gamefor many in the early days of social media. This hasn’t changed. People made entire careers out of appearing to be influential, not just actually being influential. Of course a market emerged around this so that people could buy and sell followers, friends, likes, comments, etc. Indeed, standard practice, especially in the wink-nudge world of Instagram, where monetized content is the game and so-called organic “macroinfluencers” can easily double their follower size through bots are more than happily followed by bots, paid or not.
Some sites have tried to get rid of fake accounts. Indeed, Friendster played whack-a-mole with them, killing off “Fakesters” and any account that didn’t follow their strict requirements; this prompted a mass exodus. Facebook’s real-name policy also signaled that such shenanigans would not be allowed on their site, although shhh…. lots of folks figured out how to have multiple accounts and otherwise circumvent the policy.
And let’s be honest — fake accounts are all over most online dating profiles. Ashley Madison, anyone?
Bots have been an intrinsic part of Twitter since the early days. Following the Pope’s daily text messaging services, the Vatican set up numerous bots offering Catholics regular reflections. Most major news organizations have bots so that you can keep up with the headlines of their publications. Twitter’s almost-anything-goes policy meant that people have built bots for all sorts of purposes. There are bots that do poetry, ones that argue with anti-vaxxers about their beliefs, and ones that call out sexist comments people post. I’m a big fan of the @censusAmericans bot created by FiveThirtyEight to regularly send out data from the Census about Americans.
Over the last year, sentiment towards Twitter’s bots has become decidedly negative. Perhaps most people didn’t even realize that there were bots on the site. They probably don’t think of @NYTimes as a bot. When news coverage obsesses over bots, they primarily associate the phenomenon with nefarious activities meant to seed discord, create chaos, and do harm. It can all be boiled down to: Russian bots. As a result, Congress saw bots as inherently bad and journalists keep accusing Twitter of having a “bot problem” without accounting for how their stories appear on Twitter through bots.
Although we often hear about the millions and millions of bots on Twitter as though they’re all manipulative, the stark reality is that bots can be quite fun. I had my students build Twitter bots to teach them how these things worked — they had a field day, even if they didn’t get many followers.
Of course, there are definitely bots that you can buy to puff up your status. Some of them might even be Russian built. And here’s where we get to the crux of the current conversation.
People buy bots to increase their number of followers, retweets, and likes in order to appear cooler than they are. Think of this as mascara for your digital presence. While plenty of users are happy chatting away with their friends without their makeup on, there’s an entire class of professionals who feel the need to be dolled up and giving the best impression possible. It’s a competition for popularity and status, marked by numbers.
Number games are not new, especially not in the world of media. Take a well-established firm like Nielsen. Although journalists often uncritically quote Nielsen numbers as though they are “fact,” most people in the ad and media business know that they’re crap. But they’ve long been the best crap out there. And, more importantly, they’re uniform crap so businesses can make predictable decisions off of these numbers, fully aware that they might not be that accurate. The same has long been true of page views and clicks. No major news organization should take their page views literally. And yet, lots of news agencies rank their reporters based on this data.
What makes the purchasing of Twitter bots and status so nefarious? The NYTimes story suggests that doing so is especially deceptive. Their coverage shamed Twitter into deleting a bunch of Twitter accounts, outing all of the public figures who had bought bots. It almost felt like a discussion of who had gotten Botox.
Much of this recent flurry of coverage suggests that the so-called bot problem is a new thing that is “finally” known. It boggles my mind to think that any regular Twitter user hadn’t seen automated accounts in the past. And heck, there have been services like Twitter Audit to see how many fake followers you have since at least 2012. Gilad Lotan even detailed the ecosystem of buying fake followers in 2014. I think that what’s new is that the term “bot” is suddenly toxic. And it gives us an opportunity to engage in another round of social shaming targeted at insecure people’s vanity all under the false pretense of being about bad foreign actors.
I’ve never been one to feel the need to put on a lot of makeup in order to leave the house and I haven’t been someone who felt the need to buy bots to appear cool online. But I find it deeply hypocritical to listen to journalists and politicians wring their hands about fake followers and bots given that they’ve been playing at that game for a long time. Who among them is really innocent of trying to garner attention through any means possible?
At the end of the day, I don’t really blame Twitter for giving these deeply engaged users what they want and turning a blind eye towards their efforts to puff up their status online. After all, the cosmetic industry is $55 billion. Then again, even cosmetic companies sometimes change their formulas when their products receive bad press.
Note: I’m fully aware of hypotheses that bots have destroyed American democracy. That’s a different essay. But I think that the main impact that they have had, like spam, is to destabilize people’s trust in the media ecosystem. Still, we need to contend with the stark reality that they do serve a purpose and some people do want them.
(This was originally posted on NewCo Shift)
Ever since key Apple investors challenged the company to address kids’ phone addiction, I’ve gotten a stream of calls asking me to comment on the topic. Mostly, I want to scream. I wrote extensively about the unhelpful narrative of “addiction” in my book It’s Complicated: The Social Lives of Networked Teens. At the time, the primary concern was social media. Today, it’s the phone, but the same story still stands: young people are using technology to communicate with their friends non-stop at a point in their life when everything is about sociality and understanding your place in the social world.
As much as I want to yell at all of the parents around me to chill out, I’m painfully and acutely aware of how ineffective this is. Parents don’t like to see that they’re part of the problem or that their efforts to protect and help their children might backfire. (If you want to experience my frustration in full color, watch the Black Mirror episode called “Arkangel” (trailer here).)
Lately, I’ve been trying to find smaller interventions that can make a huge different, tools that parents can use to address the problems they panic about. So let me offer two approaches for “addiction” that work at different ages.
In the early years, children learn values and norms by watching their parents and other caregivers. They emulate our language and our facial expressions, our quirky habits and our tastes. There’s nothing more satisfying and horrifying than listening to your child repeat something you say all too often. Guess what? They also get their cues about technology from people around them. A child would need to be alone in the woods to miss that people love their phones. From the time that they’re born, people are shoving phones in their faces to take pictures, turning to their phones to escape, and obsessively talking on their phones while ignoring them. Of course they want the attention that they see the phone as taking away. And of course they want the device to be special to them.
So, here’s what I recommend to parents of small people: Verbalize what you’re doing with your phone. Whenever you pick up your phone (or other technologies) in front of your kids, say what you’re doing. And involve them in the process if they’d like.
The funny thing about verbalizing what you’re doing is that you’ll check yourself about your decisions to grab that phone. Somehow, it’s a lot less comfy saying: “Mom’s going to check work email because she can’t stop looking in case something important happens.” Once you begin saying out loud every time you look at technology, you also realize how much you’re looking at technology. And what you’re normalizing for your kids. It’s like looking at a mirror and realizing what they’re learning. So check yourself and check what you have standardized. Are you cool with the values and norms you’ve set?
I can’t tell you how many parents have told me that they have a rule in their house that their kids can’t use technology until X, where X could be “after dinner” or “after homework is done” or any other markers. And yet, consistently, I ask them if they put away their phones during dinner or until after they’ve bathed and they look at me like I’m an alien. Teenagers loathe hypocrisy. It’s the biggest thing that I’ve seen to undermine trust between a parent and a child. And boy do they have a lot to say about their parents’ addiction to their phones. Oy vay.
So if you want to curb the usage of your child’s technology use, here’s what I propose: Create a household contract. This is a contract that sets the boundaries for everyone in the house — parents and kids.
Ask your teenage or tween child to write the first draft of the contract, stipulating what they think is appropriate as the rules for everyone in the house, what they’re willing to trade-off to get technology privileges and what they think that parents should trade-off. Ask them to list the consequences of not abiding by the household rules for everyone in the house. (As a parent, you can think through or sketch the terms you think are fair, but you should not present them first.). Ask your child to pitch to you what the household rules should be. You will most likely be shocked that they’re stricter and more structured than you expected. And then start the negotiation process. You may want to argue that you should have the right to look at the phone when it’s ringing in case it’s grandma calling, but then your daughter should have the right to look at her phone to see if her best friend is looking. That kind of thing. Work through the process, but have your child lead it rather than you dictate it. And then write up those rules and hang them up in the house as a contract that can be renegotiated at different types.
Many people have unhealthy habits and dynamics in their life. Some are rooted in physical addiction. Others are habitual or psychological crutches. But across that spectrum, most people are aware of when something that they’re doing isn’t healthy. They may not be able to stop. Or they may not want to stop. Untangling that is part of the challenge. When you feel as though your child has an unhealthy relationship with technology (or anything else in their life), you need to start by asking if they see this the same way you do. When parents feel as though what their child is doing is unhealthy for them, but the child does not, the intervention has to be quite different than when the child is also concerned about the issue. There are plenty of teens out there that know their psychological desire to talk non-stop with their friends for fear of missing out is putting them in a bad place. Help them through that process and work through what strategies they can develop and learn to cope. Helping them build those coping skills long term will help them a lot more than just putting rules into place.
When there is a disconnect between parent and child’s views on a situation, the best thing a parent can do is try to understand why the disconnect exists.Is it about pleasure seeking? Is it about fear of missing out? Is it about the emotional bond of friendship? Is it about a parent’s priorities being at odds with a child’s priorities? What comes next is fundamentally about values in parenting. Some parents believe that they are the masters of the house and their demands rule the day. Others acquiesce to their children’s desires with no push back. The majority of the parents are in-between. But at the end of the day, parenting is about helping children navigate the world and support them to develop agency in a healthy manner. So I would strongly recommend that parents focus their energies on negotiating a path through that allows children to be bought-in and aware of why boundaries are being set. That requires communication and energy, not a new technology to police boundaries for you. More often than not, the latter sends the wrong message and backfires, not unlike the Black Mirror episode I mentioned earlier.
Good luck parents — parenting is a non-stop adventure filled with both joy and anxiety.
(This was originally posted on Medium)
If you ever hear that implementing algorithmic decision-making tools to enable social services or other high stakes government decision-making will increase efficiency or reduce the cost to taxpayers, know that you’re being lied to. When implemented ethically, these systems cost more. And they should.
Whether we’re talking about judicial decision making (e.g., “risk assessment scoring”) or modeling who is at risk for homelessness, algorithmic systems don’t simply cost money to implement. They cost money to maintain. They cost money to audit. They cost money to evolve with the domain that they’re designed to serve. They cost money to train their users to use the data responsibly. Above all, they make visible the brutal pain points and root causes in existing systems that require an increase of services.
Otherwise, all that these systems are doing is helping divert taxpayer money from direct services, to lining the pockets of for-profit entities under the illusion of helping people. Worse, they’re helping usher in a diversion of liability because time and time again, those in powerful positions blame the algorithms.
This doesn’t mean that these tools can’t be used responsibly. They can. And they should. The insights that large-scale data analysis can offer is inspiring. The opportunity to help people by understanding the complex interplay of contextual information is invigorating. Any social scientist with a heart desperately wants to understand how to relieve inequality and create a more fair and equitable system. So of course there’s a desire to jump in and try to make sense of the data out there to make a difference in people’s lives. But to treat data analysis as a savior to a broken system is woefully naive.
Doing so obfuscates the financial incentives of those who are building these services, the deterministic rhetoric that they use to justify their implementation, the opacity that results from having non-technical actors try to understand technical jiu-jitsu, and the stark reality of how technology is used as a political bludgeoning tool. Even more frustratingly, what data analysis does well is open up opportunities for experimentation and deeper exploration. But in a zero-sum context, that means that the resources to do something about the information that is learned is siphoned off to the technology. And, worse, because the technology is supposed to save money, there is no budget for using that data to actually help people. Instead,technology becomes a mirage. Not because the technology is inherently bad, but because of how it is deployed and used.
Next week, a new book that shows the true cost of these systems is being published. Virginia Eubanks’ book“Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor” is a deeply researched accounting of how algorithmic tools are integrated into services for welfare, homelessness, and child protection. Eubanks goes deep with the people and families who are targets of these systems, telling their stories and experiences in rich detail. Further, drawing on interviews with social services clients and service providers alongside the information provided by technology vendors and government officials, Eubanks offers a clear portrait of just how algorithmic systems actually play out on the ground, despite all of the hope that goes into their implementation.
Eubanks eschews the term “ethnography” because she argues that this book is immersive journalism, not ethnography. Yet, from my perspective as a scholar and a reader, this is the best ethnography I’ve read in years. “Automating Inequality” does exactly what a good ethnography should do — it offers a compelling account of the cultural logics surrounding a particular dynamic, and invites the reader to truly grok what’s at stake through the eyes of a diverse array of relevant people. Eubanks brings you into the world of technologically mediated social services and helps you see what this really looks like on the ground. She showcases the frustration and anxiety that these implementations produce; the ways in which both social services recipientsand taxpayers are screwed by the false promises of these technologies. She makes visible the politics and the stakes, the costs and the hope. Above all, she brings the reader into the stark and troubling reality of what it really means to be poor in America today.
“Automating Inequality” is on par with Barbara Ehrenreich’s “Nickel and Dimed” or Matthew Desmond’s “Evicted.” It’s rigorously researched, phenomenally accessible, and utterly humbling. While there are a lot of important books that touch on the costs and consequences of technology through case studies and well-reasoned logic, this book is the first one that I’ve read that really pulls you into the world of algorithmic decision-making and inequality, like a good ethnography should.
I don’t know how Eubanks chose her title, but one of the subtle things about her choice is that she’s (unintentionally?) offering a fantastic backronym for AI. Rather than thinking of AI as “artificial intelligence,” Eubanks effectively builds the case for how we should think that AI often means “automating inequality” in practice.
This book should be mandatory for anyone who works in social services, government, or the technology sector because it forces you to really think about what algorithmic decision-making tools are doing to our public sector, and the costs that this has on the people that are supposedly being served. It’s also essential reading for taxpayers and voters who need to understand why technology is not the panacea that it’s often purported to be. Or rather, how capitalizing on the benefits of technology will require serious investment and a deep commitment to improving the quality of social services, rather than a tax cut.
Please please please read this book. It’s too important not to.
Data & Society will also be hosting Virginia Eubanks to talk about her book on January 17th at 4PM ET. She will be in conversation with Julia Angwin and Alondra Nelson. The event is sold out, but it will be livestreamed online. Please feel free to join us there!
The following is a transcript of my lightning talk at The People’s Disruption: Platform Co-Ops for Global Challenges — held at The New School.
When you listen to people in tech talk about the future of labor, they will tell you that AI is taking over all of the jobs. What they gloss over is the gendered dynamics of the labor force. Many of the shortages in the workforce stem from labor that is culturally gendered “feminine” and seen as low-status. There’s no conception of how workforce dynamics in tech are also gendered.
Furthermore, anxieties about automation don’t tend to focus on work that is seen as the work of immigrants, even at a time when immigration is a hotly contested conversation. As a result, when we talk about automation as the major issue in the future of work, we lose track of the broader anxiety about identities that’s shaping both technology and work.
Our current crisis around opioids offers one harrowing answer. Religious extremism offers another. Yet, we also need to consider how many people turn to activism, both healthy and destructive, as a way of finding meaning.
People often find themselves by engaging with others through collective action, but collective action isn’t always productive. Consider this in light of the broader conversation about media manipulation: for those who have grown up gaming, running a raid on America’s political establishment is thrilling. It’s exhilarating to game the media to say ridiculous things. Hacking the attention economy produces a rush. It doesn’t matter whether or not you memed the president into being if you believe you did. It doesn’t even matter if your comrades were foreign agents with a much darker agenda.
For a lot of folks in tech, being a part of tech has been a way of grounding themselves. Many who built the social media infrastructure that we know today grew up with the utopian idealism of people like John Perry Barlow. HisDeclaration of Independence of Cyberspace is now of drinking age, but today’s reality is a lot more sober. Cybernaut geeks imagined building a new world rooted in a different value structure. They wanted to resist the financialized logic of Wall Street, but ended up contributing to the latest evolution of financialized capitalism. They wanted to create a public that was more broadly accessible, but ended up enabling a new wave of corrosive populism to take hold.
You’re at this event today because you also want a new world, a sociotechnical reality that is more cooperative and equitable in nature. You see Silicon Valley as emblematic of corrosive neoliberalism and libertarianism run amok. I get it. But I can’t help but think of how social media was birthed out of idealism that got reworked by economic and political interests, by the stark realities of what people did with technology vs. what its designers hoped they would do.So many of the people that I knew in the early days of tech wanted what you want.
The early adopters of social technologies — and many of those sites’ creators — were self-identified and marginalized geeks, freaks, and queers. Early social tech was built by those who felt like outsiders in a society that valued suave masculinities. Geeks like me who flocked to the Bay felt disenfranchised and vulnerable and turned to technology to build solidarity and feel less alone. In doing so, we helped construct a form of geek masculinity that gave many geeky men in particular a sense of pride that made them feel empowered through their work and play.
But as many of you know, power corrupts. And the same geek masculinities that were once rejuvenating have spiraled out of control. Today, we’re watching as diversity becomes a wedge issue that can be used to radicalize disaffected young men in tech. The gendered nature of tech is getting ugly.
A decade ago, academics that I adore were celebrating participatory culture as emancipatory, noting that technology allowed people to engage with culture in unprecedented ways. Radical leftists were celebrating the possibilities of decentralized technologies as a form of resisting corporate power. Smart mobs were being touted as the mechanism by which authoritarian regimes could come crashing down.
Now, even the most hardened tech geek is quietly asking:
What hath we wrought?
We’ve seen massively decentralized networks coordinating and mobilizing on both for-profit and not-for-profit platforms, challenging the status quo. But the movements that they’re so strategically building are shaped by tribalistic and hate-oriented values. There are many people coordinating online who are willing to share tactic without sharing end goal, yet their tactical moves collectively achieve a form of societal gaslighting that causes unbearable pain.Tech wasn’t designed to enable this, but it did so none-the-less.
This room is filled with people who hold dear many progressive values, who see the tech sector as the new establishment, and who are pushing for a more equitable future. I share your values and desires. You rightfully want a more fair and just society. And you rage against the machine. But I also want you to know that I saw similar desires among the early developers of social media as they worked to eject the dot-com MBA culture from Silicon Valley, as they worked to resist the 1980s Wall Street culture, as they tried to operate differently than their parents.. I saw idealism corrupted, good intentions go awry, and malignant forces capitalize on weaknesses within the system.
So as you relish each other’s presence today and tomorrow, I have a favor to ask. Don’t simply focus on what would be ideal or critique the status quo.Genuinely examine how what you’re seeking could also be corrupted and abused. I believe, more than anything, that deep empathy and self-reflection is critical for us to build a healthier future.
Too often, it’s easier to rally people to tear down what we hate than it is to build a sustainable future. And yet, at this moment in time in particular, we desperately need builders. We need you.
In 1998, two graduate students at Stanford decided to try to “fix” the problems with major search engines. Sergey Brin and Larry Page wrote a paper describing how their PageRank algorithm could eliminate the plethora of “junk results.” Their idea, which we all now know as the foundation of Google, was critical. But it didn’t stop people from trying to mess with their system. In fact, the rise of Google only increased the sophistication of those invested in search engine optimization.
Fast forward to 2003, when the sitting Pennsylvania senator Rick Santorum publicly compared homosexuality to bestiality and pedophilia. Needless to say, the LGBT community was outraged. Journalist Dan Savage called on his readers to find a way to “memorialize the scandal.” One of his fans created a website to associate Santorum’s name with anal sex. To the senator’s horror, countless members of the public jumped in to link to that website in an effort to influence search engines. This form of crowdsourced SEO is commonly referred to as “Google bombing,” and it’s a form of media manipulation intended to mess with data and the information landscape.
Media manipulation is not new. As many adversarial actors know, the boundaries between propaganda and social media marketing are often fuzzy.Furthermore, any company that uses public signals to inform aspects of its product — from Likes to Comments to Reviews — knows full well that any system you create will be gamed for fun, profit, politics, ideology, and power.Even Congress is now grappling with that reality. But I’m not here to tell you what has always been happening or even what is currently happening — I’m here to help you understand what’s about to happen.
At this moment, AI is at the center of every business conversation. Companies, governments, and researchers are obsessed with data. Not surprisingly, so are adversarial actors. We are currently seeing an evolution in how data is being manipulated. If we believe that data can and should be used to inform people and fuel technology, we need to start building the infrastructure necessary to limit the corruption and abuse of that data — and grapple with how biased and problematic data might work its way into technology and, through that, into the foundations of our society.
Like search engines, social media introduced a whole new target for manipulation. This attracted all sorts of people, from social media marketers to state actors. Messing with Twitter’s trending topics or Facebook’s news feed became a hobby for many. For $5, anyone could easily buy followers, likes, and comments on almost every major site. The economic and political incentives are obvious, but alongside these powerful actors, there are also a whole host of people with less-than-obvious intentions coordinating attacks on these systems.
For example, when a distributed network of people decided to help propel Rick Astley to the top of the charts 20 years after his song “Never Gonna Give You Up” first came out, they weren’t trying to help him make a profit (although they did). Like other memes created through networks on sites like 4chan, rickrolling was for kicks. Butthrough this practice, lots of people learned how to make content “go viral” or otherwise mess with systems. In other words, they learned to hack the attention economy. And, in doing so, they’ve developed strategic practices of manipulation that can and do have serious consequences.
A story like “#Pizzagate” doesn’t happen accidentally — it was produced by a wide network of folks looking to toy with the information ecosystem. They created a cross-platform network of fake accounts known as“sock puppets” which they use to subtly influence journalists and other powerful actors to pay attention to strategically produced questions, blog posts, and YouTube videos. The goal with a story like that isn’t to convince journalists that it’s true, but to get them to foolishly use their amplification channels to negate it. This produces a “Boomerang effect,” whereby those who don’t trust the media believe that there must be merit to the conspiracy, prompting some to “self-investigate.”
Then there’s the universe of content designed to “open the Overton window” — or increase the range of topics that are acceptable to discuss in public. Journalists are tricked into spreading problematic frames. Moreover,recommendation engines can be used to encourage those who are open to problematic frames to go deeper. Researcher Joan Donovan studies white supremacy; after work, she can’t open Amazon, Netflix, or YouTube without being recommended to consume neo-Nazi music, videos, and branded objects. Radical trolls also know how to leverage this infrastructure to cause trouble. Without tripping any of Twitter’s protective mechanisms, the well-known troll weev managed to use the company’s ad infrastructure to amplify white supremacist ideas to those focused on social justice, causing outrage and anger.
Training a machine learning system requires data. Lots of it. While there are some standard corpuses, computer science researchers, startups, and big companies are increasingly hungry for new — and different — data.
The first problem is that all data is biased, most notably and recognizably by reflecting the biases of humans and of society in general. Take, for example, the popular ImageNet dataset. Because humans categorize by shape faster than they categorize by color, you end up with some weird artifacts in that data.
Things get even messier when you’re dealing with social prejudices. WhenLatanya Sweeney searched for her name on Google, she was surprised to be given ads inviting her to find out if she had a criminal record. As a curious computer scientist, she decided to run a range of common black and white names through the system to see which ads popped up. Unsurprisingly, onlyblack names produced ads for criminal justice products. This isn’t because Google knowingly treated the names differently, but because searchers were more likely to click on criminal justice ads when searching for black names.Google learned American racism and amplified it back at all of its users.
But there’s also a new challenge emerging. The same decentralized networks of people — and state actors — who have been messing with social media and search engines are increasingly eyeing the data that various companies use to train and improve their systems.
Consider, for example, the role of reddit and Twitter data as training data. Computer scientists have long pulled from the very generous APIs of these companies to train all sorts of models, trying to understand natural language, develop metadata around links, and track social patterns. They’ve trained models to detect depression, rank news, and engage in conversation. Ignoring the fact that this data is not representative in the first place, most engineers who use these APIs believe that it’s possible to clean the data and remove all problematic content. I can promise you it’s not.
I’m watching countless actors experimenting with ways to mess with public data with an eye on major companies’ systems. They are trying to fly below the radar. If you don’t have a structure in place for strategically grappling with how those with an agenda might try to route around your best laid plans, you’re vulnerable. This isn’t about accidental or natural content. It’s not even about culturally biased data. This is about strategically gamified content injected into systems by people who are trying to guess what you’ll do.
If you want to grasp what that means, consider the experiment Nicolas Papernot and his colleagues published last year. In order to understand the vulnerabilities of computer vision algorithms, they decided to alter images of stop signs so that they still resembled a stop sign to a human viewer even as the underlying neural network interpreted them as a yield sign. Think about what this means for autonomous vehicles. Will this technology be widely adopted if the classifier can be manipulated so easily?
Right now, most successful data-injection attacks on machine learning modelsare happening in the world of research, but more and more, we are seeing people try to mess with mainstream systems. Just because they haven’t been particularly successful yet doesn’t mean that they aren’t learning and evolving their attempts.
Many companies spent decades not taking security vulnerabilities seriously, until breach after breach hit the news. Do we need to go through the same pain before we start building the tools to address this new vulnerability?
In the tech industry, we have lost the culture of Test. Part of the blame rests on the shoulders of social media. Fifteen years ago, we got the bright idea to shift to a culture of the “perpetual beta.” We invited the public to be our quality assurance engineers. But internal QA wasn’t simply about finding bugs. It was about integrating adversarial thinking into the design and development process. And asking the public to find bugs in our systems doesn’t work well when some of those same people are trying to mess with our systems.Furthermore, there is currently no incentive — or path — for anyone to privately tell us where things go wrong. Only when journalists shame us by finding ways to trick our systems into advertising to neo-Nazis do we pay attention. Yet, far more maliciously intended actors are starting to play the long game in messing with our data. Why aren’t we trying to get ahead of this?
Consider, for example, the research into generative adversarial networks (or GANs). For those unfamiliar with this line of work, the idea is that you have two unsupervised ML algorithms — one is trying to generate content for the other to evaluate. The first is trying to trick the second into accepting “wrong” information. This work is all about trying to find the boundaries of your model and the latent space of your data. We need to see a lot more R&D work like this — this is the research end of a culture of Test, with true adversarial thinking baked directly into the process of building models.
But these research efforts are not enough. We need to actively and intentionally build a culture of adversarial testing, auditing, and learning into our development practice. We need to build analytic approaches to assess the biases of any dataset we use. And we need to build tools to monitor how our systems evolve with as much effort as we build our models in the first place.My colleague Matt Goerzen argues that we also need to strategically invite white hat trolls to mess with our systems and help us understand our vulnerabilities.
We no longer have the luxury of only thinking about the world we want to build. We must also strategically think about how others want to manipulate our systems to do harm and cause chaos.
In March 2013, in a flurry of days, I decided to start a research institute. I’d always dreamed of doing so, but it was really my amazing mentor and boss – Jennifer Chayes – who put the fire under my toosh. I’d been driving her crazy about the need to have more people deeply interrogating how data-driven technologies were intersecting with society. Microsoft Research didn’t have the structure to allow me to move fast (and break things). University infrastructure was even slower. There were a few amazing research centers and think tanks, but I wanted to see the efforts scale faster. And I wanted to build the structures to connect research and practices, convene conversations across sectors, and bring together a band of what I loved to call “misfit toys.” So, with the support of Jennifer and Microsoft, I put pen to paper. And to my surprise, I got the green light to help start a wholly independent research institute.
I knew nothing about building an organization. I had never managed anyone, didn’t know squat about how to put together a budget, and couldn’t even create a check list of to-dos. So I called up people smarter than I to help learn how other organizations worked and figure out what I should learn to turn a crazy idea into reality. At first, I thought that I should just go and find someone to run the organization, but I was consistently told that I needed to do it myself, to prove that it could work. So I did. It was a crazy adventure. Not only did I learn a lot about fundraising, management, and budgeting, but I also learned all sorts of things about topics I didn’t even know I would learn to understand – architecture, human resources, audits, non-profit law. I screwed up plenty of things along the way, but most people were patient with me and helped me learn from my mistakes. I am forever grateful to all of the funders, organizations, practitioners, and researchers who took a chance on me.
Still, over the next four years, I never lost that nagging feeling that someone smarter and more capable than me should be running Data & Society. I felt like I was doing the organization a disservice by not focusing on research strategy and public engagement. So when I turned to the board and said, it’s time for an executive director to take over, everyone agreed. We sat down and mapped out what we needed – a strategic and capable leader who’s passionate about building a healthy and sustainable research organization to be impactful in the world. Luckily, we had hired exactly that person to drive program and strategy a year before when I was concerned that I was flailing at managing the fieldbuilding and outreach part of the organization.
I am overwhelmingly OMG ecstatically bouncing for joy to announce that Janet Haven has agreed to become Data & Society’s first executive director. You can read more about Janet through the formal organizational announcement here. But since this is my blog and I’m telling my story, what I want to say is more personal. I was truly breaking when we hired Janet. I had taken off more than I could chew. I was hitting rock bottom and trying desperately to put on a strong face to support everyone else. As I see it, Janet came in, took one look at the duct tape upon which I’d built the organization and got to work with steel, concrete, and wood in her hands. She helped me see what could happen if we fixed this and that. And then she started helping me see new pathways for moving forward. Over the last 18 months, I’ve grown increasingly confident that what we’re doing makes sense and that we can build an organization that can last. I’ve also been in awe watching her enable others to shine.
I’m not leaving Data & Society. To the contrary, I’m actually taking on the role that my title – founder and president – signals. And I’m ecstatic. Over the last 4.5 years, I’ve learned what I’m good at and what I’m not, what excites me and what makes me want to stay in bed. I built Data & Society because I believe that it needs to exist in this world. But I also realize that I’m the classic founder – the crazy visionary that can kickstart insanity but who isn’t necessarily the right person to take an organization to the next stage. Lucky for me, Janet is. And together, I can’t wait to take Data & Society to the next level!
I left the Media Lab 15 years ago this week. At the time, I never would’ve predicted that I learned one of the most useful skills in my career there: demo-or-die.
(Me debugging an exhibit in 2002)
The culture of “demo-or-die” has been heavily critiqued over the years. In doing so, most folks focus on the words themselves. Sure, the “or-die” piece is definitely an exaggeration, but the important message there is the notion of pressure. But that’s not what most people focus on. They focus on the notion of a “demo.”
To the best that anyone can recall, the root of the term stems back from early days at the Media Lab, most likely because of Nicholas Negroponte’s dismissal of “publish-or-perish” in academia. So the idea was to focus not on writing words but producing artifacts. In mocking what it was that the Media Lab produced, many critics focused on the way in which the Lab had a tendency to create vaporware, performed to visitors through the demo. In 1987, Stewart Brand called this “handwaving.” The historian Molly Steenson has a more nuanced view so I can’t wait to read her upcoming book. But the mockery of the notion of a demo hasn’t died. Given this, it’s not surprising that the current Director (Joi Ito) has pushed people to stop talking about demoing and start thinking about deploying. Hence, “deploy-or-die.”
I would argue that what makes “demo-or-die” so powerful has absolutely nothing to do with the production of a demo. It has to do with the act of doing a demo. And that distinction is important because that’s where the skill development that I relish lies.
When I was at the Lab, we regularly received an onslaught of visitors. I was a part of the “Sociable Media Group,” run by Judith Donath. From our first day in the group, we were trained to be able to tell the story of the Media Lab, the mission of our group, and the goal of everyone’s research projects. Furthermore, we had to actually demo their quasi functioning code and pray that it wouldn’t fall apart in front of an important visitor. We were each assigned a day where we were “on call” to do demos to any surprise visitor. You could expect to have at least one visitor every day, not to mention hundreds of visitors on days that were officially sanctioned as “Sponsor Days.”
The motivations and interests of visitors ranged wildly. You’d have tour groups of VIP prospective students, dignitaries from foreign governments, Hollywood types, school teachers, engineers, and a whole host of different corporate actors. If you were lucky, you knew who was visiting ahead of time. But that was rare. Often, someone would walk in the door with someone else from the Lab and introduce you to someone for whom you’d have to drum up a demo in very short order with limited information. You’d have to quickly discern what this visitor was interested in, figure out which of the team’s research projects would be most likely to appeal, determine how to tell the story of that research in a way that connected to the visitor, and be prepared to field any questions that might emerge. And oy vay could the questions run the gamut.
I *hated* the culture of demo-or-die. I felt like a zoo animal on display for others’ benefit. I hated the emotional work that was needed to manage stupid questions, not to mention the requirement to smile and play nice even when being treated like shit by a visitor. I hated the disruptions and the stressful feeling when a demo collapsed. Drawing on my experience working in fast food, I developed a set of tricks for staying calm. Count how many times a visitor said a certain word. Nod politely while thinking about unicorns. Experiment with the wording of a particular demo to see if I could provoke a reaction. Etc.
When I left the Media Lab, I was ecstatic to never have to do another demo in my life. Except, that’s the funny thing about learning something important… you realize that you are forever changed by the experience.
I no longer produce demos, but as I developed in my career, I realized that “demo-or-die” wasn’t really about the demo itself. At the end of the day, the goal wasn’t to pitch the demo — it was to help the visitor change their perspective of the world through the lens of the demo. In trying to shift their thinking, we had to invite them to see the world differently. The demo was a prop. Everything about what I do as a researcher is rooted in the goal of using empirical work to help challenge people’s assumptions and generate new frames that people can work with. I have to understand where they’re coming from, appreciate their perspective, and then strategically engage them to shift their point of view. Like my days at the Media Lab, I don’t always succeed and it is indeed frustrating, especially because I don’t have a prop that I can rely on when everything goes wrong. But spending two years developing that muscle has been so essential for my work as an ethnographer, researcher, and public speaker.
I get why Joi reframed it as “deploy-or-die.” When it comes to actually building systems, impact is everything. But I really hope that the fundamental practice of “demo-or-die” isn’t gone. Those of us who build systems or generate knowledge day in and day out often have too little experience explaining ourselves to the wide array of folks who showed up to visit the Media Lab. It’s easy to explain what you do to people who share your ideas, values, and goals. It’s a lot harder to explain your contributions to those who live in other worlds. Impact isn’t just about deploying a system; it’s about understanding how that system or idea will be used. And that requires being able to explain your thinking to anyone at any moment. And that’s the skill that I learned from the “demo-or-die” culture.
To be honest, what surprises me most about the current conversation about the inhospitable nature of tech for women is that people are surprised. To say that discrimination, harassment, and sexual innuendos are an open secret is an understatement. I don’t know a woman in tech who doesn’t have war stories. Yet, for whatever reason, we are now in a moment where people are paying attention. And for that, I am grateful.
Like many women in tech, I’ve developed strategies for coping. I’ve had to in order to stay in the field. I’ve tried to be “one of the guys,” pretending to blend into the background as sexist speech was jockeyed about in the hopes that I could just fit in. I’ve tried to be the kid sister, the freaky weirdo, the asexual geek, etc. I’ve even tried to use my sexuality to my advantage in the hopes that maybe I could recover some of the lost opportunity that I faced by being a woman. It took me years to realize that none of these strategies would make me feel like I belonged. Many even made me feel worse.
For years, I included Ani DiFranco lyrics in every snippet of code I wrote, as well as my signature. I’ve maintained a lyrics site since I was 18 because her words give me strength for coping with the onslaught of commentary and gross behavior. “Self-preservation is a full-time occupation.” I can’t tell you how often I’ve sat in a car during a conference or after a meeting singing along off-key at full volume with tears streaming down my face, just trying to keep my head together.
What’s at stake is not about a few bad actors. There’s also a range of behaviors getting lumped together, resulting in folks asking if inescapable sexual overtures are really that bad compared to assault. That’s an unproductive conversation because the fundamental problem is the normalization of atrocious behavior that makes room for a wide range of inappropriate actions. Fundamentally, the problem with systemic sexism is that it’s not the individual people who are the problem. It’s the culture. And navigating the culture is exhausting and disheartening. It’s the collection of particles of sand that quickly becomes a mountain that threatens to bury you.
It’s having to constantly stomach sexist comments with a smile, having to work twice as hard to be heard in a meeting, having to respond to people who ask if you’re on the panel because they needed a woman. It’s about going to conferences where deals are made in the sauna but being told that you have to go to the sauna with “the wives” (a pejoratively constructed use of the word). It’s about people assuming you’re sleeping with whoever said something nice about you. It’s being told “you’re kinda smart for a chick” when you volunteer to help a founder. It’s knowing that you’ll receive sexualized threats for commenting on certain topics as a blogger. It’s giving a talk at a conference and being objectified by the audience. It’s building whisper campaigns among women to indicate which guys to avoid. It’s using Dodgeball/Foursquare to know which parties not to attend based on who has checked in. It’s losing friends because you won’t work with a founder who you watched molest a woman at a party (and then watching Justin Timberlake portray that founder’s behavior as entertainment).
Lots of people in tech have said completely inappropriate things to women. I also recognize that many of those guys are trying to fit into the sexist norms of tech too, trying to replicate the culture that they see around them because they too are struggling for status. But that’s the problem. Once guys receive power and status within the sector, they don’t drop their inappropriate language. They don’t change their behavior or call out others on how insidious it is. They let the same dynamics fester as though it’s just part of the hazing ritual.
For women who succeed in tech, the barrage of sexism remains. It just changes shape as we get older.
On Friday night, after reading the NYTimes article on tech industry harassment, I was deeply sad. Not because the stories were shocking — frankly, those incidents are minor compared to some of what I’ve seen. I was upset because stories like this typically polarize and prompt efforts to focus on individuals rather than the culture. There’s an assumption that these are one-off incidents. They’re not.
I appreciate that Dave and Chris owned up to their role in contributing to a hostile culture. I know that it’s painful to hear that something you said or did hurt someone else when you didn’t intend that to be the case. I hope that they’re going through a tremendous amount of soul-searching and self-reflection. I appreciate Chris’ willingness to take to Medium to effectively say “I screwed up.” Ideally, they will both come out of this willing to make amends and right their wrongs.
Unfortunately, most people don’t actually respond productively when they’re called out. Shaming can often backfire.
One of the reasons that most people don’t speak up is that it’s far more common for guys who are called out on their misdeeds to respond the way that Marc Canter appeared to do, by justifying his behavior and demonizing the woman who accused him of sexualizing her. Given my own experiences with his sexist commentary, I decided to tweet out in solidarity by publicly sharing how he repeatedly asked me for a threesome with his wife early on in my career. At the time, I was young and I was genuinely scared of him; I spent a lot of time and emotional energy avoiding him, and struggled with how to navigate him at various conferences. I wasn’t the only one who faced his lewd comments, often framed as being sex-positive even when they were an abuse of power. My guess is that Marc has no idea how many women he’s made feel uncomfortable, ashamed, and scared. The question is whether or not he will admit that to himself, let alone to others.
I’m not interested in calling people out for sadistic pleasure. I want to see the change that most women in tech long for. At its core, the tech industry is idealistic and dreamy, imagining innovations that could change the world. Yet, when it comes to self-reflexivity, tech is just as regressive as many other male-dominated sectors. Still, I fully admit that I hold it to a higher standard in no small part because of the widespread commitment in tech to change the world for the better, however flawed that fantastical idealism is.
Given this, what I want from men in tech boils down to four Rs: Recognition. Repentance. Respect. Reparation.
Recognition. I want to see everyone — men and women — recognize how contributing to a culture of sexism takes us down an unhealthy path, not only making tech inhospitable for women but also undermining the quality of innovation and enabling the creation of tech that does societal harm. I want men in particular to reflect on how the small things that they do and say that they self-narrate as part of the game can do real and lasting harm, regardless of what they intended or what status level they have within the sector. I want those who witness the misdeeds of others to understand that they’re contributing to the problem.
Repentance. I want guys in tech — and especially those founders and funders who hold the keys to others’ opportunity — to take a moment and think about those that they’ve hurt in their path to success and actively, intentionally, and voluntarily apologize and ask for forgiveness. I want them to reach out to someone they said something inappropriate to, someone whose life they made difficult and say “I’m sorry.”
Respect. I want to see a culture of respect actively nurtured and encouraged alongside a culture of competition. Respect requires acknowledging others’ struggles, appreciating each others’ strengths and weaknesses, and helping each other through hard times. Many of the old-timers in tech are nervous that tech culture is being subsumed by financialization. Part of resisting this transformation is putting respect front and center. Long-term success requires thinking holistically about society, not just focusing on current capitalization.
Reparation. Every guy out there who wants to see tech thrive owes it to the field to actively seek out and mentor, support, fund, open doors for, and otherwise empower women and people of color. No excuses, no self-justifications, no sexualized bullshit. Just behavior change. Plain and simple. If our sector is about placing bets, let’s bet on a better world. And let’s solve for social equity.
I have a lot of respect for the women who are telling their stories, but we owe it to them to listen to the culture that they’re describing. Sadly, there are so many more stories that are not yet told. I realize that these stories are more powerful when people are named. My only hope is that those who are risking the backlash to name names will not suffer for doing so. Ideally, those who are named will not try to self-justify but acknowledge and accept that they’ve caused pain. I strongly believe that changing the norms is the only path forward. So while I want to see people held accountable, I especially want to see the industry work towards encouraging and supporting behavior change. At the end of the day, we will not solve the systemic culture of sexism by trying to weed out bad people, but we can work towards rendering bad behavior permanently unacceptable.
I was 19 years old when a some configuration of anonymous people came after me. They got access to my email and shared some of the most sensitive messages on an anonymous forum. This was after some of my girl friends received anonymous voice messages describing how they would be raped. And after the black and Latinx high school students I was mentoring were subject to targeted racist messages whenever they logged into the computer cluster we were all using. I was ostracized for raising all of this to the computer science department’s administration. A year later, when I applied for an internship at Sun Microsystems, an alum known for his connection to the anonymous server that was used actually said to me, “I thought that they managed to force you out of CS by now.”
Needless to say, this experience hurt like hell. But in trying to process it, I became obsessed not with my own feelings but with the logics that underpinned why some individual or group of white male students privileged enough to be at Brown University would do this. (In investigations, the abusers were narrowed down to a small group of white men in the department but it was never going to be clear who exactly did it and so I chose not to pursue the case even though law enforcement wanted me to.)
My first breakthrough came when I started studying bullying, when I started reading studies about why punitive approaches to meanness and cruelty backfire. It’s so easy to hate those who are hateful, so hard to be empathetic to where they’re coming from. This made me double down on an ethnographic mindset that requires that you step away from your assumptions and try to understand the perspective of people who think and act differently than you do. I’m realizing more and more how desperately this perspective is needed as I watch researchers and advocates, politicians and everyday people judge others from their vantage point without taking a moment to understand why a particular logic might unfold.
A few days ago, my networks were on fire with condescending comments referencing an article in The Guardian titled “Scraping by on six figures? Tech workers feel poor in Silicon Valley’s wealth bubble.” I watched as all sorts of reasonably educated, modestly but sustainably paid people mocked tech folks for expressing frustration about how their well-paid jobs did not allow them to have the sustainable lifestyle that they wanted. For most, Silicon Valley is at a distance, a far off land of imagination brought to you by the likes of David Fincher and HBO. Progressive values demand empathy for the poor and this often manifests as hatred for the rich. But what’s missing from this mindset is an understanding of the local perception of wealth, poverty, and status. And, more importantly, the political consequences of that local perception.
Think about it this way. I live in NYC where the median household income is somewhere around $55K. My network primarily makes above the median and yet they all complain that they don’t have enough money to achieve what they want in NYC, whether they’re making $55K, $70K, or $150K. Complaining about being not having enough money is ritualized alongside complaining about the rents. No one I know really groks that they’re making above the median income for the city (and, thus, that most people are much poorer than they are), let alone how absurd their complaints might sound to someone from a poorer country where a median income might be $1500 (e.g., India).
The reason for this is not simply that people living in NYC are spoiled, but that people’s understanding of prosperity is shaped by what they see around them. Historically, this has been understood through word-of-mouth and status markers. In modern times, those status markers are often connected to conspicuous consumption. “How could HE afford a new pair of Nikes!?!?”
The dynamics of comparison are made trickier by media. Even before yellow journalism, there has always been some version of Page Six or “Lifestyles of the Rich and Famous.” Stories of gluttonous and extravagant behaviors abound in ancient literature. Today, with Instagram and reality TV, the idea of haves and havenots is pervasive, shaping cultural ideas of privilege and suffering. Everyday people perform for the camera and read each other’s performances critically. And still, even as we watch rich people suffer depression or celebrities experience mental breakdowns, we don’t know how to walk in each other’s shoes. We collectively mock them for their privilege as a way to feel better for our own comparative struggles.
In other words, in a neoliberal society, we consistently compare ourselves to others in ways that make us feel as though we are less well off than we’d like. And we mock others who are more privileged who do the same. (And, horribly, we often blame others who are not for making bad decisions.)
I grew up with identity politics, striving to make sense of intersectional politics and confused about what it meant to face oppression as a woman and privilege as a white person. I now live in a world of tech wealth while my family does not. I live with contradictions and I work on issues that make those contradictions visible to me on a regular basis. These days, I am surrounded by civil rights advocates and activists of all stripes. Folks who remind me to take my privilege seriously. And still, I struggle to be a good ally, to respond effectively to challenges to my actions. Because of my politics and ideals, I wake up each day determined to do better.
Yet, with my ethnographer’s hat on, I’m increasingly uncomfortable with how this dynamic is playing out. Not for me personally, but for affecting change. I’m nervous that the way that privilege is being framed and politicized is doing damage to progressive goals and ideals. In listening to white men who see themselves as “betas” or identify as NEETs (“Not in Education, Employment, or Training”) describe their hatred of feminists or social justice warriors, I hear the cost of this frame. They don’t see themselves as empowered or privileged and they rally against these frames. And they respond antagonistically in ways that further the divide, as progressives feel justified in calling them out as racist and misogynist. Hatred emerges on both sides and the disconnect produces condescension as everyone fails to hear where each other comes from, each holding onto their worldview that they are the disenfranchised, they are the oppressed. Power and wealth become othered and agency becomes understood through the lens of challenging what each believes to be the status quo.
It took me years to understand that the boys who tormented me in college didn’t feel powerful, didn’t see their antagonism as oppression. I was even louder and more brash back then than I am now. I walked into any given room performing confidence in ways that completely obscured my insecurities. I took up space, used my sexuality as a tool, and demanded attention. These were the survival skills that I had learned to harness as a ticket out. And these are the very same skills that have allowed me to succeed professionally and get access to tremendous privilege. I have paid a price for some of the games that I have played, but I can’t deny that I’ve gained a lot in the process. I have also come to understand that my survival strategies were completely infuriating to many geeky white boys that I encountered in tech. Many guys saw me as getting ahead because I was a token woman. I was accused of sleeping my way to the top on plenty of occasions. I wasn’t simply seen as an alpha — I was seen as the kind of girl that screwed boys over. And because I was working on diversity and inclusion projects in computer science to attract more women and minorities as the field, I was seen as being the architect of excluding white men. For so many geeky guys I met, CS was the place where they felt powerful and I stood for taking that away. I represented an oppressor to them even though I felt like it was they who were oppressing me.
Privilege is complicated. There is no static hierarchical structure of oppression. Intersectionality provides one tool for grappling with the interplay between different identity politics, but there’s no narrative for why beta white male geeks might feel excluded from these frames. There’s no framework for why white Christians might feel oppressed by rights-oriented activists. When we think about privilege, we talk about the historical nature of oppression, but we don’t account for the ways in which people’s experiences of privilege are local. We don’t account for the confounding nature of perception, except to argue that people need to wake up.
We live in a complex interwoven society. In some ways, that’s intentional. After WWII, many politicians and activists wanted to make the world more interdependent, to enable globalization to prevent another world war. The stark reality is that we all depend on social, economic, and technical infrastructures that we can’t see and don’t appreciate. Sure, we can talk about how our food is affordable because we’re dependent on underpaid undocumented labor. We can take our medicine for granted because we fail to appreciate all of the regulatory processes that go into making sure that what we consume is safe. But we take lots of things for granted; it’s the only way to move through the day without constantly panicking about whether or not the building we’re in will collapse.
Without understanding the complex interplay of things, it’s hard not to feel resentful about certain things that we do see. But at the same time, it’s not possible to hold onto the complexity. I can appreciate why individuals are indignant when they feel as though they pay taxes for that money to be given away to foreigners through foreign aid and immigration programs. These people feel like they’re struggling, feel like they’re working hard, feel like they’re facing injustice. Still, it makes sense to me that people’s sense of prosperity is only as good as their feeling that they’re getting ahead. And when you’ve been earning $40/hour doing union work only to lose that job and feel like the only other option is a $25/hr job, the feeling is bad, no matter that this is more than most people make. There’s a reason that Silicon Valley engineers feel as though they’re struggling and it’s not because they’re comparing themselves to everyone in the world. It’s because the standard of living keeps dropping in front of them. It’s all relative.
It’s easy to say “tough shit” or “boo hoo hoo” or to point out that most people have it much worse. And, at some levels, this is true. But if we don’t account for how people feel, we’re not going to achieve a more just world — we’re going to stoke the fires of a new cultural war as society becomes increasingly polarized.
The disconnect between statistical data and perception is astounding. I can’t help but shake my head when I listen to folks talk about how life is better today than it ever has been in history. They point to increased lifespan, new types of medicine, decline in infant mortality, and decline in poverty around the world. And they shake their heads in dismay about how people don’t seem to get it, don’t seem to get that today is better than yesterday. But perception isn’t about statistics. It’s about a feeling of security, a confidence in one’s ecosystem, a belief that through personal effort and God’s will, each day will be better than the last. That’s not where the vast majority of people are at right now. To the contrary, they’re feeling massively insecure, as though their world is very precarious.
I am deeply concerned that the people whose values and ideals I share are achieving solidarity through righteous rhetoric that also produces condescending and antagonistic norms. I don’t fully understand my discomfort, but I’m scared that what I’m seeing around me is making things worse. And so I went back to some of Martin Luther King Jr.’s speeches for a bit of inspiration today and I started reflecting on his words. Let me leave this reflection with this quote:
The ultimate weakness of violence is that it is a descending spiral,
begetting the very thing it seeks to destroy.
Instead of diminishing evil, it multiplies it.
Through violence you may murder the liar,
but you cannot murder the lie, nor establish the truth.
Through violence you may murder the hater,
but you do not murder hate.
In fact, violence merely increases hate.
So it goes.
Returning violence for violence multiplies violence,
adding deeper darkness to a night already devoid of stars.
Darkness cannot drive out darkness:
only light can do that.
Hate cannot drive out hate: only love can do that.
— Dr. Martin Luther King, Jr.
Image from Flickr: Andy Doyle
There’s a joke out there that when you’re having your first child, you tell everyone personally and update your family and friends about every detail throughout the pregnancy. With Baby #2, there’s an abbreviated notice that goes out about the new addition, all focused on how Baby #1 is excited to have a new sibling. And with Baby #3, you forget to tell people.
I’m a living instantiation of that. If all goes well, I will have my third child in early March and I’ve apparently forgotten to tell anyone since folks are increasingly shocked when I indicate that I can’t help out with XYZ because of an upcoming parental leave. Oops. Sorry!
As noted when I gave a heads up with Baby #1 and Baby #2, I plan on taking parental leave in stride. I don’t know what I’m in for. Each child is different and each recovery is different. What I know for certain is that I don’t want to screw over collaborators or my other baby – Data & Society. As a result, I will be not taking on new commitments and I will be actively working to prioritize my collaborators and team over the next six months.
In the weeks following birth, my response rates may get sporadic and I will probably not respond to non-mission-critical email. I also won’t be scheduling meetings. Although I won’t go completely offline in March (mostly for my own sanity), but I am fairly certain that I will take an email sabbatical in July when my family takes some serious time off** to be with one another and travel.
A change in family configuration is fundamentally walking into the abyss. For as much as our culture around maternity leave focuses on planning, so much is unknown. After my first was born, I got a lot of work done in the first few weeks afterwards because he was sleeping all the time and then things got crazy just as I was supposedly going back to work. That was less true with #2, but with #2 I was going seriously stir crazy being home in the cold winter and so all I wanted was to go to lectures with him to get out of bed and soak up random ideas. Who knows what’s coming down the pike. I’m fortunate enough to have the flexibility to roll with it and I intend to do precisely that.
What’s tricky about being a parent in this ecosystem is that you’re kinda damned if you do, damned if you don’t. Women are pushed to go back to work immediately to prove that they’re serious about their work – or to take serious time off to prove that they’re serious about their kids. Male executives are increasingly publicly talking about taking time off, while they work from home. The stark reality is that I love what I do. And I love my children. Life is always about balancing different commitments and passions within the constraints of reality (time, money, etc.). And there’s nothing like a new child to make that balancing act visible.
So if you need something from me, let me know ASAP! And please understand and respect that I will be navigating a lot of unknown and doing my best to achieve a state of balance in the upcoming months of uncertainty.
** July 2017 vacation. After a baby is born, the entire focus of a family is on adjustment. For the birthing parent, it’s also on recovery because babies kinda wreck your body no matter how they come out. Finding rhythms for sleep and food become key for survival. Folks talk about this time as precious because it can enable bonding. That hasn’t been my experience and so I’ve relished the opportunity with each new addition to schedule some full-family bonding time a few months after birth where we can do what our family likes best – travel and explore as a family. If all goes well in March, we hope to take a long vacation in mid-July where I intend to be completely offline and focused on family. More on that once we meet the new addition.
I am surrounded by people who are driven by good intentions. Educators who want to inform students, who passionately believe that people can be empowered through knowledge. Activists who have committed their lives to addressing inequities, who believe that they have a moral responsibility to shine a spotlight on injustice. Journalists who believe their mission is to inform the public, who believe that objectivity is the cornerstone of their profession. I am in awe of their passion and commitment, their dedication and persistence.
Yet, I’m existentially struggling as I watch them fight for what is right. I havelearned that people who view themselves through the lens of good intentions cannot imagine that they could be a pawn in someone else’s game. They cannot imagine that the values and frames that they’ve dedicated their lives towards — free speech, media literacy, truth — could be manipulated or repurposed by others in ways that undermine their good intentions.
I find it frustrating to bear witness to good intentions getting manipulated,but it’s even harder to watch how those who are wedded to good intentions are often unwilling to acknowledge this, let alone start imagining how to develop the appropriate antibodies. Too many folks that I love dearly just want to double down on the approaches they’ve taken and the commitments they’ve made. On one hand, I get it — folks’ life-work and identities are caught up in these issues.
But this is where I think we’re going to get ourselves into loads of trouble.
The world is full of people with all sorts of intentions. Their practices and values, ideologies and belief systems collide in all sorts of complex way. Sometimes, the fight is about combating horrible intentions, but often it is not. In college, my roommate used to pound a mantra into my head whenever I would get spun up about something: “Do not attribute to maliciousness what you can attribute to stupidity.” I return to this statement a lot when I think about how to build resilience and challenge injustices, especially when things look so corrupt and horribly intended — or when people who should be allies see each other as combatants. But as I think about how we should resist manipulation and fight prejudice, I also think that it’s imperative to move away from simply relying on “good intentions.”
I don’t want to undermine those with good intentions, but I also don’t want good intentions to be a tool that can be used against people. So I want to think about how good intentions get embedded in various practices and the implications of how we view the different actors involved.
The Good Intentions of Media Literacy
When I penned my essay “Did Media Literacy Backfire?”, I wanted to ask those who were committed to media literacy to think about how their good intentions — situated in a broader cultural context — might not play out as they would like. Folks who critiqued my essay on media literacy pushed back in all sorts of ways, both online and off. Many made me think, but some also reminded me that my way of writing was off-putting. I was accused of using the question “Did media literacy backfire?” to stoke clicks.Some snarkily challenged my suggestion that media literacy was even meaningfully in existence, asked me to be specific about which instantiations I meant (because I used the phrase “standard implementations”), and otherwise pushed for the need to double down on “good” or “high quality” media literacy. The reality is that I’m a huge proponent of their good intentions — and have long shared them, but I wrote this piece because I’m worried that good intentions can backfire.
While I was researching youth culture, I never set out to understand what curricula teachers used in the classroom. I wasn’t there to assess the quality of the teachers or the efficacy of their formal educational approaches. I simply wanted to understand what students heard and how they incorporated the lessons they received into their lives. Although the teens that I met had a lot of choice words to offer about their teachers, I’ve always assumed that most teachers entered the profession with the best of intentions, even if their students couldn’t see that. But I spent my days listening to students’ frustrations and misperceptions of the messages teachers offered.
I’ve never met an educator who thinks that the process of educating is easy or formulaic. (Heck, this is why most educators roll their eyes when they hear talk of computerized systems that can educate better than teachers.) So why do we assume that well-intended classroom lessons — or even well-designed curricula — might not play out as we imagine? This isn’t simply about the efficacy of the lesson or the skill of the teacher, but the cultural context in which these conversations occur.
In many communities in which I’ve done research, the authority of teachers is often questioned. Nowhere is this more painfully visible than when well-intended highly educated (often white) teachers come to teach in poorer communities of color. Yet, how often are pedagogical interventions designed by researchers really taking into account the doubt that students and their parents have of these teachers? And how do we as educators and scholars grapple with how we might have made mistakes?
I’m not asking “Did Media Literacy Backfire?” to be a pain in the toosh, but to genuinely highlight how the ripple effects of good intentions may not play out as imagined on the ground for all sorts of reasons.
From the outside, companies like Facebook and Google seem pretty evil to many people. They’re situated in a capitalist logic that many advocates and progressives despise. They’re opaque and they don’t engage the public in their decision-making processes, even when those decisions have huge implications for what people read and think. They’re extremely powerful and they’ve made a lot of people rich in an environment where financial inequality and instability is front and center. Primarily located in one small part of the country, they also seem like a monolithic beast.
As a result, it’s not surprising to me that many people assume that engineers and product designers have evil (or at least financially motivated) intentions. There’s an irony here because my experience is the opposite.Most product teams have painfully good intentions, shaped by utopic visions of how the ideal person would interact with the ideal system. Nothing is more painful than sitting through a product design session with design personae that have been plucked from a collection of clichés.
I’ve seen a lot of terribly naive product plans, with user experience mockups that lack any sense of how or why people might interact with a system in unexpected ways. I spent years tracking how people did unintended things with social media, such as the rise of “Fakesters,” or of teenagers who gamed Facebook’s system by inserting brand names into their posts, realizing that this would make their posts rise higher in the social network’s news feed. It has always boggled my mind how difficult it is for engineers and product designers to imagine how their systems would get gamed. I actually genuinely loved product work because I couldn’t help but think about how to break a system through unexpected social practices.
Most products and features that get released start with good intentions, but they too get munged by the system, framed by marketing plans, and manipulated by users. And then there’s the dance of chaos as companies seek to clean up PR messes (which often involves non-technical actors telling insane fictions about the product), patch bugs to prevent abuse, and throw bandaids on parts of the code that didn’t play out as intended. There’s a reason that no one can tell you exactly how Google’s search engine or Facebook’s news feed works. Sure, the PR folks will tell you that it’s proprietary code. But the ugly truth is that the code has been patched to smithereens to address countless types of manipulation and gamification(e.g., SEO to bots). It’s quaint to read the original “page rank” paper that Brin and Page wrote when they envisioned how a search engine could ideally work. That’s so not how the system works today.
The good intentions of engineers and product people, especially those embedded in large companies, are often doubted as sheen for a capitalist agenda. Yet, like many other well-intended actors, I often find that makers feel misunderstood and maligned, assumed to have evil thoughts. And I often think that when non-tech people start by assuming that they’re evil, we lose a significant opportunity to address problems.
I’ve been harsh on journalists lately, mostly because I find it so infuriating that a profession that is dedicated to being a check to power could be so ill-equipped to be self-reflexive about its own practices.
Yet, I know that I’m being unfair. Their codes of conduct and idealistic visions of their profession help journalists and editors and publishers stay strong in an environment where they are accustomed to being attacked. It just kills me that the cultural of journalism makes those who have an important role to play unable to see how they can be manipulated at scale.
Sure, plenty of top-notch journalists are used to negotiating deception and avoidance. You gotta love a profession that persistently bangs its head against a wall of “no comment.” But journalism has grown up as an individual sport; a competition for leads and attention that can get fugly in the best of configurations. Time is rarely on a journalist’s side, just as nuance is rarely valued by editors. Trying to find “balance” in this ecosystem has always been a pipe dream, but objectivity is a shared hallucination that keeps well-intended journalists going.
Powerful actors have always tried to manipulate the news media, especially State actors. This is why the fourth estate is seen as so important in the American context. Yet, the game has changed, in part because of the distributed power of the masses. Social media marketers quickly figured out that manufacturing outrage and spectacle would give them a pathway to attention, attracting news media like bees to honey. Most folks rolled their eyes, watching as monied people played the same games as State actors. But what about the long tail? How do we grapple with the long tail? How should journalists respond to those who are hacking the attention economy?
I am genuinely struggling to figure out how journalists, editors, and news media should respond in an environment in which they are getting gamed.What I do know from 12-steps is that the first step is to admit that you have a problem. And we aren’t there yet. And sadly, that means that good intentions are getting gamed.
I’m in awe of how many of the folks I vehemently disagree with are willing to align themselves with others they vehemently disagree with when they have a shared interest in the next step. Some conservative and hate groups are willing to be odd bedfellows because they’re willing to share tactics, even if they don’t share end goals. Many progressives can’t even imagine coming together with folks who have a slightly different vision, let alone a different end goal, to even imagine various tactics. Why is that?
My goal in writing these essays is not because I know the solutions to some of the most complex problems that we face — I don’t — but because I think that we need to start thinking about these puzzles sideways, upside down, and from non-Euclidean spaces. In short, I keep thinking that we need more well-intended folks to start thinking like hackers.
Think just as much about how you build an ideal system as how it might be corrupted, destroyed, manipulated, or gamed. Think about unintended consequences, not simply to stop a bad idea but to build resilience into the model.
As a developer, I always loved the notion of “extensibility” because it was an ideal of building a system that could take unimagined future development into consideration. Part of why I love the notion is that it’s bloody impossible to implement. Sure, I (poorly) comment my code and build object-oriented structures that would allow for some level of technical flexibility. But, at the end of the day, I’d always end up kicking myself for not imagining a particular use case in my original design and, as a result, doing a lot more band-aiding than I’d like to admit. The masters of software engineering extensibility are inspiring because they don’t just hold onto the task at hand, but have a vision for all sorts of different future directions that may never come into fruition. That thinking is so key to building anything, whether it be software or a campaign or a policy. And yet, it’s not a muscle that we train people to develop.
If we want to address some of the major challenges in civil society, we need the types of people who think 10 steps ahead in chess, imagine innovative ways of breaking things, and think with extensibility at their core. More importantly, we all need to develop that sensibility in ourselves. This is the hacker mindset.
This post was originally posted on Points. It builds off of a series of essays on topics affecting the public sphere written by folks at Data & Society. As expected, my earlier posts ruffled some feathers, and I’ve been trying to think about how to respond in a productive manner. This is my attempt.
Yesterday, Steve Bannon clearly articulated what many people have felt and known for quite some time when he told journalists, “You’re the opposition party. Not the Democratic Party… The media’s the opposition party.” This builds on earlier remarks by Trump, who said, “I have a running war with the media.”
Journalists have covered this with their “objective” voice as though it was another news story in the crazy first week of WTF moments. Many of those who value the media have looked at this with wide eyes, struggling to assess which of the many news stories they should be more horrified by. Far too few are getting the point:
The news media have become a pawn in a big chess game of an information war.
News agencies, long trained to focus on reporting information and maintaining a conceptual model of standards, are ill-equipped to understand that they may have a role in this war, that their actions and decisions are shaping the way the war plays out.
When Kellyanne Conway argued that they were operating with “alternative facts,” the media mocked her. They tried to dismiss her comment that the media has a 14% approval rating by fact-correcting this to point out that this was only a Gallup poll concerning the media’s approval rating among Republicans. But they missed her greater point: there’s no cost to the administration to be helpful to the media because the people the Trump Administration cares about don’t trust the media anyhow.
How many years did it take for the US military to learn that waging war with tribal networks couldn’t be fought with traditional military strategies? How long will it take for the news media to wake up and recognize that they’re being played? And how long after that will it take for editors and publishers to start evolving their strategies?
As I wrote in “Hacking the Attention Economy,” manipulating the media for profit, ideology, and lulz has evolved over time. The strategies that hackers, hoaxers, and haters have taken have become more sophisticated. The campaigns have gotten more intense. And now many of the actors most set on undermining institutionalized information intermediaries are in the most powerful office in the land. They are waging war on the media and the media doesn’t know what to do other than to report on it.
We’ve built an information ecosystem where information can fly through social networks (both technical and personal). Folks keep looking to the architects of technical networks to solve the problem. I’m confident that these companies can do a lot to curb some of the groups who have capitalized on what’s happening to seek financial gain. But the battles over ideology and attention are going to be far trickier. What’s at stake isn’t “fake news.” What’s at stake is the increasing capacity of those committed to a form of isolationist and hate-driven tribalism that has been around for a very long time. They have evolved with the information landscape, becoming sophisticated in leveraging whatever tools are available to achieve power, status, and attention. And those seeking a progressive and inclusive agenda, those seeking to combat tribalism to form a more perfect union — they haven’t kept up.
The information war has begun. Normative approaches to challenging the system will not work. What will it take for news media to wake up? What will it take for progressives to start developing skills to fight back?
The United States has always been a diverse but segregated country. This has shaped American politics profoundly. Yet, throughout history, Americans have had to grapple with divergent views and opinions, political ideologies, and experiences in order to function as a country. Many of the institutions that underpin American democracy force people in the United States to encounter difference. This does not inherently produce tolerance or result in healthy resolution. Hell, the history of the United States is fraught with countless examples of people enslaving and oppressing other people on the basis of difference. This isn’t about our past; this is about our present. And today’s battles over laws and culture are nothing new.
Ironically, in a world in which we have countless tools to connect, we are also watching fragmentation, polarization, and de-diversification happen en masse. The American public is self-segregating, and this is tearing at the social fabric of the country.
Many in the tech world imagined that the Internet would connect people in unprecedented ways, allow for divisions to be bridged and wounds to heal.It was the kumbaya dream. Today, those same dreamers find it quite unsettling to watch as the tools that were designed to bring people together are used by people to magnify divisions and undermine social solidarity. These tools were built in a bubble, and that bubble has burst.
Nowhere is this more acute than with Facebook. Naive as hell, Mark Zuckerberg dreamed he could build the tools that would connect people at unprecedented scale, both domestically and internationally. I actually feel bad for him as he clings to that hope while facing increasing attacks from people around the world about the role that Facebook is playing in magnifying social divisions. Although critics love to paint him as only motivated by money, he genuinely wants to make the world a better place and sees Facebook as a tool to connect people, not empower them to self-segregate.
The problem is not simply the “filter bubble,” Eli Pariser’s notion that personalization-driven algorithmic systems help silo people into segregated content streams. Facebook’s claim that content personalization plays a small role in shaping what people see compared to their own choices is accurate.And they have every right to be annoyed. I couldn’t imagine TimeWarner being blamed for who watches Duck Dynasty vs. Modern Family. And yet, what Facebook does do is mirror and magnify a trend that’s been unfolding in the United States for the last twenty years, a trend of self-segregation that is enabled by technology in all sorts of complicated ways.
The United States can only function as a healthy democracy if we find a healthy way to diversify our social connections, if we find a way to weave together a strong social fabric that bridges ties across difference.
Yet, we are moving in the opposite direction with serious consequences. To understand this, let’s talk about two contemporary trend lines and then think about the implications going forward.
The voluntary US military is, in many ways, a social engineering project. The public understands the military as a service organization, dedicated to protecting the country’s interests. Yet, when recruits sign up, they are promised training and job opportunities. Individual motivations vary tremendously, but many are enticed by the opportunity to travel the world, participate in a cause with a purpose, and get the heck out of dodge. Everyone expects basic training to be physically hard, but few recognize that some of the most grueling aspects of signing up have to do with the diversification project that is central to the formation of the American military.
When a soldier is in combat, she must trust her fellow soldiers with her life. And she must be willing to do what it takes to protect the rest of her unit. In order to make that possible, the military must wage war on prejudice. This is not an easy task. Plenty of generals fought hard to fight racial desegregation and to limit the role of women in combat. Yet, the US military was desegregated in 1948, six years before Brown v. Board forced desegregation of schools. And the Supreme Court ruled that LGB individuals could openly serve in the military before they could legally marry.
Morale is often raised as the main reason that soldiers should not be forced to entrust their lives to people who are different than them. Yet, time and again, this justification collapses under broader interests to grow the military. As a result, commanders are forced to find ways to build up morale across difference, to actively and intentionally seek to break down barriers to teamwork, and to find a way to gel a group of people whose demographics, values, politics, and ideologies are as varied as the country’s.
In the process, they build one of the most crucial social infrastructures of the country. They build the diverse social fabric that underpins democracy.
Tons of money was poured into defense after 9/11, but the number of people serving in the US military today is far lower than it was throughout the 1980s. Why? Starting in the 1990s and accelerating after 9/11, the US privatized huge chunks of the military. This means that private contractors and their employees play critical roles in everything from providing food services to equipment maintenance to military housing. The impact of this on the role of the military in society is significant. For example, this undermine recruits’ ability to get training to develop critical skills that will be essential for them in civilian life. Instead, while serving on active duty, they spend a much higher amount of time on the front lines and in high-risk battle, increasing the likelihood that they will be physically or psychologically harmed. The impact on skills development and job opportunities is tremendous, but so is the impact on the diversification of the social fabric.
Private vendors are not engaged in the same social engineering project as the military and, as a result, tend to hire and fire people based on their ability to work effectively as a team. Like many companies, they have little incentive to invest in helping diverse teams learn to work together as effectively as possible. Building diverse teams — especially ones in which members depend on each other for their survival — is extremely hard, time-consuming, and emotionally exhausting. As a result, private companies focus on “culture fit,” emphasize teams that get along, and look for people who already have the necessary skills, all of which helps reinforce existing segregation patterns.
The end result is that, in the last 20 years, we’ve watched one of our major structures for diversification collapse without anyone taking notice. And because of how it’s happened, it’s also connected to job opportunities and economic opportunity for many working- and middle-class individuals, seeding resentment and hatred.
If you ask a college admissions officer at an elite institution to describe how they build a class of incoming freshman, you will quickly realize that the American college system is a diversification project. Unlike colleges in most parts of the world, the vast majority of freshman at top tier universities in the United States live on campus with roommates who are assigned to them. Colleges approach housing assignments as an opportunity to pair diverse strangers with one another to build social ties. This makes sense given how many friendships emerge out of freshman dorms. By pairing middle class kids with students from wealthier families, elite institutions help diversify the elites of the future.
This diversification project produces a tremendous amount of conflict. Although plenty of people adore their college roommates and relish the opportunity to get to know people from different walks of life as part of their college experience, there is an amazing amount of angst about dorm assignments and the troubles that brew once folks try to live together in close quarters. At many universities, residential life is often in the business of student therapy as students complain about their roommates and dormmates. Yet, just like in the military, learning how to negotiate conflict and diversity in close quarters can be tremendously effective in sewing the social fabric.
In the springs of 2006, I was doing fieldwork with teenagers at a time when they had just received acceptances to college. I giggled at how many of them immediately wrote to the college in which they intended to enroll, begging for a campus email address so that they could join that school’s Facebook (before Facebook was broadly available). In the previous year, I had watched the previous class look up roommate assignments on MySpace so I was prepared for the fact that they’d use Facebook to do the same. What I wasn’t prepared for was how quickly they would all get on Facebook, map the incoming freshman class, and use this information to ask for a roommate switch. Before they even arrived on campus in August/September of 2006, they had self-segregated as much as possible.
A few years later, I watched another trend hit: cell phones. While these were touted as tools that allowed students to stay connected to parents (which prompted many faculty to complain about “helicopter parents” arriving on campus), they really ended up serving as a crutch to address homesickness, as incoming students focused on maintaining ties to high school friends rather than building new relationships.
Students go to elite universities to “get an education.” Few realize that the true quality product that elite colleges in the US have historically offered is social network diversification. Even when it comes to job acquisition, sociologists have long known that diverse social networks (“weak ties”) are what increase job prospects. By self-segregating on campus, students undermine their own potential while also helping fragment the diversity of the broader social fabric.
Diversity is often touted as highly desirable. Indeed, in professional contexts, we know that more diverse teams often outperform homogeneous teams. Diversity also increases cognitive development, both intellectually and socially. And yet, actually encountering and working through diverse viewpoints, experiences, and perspectives is hard work. It’s uncomfortable. It’s emotionally exhausting. It can be downright frustrating.
Thus, given the opportunity, people typically revert to situations where they can be in homogeneous environments. They look for “safe spaces” and “culture fit.” And systems that are “personalized” are highly desirable. Most people aren’t looking to self-segregate, but they do it anyway. And, increasingly, the technologies and tools around us allow us to self-segregate with ease. Is your uncle annoying you with his political rants? Mute him. Tired of getting ads for irrelevant products? Reveal your preferences. Want your search engine to remember the things that matter to you? Let it capture data. Want to watch a TV show that appeals to your senses? Here are some recommendations.
Any company whose business model is based on advertising revenue and attention is incentivized to engage you by giving you what you want. And what you want in theory is different than what you want in practice.
Consider, for example, what Netflix encountered when it started its streaming offer. Users didn’t watch the movies that they had placed into their queue. Those movies were the movies they thought they wanted, movies that reflected their ideal self — 12 Years a Slave, for example. What they watched when they could stream whatever they were in the mood for at that moment was the equivalent of junk food — reruns of Friends, for example. (This completely undid Netflix’s recommendation infrastructure, which had been trained on people’s idealistic self-images.)
The divisions are not just happening through commercialism though. School choice has led people to self-segregate from childhood on up. The structures of American work life mean that fewer people work alongside others from different socioeconomic backgrounds. Our contemporary culture of retail and service labor means that there’s a huge cultural gap between workers and customers with little opportunity to truly get to know one another. Even many religious institutions are increasingly fragmented such that people have fewer interactions across diverse lines. (Just think about how there are now “family services” and “traditional services” which age-segregate.) In so many parts of public, civic, and professional life, we are self-segregating and the opportunities for doing so are increasing every day.
By and large, the American public wants to have strong connections across divisions. They see the value politically and socially. But they’re not going to work for it. And given the option, they’re going to renew their license remotely, try to get out of jury duty, and use available data to seek out housing and schools that are filled with people like them. This is the conundrum we now face.
Many pundits remarked that, during the 2016 election season, very few Americans were regularly exposed to people whose political ideology conflicted with their own. This is true. But it cannot be fixed by Facebook or news media. Exposing people to content that challenges their perspective doesn’t actually make them more empathetic to those values and perspectives. To the contrary, it polarizes them. What makes people willing to hear difference is knowing and trusting people whose worldview differs from their own. Exposure to content cannot make up for self-segregation.
If we want to develop a healthy democracy, we need a diverse and highly connected social fabric. This requires creating contexts in which the American public voluntarily struggles with the challenges of diversity to build bonds that will last a lifetime. We have been systematically undoing this, and the public has used new technological advances to make their lives easier by self-segregating. This has increased polarization, and we’re going to pay a heavy price for this going forward. Rather than focusing on what media enterprises can and should do, we need to focus instead on building new infrastructures for connection where people have a purpose for coming together across divisions. We need that social infrastructure just as much as we need bridges and roads.
This piece was originally published as part of a series on media, accountability, and the public sphere. See also:
Anxious about the widespread consumption and spread of propaganda and fake news during this year’s election cycle, many progressives are calling for an increased commitment to media literacy programs. Others are clamoring for solutions that focus on expert fact-checking and labeling. Both of these approaches are likely to fail — not because they are bad ideas, but because they fail to take into consideration the cultural context of information consumption that we’ve created over the last thirty years. The problem on our hands is a lot bigger than most folks appreciate.
What Are Your Sources?
I remember a casual conversation that I had with a teen girl in the midwest while I was doing research. I knew her school approached sex ed through an abstinence-only education approach, but I don’t remember how the topic of pregnancy came up. What I do remember is her telling me that she and her friends talked a lot about pregnancy and “diseases” she could get through sex. As I probed further, she matter-of-factly explained a variety of “facts” she had heard that were completely inaccurate. You couldn’t get pregnant until you were 16. AIDS spreads through kissing. Etc. I asked her if she’d talked to her doctor about any of this, and she looked me as though I had horns. She explained that she and her friends had done the research themselves, by which she meant that they’d identified websites online that “proved” their beliefs.
For years, that casual conversation has stuck with me as one of the reasons that we needed better Internet-based media literacy. As I detailed in my book It’s Complicated: The Social Lives of Networked Teens, too many students I met were being told that Wikipedia was untrustworthy and were, instead, being encouraged to do research. As a result, the message that many had taken home was to turn to Google and use whatever came up first. They heard that Google was trustworthy and Wikipedia was not.
Understanding what sources to trust is a basic tenet of media literacy education. When educators encourage students to focus on sourcing quality information, they encourage them to critically ask who is publishing the content. Is the venue a respected outlet? What biases might the author have? The underlying assumption in all of this is that there’s universal agreement that major news outlets like the New York Times, scientific journal publications, and experts with advanced degrees are all highly trustworthy.
Think about how this might play out in communities where the “liberal media” is viewed with disdain as an untrustworthy source of information…or in those where science is seen as contradicting the knowledge of religious people…or where degrees are viewed as a weapon of the elite to justify oppression of working people. Needless to say, not everyone agrees on what makes a trusted source.
Students are also encouraged to reflect on economic and political incentives that might bias reporting. Follow the money, they are told. Now watch what happens when they are given a list of names of major power players in the East Coast news media whose names are all clearly Jewish. Welcome to an opening for anti-Semitic ideology.
We’ve been telling young people that they are the smartest snowflakes in the world. From the self-esteem movement in the 1980s to the normative logic of contemporary parenting, young people are told that they are lovable and capable and that they should trust their gut to make wise decisions. This sets them up for another great American ideal: personal responsibility.
In the United States, we believe that worthy people lift themselves up by their bootstraps. This is our idea of freedom. What it means in practice is that every individual is supposed to understand finance so well that they can effectively manage their own retirement funds. And every individual is expected to understand their health risks well enough to make their own decisions about insurance. To take away the power of individuals to control their own destiny is viewed as anti-American by so much of this country. You are your own master.
Children are indoctrinated into this cultural logic early, even as their parents restrict their mobility and limit their access to social situations. But when it comes to information, they are taught that they are the sole proprietors of knowledge. All they have to do is “do the research” for themselves and they will know better than anyone what is real.
Combine this with a deep distrust of media sources. If the media is reporting on something, and you don’t trust the media, then it is your responsibility to question their authority, to doubt the information you are being given. If they expend tremendous effort bringing on “experts” to argue that something is false, there must be something there to investigate.
Now think about what this means for #Pizzagate. Across this country, major news outlets went to great effort to challenge conspiracy reports that linked John Podesta and Hillary Clinton to a child trafficking ring supposedly run out of a pizza shop in Washington, DC. Most people never heard the conspiracy stories, but their ears perked up when the mainstream press went nuts trying to debunk these stories. For many people who distrust “liberal” media and were already primed not to trust Clinton, the abundant reporting suggested that there was something to investigate.
Most people who showed up to the Comet Ping Pong pizzeria to see for their own eyes went undetected. But then a guy with a gun decided he “wanted to do some good” and “rescue the children.” He was the first to admit that “the intel wasn’t 100%,” but what he was doing was something that we’ve taught people to do — question the information they’re receiving and find out the truth for themselves.
Many marginalized groups are justifiably angry about the ways in which their stories have been dismissed by mainstream media for decades. This is most acutely felt in communities of color. And this isn’t just about the past. It took five days for major news outlets to cover Ferguson. It took months and a lot of celebrities for journalists to start discussing the Dakota Pipeline. But feeling marginalized from news media isn’t just about people of color. For many Americans who have watched their local newspaper disappear, major urban news reporting appears disconnected from reality. The issues and topics that they feel affect their lives are often ignored.
For decades, civil rights leaders have been arguing for the importance of respecting experience over expertise, highlighting the need to hear the voices of people of color who are so often ignored by experts. This message has taken hold more broadly, particularly among lower and middle class whites who feel as though they are ignored by the establishment. Whites also want their experiences to be recognized, and they too have been pushing for the need to understand and respect the experiences of “the common man.” They see “liberal” “urban” “coastal” news outlets as antithetical to their interests because they quote from experts, use cleaned-up pundits to debate issues, and turn everyday people (e.g., “red sweater guy”) into spectacles for mass enjoyment.
Consider what’s happening in medicine. Many people used to have a family doctor whom they knew for decades and trusted as individuals even more than as experts. Today, many people see doctors as arrogant and condescending, overly expensive and inattentive to their needs. Doctors lack the time to spend more than a few minutes with patients, and many people doubt that the treatment they’re getting is in their best interest. People feel duped into paying obscene costs for procedures that they don’t understand. Many economists can’t understand why so many people would be against the Affordable Care Act because they don’t recognize that this “socialized” medicine is perceived as experts over experience by people who don’t trust politicians who tell them what’s in their best interest any more than they trust doctors. And public trust in doctors is declining sharply.
Why should we be surprised that most people are getting medical information from their personal social network and the Internet? It’s a lot cheaper than seeing a doctor, and both friends and strangers on the Internet are willing to listen, empathize, and compare notes. Why trust experts when you have at your fingertips a crowd of knowledgeable people who may have had the same experience as you and can help you out?
Consider this dynamic in light of discussions around autism and vaccinations. First, an expert-produced journal article was published linking autism to vaccinations. This resonated with many parents’ experience. Then, other experts debunked the first report, challenged the motivations of the researcher, and engaged in a mainstream media campaign to “prove” that there was no link. What unfolded felt like a war on experience, and a network of parents coordinated to counter this new batch of experts who were widely seen as ignorant, moneyed, and condescending. The more that the media focused on waving away these networks of parents through scientific language, the more the public felt sympathetic to the arguments being made by anti-vaxxers.
Keep in mind that anti-vaxxers aren’t arguing that vaccinations definitively cause autism. They are arguing that we don’t know. They are arguing that experts are forcing children to be vaccinated against their will, which sounds like oppression. What they want is choice — the choice to not vaccinate. And they want information about the risks of vaccination, which they feel are not being given to them. In essence, they are doing what we taught them to do: questioning information sources and raising doubts about the incentives of those who are pushing a single message. Doubt has become tool.
Since the election, everyone has been obsessed with fake news, as experts blame “stupid” people for not understanding what is “real.” The solutionism around this has been condescending at best. More experts are needed to label fake content. More media literacy is needed to teach people how not to be duped. And if we just push Facebook to curb the spread of fake news, all will be solved.
I can’t help but laugh at the irony of folks screaming up and down about fake news and pointing to the story about how the Pope backs Trump. The reason so many progressives know this story is because it was spread wildly among liberal circles who were citing it as appalling and fake. From what I can gather, it seems as though liberals were far more likely to spread this story than conservatives. What more could you want if you ran a fake news site whose goal was to make money by getting people to spread misinformation? Getting doubters to click on clickbait is far more profitable than getting believers because they’re far more likely to spread the content in an effort to dispel the content. Win!
People believe in information that confirms their priors. In fact, if you present them with data that contradicts their beliefs, they will double down on their beliefs rather than integrate the new knowledge into their understanding. This is why first impressions matter. It’s also why asking Facebook to show content that contradicts people’s views will not only increase their hatred of Facebook but increase polarization among the network. And it’s precisely why so many liberals spread “fake news” stories in ways that reinforce their belief that Trump supporters are stupid and backwards.
Labeling the Pope story as fake wouldn’t have stopped people from believing that story if they were conditioned to believe it. Let’s not forget that the public may find Facebook valuable, but it doesn’t necessarily trust the company. So their “expertise” doesn’t mean squat to most people. Of course, it would be an interesting experiment to run; I do wonder how many liberals wouldn’t have forwarded it along if it had been clearly identified as fake. Would they have not felt the need to warn everyone in their network that conservatives were insane? Would they have not helped fuel a money-making fake news machine? Maybe.
But I think labeling would reinforce polarization — but it would feel like something was done. Nonbelievers would use the label to reinforce their view that the information is fake (and minimize the spread, which is probably a good thing), while believers would simply ignore the label. But does that really get us to where we want to go?
Addressing so-called fake news is going to require a lot more than labeling.It’s going to require a cultural change about how we make sense of information, whom we trust, and how we understand our own role in grappling with information. Quick and easy solutions may make the controversy go away, but they won’t address the underlying problems.
As a huge proponent for media literacy for over a decade, I’m struggling with the ways in which I missed the mark. The reality is that my assumptions and beliefs do not align with most Americans. Because of my privilege as a scholar, I get to see how expert knowledge and information is produced and have a deep respect for the strengths and limitations of scientific inquiry. Surrounded by journalists and people working to distribute information, I get to see how incentives shape information production and dissemination and the fault lines of that process. I believe that information intermediaries are important, that honed expertise matters, and that no one can ever be fully informed. As a result, I have long believed that we have to outsource certain matters and to trust others to do right by us as individuals and society as a whole. This is what it means to live in a democracy, but, more importantly, it’s what it means to live in a society.
In the United States, we’re moving towards tribalism, and we’re undoing the social fabric of our country through polarization, distrust, and self-segregation. And whether we like it or not, our culture of doubt and critique, experience over expertise, and personal responsibility is pushing us further down this path.
Media literacy asks people to raise questions and be wary of information that they’re receiving. People are. Unfortunately, that’s exactly why we’re talking past one another.
The path forward is hazy. We need to enable people to hear different perspectives and make sense of a very complicated — and in many ways, overwhelming — information landscape. We cannot fall back on standard educational approaches because the societal context has shifted. We also cannot simply assume that information intermediaries can fix the problem for us, whether they be traditional news media or social media. We need to get creative and build the social infrastructure necessary for people to meaningfully and substantively engage across existing structural lines. This won’t be easy or quick, but if we want to address issues like propaganda, hate speech, fake news, and biased content, we need to focus on the underlying issues at play. No simple band-aid will work.
Special thanks to Amanda Lenhart, Claire Fontaine, Mary Madden, and Monica Bulger for their feedback!
This post was first published as part of a series on media, accountability, and the public sphere. See also:
For most non-technical folks, “hacking” evokes the notion of using sophisticated technical skills to break through the security of a corporate or government system for illicit purposes. Of course, most folks who were engaged in cracking security systems weren’t necessarily in it for espionage and cruelty. In the 1990s, I grew up among teenage hackers who wanted to break into the computer systems of major institutions that were part of the security establishment, just to show that they could. The goal here was to feel a sense of power in a world where they felt pretty powerless. The rush was in being able to do something and feel smarter than the so-called powerful. It was fun and games. At least until they started getting arrested.
Hacking has always been about leveraging skills to push the boundaries of systems. Keep in mind that one early definition of a hacker (from the Jargon File) was “A person who enjoys learning the details of programming systems and how to stretch their capabilities, as opposed to most users who prefer to learn only the minimum necessary.” In another early definition (RFC:1392), a hacker is defined as “A person who delights in having an intimate understanding of the internal workings of a system, computers and computer networks in particular.” Both of these definitions highlight something important: violating the security of a technical system isn’t necessarily the primary objective.
Indeed, over the last 15 years, I’ve watched as countless hacker-minded folks have started leveraging a mix of technical and social engineering skills to reconfigure networks of power. Some are in it for the fun. Some see dollar signs. Some have a much more ideological agenda. But above all, what’s fascinating is how many people have learned to play the game. And in some worlds, those skills are coming home to roost in unexpected ways, especially as groups are seeking to mess with information intermediaries in an effort to hack the attention economy.
In 2003, a 15-year-old named Chris Poole started an image board site based on a Japanese trend called 4chan. His goal was not political. Rather, like many of his male teenage peers, he simply wanted a place to share pornography and anime. But as his site’s popularity grew, he ran into a different problem — he couldn’t manage the traffic while storing all of the content. So he decided to delete older content as newer content came in. Users were frustrated that their favorite images disappeared so they reposted them, often with slight modifications. This gave birth to a phenomenon now understood as “meme culture.” Lolcats are an example. These are images of cats captioned with a specific font and a consistent grammar for entertainment.
Those who produced meme-like images quickly realized that they could spread like wildfire thanks to new types of social media (as well as older tools like blogging). People began producing memes just for fun. But for a group of hacker-minded teenagers who were born a decade after I was, a new practice emerged. Rather than trying to hack the security infrastructure, they wanted to attack the emergent attention economy. They wanted to show that they could manipulate the media narrative, just to show that they could. This was happening at a moment when social media sites were skyrocketing, YouTube and blogs were challenging mainstream media, and pundits were pushing the idea that anyone could control the narrative by being their own media channel. Hell, “You” was TIME Magazine’s person of the year in 2006.
Taking a humorist approach, campaigns emerged within 4chan to “hack” mainstream media. For example, many inside 4chan felt that widespread anxieties about pedophilia were exaggerated and sensationalized. They decided to target Oprah Winfrey, who, they felt, was amplifying this fear-mongering. Trolling her online message board, they got her to talk on live TV about how “over 9,000 penises” were raping children. Humored by this success, they then created a broader campaign around a fake character known as Pedobear. In a different campaign, 4chan “b-tards” focused on gaming the TIME 100 list of “the world’s most influential people” by arranging it such that the first letter of each name on the list spelled out “Marblecake also the game,” which is a known in-joke in this community. Many other campaigns emerged to troll major media and other cultural leaders. And frankly, it was hard not to laugh when everyone started scratching their heads about why Rick Astley’s 1987 song “Never Gonna Give You Up” suddenly became a phenomenon again.
By engaging in these campaigns, participants learned how to shape information within a networked ecosystem. They learned how to design information for it to spread across social media.
They also learned how to game social media, manipulate its algorithms, and mess with the incentive structure of both old and new media enterprises. They weren’t alone. I watched teenagers throw brand names and Buzzfeed links into their Facebook posts to increase the likelihood that their friends would see their posts in their News Feed. Consultants starting working for companies to produce catchy content that would get traction and clicks. Justin Bieber fans ran campaign after campaign to keep Bieber-related topics in Twitter Trending Topics. And the activist group Invisible Children leveraged knowledge of how social media worked to architect the #Kony2012 campaign. All of this was seen as legitimate “social media marketing,” making it hard to detect where the boundaries were between those who were hacking for fun and those who were hacking for profit or other “serious” ends.
Running campaigns to shape what the public could see was nothing new, but social media created new pathways for people and organizations to get information out to wide audiences. Marketers discussed it as the future of marketing. Activists talked about it as the next frontier for activism. Political consultants talked about it as the future of political campaigns. And a new form of propaganda emerged.
In her phenomenal account of Anonymous — “Hacker, Hoaxer, Whistleblower, Spy” — Gabriella Coleman describes the interplay between different networks of people playing similar hacker-esque games for different motivations. She describes the goofy nature of those “Anons” who created a campaign to expose Scientology, which many believed to be a farcical religion with too much power and political sway. But she also highlights how the issues became more political and serious as WikiLeaks emerged, law enforcement started going after hackers, and the Arab Spring began.
Anonymous was birthed out of 4chan, but because of the emergent ideological agendas of many Anons, the norms and tactics started shifting. Some folks were in it for fun and games, but the “lulz” started getting darker and those seeking vigilante justice started using techniques like “doxing”to expose people who were seen as deserving of punishment. Targets changed over time, showcasing the divergent political agendas in play.
Perhaps the most notable turn involved “#GamerGate” when issues of sexism in the gaming industry emerged into a campaign of harassment targeted at a group of women. Doxing began being used to enable “swatting” — in which false reports called in by perpetrators would result in SWAT teams sent to targets’ homes. The strategies and tactics that had been used to enable decentralized but coordinated campaigns were now being used by those seeking to use the tools of media and attention to do serious reputational, psychological, economic, and social harm to targets. Although 4chan had long been an “anything goes” environment (with notable exceptions), #GamerGate became taboo there for stepping over the lines.
As #GamerGate unfolded, men’s rights activists began using the situation to push forward a long-standing political agenda to counter feminist ideology, pushing for #GamerGate to be framed as a serious debate as opposed to being seen as a campaign of hate and harassment. In some ways, the resultant media campaign was quite successful: major conferences and journalistic enterprises felt the need to “hear both sides” as though there was a debate unfolding. Watching this, I couldn’t help but think of the work of Frank Luntz, a remarkably effective conservative political consultant known for reframing issues using politicized language.
As doxing and swatting have become more commonplace, another type of harassment also started to emerge en masse: gaslighting. This term refers to a 1944 Ingrid Bergman film called “Gas Light” (which was based on a 1938 play). The film depicts psychological abuse in a domestic violence context, where the victim starts to doubt reality because of the various actions of the abuser. It is a form of psychological warfare that can work tremendously well in an information ecosystem, especially one where it’s possible to put up information in a distributed way to make it very unclear what is legitimate, what is fake, and what is propaganda. More importantly, as many autocratic regimes have learned, this tactic is fantastic for seeding the public’s doubt in institutions and information intermediaries.
In the early days of blogging, many of my fellow bloggers imagined that our practice could disrupt mainstream media. For many progressive activists, social media could be a tool that could circumvent institutionalized censorship and enable a plethora of diverse voices to speak out and have their say. Civic minded scholars were excited by “smart mobs” who leveraged new communications platforms to coordinate in a decentralized way to speak truth to power. Arab Spring. Occupy Wall Street. Black Lives Matter. These energized progressives as “proof” that social technologies could make a new form of civil life possible.
I spent 15 years watching teenagers play games with powerful media outlets and attempt to achieve control over their own ecosystem. They messed with algorithms, coordinated information campaigns, and resisted attempts to curtail their speech. Like Chinese activists, they learned to hide their traces when it was to their advantage to do so. They encoded their ideas such that access to content didn’t mean access to meaning.
Of course, it wasn’t just progressive activists and teenagers who were learning how to mess with the media ecosystem that has emerged since social media unfolded. We’ve also seen the political establishment, law enforcement, marketers, and hate groups build capacity at manipulating the media landscape. Very little of what’s happening is truly illegal, but there’s no widespread agreement about which of these practices are socially and morally acceptable or not.
The techniques that are unfolding are hard to manage and combat. Some of them look like harassment, prompting people to self-censor out of fear. Others look like “fake news”, highlighting the messiness surrounding bias, misinformation, disinformation, and propaganda. There is hate speech that is explicit, but there’s also suggestive content that prompts people to frame the world in particular ways. Dog whistle politics have emerged in a new form of encoded content, where you have to be in the know to understand what’s happening. Companies who built tools to help people communicate are finding it hard to combat the ways their tools are being used by networks looking to skirt the edges of the law and content policies. Institutions and legal instruments designed to stop abuse are finding themselves ill-equipped to function in light of networked dynamics.
The Internet has long been used for gaslighting, and trolls have long targeted adversaries. What has shifted recently is the scale of the operation, the coordination of the attacks, and the strategic agenda of some of the players.
For many who are learning these techniques, it’s no longer simply about fun, nor is it even about the lulz. It has now become about acquiring power.
A new form of information manipulation is unfolding in front of our eyes. It is political. It is global. And it is populist in nature. The news media is being played like a fiddle, while decentralized networks of people are leveraging the ever-evolving networked tools around them to hack the attention economy.
I only wish I knew what happens next.
This post was first published as part of a series on media, accountability, and the public sphere. See also:
This post was also translated to Portuguese.