About This Episode
In this episode, our guest, Dr. Safiya U. Noble, author of Algorithms of Oppression: How Search Engines Reinforce Racism, discusses the impact of algorithmic injustice. She unpacks the hidden biases built into the platforms we use daily. This conversation sheds light on why reclaiming the power to tell our own stories is essential in the digital age.
About Dr. Safiya U. Noble
Dr. Safiya U. Noble is the David O. Sears Presidential Endowed Chair of Social Sciences and Professor of Gender Studies, African American Studies, and Information Studies at the University of California, Los Angeles (UCLA). She is the Director of the Center on Resilience & Digital Justice and Co-Director of the Minderoo Initiative on Tech & Power at the UCLA Center for Critical Internet Inquiry (C2i2). She currently serves as a Director of the UCLA DataX Initiative, leading work in critical data studies for the campus. Professor Noble is the author of the best-selling book on racist and sexist algorithmic harm in commercial search engines, entitled Algorithms of Oppression: How Search Engines Reinforce Racism (NYU Press), which has been widely reviewed in scholarly and popular publications. In 2021, she was recognized as a MacArthur Foundation Fellow for her groundbreaking work on algorithmic discrimination.
In her words…
[Technology of this kind doesn’t] “look at the history of discrimination against African-Americans, or people of color, or poor people, or women. That’s not a mitigating factor in the way in which the systems are programmed. So this is one of the huge opportunities when we think about what technology could be in our society.”
“Imagine if we had more pro-social, pro-human rights, pro-civil rights kinds of technology design that accounted for histories of discrimination and oppression. Then we wouldn’t actually have the prevalence of technological redlining that we do and at the rate that we have it now.”
“We have to think about other ways of living and doing and being that can resist these totalizing kinds of systems that are about technological redlining or they are algorithms of oppression. And that might even mean that we have a whole pivot. I can see and I can believe that in my own lifetime, we might have a radical pivot away from technology and doing things differently because of a sense of future and possibility that is really dependent upon it.”
“We need deeper investment in pro-social, rights-protecting kinds of technologies and communications systems.”
“We underestimate the power of the local. We need to cultivate a range of skills, a range of networks, and a range of access points to each other.”
Questions Answered on this Episode
- In the context of our theme, Who Shapes the Story?, how do you see algorithms acting as hidden storytellers in society?
- From a perspective of social change, what should the person who is trying to create a narrative, or share a narrative, be thinking of? How can they be mindful, understanding the constraints of this particular platform? What do they need to know?
- For the person consuming the media, what can they do to sort of attempt to sift through fact or fiction, knowing that the algorithm is only feeding them certain things? How can they do their own due diligence within that? Is there anything folks can be thinking about?
- How should the platforms be leveraged? How should people be thinking about themselves as part of an ecosystem of storytellers?
- Can you talk to me a little bit about your definition of technological redlining? I think it’s really important for people to hear that term and to understand it. And hopefully, put some structure around that from a storytelling context.
- What advice would you give if you were sitting in an auditorium of communications leaders, or CEOs of foundations and nonprofits, who don’t want this story to die? What advice would you give to them based on everything you know and understand about the platforms? What’s at risk, and what are the opportunities?
Transcript
Vanessa: Hello, and welcome to the Social Change Diaries. I am your host, Vanessa Wakeman, and I am excited to be bringing this show to you all after a very long and much-needed hiatus. So, the Social Change Diaries has been on hiatus for a couple of years as we were knee-deep in the work with assisting nonprofits, foundations, and socially responsible companies with the business of communications. We decided to return because communications is shifting and so is how people understand communications and narrative, and the biases that are inherent in a lot of what we’re seeing. We wanted to be in conversation with some of the experts who have examined and interrogated narrative and storytelling. And so, for our very first episode, I am delighted to have with us Dr. Safiya Noble. She is the David O. Sears Presidential Endowed Chair of Social Sciences and Professor of Gender Studies, African American Studies, and Information Studies at UCLA. And she is also the author of the bestselling book on racist and sexist algorithmic harm in commercial search engines, entitled Algorithms of Oppression: How Search Engines Reinforce Racism. If you have not read this book, please get it. It’s an important, important book and definitely reflective of our times and things that we need to be thinking about. And so I want to say welcome to Dr. Noble and thank you for being here with us today.
Safiya: It’s my pleasure. So great to be here with you today.
Vanessa: Thank you. So, as an organization, the Wakeman Agency is a strategic communications firm, and much of our work is rooted in narratives and story. And one of the things we have been confronted with more so in the last year than ever before in our 22-year history is really trying to get to the root of who shapes the story, like who has the power to shape the story. And we’ve explored that in different ways over the years, but today, we really want to understand who holds the power to tell the stories, how the narratives are constructed, and who benefits. And so one of the things that I think is really important about your work is you sort of help us to uncover and think about who benefits from the way stories are told. So often, people are not thinking about the algorithms and the platforms that we use as a tool for shaping narrative in our belief system. And so I guess my first question for you is in the context of our theme, Who Shapes the Story, how do you see algorithms acting as hidden storytellers in society?
Safiya: This is such a great question. And I love the entire premise of your question about algorithms as storytellers, because I still think we are, even though, let’s say, we’re more than a decade into research that uncovered and demonstrated again and again how algorithms are controlling narratives about people and products and businesses and communities. I think people still often believe that what happens on the internet, especially in a social media platform or a search platform, is really just free speech. That if we experience stories and ideas that we know are everything from fact and evidence-based research to full-blown propaganda, it’s just a reflection of what users are doing on the internet. But actually, the companies that control the flow of information through their systems, they really are constantly shaping narratives around us about all kinds of things. And so my work started out back in, I would say, I started in 2010 as a graduate student. I was looking at large digital media platforms. I was especially interested in Google because I was in library and information science grad school, and Google was this new technology, the search platform that was basically starting to become a proxy for librarians. Young people and the public would go to the public library, they’d go to their school library, their university library, and they’d try to deeply research something. Librarians started reporting out that everybody was using Google. The students were all using Google, and they were interested. And of course, this incredible book was written by Siva Fideyanathan called The Googolization of Everything and Why We Should Worry. And I loved that book and I thought, oh, yes, we should worry, but it’s actually even worse.
What he’s talking about is if you point something like a search engine or these kinds of technologies toward minoritized communities, right? Toward people of color or toward women and girls. And so I just did a deep dive into that. Now, I gotta tell you that back then, when you said that, first of all, that there was an algorithm doing something, people didn’t even know what that word was. I mean, that was not an everyday word like we use today. I mean, I absolutely love it, it’s music to my ears when you say, “How was the algorithm, the storyteller?” I’m like, my gosh, remember just a minute ago when that was computer science and information science. So, a number of us scholars and investigative journalists have spent the last decade and a half demonstrating again and again and again how algorithms or computers, the way that computers are coded and programs are coded, overdetermine what we might experience. And fundamentally, that’s because most of these technologies are advertising technologies; they’re ad tech. So they are absolutely designed to prioritize the messages and the clicks and the stories and reels and everything else, the tweets of their advertisers. And once you understand that these platforms that we engage with are advertising technologies, and that the algorithm is looking to find audiences for the content that is sponsored by those advertisers on those platforms, then you absolutely understand that these are, you know, anything but just like a free speech zone where users are deciding what gets seen and what goes viral. It’s actually completely for the most part in the control of the companies. And that is very important because so many other dimensions of speech and news and information are in decline. Libraries under attack, newspapers almost completely out of business in the United States, the major media organizations, let’s say in terms of cable news, mostly consolidated, controlled by a handful of companies. So the algorithm is doing a lot of work right now to shape our societies in the interest of the people who own and control those algorithms.
And I think this is one of the most important things we have to understand in a democracy where the free flow of information and news, what we call the fourth estate, is absolutely crucial. And the platforms that we’re on are not news platforms, they’re ad platforms. And that’s fundamentally different.
Vanessa: So that part, right? Like the understanding of what the platform is. I feel that so often people are, I like to say people look at Google as Bible, right? Like if it’s on Google, it’s real for me, it’s the truth. Also, as you were speaking, I thought about a scene from The Devil Wears Prada where Meryl Streep is the Anna Wintour character. There’s a conversation about Cerulean blue, and Anne Hathaway sort of scoffs at her.
It’s, you know, it’s just blue. And Anna reminds her of how an entire industry has created this sort of perception around what’s important, what’s relevant, what you should care about. And in this particular case, what’s hot. I think so much of what you said to sort of simplify or sort of give an example in a different lane. That’s exactly what’s happening with the algorithms, right? There are multiple forces sort of leveraging the platform to create the illusion of a reality that they want us to buy into, right? Around this is, you know, how this particular demographic of people should be seen and perceived. This is how these geographies should be considered. And so I think that there are two points here that are really important for us to think about. One, the narrator, right? Like, is there an opportunity for the person who’s creating the narrative to be more mindful? I feel like for the advertisers, we have no control over that in this conversation and context. But I think from a perspective of social change, thinking about what a person who is trying to create a narrative or share a narrative needs to be thinking of?
How can I be mindful, understanding the constraints of this particular platform? What do I need to know? And then for the person consuming the media, what can I do to sort of attempt to sift through fact or fiction, or knowing that the algorithm is only feeding me certain things? How can I do my own due diligence within that? Do you have any thoughts about that? Iis there anything that folks can be thinking about?
Safiya: Yeah, I mean, you really just so powerfully illustrated the range of factors that are at play when we are online, when we’re on the internet, right? And so on one hand, yes, you’re right. There are people who are the creators and who develop the logics that guide these systems that then determine that we like the ground upon which we play.
Right? And so that’s a perfect metaphor from the Devil Wears Prada in the sense that, you find the sweater at TJ Maxx and you think like it just got there. And in fact, there’s been like a whole curatorial process before it ever did, right? When it goes from like high fashion couture to the Target bin or the TJ – we’re not doing Target anymore – so the TJ Maxx bin. And so, you have this ecosystem within which a lot of decisions are made that are not apparent to you. Think of one of the most profound, right after the algorithmic kind of design, and the decisions, the values, and the determinants that are considered, and when.
Vanessa: Right?
Safiya: Writing the algorithms, and there are increasingly more of these kinds of systems, what people are shorthand calling AI systems. But there are also, let’s say, you know, we know this from the work of people like Professor Sarah Roberts. There are tens of thousands, if not more, content moderators who work all over the world who are constantly screening content and bringing it down, taking it down off a platform because there are different kinds of speech laws all over the world. So, this idea of the First Amendment doesn’t apply all over the world. It’s not a globally understood and recognized concept. And I will dare say it is losing ground in the United States as a concept. We certainly are not living in a free speech zone with the full protection of the First Amendment and the Constitution right now in the United States in 2025. So I think we have that as what feels like a little bit of an invisible force, let’s say that’s shaping the environment that people are working in. There’s the product design itself. Is it 140 characters? Is it, you know, 90 seconds? Yeah. Like what are the parameters within which you can communicate? How much space and time do you have? How long will it be visible? Who will it be visible to? So, again, there are many of these things that we don’t have any control over. We may not even notice that those are constraints, right? It’s actually hard to give a deep, detailed analysis in 140 characters, right? We learned this from Twitter. So those things are important as factors, but then there’s like, okay, what do we do as users of these platforms?
This is like the second part of your great analysis. And I would say that part of the challenge is that there have always been, well, primarily the kind of brand management or the trust and safety concerns that a platform has, so that it does not alienate its advertisers, who again are the customer. We are the product, not the customer, to these platforms. So that means if you are a Nazi and you’re putting out your Nazi music, Procter & Gamble doesn’t want those videos to be queued up or right next to their Pampers. I don’t know if Procter & Gamble owns Pampers, but you know what I’m saying, like to one of their products.
Vanessa: Right.
Safiya: So, they’re Pantene or whatever it is. So you have these kinds of issues that are also happening algorithmically and by humans who are looking at it. Listen, certain types of material actually can’t be associated with certain types of advertisers. So we’re constantly trying to manage. And of course, people are always gaming the system, right? Because if the Procter and Gamble ad is next to your content, it’s probably more likely to be algorithmically boosted because they’re paying for that boost to have as many eyeballs as possible. See, so you always have people who are kind of gaming these systems in that way, too. And then platforms, quite frankly, you know, the last statistic I had from several years ago on YouTube was that there were like 400 hours per minute of content being uploaded to YouTube. 400 hours per minute.
So that means there’s no way that human beings are reviewing all of that material and making sure that it’s not child sexual exploitation material, that it’s not predatory and being targeted to kids, that it’s not Nazis up against some product. There’s just no way. They try to use AI and software to do that, but they cannot do that at scale. And they can’t hire enough people to review it.
So this is one of the reasons why we also, for those of us who are often targeted by some of the most dangerous and harmful and hateful, misogynistic kinds of content, homophobic, online, why we’re often assaulted with this kind of stuff coming at us. And it feels like we don’t have a lot of control, and it’s very difficult. There aren’t a lot of controls to get that out of your feed. Although I love that there are new projects. Like Black Sky, which is a version, kind of a BlueSky where they are working on designing algorithms and their platforms so that people are not trolled by racist bots that have been programmed in racist and misogynistic ways. So these things are, you know, like these things can be done, but for the most part, venture capital is not interested in investing in those kinds of pro-social, pro-rights, democratic types of technology, right? Because the more of an “anything goes” environment, like let’s say on Discord or Truth, those kinds of things, the more you’re going to attract millions of people to it. And, ultimately, that’s the moneymaker – it’s where all the people are. So, you know, I’ll just say one last thing on this, which is, people still find a way to get their words out there and their thoughts out there. And so you see, for example, a lot of people will increasingly use clever kinds of storytelling tactics to talk about Palestine when Palestine or Gaza are words that are increasingly throttled and down ranked by social media platforms, particularly since, you know, January 19th of 2025.
So you have that kind of dynamic where people are like, I know the algorithm doesn’t like certain words. So I’m going to creatively storytell so that the, I, you know, replace a letter in that with a, you know, with a, you know, a smiley face, right. Or an at symbol, or I’m going to write it on a piece of paper and hold it up while I talk about my makeup tutorial that has nothing to do with actually what I’m telling you with the pieces of paper. There are so many clever and amazing ways that people work around it because now I think unlike, you know, in 2012, where people were like, “what’s an algorithm now?”, people are like, “I know the algorithms are doing things and I know certain words are gonna be scanned and certain types of speech will be censored and I am still going to say the things I wanna say”.
Vanessa: Thank you. That’s really helpful. And I think the idea of finding creative ways to tell the story is really important. You reminded me of the historical context when enslaved people had to find ways to let others know, like, “hey, we’re fleeing tomorrow”. There’s always been a way to work outside of the system. And so, it feels as those words came out of my mouth, I was like, oh, that doesn’t feel good to be saying in the context of how we need to sort of talk about narrative and storytelling. But I think it is important to think about how there’s no one way to sort of get your message across, and thinking within the constraints of the algorithms, there are ways to do that. And for us to constantly be examining and paying attention to what’s shifting because as we get wiser, the algorithms also get wiser, and sort of like, this is what they’re doing now. So there’ll be some other criteria that will be used to sort of silence certain voices and perspectives. And so making sure that we are intentional and paying attention to what’s happening there. I think when we think about story and narrative power, many people assume it lies in the hands of journalists, authors, and public figures, but your work suggests otherwise, that the tech platforms are also key narrators. Understanding that and how media is consumed, what advice would you give to those in the social change sector, like how they think about the relevance and importance of the platform, as opposed to some of the more traditional platforms for storytelling? We started the agency 22 years ago, and the big push was “we want to be on this news program” or “we want to be interviewed by this particular newspaper or media outlet”. And now, people still find great value in that, but they’re also looking at the different platforms, like, if we can, you know, have this many people consume our video on YouTube or connect with this audience on Instagram, like, there’s value to that, the expediency of how the message is shared and ideally people sort of returning to it and being able to get those incremental updates. What would you say, how should the platforms be leveraged in addition to what you already share? How should the people be thinking about that as part of an ecosystem of storytellers?
Safiya: Yeah, I mean, I think that for me, what I want to start with is what I think of as the kind of most dangerous and misunderstood dynamics right now around stories that are manufactured online. That would be large language models like chat, GPT, Gemini, Brock, these kinds of projects that are taking all kinds of stories, science, evidence, fact, subreddits, the best of what humans have to offer by way of knowledge and information, both past and present, combining that with all kinds of forms of other data, which is anything they can scrape off the internet and increasingly synthetic data that they just make up in order to train the model. All of that collapsing in together and being indiscernible, like undisambiguated, where you cannot, there’s no way you can know how this model has been trained, what it’s been trained on, and how reliable it is or not. More and more people are turning to those kinds of large language models, like in generative AI, to help them write their own stories. And to me, this is one; it’s a loss of our stories. As soon as we put them into those kinds of systems, they’re gone. They belong to the companies. But two, they’re kind of co-mingled with all kinds of other things that might actually be quite in opposition to the story you’re trying to tell. And so, the question to me is, as these type of systems are increasingly legitimated in our society and perceived as reliable and trustworthy, which let me tell you a little bit about the trustworthy and reliability of these systems.Researchers are showing that about 50%, just a little less than 50% of what you find in these systems is reliable, is actually factually correct. Okay, so that seems dangerous if you want to build your sales funnel, or you’re going to write a press release, or you’re going to do anything that requires some facts and figures and knowledge, and you’re turning to these kinds of systems that are already proven to be deeply unreliable. Second, they have points of view, and those points of view because of the way in which the interface kind of anthropomorphizes the result or makes it seem human, which means that they feel convincing. They feel as convincing as a person that you trust, but they’re not trustworthy. And we see in the most extreme cases where people use these kinds of chatbots for things like companionship or friendship, where children are adopting these technologies through platforms like Character AI or OtherFriend, you know, like these kinds of technologies. And they’re so dangerous, you know, they’re encouraging children and people to end their lives or their marriages. Like, I mean, building a false kind of intimacy with you, and you don’t even know what the problem is. Okay, so it’s the worst. I think people really fail to understand what these projects are.
And what a capture they are of both the human spirit, human creativity, the human imagination, in some cases, human life. And so that feels like, you know, we’re in the hype cycle of the advertising and news stories about these technologies, like they’re so amazing, but they’re opaque. We see corporate America, for example, for whom these chatbots were created, starting to wholly reject using them. They actually now are reporting out, I read in Forbes just a couple of weeks ago, that 70% of corporate America is disinterested in these types of chatbots because it creates so much more work for them because they have to hire more people, 30 % more employees to check the veracity, the worthiness, the credibility of what’s spit out of these systems. And that’s dangerous to the bottom line.
Vanessa: Yeah.
Safiya: But as that’s happening, those companies are looking for markets. And so guess where they’re moving? They’re moving into education. They’re moving into journalism. They’re moving into other places that are already strapped for resources and where you don’t have enough people to be a counterweight to what these technologies say or do, right? Where people can’t actually check on it because they’re already resource and labor-strapped.
Vanessa: Resources, right? That to me is really important. Of course, there’s all the other kinds of moral and ethical issues about stealing everybody’s art and then using it in mid-journey art to make a film. Now we have a new Tilly Norwood, the new AI actress that is created by the tech company, you know, because I guess Hollywood doesn’t have enough anorexic white women actresses, I don’t know, do you know what I’m saying? But then let’s make one, and that’s AI, you know, try to make it the next Scarlett Johansson, which is happening right now in real time. So, you know, you have all these kinds of things that I think ultimately for me, the antidote or the real power is human beings, human creativity, the human voice, spirit, what human beings make, what we say, what we do.
That’s actually so much more impactful than the inauthentic, synthetic, posing kinds of technologies. And when you think about what human beings value the most in other people, it’s authenticity, right? So if we are relinquishing our authenticity to machines, to me, we are, you know…
Safiya: We will fundamentally reshape humanity, probably in ways that we don’t want. And we see it already if you’re a Gen X manager somewhere and you hire a Gen Z, and you’re like, who are you? And they struggle to actually communicate who they are. And I see this in the classroom with students who are like, I just want to give the right answer because no one’s ever asked me what I think as a person.
Vanessa: Right.
Safiya: You know, so they’re programmed to pass, but not programmed in K-12 to think and to be critical thinkers and to be who they are. Like, they don’t even know who they are. And to me, this is such a powerful moment in human history for us to cultivate the critical thinking, the voice, the authentic self. We have very serious problems to solve in the world, and we need each other, and we don’t need those things mediated by ad tech that just wants to make money by extracting from us at all costs.
Vanessa: Yeah, as you were talking about the authenticity piece, I think that that is so important in the sector that we work with, with nonprofits, right? People want to understand social issues through a lens of authenticity and transparency, not extractive, right? We don’t want to exploit people, but like…where’s the soul, if you will, of the story? And so often, like one of the big complaints about a lot of the AI platforms is that it feels robotic and very impersonal versus the ability for someone to share an experience in a way that you feel that emotional connection, right? Like, how can I tell this story in a way that you now feel like you know me, you understand who I am, what I’m about, my challenges, my dreams, et cetera. And so I, if I’m sort of summarizing, like your beautiful sort of analysis and definition here, it’s like we should not be thinking about combating these sorts of platforms in any way outside. One of the most powerful ways we can do that is by telling more stories, right? Like being consistent and, you know, resolute in our desire to tell our stories, to not allow these platforms to take the parts that are of benefit to them in the larger ecosystem of information, crowd sharing, if you will, but for us to tell our stories. I think, in this moment, what we’re finding is that a lot of organizations are very concerned about how to put forth some of these narratives. How do we tell these stories against the backdrop of, you know, a very tense political climate when there’s so much disinformation and so many people who don’t believe that equity is important or that people of certain demographics should have opportunities for certain things. And so there’s this hesitation about, like, will this go into the chamber of just more information that people aren’t sure? And I think that the consistency in creating that drumbeaten cadence of like, we’re gonna continue to talk about equity, education equity, we’re going to continue to talk about, you know, women’s reproductive rights, we’re going to continue to talk about, you know, voter rights, all of these things. I think we may not have the same power structure behind us, but I think that the consistency of people hearing that we have an opportunity to educate people and for our stories to not get silenced or, even more importantly, erased, particularly because there’s so much focus and a desire to do that. We need to make sure that we’re not, you know, that we’re holding the boundary as best we can by just continuing to talk about it. Let’s remind people, let’s tell people. So thank you so much for sharing that. I feel like I have a hundred more questions, but I’m going to try to stick to our timeframe here. Can you talk to me a little bit about your definition of technological redlining? I think it is so important when, you know, we all know redlining, but when I heard that term and read it in your book and how you were thinking about it, it made me think about all of the ways that we make assumptions about what’s real and what’s true and how we have not began to sort of assess all of the things that were put in place to make us sort of believe certain things and how, as you talked about, the biases and the intention and where the product versus, you know, the advertisers. I think it’s really important for people to hear that term and to understand it and put some structure around that from a storytelling context.
Safiya: Yeah, I love that. I mean, I will say that there are certain words that help us understand conceptually a practice, a social practice, or a political practice. You know, one of those words is oppression. And it’s one of the reasons why I named the book Algorithms of Oppression, because oppression is actually different than other kinds of words.
It could have just been like algorithms of discrimination, but really it’s not just about discrimination in terms of the violation of civil rights laws, let’s say, or kind of legal ways in which people are harmed and locked out of opportunity. But we understand oppression as a system that involves the legal and political practices of a society, the social practices and cultural practices of a society, and the economic practices of a society to keep some people suppressed and at the bottom and not benefiting or to work in service of benefiting those at the top, right? Who benefits from their labor and extracts from them. So, we understand oppression in the United States because the United States was built upon the occupation of indigenous lands and the enslavement of African peoples. The largesse of what we experience in this country is off the backs of those practices for hundreds of years, which also continue. I try to talk about the ways in which those practices continue vis-a-vis technology by saying, listen, we understand redlining as a process of economic discrimination, right? That is often legal and has been made legal in the United States. And where we understand that the most is in, let’s say, insurance and banking, where you live at a particular, even the invention of zip codes was so that you could implement a process and a practice of devaluing certain zip codes or neighborhoods where, for the most part, African-Americans lived, but also Latinos at different moments in history.
Asian Americans, so Indigenous people, right, even more profoundly because they were, you know, put on reservations. So you have this kind of segregation of the society, an organization of it numerically, with something like zip codes as a practice or a census tract, all the way to the level of the block, and in some cases, the home. And then you use that data, and you feed all this historical data into a predictive analytic, a technical system. And what you now have is a kind of technological redlining where it’s quite opaque. It’s very difficult to see because the algorithm has basically spit out an approval or a disapproval for credit for a mortgage, for a small business loan, a personal loan, for an educational loan, for access to healthcare, you know, I mean, you name it, insurance premiums. And so this to me was a huge worry, and it was something that I was researching and kind of seeing that many of the products and the companies that we engage with use technology. They have automated a lot of decision making. So human beings are no longer really in the loop, and an algorithm will look at a set of conditions in your profile or in a set of consumers who you look most like, and then will, in an automated way, determine whether you are in or out of an opportunity, or what your interest rate will be, right? It will make predatory kinds of decisions less obvious because you entered your information into a form online, and in 30 seconds, you got a decision, and there’s no person involved. There’s no way to describe or mediate.
Vanessa: Right.
Safiya: You can’t, you’re not, even if you made a mistake, it would be hard to know. You don’t know if what they’re evaluating you against is actually fair or appropriate. We certainly know that most of these models are trained on historical data. So that means if you are, and this is one of the more famous cases, you know, that happened a few years ago when the Apple credit card was introduced, and all of a sudden, men were getting 10 times the credit limit that their wives were getting. And these were white men and white women. We know what happens with people of color in terms of like higher interest rates or the denials. And this story made national news. And the reason was because, and of course, these couples are calling up to Barclays Bank and they’re saying, “What’s going on?” I have the same exact tax, you know, taxes, credit profile. In fact, in many cases, the wives had better credit than the husbands. Like, explain to me why my wife got one-tenth of the credit that I got. And for me, this was like a perfect example of technological redlining. And the customer service reps, I mean, you know, bless them. They’re like, I don’t know, the algorithm decided, right? The computer said so, you know, they have no idea what’s happening because they’re not actually.
Vanessa: Not involved.
Safiya: They have no agency in this. The computer is deciding. And it turns out that, like we can imagine. Women didn’t get the right to have bank accounts and credit in our own names until the 90s. So if you train a model on all the historical financial data, but men have had financial data for a hundred years longer than women, guess what? The model is going to predict that the men are more reliable and more creditworthy and have more financial stability because of what? Patriarchy.
Vanessa: Right.
Safiya: — and sexism and discrimination against women in our society. And the systems do not account for that. So they don’t look at the history of discrimination against African-Americans or people of color or poor people or women, right? That is not a mitigating factor in the way in which the systems are programmed. And so this is one of the huge opportunities for me when we think about what technology could be in our society.
Vanesa: Right?
Safiya: You know, imagine if we had more pro-social, pro-human rights, pro-civil rights kinds of technology design that accounted for histories of discrimination and oppression, then we wouldn’t actually have the prevalence of technological redlining that we do, and at the rate that we have it now. And to me, these things are only getting increasingly worse. And I’ll tell you why. Because we have companies like Palantir, that are now taking your social media likes and dislikes and comments and cross-checking that and building a profile against every financial transaction. So anything you’ve ever done that wasn’t in cash, all your banking credit transactions, your GPS location, I mean, every data point that can be collected about you that is available, that can be scraped or bought by a data broker. Now we have these digital profiles that stand in as a proxy for us that the government and other kinds of companies are using. And this is so, so, so, and let me add to it just for listeners, that includes everything you’ve ever asked ChatGPT, every search you’ve ever done, things you think are private that you’re doing on the internet that are 100 % not private, okay? Anything you do on the internet is not private. Anything you’ve written on the internet, it’s written in ink.
So you have these digital profiles now that are standing in as a proxy for who we are. And this is the genesis of things like social credit, which the United States has been very critical of countries like China and their social credit. I mean, it’s like the episode of Black Mirror where you’re accumulating social credit points or losing them by how friendly or not you are or by whether you are well liked by others, how you’re judged, all of these things. I think that Black Mirror episode is actually not that far away. These dystopian kinds of futures, we are living them right now, and we’re really living them under this administration, this political administration at the federal level.
I say all this to say we have to think about other ways of living and doing and being that can resist these totalizing kinds of systems that are about technological redlining, or they are algorithms of oppression. And that might mean that we have a whole pivot. I mean, I can see, and I can believe in my own lifetime that we might have a radical pivot away from technology and doing things differently, very differently, because our sense of future and possibility is really dependent upon it. I mean, we don’t want everything from our past to over-determine our future, because when that happens, we have no redemption. We have no sense of agency. There is no ability to grow because anything you’ve done and who you’ve been in the past determines who you can be in the future. And what a horrible thought for children and young people whose brains aren’t even developed until they’re 28 years old, right? That everything they’ve done, I mean, we want a culture where we can make mistakes and we can recover, not that we are penalized and harmed by these kinds of technological decisions.
Vanessa: Yeah. And to your point about oppression, right? Some of the things that we’re being penalized for are not things of our own doing, right? And so like, I think that’s important to consider as well. As you were talking, I was thinking about this idea of culture and identity, right? The role that it plays in how we consume information. So to your point about the algorithm, if this is the profile, then these are the things that I’m going to feed you, right? Like if I believe that you love cats who have had seven out of nine lives, I’m going to feed you into this group. And so thinking, trying to like filter the idea of the way the algorithm works and considering communications and storytelling, wanting to say to folks who are listening to this, possibly consider where are the places that you can be telling your stories. Like if you feel that these various platforms are valuable because maybe you found responsiveness or were able to connect to people, to think about where the points of identity are where you can find like-minded people to maybe tell those stories and to begin to build community, right? To strengthen where your cause is shared and how people understand it. And so, yes, there is probably some value in sort of like, hey, I want to share this everywhere on every platform, but there probably is more meaning in sharing it and trying to identify what the points of interest are? Like where do the interests intersect between people who care about education equity and maybe, you know, a poverty conversation or a hunger conversation, and trying to bring those things together and seeing if that is a way to grow understanding and strengthen the activism behind a particular cause. And so from a storytelling perspective, I know that culture and identity are a really important place to be thinking about how stories are told and what the connection is and that connective tissue. I just wanted to bring that up.
Safiya: Yeah.
Vanessa: For my last question for you, which probably means I’m gonna be begging you for a part two on another season. But my last question to you is with the understanding that 95 % of the audience of this podcast are people who are working on some of the most pressing issues of our time, knowing that story is important, that narrative is important.
What advice, if you were sitting in an auditorium of communications, leaders, or CEOs of foundations and nonprofits, who are like, we don’t want this story to die. We want people to understand why we’re continuing to advocate for these things. What advice would you give to them based on everything you know and understand about the platforms, what’s at risk, and what the opportunities are? What would your sort of remarks in abbreviated form be?
Safiya: Yes. Well, I think for investors, people who have resources to help us build alternatives, this is your moment to stand up. We need you. We need deeper investment in pro-social rights protecting kinds of technologies and communication systems. So, you know, I think people should be taking very seriously moving to end-to-end encrypted kinds of technologies, privacy-protecting technologies like Signal. I think more of that kind of old school one-to-many, one-to-one group kinds of communications in protected space right now. That is just a requirement of the times, especially for sensitive stories that you want to make sure people understand. I think that
In-person events and connections are very important, especially when you can. And I know that that is a luxury that isn’t always viable. People need to be joining organizations, and we need to be strengthening organizations, joining unions, making unions, bargaining around AI and how much it can encroach upon a workplace or upon workers.
And these are things like strengthening those kinds of associations, which also allow people to tell their stories about what’s happening to them around their work or the devaluation of their labor. And that’s a very important conversation we should be having right now in this moment of, you know, like a push to automate labor and employment. Workers need to be able to feed ourselves.
And we should be very, very involved in anywhere that technology seeks to replace us or displace us. So those are conversations and stories that need to be facilitated and shared across industries and occupations because we have lots of stories to tell each other and successes as well that we can learn from people who have been able to hold some of these projects at bay. I also think that you know, we underestimate the power of the local. You know, technologies really tried to convince us that everything is global and that all communication, you know, this one to millions kind of thing, but really, problems get solved locally in many, many, most cases. I mean, we have to figure out how to put all of our eggs.
Vanessa: Mm-hmm.
Safiya: Let’s just say that the digital basket is pretty dangerous when you live, like I live in California, I live in Los Angeles, we are in a climate crisis in California. I just got an announcement from the city yesterday that we may have blackouts. So guess what? When we have power blackouts, there’s no digital anything happening. So I’m always thinking about like, what does it mean to know my neighbors, for us to be connected, to be able to help the elders in our community and our block and our neighborhood?
Vanessa: Mm-hmm.
Safiya: When the power goes out, or when there’s a need. So we need to cultivate a range of skills, okay? A range of networks, a range of access points to each other. They can’t all just be digital because the minute the power goes out, we got nothing. The minute the cell tower goes down, we got nothing. So those things to me, I mean, I’m not a prepper by any stretch, but I do know that the incredible energy intensity of things like generative AI mean that we should absolutely be expecting more environmental catastrophe as data centers roll into black neighborhoods in Memphis or neighborhoods in Austin or South America and all the places where these companies have to put their dangerous polluting systems in order to make them work. And we are gonna have to respond to those things as people are. And I think that having networks, being in organizations, I don’t know, I’m very Gen X. Like remember in the 90s when we just said we were gonna meet up at this time and place and we did, I don’t know. There’s lots of parties. Let’s not lose that as one of our tools in our toolkit of living and being human, which is finding each other, making connections, doing work together—
Vanessa: Can we try that again, please?
Safiya: —and everything doesn’t have to have a digital trace connected to it.
Vanessa: So I think that one of the things that I’m pulling from that and will certainly share with our clients is using story to create connection, right? Like, how do we call people in or remind people of what it was like when we did sort of come together in that way?
Thank you so very much for this conversation. I think that what you shared was really enlightening and timely for a lot of the conversations that we’re having with our clients and others in the industry to think about how do we sort of push through some of the counter narratives that are coming up against things that we know to be true and important for you know, people to have the freedoms that we’re supposed to have under our democracy. And so, really wanting to give people another opportunity to explore some of those challenges, but also a way forward. And so thank you, thank you, thank you. And yeah, I just want to say thank you for your work. Thank you for your time. And I look forward to us chatting again in the future.
Safiya: So thanks for having me. It’s really been a pleasure.
Vanessa: Now, this episode is how one launches after a long hiatus. Thank you, thank you, thank you. There were bombs dropped and just incredible insights shared in this conversation. So we covered a lot of ground, but the one thing that stands out to me, well, there are a lot of things that stand out to me. One of the things that I feel like I will be exploring and thinking through for a bit after the conversation is this idea of what Safiya described as a radical pivot away from technology and doing things differently. Like, what would happen if we did not orient most of our storytelling around technology, right? We’re on the platforms, trying to reach as many people as we can. But what if we were not so reliant on that approach and instead thought of some of the more traditional ways to gather and be in community and connect, right? Will that allow us an opportunity to strengthen the narrative, to make it more resilient, to educate and create awareness about the issues that are important right now? What are the benefits of doing a more traditional, old school approach to narrative building and narrative sharing, versus what we’re doing today? Like is a return to yesteryear right now, like the way to go. I don’t know, I’m thinking about some of the projects that we are working on and some of the things we’re doing with communications and thinking about clients, like how amazing it could be. And yes, it will be a lot of work, but what’s the gain here? And can we sort of cover more ground and offer more support if we have more control over the way narratives are told? We’re shaping them, and then we have a direct line to who is receiving it and how they’re receiving it without any sort of interference from outside sources. Just something to think about.
This was our first episode. Next episode, we will be joined by Tracie Powell, the founder and CEO of the Pivot Fund. So tune in again next week for another exciting episode of the Social Change Diaries.