AI is everywhere. You can’t avoid it. Whether you do an internet search, ask it for advice, use it at the office, or simply scroll through your social media and news feeds – it’s always there.
It’s moving fast, with far-reaching consequences for society. The “good stuff” from AI could lead to unimaginable wealth and abundance. But if AI goes rogue, it could wipe out humanity as we know it, some AI industry leaders have publicly said.
As the founder of the AI development platform Miriel, Political Economy Alumnus Andy Barkett has spent a lot of time thinking about AI’s impact on humanity’s existence, jobs, wealth inequality as well as government regulation and artificial general intelligence.
Barkett, who was also a former engineering manager at Google, Nvidia and Facebook, told Berkeley Social Sciences what he thinks about the risks AI poses; where he thinks AI is headed; what people are getting wrong about the technology; and how college students should prepare for an AI world. This interview was edited for clarity.
Should we worry that there is a 10% chance that AI will kill us all?
Andy Barkett: I think assigning a specific probability to something like that is misleading. There are certainly real risks associated with AI, including potentially existential ones. But humanity has been through exactly zero AI revolutions. We simply don’t have the historical experience or empirical foundation to calculate a meaningful probability.
The range of possible outcomes is extraordinarily broad. It includes everything from unprecedented prosperity and scientific breakthroughs, to serious economic and geopolitical disruptions, and even extreme scenarios involving autonomous weapons or nuclear conflict.
Rather than focusing on an arbitrary percentage, I think it’s more useful to recognize that we’re entering a period of extraordinary uncertainty, perhaps unlike anything we’ve experienced in our lifetimes. And uncertainty cuts both ways: the possibilities for positive transformation are just as important to consider as the risks.
What are some other scenarios of AI wreaking havoc on society?
Andy Barkett: I’m actually more concerned about the potential for AI to amplify problems we already face than about some science-fiction scenario in which AI independently decides to destroy humanity.
For example, we’ve seen declining confidence in governments and institutions over the past couple of decades. AI could accelerate that trend by making it harder to distinguish reliable information from misinformation, or by disrupting economic and political systems faster than those institutions can adapt.Economic dislocation is another possibility. If technological change outpaces our ability to adjust, we could experience periods of serious unemployment, instability, or even economic depression.
But it’s important to distinguish between disruption and catastrophe. The Industrial Revolution brought enormous social and economic upheaval, but it also laid the foundation for unprecedented improvements in living standards. The challenge is to manage the transition effectively so that we capture the benefits of AI while minimizing the costs.
Do we need the government to regulate AI?
Andy Barkett: Absolutely, but I would approach regulation differently from much of the current public debate. I think the idea that we can simply slow down AI development or tightly control who has access to the technology is largely misguided. The technology is advancing rapidly, it’s becoming more accessible, and much of that progress is happening across national borders.
The more important question is how we establish accountability in a world where AI systems increasingly participate in decisions and actions. For example, AI challenges some of our most fundamental assumptions about agency and responsibility. When an AI system makes a decision, provides misleading information, or takes an action that causes harm, who is accountable? The developer? The company deploying it? The person instructing it?
Those are questions that governments and legal institutions need to address. I would much rather see us focus on establishing clear rules around responsibility, liability, transparency, and accountability than trying to prevent technological progress. We should be building institutions capable of functioning in an AI-driven world, not simply trying to delay its arrival.
Are you worried about recursive self-improvement (AI improving itself) and the road to artificial general intelligence (AGI)?
Andy Barkett: Not particularly, at least not in the way these issues are often presented. I think we need to be much more precise about what we mean by general intelligence. Intelligence isn’t just the ability to solve problems or recall information. In humans, it also involves motivations, goals, perception, and the ability to interact with and learn from the physical and social world.
Today’s AI systems have become extraordinarily capable at certain kinds of reasoning and problem-solving. In some domains, they’ve already surpassed human performance. But that doesn’t mean they possess the full range of capabilities we associate with human intelligence.
So if you define intelligence narrowly as performance on certain cognitive tasks, we’ve already crossed some remarkable thresholds. If you define it more broadly, as the integrated set of capabilities that characterize human intelligence, there’s still a great deal of development ahead. I think recursive self-improvement is an important technical possibility, but I don’t see it as inherently synonymous with existential danger.
Will AI take our jobs away?
Andy Barkett: I think AI will fundamentally transform employment, but I’m much more optimistic about the long-term outlook for jobs than many people are. So far, we haven’t seen the kind of widespread employment collapse that some predicted. And historically, technological revolutions have tended to create new categories of work even as they eliminate existing ones.
I expect AI to follow a similar pattern, although the transition could be unusually rapid and disruptive. The important distinction is between jobs as we currently define them and the broader concept of productive human work.
Many of today’s occupations will change dramatically, and some will disappear altogether. But entirely new fields, industries and forms of employment will emerge — some of which we can barely imagine today. When people predict mass unemployment, they often start with an existing occupation, identify the tasks AI could automate, and assume the job disappears. What they miss is everything that becomes possible when those tasks become dramatically cheaper or easier to perform.
AI won’t simply replace existing work. It will make entirely new kinds of work economically viable. I wouldn’t assume that the transition will be painless, but I think there’s good reason to be optimistic about the long-term potential for employment and human productivity.
How should UC Berkeley students prepare for an AI world?
Andy Barkett: I think this is one of the most exciting times in history to be a young person entering the workforce. For the first time, individuals have access to tools that dramatically expand what they can learn, build and accomplish independently.
You can teach yourself about microfinance, learn how to operate a hotel, build software, develop a business plan, or explore an entirely new field without waiting for someone to give you permission or hire you. That changes how I think students should approach their education and early careers.
Traditionally, students have focused on acquiring credentials and finding internships at established organizations. Those things still have value, but AI creates an entirely new opportunity: demonstrating your abilities by actually solving problems. You might help a museum attract more visitors, improve a restaurant’s online ordering system, or develop a tool that makes a small business more efficient.
You don’t necessarily need years of technical training or a large team to accomplish something meaningful anymore. And when you approach a potential employer, you’re not just presenting a résumé that says you studied a subject or participated in an internship. You’re showing them something you actually built and a problem you actually solved.
I think universities have a tremendous opportunity to facilitate this kind of experiential learning by connecting students with real-world problems and giving them the tools and support to address them. To me, that’s one of the most empowering aspects of AI. You no longer have to wait for an institution to decide you’re qualified to do something. You can go out and demonstrate that you are.
Could AI actually make wealth inequality worse?
Andy Barkett: Absolutely. But it could also reduce inequality, and I don’t think we have enough evidence yet to know which effect will dominate.
On one hand, AI gives people access to capabilities they might otherwise lack. Someone with a great business idea but limited technical expertise can now build software. Someone without a financial background can analyze business opportunities. Someone who can’t afford a team of specialists can accomplish things that previously required substantial resources. That’s potentially an enormous equalizing force.
But there’s another dynamic operating in the opposite direction. At our company, for example, we’ve found that our strongest software engineers often get the greatest productivity gains from AI. They already understand the underlying problems, so they’re particularly effective at directing and evaluating the technology.
In that sense, AI can amplify existing advantages. People who already have valuable skills, knowledge, or resources may become disproportionately more productive. So there are two competing forces: AI democratizes access to expertise and capabilities, but it can also magnify the advantages of people who already possess them.
Which effect ultimately dominates will depend partly on how the technology develops, but also on how effectively our educational, economic and political institutions adapt. I think that’s one of the most important questions for the political economy in the coming decades.