Discover more from Navigating AI Risks
#2 - Regulate or Compete? The China Factor in U.S. AI Policy
Greetings, and welcome to the 2nd edition of Navigating AI Risks, where we explore how to govern the risks posed by transformative artificial intelligence.
Starting from the latest developments in AI industry, governance, and policy, we reflect on the important questions surrounding the transition to a world with advanced AI.
This week, we’ll talk about the China argument in US AI policy debates, “God-like AI,” the EU’s latest moves, and more. Let’s jump into it!
Regulate or Compete? The China Factor in U.S. AI Policy
Lead author: Charles M.
The modern story of tech regulation in the United States is a story about China. Worries that regulation may damage the US's technology leadership vis-à-vis China have often been cited as a reason not to regulate. This concern played an important role in debates on privacy rules, breaking up Big Tech companies, and now AI.
This trade-off between regulation and technological preeminence seems obvious. But the situation is more complicated. The preservation of AI leadership over China need not come at the cost of effective AI governance.
There are two key variables at play here:
The desirability of regulation and its impact on innovation: Are the risks posed by AI significant enough to justify additional regulation? Will such regulations impede innovation?
Whether the United States is on course to maintain (or increase) its lead over China: Is the US AI lead over China large enough to provide sufficient reassurance that (innovation-stifling) regulation would not place them at a competitive disadvantage?
These multifaceted and critical questions merit thorough consideration. Is it justifiable for the US to avoid regulating AI because that may allow China to become the world's leading AI powerhouse? Here are some tentative observations:
Regulation can promote and shape innovation: In the 1960s, the automotive industry saw “a host of new or stronger safety requirements [that] led — often after stiff opposition — to new technologies like airbags, antilock brakes, electronic stability control and, recently, rearview cameras and automatic braking.”A similar dynamic was at work when liability for fraudulent credit card transactions shifted from consumers to payment networks in the 1970s. Innovation is not unidirectional; regulation can help shape it in desirable directions.
Increasingly powerful systems are rendering existing laws insufficient: The increasing performance, diverse abilities, and potential for high consequences misuse of general-purpose AI systems makes it increasingly urgent to put in place guardrails on the underlying model (e.g. GPT-4) and the company that develops it (OpenAI), rather than only its specific applications (ChatGPT), or use cases (writing an essay), which are often already regulated by existing laws. A broad range of stakeholders now agree that the pace of AI progress is creating a real need for guardrails (including the CEO of the world’s leading AI lab, OpenAI). Many of these calls come from a growing recognition that a race to the bottom is undesirable. When do the risks posed by frontier general-purpose AI become large enough to warrant regulation, even if it stifles innovation?
Regulate for better regulation: Mandating algorithmic audits, or at least reporting and transparencyrequirements would allow policymakers more insights into how these models work, and use these insights to create carefully targeted regulation that minimize AI risks while seizing its benefits. These procedures will have a minimal impact on the pace of research and development.
Regulate to avoid large-scale accidents: An increasing number of experts worry about large-scale accidents, going up to human extinction. Microsoft’s Bing threatening users is a low-stakes example of an issue called “goal misgeneralization”, when an AI “system competently pursues the wrong goal”. This challenge is just one of many obstacles in making advanced systems do what we want them to do. Much ink has already been spilled by the AI safety community on the topic, but progress remains slow. Many worry that if companies trying to develop what AI investor Ian Hogarth calls “God-like AI”, are not able to avoid current accidents, they will almost certainly not be able to solve future, potentially existential issues that arise with more powerful systems (which may come sooner than we think, warns MIT professor Max Tegmark)
Regulation facilitates adoption: People and companies won’t widely adopt AI until they trust that advanced systems are not biased, unsafe, or inaccurate. As shown by the wide array of failed AI model releases, commercial incentives don’t seem to be enough to ensure sustainable efforts on making systems aligned with ethical principles. Targeted regulation can also help avoid disasters, and thus ensure long-term public buy-in for the technology. For instance, insufficiently enforced safety rules were a key reason why the 2011 tsunami in Japan led to an incident at the Fukushima power plant. The disaster led to public backlash and dealt a devastating blow to the country's nuclear industry.
Regulation can also unnecessarily hurt innovation. For instance, setting uniform rules for both small models like Stable Diffusion or BERT, and very large models like the one behind ChatGPT, could lead to excessive regulation of models with minimal risks and stifle innovative advances. Badly designed regulations can lead companies to avoid pursuing potentially beneficial & innocuous AI advancements to avoid legal repercussions. That could be net harmful for the industry's competitiveness and citizens' welfare alike. But these are features of ill-conceived regulations, not regulation in general.
So: there’s a growing consensus that we need guardrails to prevent the misuse and accident risks of increasingly powerful AI systems; and regulation can promote both AI adoption and innovation. Still, the prospect of China gaining a competitive advantage might not convince everyone that regulation would be a net good for the American public and national interests. Would it be convincing if China still had a lot to do before it catches up?
Can China surpass the US?
China's efforts reflect its ambitious goals: China seems determined — its 2017 New Generation AI Development Plan set the goal of becoming the 'major AI innovation centre in the world' by 2030 — and dedicates huge sums of public spending to that end. The country has well-developed science & technology intelligence and tech transfer strategies. Its population size allows the country’s government and companies to access a large amount of data, and its large domestic market creates wide opportunities to commercialize AI products and services. Chinese citizens are also enthusiastic about AI, which may facilitate adoption; 78% of them agree that “products and services using AI have more benefits than drawbacks,” compared with 35% of US citizens. Overall, China has come a long way and is now an innovation and science superpower. Does it mean it’s close to the US in leading edge AI, though?
China remains behind the US: Assessments by AI researchers, investors, and CEOs generally see Chinese firms as behind US firms, often by a lot. Most national AI innovation and commercialization indexes show the United States ahead, especially on two crucial foundations of AI innovation, talent and investment. Finally, US export controls on advanced AI chips all but guarantee that China won’t have access to the semiconductors it needs to train the next generation of AI systems. As a (not necessarily anecdotal) example, ChatGPT’s most performant alternative in China is simply subpar.
But the autocratic advantage is overestimated: The characteristics of China’s political system may also afford it structural advantages. AI and autocracy may be mutually reinforcing, creating a synergy whereby “new technology bolsters autocratic power, and autocratic demand for technology stimulates innovation across sectors” (AI-autocracy mutual reinforcement argument). In addition to its large population, the Chinese government has access to the industrial and personal data harvested by the private sector. In turn, it may be easier for China to access more data than democracies (data advantage argument). But it seems like these 2 arguments apply mostly to facial recognition and other AI fields that underpin surveillance; less so to other segments of the AI industry. Finally, only 1.4% of the Internet is in Chinese - compared with 60.4% in English. Because the Internet is now the dataset for the most powerful models, useful Chinese language texts may be a bottleneck.
China doesn’t prioritize AI leadership at all costs: A 2022 regulation requires companies to submit security assessments of recommendation algorithms to the government and to disclose the datasets used; its proposed generative AI rules would impose stringent content restrictions coming from ChatGPT-like applications, and make AI companies liable for this. In 2021, Xi Jinping's regulatory overhaul of China's tech industry resulted in a total loss of $1 trillion in total market capitalization. Journalist Ezra Klein said it well: “China seems perfectly willing to cripple the development of general A.I. so it can concentrate on systems that will more reliably serve state interests”.
Innovation is insufficient by itself: Research and development is not the only important factor in AI competition. The diffusion of AI throughout the economy and society is another key pillar of technological power. Simply having a lead in research and development doesn't guarantee long-term success. You also need a sufficiently large pool of skilled engineers capable of deploying the latest innovations across sectors of the economy. Seen through this prism, “China is far from being a science and technology superpower” compared with the United States.
The US has a well-developed network of alliances: Most other AI powerhouses, such as the UK, Japan, or Israel, work closely with the US. In comparison, China is more isolated. The US could use its network to coordinate on export controls, conduct international AI R&D projects, or pool its resources. Even if China came dangerously close, the US would have many ways to act quickly to maintain its lead. There are other, more effective ways than an unregulated AI sector to delay China’s lead and grow the US’ own.
The US hasn’t lost its edge. America may believe to be in a tight race, when it is not. Its AI lead seems large enough to take the time to set up guardrails around this emerging technology. Even if it stifles innovation (which can very well be avoided if carefully designed), regulation is needed to mitigate AI risks.
‘Regulate or compete?’ may be a false dilemma. The United States doesn’t have to choose between maintaining AI leadership over China and hardwiring both safety procedures and democratic values into its AI governance ecosystem. It should do both.
In the Loop
Lead Author: Henry Papadatos
In a recent Financial Times article, Ian Hogarth, writer of the State of AI 2022 report and AI investor in more than 50 companies, details the concept of "God-like" AI – an extremely capable, autonomously learning computer with the ability to comprehend its environment without supervision. Though such technology may be distant, its arrival is difficult to predict and some believe that it could arise in the next decade. That’s because recursive self-improvement (AI systems making better AI systems) might be technically feasible, and even probably, a few years away from now, as the AI company Anthropic is pitching to its investors. The rapid development of AI till now can be attributed to the exponential increase in computing power. The past decade has seen a 100-million-fold increase in computing power thrown at AI models, with even larger models expected to emerge due to recent hardware developments and unprecedented monetary incentives.
Advanced AI models present significant risks, such as enhancing cyber-warfare capabilities, making it easier to build bioweapons, and allowing better misinformation campaigns. Some risks are already arising with current AI systems. A recent case in Belgium involved an individual committing suicide after conversations with a chatbot. These potentially hazardous models are primarily controlled by private companies without regulatory oversight or government intervention.
To diminish the risks associated with "God-like" AI, exceptional collaboration between research labs and nations, as well as proactive regulatory measures, will be necessary. This entails developing an unusually high degree of political will and urgency to control and oversee such AI advancements. As models developed next year could use up to 100 times the computing power of current models, new emergent capabilities are expected. This scenario presents a window of opportunity for governments to assume control through 2023 by regulating access to cutting-edge hardware.
AutoGPT: an autonomous AI agent
AutoGPT is an open-source project utilizing ChatGPT and GPT-4 to create an autonomous agent capable of performing tasks independently. It works by dividing complex tasks into smaller sub-tasks and then using different instances of large language models such as GPT to tackle them.
Recent statements by Sam Altman that the scaling era is over suggest that OpenAI is likely going down the same route.
From an AI risk management perspective, these architectures are not good news because they might foster big discontinuous jumps in capabilities, thus giving less time to developers to make sure that their systems are safe.
One open source project, “ChaosGPT”, assigned to an autonomous agent the task to "destroy humanity". Its first sub-task was researching the most powerful atomic weapons in history.
Although AutoGPT is not currently very powerful, it demonstrates how quickly some people use AI for harmful applications. This emphasizes the need for caution as general-purpose AI models continue to grow.
EU launches a research center on algorithmic transparency
This new center, part of the European Commission, will focus on deciphering algorithmic black boxes, positioning itself as an international research hub.
Its mission is to translate the EU’s comprehensive digital legislation into practical enforcement against some of the world's most influential companies, now mandated by the EU’s Digital Services Act to undergo “algorithmic accountability and transparency audits.” Its scientists and experts will “analyze transparency, assess risks, and propose new transparent approaches and best practices.”
For instance, the center will aim to provide evidence of discriminatory results produced by AI-based recommender algorithms, ensuring compliance with EU digital standards.
Provisional agreement on the EU “Chips Act”
A provisional political agreement was reached between the European Parliament and the Council regarding the regulation to bolster Europe's semiconductor ecosystem, commonly referred to as the "Chips Act."
The agreement aims to create an environment that fosters industrial growth, ultimately doubling the EU's global market share in semiconductors from 10% to at least 20% by 2030.
Chips for Europe Initiative: The initiative, as part of this agreement, is expected to mobilize €43 billion in combined public and private investments, with €3.3 billion contributed directly from the EU budget.
EU: The European Parliament has reached a deal on their position on the EU’s AI Act. There will be a formal vote in the coming weeks. This followed a call to action on very powerful Artificial Intelligence by MEPs Dragos Tudorache and Brando Benifei, co-rapporteurs on the EU Artificial Intelligence Act.
Industry: Social websites Reddit, Twitter, and StackOverflow are all starting to charge for access to their data and API, notably to avoid giving it away for free to AI companies that want to train their models.
US: Senate Majority Leader Chuck Schumer announced plans to regulate AI.
EU: Europe’s data protection authorities have created a taskforce to discuss enforcement of the EU’s GDPR on OpenAI and ChatGPT.
US: In a sign of technology’s growing importance for geopolitics, the US State Department “plans to add tech experts to every embassy by next year”.
US: Over 200 companies have already signaled their desire to receive funding as part of the US Chips Act.
EU: The President of the European Commission, Ursula Von der Leyen, announced the release of an EU economic security strategy in the coming months, notably “to look at where there are gaps in our toolbox, which allow the leakage of emerging and sensitive technologies through investments in other countries” (read: China).
Industry: One less, one more: Alphabet announced that one of its subsidiaries, DeepMind, and Google’s AI research team, Google Brain, would merge to form a single entity, Google Deepmind. Elon Musk announces the creation of his own AI company, X.AI.
UK: British parliamentarian Darren Jones, chair of the Business and Trade Select Committee, sent a letter to the UK secretary of state for Science, Innovation and Technology to call for the country to “play a fundamental role in establishing a new intergovernmental organization for the safe and secure development of advanced artificial intelligence”.
What We’re Reading
Emergent autonomous scientific research capabilities of large language models (Boiko et al. 2023)
How to design an AI ethics board (Schuett et al. 2023)
Regulatory Markets: The Future of AI Governance (Hadfield & Clark)
That’s a wrap for this second edition. You can share it using this link. Thanks a lot for reading us!
— Siméon, Henry, & Charles.
Be at the forefront of AI governance. Subscribe below and never miss one edition.
(If you want to meet us, you can book a 15-minute call with us right here.)
During a hearing before the U.S. Congress on the topic of data privacy rules, then-Facebook CEO Mark Zuckerberg said that “we still need to make it so that American companies can innovate in areas” like facial recognition, “or else we’re going to fall behind Chinese competitors”. As documented by the AI Now Institute, a think-tank argued in 2021 that the “U.S. Congress should ensure any change to federal data privacy legislation does not limit data collection and use of AI.”
In a 2021 white paper entitled ‘National Security Issues posed by House Antitrust Bills’, the CCIA, a trade association composed of members such as Google, Amazon, Uber, Meta, or Apple, cited “Undermining U.S. tech leadership” as a key risk of those proposed antitrust bills.
Notably by former Google CEO Eric Schmidt, who is influential in both tech industry and national security circles, as reported by Protocol: “‘Why don’t we wait until something bad happens and then we can figure out how to regulate it — otherwise, you’re going to slow everybody down. Trust me, China is not busy stopping things because of regulation. They’re starting new things,’ he said during an interview last year. Schmidt reiterated his antiregulation stance in October when the White House unveiled a nonbinding ‘Blueprint for an AI Bill of Rights.’”
This point was hinted at by US senator Chuck Schumer, who recently came forward with plans to regulate AI: “We don’t want to let China get ahead of us, but at the same time, we’ve got to make sure there’s safety and protection.”
A book on automotive safety in the 1960s had a large impact in terms of public and policy perception. A few years later, the US government created the National Highway Traffic Safety Administration: “The book had a seminal effect,” Robert A. Lutz, who was a top executive at BMW, Ford Motor, Chrysler and General Motors, said in a telephone interview. “I don’t like Ralph Nader and I didn’t like the book, but there was definitely a role for government in automotive safety.” Maybe, in a few years, we’ll see this in the news: “The letter had a seminal effect,” said Sam Altman, CEO of OpenAI. “I don’t like the Future of Life Institute and I didn’t like the open letter, but there was definitely a role for government in AI safety.”
New York University professor Gary Marcus has more examples: “Regulation is NOT always bad for technology, e.g., regulation around the environmental impact of cars has spurred advances in electric cars; rising fuel standards have had a positive impact as well. A 1966 US Army restriction on fixed-wing aircraft for airlifts presumably inspired innovation in helicopters, etc.”
“In the early 1970s, the fledgling credit card industry routinely and shortsightedly held cardholders liable for fraudulent transactions, even if their cards had been lost or stolen. In response, Congress passed the 1974 Fair Credit Billing Act to limit cardholder liability. This protection increased public trust in the new payment system and spurred growth and innovation. Because they could no longer just pass fraud losses on to cardholders, payment networks devised one of the first commercial applications of neural networks to detect out-of-pattern card usage and reduce their fraud losses”. (source)
Shaping the direction of innovation is a key element of ‘differential technological development’, a strategy where risky or harmful technologies are delayed compared with risk-reducing or otherwise beneficial technologies. For example, higher carbon taxes lead to “lower levels of innovation in high-emissions technologies and higher levels of innovation in green technologies”Differential technology development: A responsible innovation principle for navigating technology risks
Ezra Klein talks about “alignment risk”, “the danger that what we want the systems to do and what they will actually do could diverge, and perhaps do so violently. Curbing alignment risk requires curbing the systems themselves, not just the ways we permit people to use them”.
Branches of law that apply to AI systems are numerous; they include intellectual property law, privacy law, internet law, product liability law, etc. Recently, officials from several U.S. government departments agencies, including the Federal Trade Commission and the Department of Justice released an announcement reiterating that “Existing legal authorities apply to the use of automated systems and innovative new technologies.”
This includes the following: “When Meta released its large language model BlenderBot 3 in August 2022, it immediately faced problems of making inappropriate and untrue statements. Meta’s Galactica was only up for three days in November 2022 before it was withdrawn after it was shown confidently ‘hallucinating’ (making up) academic papers that didn’t exist. Most recently, in February 2023, Meta irresponsibly released the full weights of its latest language model, LLaMA. As many experts predicted would happen, it proliferated to 4chan, where it will be used to mass-produce disinformation and hate.”
An independent Commission found that “the causes of the accident had been foreseeable, and that the plant operator had failed to meet basic safety requirements such as risk assessment [or] preparing for containing collateral damage“. The Prime Minister of Japan at the time said the disaster “laid bare a host of an even bigger man-made vulnerabilities in Japan's nuclear industry and regulation, from inadequate safety guidelines to crisis management, all of which he said need to be overhauled.“ Another example is the low passenger numbers following the 2003 Concorde crash, a key reason why the airliner was retired. And this, despite the otherwise perfect record of the Concorde, with 0 deaths in the more than 20 years that preceded the crash.
This is why it’s extremely important to have a very precise definition that only targets the riskiest models.
“Training complex AI systems is not easy. OpenAI is ahead of its US competitors (including Google and Meta), and developers in China and other countries also lag behind. It’s unlikely that “rogue groups” or governments will surpass GPT-4’s capabilities in the foreseeable future. Most AI talent, knowledge, and computing infrastructure is concentrated in a handful of top [US] labs,” says AI researcher Rodolfo Ocampo.
“US and US-allied sanctions on advanced semiconductors, in particular the next generation of Nvidia hardware needed to train the largest AI systems, mean China is not likely in a position to race ahead of DeepMind or OpenAI,” says AI investor Ian Ogarth.
According to Conor Leahy, CEO of AI lab Conjecture, Chinese tech giants are irrelevant and probably won't catch up with US Big Tech Companies and large AI labs.
See, among others, the Emerging Technology Observatory’s Country Activity Tracker, the Stanford Institute for Human-Centered AI’s Global AI Vibrancy Tool, and ASPI’s Critical Technology Tracker. Keep in mind, however, that indexes are very imprecise tools for making such complex assessments (pp. 26-27).
According to Paul Scharre, director of studies at the Center for a New American Security, “China produces the most top AI scientists in the world. More researchers publishing in top AI conferences come from China than any other country. But they don’t stay in China. Over half of China’s best undergraduates studying AI come to the U.S. for graduate school. And they stay. Over 90 percent of Chinese undergraduates who move to the U.S. for their PhD stay in the U.S. after graduation. The biggest beneficiary of Chinese talent isn’t China—it’s the United States.”
In the long-term, however, export controls might incentivize the creation of a US-free semiconductor supply chain and lead China to double down on its self-sufficiency agenda. See CSET’s report on ‘Decoupling in Strategic Technologies’ for historical lessons on export controls and their implications for AI.
China may have other advantages because of its autocratic nature, though these arguments rest on more tenuous ground. Some see the ability of autocracies to plan long-term and to control the direction of the economy as an advantage in competition for AI. Others argue that collectivist cultures like China are by nature less inventive than individualist ones like in the west. Finally, other structural features, this time less related to political regime type, are that the Chinese economy faces some dampening economic prospects, including because of its aging population, as well as housing and debt situations.
It may also have a ‘deployment advantage’, as the country faces fewer constraints than democracies in deciding which technologies get rolled out and for what purpose.
Per Paul Scharre: “Data on facial recognition doesn’t necessarily help you in other areas. One of the arguments in favor of China’s alleged data advantage is that China is a larger country with a much larger user base. That’s true. But what probably matters much more is the user base that tech companies have, and U.S. tech companies have global reach. [...] The conclusion is counter to what a lot of people might initially think — I don’t think that China has a data advantage”; furthermore, certain technical advancements presented by Tim Hwang that reduce the importance of data for ML suggest that these advantages may be eroded.
Though this is not a shopping list, and the second-order consequences of taking such actions have to be carefully considered, this includes “export and import controls, inbound and outbound investment restrictions, telecommunications and electronics licensing regimes, visa bans, financial sanctions, technology transaction rules, federal spending limits,” most of which can have immediate effects on China’s AI industry.
Similarly to the point made above that regulation can help influence the direction of innovation, one such way to shift the dynamics that underpin US-China tech competition is through Tim Hwang’s “terrain strategy”, which consists in shaping “the direction of [AI] to provide structural advantages to itself and other democracies. This effort involves accelerating the development of certain areas within machine learning (ML)—the core technology driving the most dramatic advances in AI—to alter the global playing field.”
For sure, there remain some barriers to beneficial and robust AI regulations. For one, technical standards are not yet operational and/or designed. The pace of technological progress is a constant challenge for policy, as EU policymakers are experiencing as generative AI challenges the rules of its AI Act. As lamented by US sociobiologist E.O. Wilson, humans have “Paleolithic emotions, medieval institutions, and god-like technology.” But several sound policy proposals would help update the legal environment within which AI operates. Ezra Klein’s set of key issues to resolve is a good roadmap, as are policy recommendations by the AI Now Institute or the Future of Life Institute.