ClearFactAll topics

Should AI companies be required to conduct impact assessments before deploying large-scale models?

7 peer-reviewed studies for, 5 against

Key takeaways

Stronger evidence: For

Verdict

The evidence supports requiring AI impact assessments before large-scale deployment, though implementation details matter significantly. Research has documented concrete harms from deploying AI systems without adequate evaluation—facial recognition systems with dramatic accuracy disparities across demographics, language models amplifying societal biases, and autonomous systems facing ethical dilemmas affecting public safety. The Gender Shades study demonstrated that commercial systems shipped with error rates 34 times higher for some groups, harms that pre-deployment testing would have identified. The Stochastic Parrots paper outlines systematic risks from large language models that scale with deployment. While economic research validly warns that excessive regulation could slow innovation and shift development to less regulated jurisdictions, this argues for well-designed assessments rather than no assessments. Impact assessments need not be prohibitively burdensome—environmental impact assessments and drug approval processes show regulatory review can coexist with innovation. The key is proportionality: lightweight assessments for low-risk applications, rigorous evaluation for systems affecting employment, healthcare, criminal justice, or other high-stakes domains. Voluntary industry standards have proven insufficient given competitive pressures to deploy quickly. Mandatory assessments create level playing fields and build public trust essential for long-term AI adoption.

Strongest study on each side

Ranked by citation count.

Strongest study for

The social dilemma of autonomous vehicles (Science, 2016)

Jean-François Bonnefon, Azim Shariff et al. · 1,292 citations

Study demonstrates AI systems face complex ethical tradeoffs with significant public safety implications, supporting requirement for systematic impact evaluation before deployment of autonomous systems affecting human welfare.

Strongest study against

Open the interactive page →

The evidence

For (7)

The social dilemma of autonomous vehicles (Science, 2016)

Jean-François Bonnefon, Azim Shariff et al. · 1,292 citations

Study demonstrates AI systems face complex ethical tradeoffs with significant public safety implications, supporting requirement for systematic impact evaluation before deployment of autonomous systems affecting human welfare.

Algorithmic Impact Assessments and Accountability: The Co-Construction of Impacts (FAccT, 2021)

Jacob Metcalf, Emanuel Moss et al. · 735 citations

This peer-reviewed conference paper by Metcalf et al. examines how algorithmic impact assessments (AIAs) can establish accountability for AI systems. The authors argue that AIAs are crucial governance mechanisms that bundle accounts of potential harms with identification of responsibility for remedy, requiring thoughtful engagement with social and political contexts rather than self-assessment alone.

Opportunities and challenges of artificial intelligence in public health: a systematic review on technological efficacy, ethical dilemmas, and governance pathways (Frontiers in Public Health, 2026)

Qin Gao, Lin Chen et al. · 136 citations

Systematic review of 136 studies examining AI in public health, revealing that while AI enhances efficiency in surveillance and emergency response, it also creates risks including algorithmic bias, data privacy concerns, health inequities, and erosion of public trust, necessitating robust governance pathways.

Navigating Responsible AI: A Systematic Review of Governance Mechanisms and Future Co-Governance Scenarios (Procedia Computer Science, 2025)

Imen Ferjani, Fatimah Abdulraheem Alzahrani et al. · 75 citations

Systematic review analyzing AI governance literature from 2010-2024 across 75 peer-reviewed studies, revealing four key governance themes (ethics, regulation, technology, global coordination) and identifying critical gaps between theoretical principles and practical implementation, supporting the need for mandatory impact assessments.

Against (5)

Do AI Laws Inhibit Innovation? (SSRN Electronic Journal, 2026)

Empirical study finding that AI laws and regulations are associated with reduced innovation and create regulatory uncertainty that negatively affects firms' willingness to innovate, suggesting current AI regulations impose significant costs with long-term implications for economic growth.

The debate

Daniel: We have seen what happens when AI ships without proper testing. Facial recognition that barely works for dark-skinned women. Language models spewing toxic content. We need mandatory impact assessments.

Marcus: And who decides what counts as passing? Government bureaucrats who do not understand the technology? You will create a bottleneck that kills innovation.

Daniel: The Gender Shades research found 34% error rate differences. Those systems were deployed commercially. Companies clearly cannot police themselves when racing competitors to market.

Marcus: So one study justifies a whole regulatory apparatus? AI moves fast. By the time your assessment is complete, the technology has already evolved three generations.

Daniel: That is exactly the problem. Move fast and break things does not work when you are breaking people lives—denying them jobs, loans, parole based on biased algorithms.

Marcus: Then regulate high-stakes applications specifically. Do not require impact assessments for every chatbot and image filter. You will strangle startups while big tech has compliance departments to handle paperwork.

Daniel: Proportional regulation is exactly what I am proposing. Light touch for low risk, rigorous review for systems affecting healthcare, criminal justice, employment. The EU AI Act takes this approach.

Marcus: The EU AI Act is already driving AI companies to relocate. You will just offshore innovation to countries without these requirements.

Daniel: That is a race to the bottom argument. We do not abandon drug safety because other countries have lower standards. And companies want access to large regulated markets.

Marcus: Drugs take a decade to develop. AI models iterate weekly. You cannot apply pharmaceutical regulation timelines to software.

Daniel: I am not suggesting decade-long reviews. Impact assessments can be fast—document training data, test for bias, evaluate failure modes. Weeks, not years.

Marcus: Voluntary frameworks could accomplish the same thing without government mandates. Industry has every incentive to avoid deploying harmful systems.

Daniel: History says otherwise. Every industry claims self-regulation works until disasters prove it does not. We need enforceable standards before the next large-scale harm, not after.

Frequently asked questions

What is an AI impact assessment?

An AI impact assessment is a structured review of an AI system's potential harms before it is deployed. Research describes it as a way to document possible harms and identify who is responsible for fixing them, ideally with outside input rather than self-assessment alone.

Why do some researchers support mandatory AI impact assessments?

Studies have documented harms from systems released without adequate evaluation, such as facial recognition accuracy gaps across demographic groups and bias in language models. Supporters argue pre-deployment testing could catch these problems, and that voluntary standards have fallen short under competitive pressure.

Do AI regulations slow down innovation?

Some economic research finds AI laws are associated with reduced innovation and regulatory uncertainty, and that AI progress depends on rapid trial and error. Supporters of assessments respond that this argues for well-designed, proportionate rules rather than none.

How much does AI regulatory compliance cost?

One study of the EU AI Act estimated compliance costs of €29,277 per year for each high-risk medical AI unit. It found these costs weigh more heavily on smaller organizations and could concentrate market power among well-resourced firms.

Related questions