by Rahi Shah and Sanchita Teeka
From left to right: Rahi Shah, and Sanchita Teeka. Photos courtesy of authors.
Artificial intelligence has fundamentally reshaped how lawyers perform core litigation and transactional work, forcing firms to reconsider the billable hour, while public and private organizations wrestle with how to implement effective attorney training and their professional responsibility obligations.[1] Its use is only projected to increase as the technology develops to better aid with legal tasks while increasing efficiency and reducing costs. Sixty-nine percent of legal professionals report personally using GenAI tools for work, but most firms still lack formal policies, training, and governance frameworks to manage the risk associated with AI use.[2] From clerical errors to hallucinated cases, AI is far from infallible when it comes to legal research and case work. Unsurprisingly, while widespread use of AI is met with a lack of uniform legal regulations governing the space and rather confronted with a fragmented array of semi-formal rules from different law firms, courts, and professional associations, the industry is not waiting for a set of formalized rules to continue rapidly integrating AI in the field. The question is now how the profession, and the courts that oversee it, will manage the risk that comes with AI use.
A. Private Practice
While regulations regarding AI use for private practitioners are firm-dependent, the American Bar Association Standing Committee on Ethics and Professional Responsibility has stated that its model rules related to competency, informed consent, confidentiality and fees principally apply to generative AI usage.[3] Specifically, they have highlighted the rules below.
- Model Rule 1.1 (Competence). Lawyers should understand “the benefits and risks associated” with the technologies used to deliver legal services to clients.
- Model Rule 1.6 (Confidentiality of Information).
- Model Rule 1.4(a)(2) (Communications). Lawyers should “reasonably consult” with the client about the means by which the client’s objectives are to be accomplished.
- Model Rule 1.5 (Fees). Fees and expenses should be reasonable which means that if a lawyer uses an AI tool to draft a pleading and expends 15 minutes to input the relevant information into the program, the lawyer may charge for that time as well as for the time necessary to review the resulting draft for accuracy and completeness. However, in most circumstances, the lawyer cannot charge a client for learning how to work an AI tool.
These are principles borrowed from an earlier technological era and stretched to fit a new one. The rules assume a “tech-neutral” approach to regulating technology and lack the specificity needed to effectively oversee AI use, and thus, leave open significant questions like verification standards, vendor selection, and firm-level governance that the Model Rules alone were never built to resolve. This is precisely the gap legislation must fill: establishing clear standards to govern the use of this highly consequential and potentially high-risk technology.
B. Federal Prosecutors
The Department of Justice has been equally proactive in expanding AI capabilities across its components. The DOJ’s broader AI strategy reflects this commitment, emphasizing that AI can improve operational efficiency by accelerating information processing, supporting decision-making, and reducing administrative burdens, while also recognizing the need for governance structures to ensure responsible deployment.[4] DOJ’s Civil Division saw the second-highest increase in AI use between 2024 and 2025 and it has only continued to increase. The DOJ has focused on exploring how AI can be better used to synthesize records, summarize expert reports and depositions, and identify duplicate claims.[5] The Executive Office for United States Attorneys (EOUSA), which supports U.S. attorneys nationwide, has likewise integrated AI use by partnering with Palantir to integrate generative AI for analysis of case information and to update and maintain an accurate case management system.
These developments suggest that AI is increasingly being integrated into core enforcement functions. The ability to quickly analyze large volumes of documents, identify patterns across evidence, and synthesize complex factual records may enable prosecutors to investigate cases more efficiently and devote greater attention to litigation strategy. As government agencies continue adopting these technologies, AI has the capability to reshape how enforcement actions are investigated and litigated if harnessed properly.
At the same time, the government’s growing use of AI raises broader questions about parity in the enforcement process. While large corporations are increasingly investing in comparable AI capabilities for internal investigations and regulatory responses, smaller companies, individual defendants, and many public defenders may lack the resources to deploy similar tools. As AI becomes a competitive advantage in reviewing evidence and developing legal strategy, disparities in access could widen existing resource gaps between enforcement agencies and those responding to government investigations. More fundamentally, the government’s commitments to responsible AI adoption have not yet resolved many of these practical and institutional questions that widespread deployment will create.
The DOJ and other government bodies have recognized concerns surrounding privacy, transparency, bias training and human oversight, but acknowledging these concerns is not the same as establishing workable standards for addressing them.[6] For example, how will courts meaningfully scrutinize increasingly complex and opaque models when even experts may struggle to explain how a particular output was produced? What meaningful transparency can be required when AI systems rely on proprietary models or commercial vendors? And what are the financial realities of training federal, state, and local personnel to understand the systems government agencies increasingly plan to incorporate?
The need for clear guidance is particularly acute in high-stakes enforcement contexts, where vague commitments to “human oversight” provide little direction about when AI may appropriately be used, what degree of verification is required, or who bears responsibility when an AI-assisted decision causes harm. These unresolved questions suggest that the government’s current approach may be more focused on adoption than toward the development of the concrete governance structures necessary to ensure that such adoption is fair, transparent, and meaningfully accountable.
Jacob Hoffman-Andrews, a senior staff technologist at the Electronic Frontier Foundation, highlighted this concern in an interview with FedScoop:
“If the prosecuting side has access to tools that allow them to analyze a ton of evidence and come up with the most useful stuff for them, the question is, does the defense have the same ability to synthesize all the possible evidence and find all the possible best cases for the defendant to be exonerated?”
Whether AI ultimately promotes more effective enforcement or creates new asymmetries may depend less on the technology itself than on how broadly and uniformly it is adopted across the legal system. As both government agencies and private parties continue integrating AI into their workflows, ensuring that technological capabilities do not outpace procedural fairness will become an increasingly important consideration for the future of enforcement.
C. Judicial Use of AI
Judges are adopting AI as well, if unevenly. A March 2026 Northwestern University study conducted with the Sedona Conference and the New York City Bar was the first random-sample survey of federal judges on AI use.[7] The study found that more than 60% of the judges who responded had used at least one AI tool in their judicial work, mainly for legal research and document review, though only about 22% used AI weekly or daily, and nearly half reported that their court had provided no AI training at all.
Roughly 38% of respondents reported no AI use whatsoever, suggesting the technology has entered the courthouse without yet becoming a routine, embedded part of judicial decision-making.[8] Judges also reported far greater comfort with AI tools embedded in established research platforms, such as Westlaw’s AI-Assisted Research, than with general-purpose tools like ChatGPT, Copilot, or Gemini, a pattern suggesting that vendor familiarity, not technical sophistication, is driving adoption.[9] This highlights a limitation of judges’ AI adaptation focusing less on the AI’s technological ability which could include impact legal accuracy, recency of information, and efficiency. Furthermore, if the judges’ AI use and comfort levels has prioritized familiar platforms like Westlaw that aren’t pioneering the AI space rather than companies driving AI innovation such as ChatGPT and Gemini, will the same reluctance that applies to mass-adapting general AI vendors extend to tools such as Harvey and Legora?
Judicial sentiment on AI is nearly evenly split between optimism and concern, and in free-response answers, judges repeatedly flagged two distinct risks. First, “zombie cases,” which are citations to decisions that were once good law but have since been overturned, vacated, or superseded. Second, a rising volume of AI-drafted, facially sophisticated but frivolous filings from pro se litigants, which several judges said increases rather than decreases their workload.[10]
Individual judges are also experimenting on their own initiative. One Texas federal judge reportedly feeds case filings into an AI tool to generate a chronology and flag weaknesses in a party’s arguments before hearings work that previously consumed a law clerk thirty to sixty minutes per case.[11] These uses demonstrate AI’s potential to reduce administrative burdens while preserving judicial oversight, but they also underscore the importance of transparency and procedural safeguards when AI becomes integrated into litigation preparation. The emerging judicial approach appears to be one of cautious experimentation: embracing AI’s potential to improve efficiency while maintaining human responsibility for legal judgment.
A. Hallucinated cases
No AI-related litigation risk has drawn more judicial attention than the fabricated citation. The now-canonical example is Mata v. Avianca, Inc., in which two New York attorneys submitted an opposition brief citing cases that did not exist. When opposing counsel and the court could not locate the citations, the attorneys did not withdraw the filing; instead, one told the court he was on vacation, and the other ultimately submitted an affidavit attaching “decisions” that had been fabricated by ChatGPT. These decisions were riddled with gibberish reasoning and internally inconsistent procedural histories, one of which confused the District of Columbia with the state of Washington while purporting to cite itself as precedent.[12] Judge P. Kevin Castel found that both attorneys had acted in subjective bad faith and imposed a $5,000 sanction under Rule 11 on the attorneys and their firm, while emphasizing that “there is nothing inherently improper about using a reliable artificial intelligence tool for assistance” and that the failure was the attorneys’ abdication of their gatekeeping role, not their use of the tool itself.[13]
B. Attorney-Client Privilege and Discovery
AI also complicates two of litigation’s oldest doctrines. First, sharing client information with a third-party AI platform raises confidentiality concerns distinct from traditional cloud storage, since some consumer-grade tools may retain or train on submitted prompts absent contractual protections to the contrary.[14] Second, courts are increasingly being asked whether the AI tool a party or witness used to prepare for litigation is itself discoverable. In Morgan v. V2X, Inc., a federal court considered a motion to compel disclosure of the specific AI tool a pro se plaintiff used in litigation preparation and to restrict AI use of confidential material; the court held that Federal Rule of Civil Procedure 26(b)(3) applies to pro se litigants using AI tools and, thus, disclosure to an AI platform provider, without more, does not waive work-product protection because the disclosure is not made to an adversary or under circumstances substantially likely to reach one.[15] However, the court also ruled that this privilege was not absolute and the pro-se litigant was compelled to disclose which AI tool he used.[16] Separately, in the discovery context, at least one court has treated a generative AI review tool as a form of technology-assisted review governed by existing case law rather than a novel technology requiring new procedural rules, permitting a party to apply search terms before layering in generative AI review of the results. [17] Both threads point in the same direction: AI is being folded into existing doctrine, albeit imperfectly, and litigants should expect discovery disputes over AI tools, prompts, and outputs to become commonplace.
C. Bias
Generative AI models are trained on vast datasets drawn from human-generated information, which can encode and reproduce existing social biases. Research has demonstrated that large language models can produce different outputs based on demographic, linguistic, and identity-related signals even when the underlying information is substantively identical. For example, a study of large language models found that AI systems associated African American English with less prestigious occupations and assigned higher-status occupations, including lawyer and judge, to equivalent statements written in Standard American English.[18] Other studies have found that language models reproduce gender and racial stereotypes, including associating certain occupations, emotions, and social roles with particular demographic groups.[19] These concerns are especially significant in legal settings, where AI tools may influence research, compliance assessments, investigations, and litigation strategy. A biased output may not present as an obvious error; instead, it may subtly affect which information is prioritized, which risks are identified, or which conclusions appear most supported.[20] As organizations increasingly incorporate AI into legal and compliance functions, understanding and mitigating these forms of algorithmic bias will become a central component of responsible AI governance[21]
As of April 2025, only nine states had released informal guidance on lawyers’ responsibilities regarding awareness of bias in AI technologies.[22] However, while more states and firms continue to issue memorandums acknowledging the risk of bias and courts continue to tackle disparate impact of AI, it is a blind spot for litigation regulation and professional responsibility, especially regarding AI use in the legal profession, as there is no federal law in the US that specifically addresses fairness, bias, or other forms of algorithmic discrimination in AI systems.[23]
A. Sanctions
As courts are faced with the challenge of generative AI used in litigation both by government lawyers and private practitioners, there are a growing number of judges who have ordered sanctions against attorneys who have submitted incorrect information in their briefs due to reliance on AI tools. Sanctions for AI-related filing errors have ranged from $5,000 penalties to case dismissal to referral for disciplinary proceedings.[24] Recently, the Ninth Circuit sanctioned two attorneys after they filed briefs containing nonexistent cases, misattributed quotations, and materially misrepresented actual cases. The court said it was not sanctioning the attorneys simply for using AI.[25] The problem arose when the attorneys filed the AI-generated material without verifying it. Ultimately, the emerging rule suggests that AI does not change the lawyer’s existing obligations of accuracy, candor, and verification. Notably, in the internet era, the reputational consequences might be more impactful, tending to follow an attorney indefinitely.
B. Rule 11 Amendment
Federal judges are also considering amending Federal Rule 11 of the rules of civil procedure to address concerns posed by AI. Rule 11 governs representations attorneys make to the Federal Courts in pleadings, motions, and other writings. It requires that attorneys certify that, to the best of their knowledge, information and belief, the legal arguments contained within are “warranted by existing law or by a nonfrivolous argument for extending…existing law.” The proposed amendment would expand this certification requirement to include that the legal authorities exist and are accurately cited to directly address the issues with hallucinated cases and gaps in citations.[26]
C. AI Disclosure Certification
From Judge Brantley Starr’s certification requirement in the Northern District of Texas to Magistrate Judge Gabriel Fuentes’s disclosure order in the Northern District of Illinois to outright AI bans in other chambers, many courts have issued standing orders requiring parties to disclose AI use, certify human verification of AI-assisted content, or forgo AI altogether.[27] As one commentary has put it, courts are “shifting the burden to counsel at the outset. Disclose it, verify it, stand behind it.” This shift has consequences because a certification is itself a fact opposing counsel can probe to test the reliability of a filing and to lay the groundwork for a Rule 11 motion.[28] The absence of a uniform rule means practitioners appearing in multiple districts must track a fragmented landscape of local and even judge-specific requirements, with potential for real professional exposure for missing one.
Given this uncertain landscape, several steps have emerged as best practice for litigators and their firms:
- Address AI at the outset of discovery. In civil matters, raise AI usage at the Rule 26(f) conference. Discuss what tools are being used, how privilege will be protected, whether AI metadata is relevant, and what a proportional scope of any AI-related requests looks like. Narrow any requests you make to specific, case-linked materials rather than sweeping demands for a party’s internal drafting process.[29]
- Know your court’s disclosure obligations before you file. Because AI disclosure requirements vary by district and even by individual judges, confirm applicable standing orders. In any case, default toward voluntary disclosure in jurisdictions without a formal rule, since proactive disclosure carries essentially no downside.
- Verify before you file and correct promptly if you don’t. The Mata sanctions turned less on the attorneys’ initial reliance on ChatGPT than on their failure to acknowledge and correct the error once it surfaced. Build independent-verification steps into any AI-assisted workflow, and train attorneys to treat a suspected AI error the way they would treat any other mistake: disclosed and corrected immediately.
- Build AI governance into firm policy now, rather than after an incident. Formal training, careful tool vetting, particularly around the confidentiality protections available in proprietary platforms versus open consumer tools, and clear guidance on which tasks may be delegated to AI can help to prevent a Mata-style event.
AI is not going anywhere, and prosecutors, judges, and private practitioners are already using it without waiting for perfect rules to emerge. The professional responsibility framework developed for an earlier technological era is now being stretched to address AI, while courts are moving quickly through standing orders, sanctions, and proposed amendments to Rule 11. For practitioners, the lesson from the past several years is that lawyers remain responsible for checking the work they rely on, and AI has added a new and increasingly difficult set of facts, citations, and legal analysis that must be verified.
[1] Harvard Center on the Legal Profession, The Impact of Artificial Intelligence on Law Firm Business Models, https://clp.law.harvard.edu/knowledge-hub/insights/the-impact-of-artificial-intelligence-on-law-law-firms-business-models/.
[2] LawPay on ABA – AI for Law Firms: What the 8am Legal Industry Report Tells Us About AI Use. https://www.americanbar.org/groups/law_practice/resources/law-practicemagazine/2026/march-april-2026/8am-legal-industry-report/.
[3] Harvard Center on the Legal Profession, The Impact of Artificial Intelligence on Law Firm Business Models, https://clp.law.harvard.edu/knowledge-hub/insights/the-impact-of-artificial-intelligence-on-law-law-firms-business-models/.
[4] U.S. Dep’t of Justice, Artificial Intelligence Use Case Inventory (2025), https://www.justice.gov/ai.
[5] FedScoop, DOJ’s AI Inventory Shows Growing Use Across Divisions, https://fedscoop.com/justice-department-artificial-intelligence-ai-surveillance-inventory-predictive-technology-algorithm-bias/.
[6] Council on Criminal Justice, DOJ Report on AI in Criminal Justice: Key Takeaways (2024), https://counciloncj.org/doj-report-on-ai-in-criminal-justice-key-takeaways/.
[7] Northwestern Univ., Study Finds a Significant Number of Federal Judges Are Already Using AI Tools (Mar. 2026), https://news.northwestern.edu/stories/2026/03/northwestern-study-finds-a-significant-number-of-federal-judges-are-already-using-ai-tools.
[8] N.Y.C. Bar Ass’n & Sedona Conference, Artificial Intelligence in Federal Courts: A Random Sample Survey of Judges (2026), https://www.nycbar.org/reports/artificial-intelligence-in-federal-courts-a-random-sample-survey-of-judges/.
[9] Id.
[10] Id.
[11] The Washington Post, Judges Are Increasingly Using AI to Draft Rulings and Prepare for Hearings ( 2026), https://www.washingtonpost.com/nation/2026/04/02/judges-ai-hearings-rulings/.
[12] Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), 2023 WL 4114965 (S.D.N.Y. June 22, 2023); see also William A. Ryan, Allen Garrett & Brad Sears, Practical Lessons from Attorney AI Missteps: Mata v. Avianca, Ass’n of Corp. Counsel, https://www.acc.com/resource-library/practical-lessons-attorney-ai-missteps-mata-v-avianca.
[13] Id.
[14] Am. Bar Ass’n, AI Legal Issues and Concerns for Legal Practitioners, Law Tech. Today (2025), https://www.americanbar.org/groups/law_practice/resources/law-technology-today/2025/ai-legal-issues-and-concerns-for-legal-practitioners/.
[15] Morgan v. V2X, Inc., No. 25-cv-01991-SKC-MDB, 2026 WL 864223 (D. Colo. Mar. 30, 2026), discussed in Think Before You Prompt: What Recent Case Law Tells Us About Privilege, Work Product, and Your AI Interactions, Drug & Device Law Blog (Apr. 29, 2026), https://www.druganddevicelawblog.com/2026/04/guest-post-think-before-you-prompt-what-recent-case-law-tells-us-about-privilege-work-product-and-your-ai-interactions.html.
[16] Morgan v. V2X, Inc., No. 25-cv-01991-SKC-MDB, 2026 WL 864223 (D. Colo. Mar. 30, 2026), Should Protective Orders Expressly Restrict Using AI with Confidential Information? Lessons from Morgan v. V2X (Part II), Greenberg Traurig (May 19, 2026), https://www.gtlaw-ediscoverywatch.com/2026/05/should-protective-orders-expressly-restrict-using-ai-with-confidential-information-lessons-from-morgan-v-v2x-part-ii/.
[17] Schulte v. LinkedIn Corp., No. 22-cv-00237-HSG (LB), 2026 WL 1905851 (N.D. Cal. June 30, 2026), discussed in Generative AI Does Not Eliminate Discovery Burden: Key Lessons for White Collar Practitioners, Holland & Knight (2026), https://www.hklaw.com/en/insights/publications/2026/08/generative-ai-does-not-eliminate-discovery-burden.
[18] Valentin Hofmann et al., AI Generates Discriminatory Decisions About People Based on Their Dialect, Proc. Nat’l Acad. Sci. (2024), https://www.pnas.org/doi/10.1073/pnas.2404914121.
[19] Emily M. Bender et al., On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?, Proc. ACM Conf. on Fairness, Accountability, & Transparency (2021), https://doi.org/10.1145/3442188.3445922.
[20] Margaret Mitchell et al., Model Cards for Model Reporting, Proc. ACM Conf. on Fairness, Accountability, & Transparency (2019),https://dl.acm.org/doi/10.1145/3287560.3287596
[21] Nat’l Inst. of Standards & Tech., Artificial Intelligence Risk Management Framework (AI RMF 1.0) (2023), https://www.nist.gov/itl/ai-risk-management-framework.
[22] AI and Attorney Ethics Rules: 50-State Survey (April 2025), https://www.justia.com/trials-litigation/ai-and-attorney-ethics-rules-50-state-survey/.
[23] Fairness / unlawful bias in the United States, DLA Piper (July 29, 2026), https://intelligence.dlapiper.com/artificial-intelligence/?t=10-fairness-or-unlawful-bias&c=US.
[24] AI Court Disclosure Map: All Rules (2026), AI Vortex, https://www.aivortex.io/legal/guides/ai-court-disclosure-map-2026/.
[25] LNU v. Blanche, No. 24-4790 (9th Cir. June 3, 2026). https://law.justia.com/cases/federal/appellate-courts/ca9/24-4790/24-4790-2026-06-03.html?utm_source=chatgpt.com.
[26] Barnes & Thornburg LLP, Federal Judge Proposes Rule 11 Amendment to Address Generative AI in Court Filings (2026), https://btlaw.com/en/insights/alerts/2026/federal-judge-proposes-rule-11-amendment-to-address-generative-ai-in-court-filings.
[27] N.D. Tex. Judge Brantley Starr, Standing Order on Artificial Intelligence, discussed in What You Need to Know: AI Disclosure Rules in Legal Filings, https://www.eve.legal/blogs/what-you-need-to-know-ai-disclosure-rules-in-legal-filings; N.D. Ill. Magistrate Judge Gabriel A. Fuentes, Standing Order (May 31, 2023), https://guides.lib.uchicago.edu/AI/Practice.
[28] Courts Get Proactive on AI: Disclosure, Certification, and Consequences, Drug & Device Law Blog (Apr. 14, 2026), https://www.druganddevicelawblog.com/2026/04/courts-get-proactive-on-ai-disclosure-certification-and-consequences.html.
[29] Dentons, Landmark AI Rulings Impacting All (Mar. 3, 2026), https://www.dentons.com/en/insights/alerts/2026/march/3/landmark-ai-rulings-impacting-all.
Rahi Shah and Sanchita Teeka are PCCE Student Fellows and JD Candidates at NYU School of Law.
The views, opinions and positions expressed within all posts are those of the author(s) alone and do not represent those of the Program on Corporate Compliance and Enforcement (PCCE) or of the New York University School of Law. PCCE makes no representations as to the accuracy, completeness and validity or any statements made on this site and will not be liable any errors, omissions or representations. The copyright of this content belongs to the author(s) and any liability with regards to infringement of intellectual property rights remains with the author(s).
by Rahi Shah and Sanchita Teeka
From left to right: Rahi Shah, and Sanchita Teeka. Photos courtesy of authors.
Artificial intelligence has fundamentally reshaped how lawyers perform core litigation and transactional work, forcing firms to reconsider the billable hour, while public and private organizations wrestle with how to implement effective attorney training and their professional responsibility obligations.[1] Its use is only projected to increase as the technology develops to better aid with legal tasks while increasing efficiency and reducing costs. Sixty-nine percent of legal professionals report personally using GenAI tools for work, but most firms still lack formal policies, training, and governance frameworks to manage the risk associated with AI use.[2] From clerical errors to hallucinated cases, AI is far from infallible when it comes to legal research and case work. Unsurprisingly, while widespread use of AI is met with a lack of uniform legal regulations governing the space and rather confronted with a fragmented array of semi-formal rules from different law firms, courts, and professional associations, the industry is not waiting for a set of formalized rules to continue rapidly integrating AI in the field. The question is now how the profession, and the courts that oversee it, will manage the risk that comes with AI use.
A. Private Practice
While regulations regarding AI use for private practitioners are firm-dependent, the American Bar Association Standing Committee on Ethics and Professional Responsibility has stated that its model rules related to competency, informed consent, confidentiality and fees principally apply to generative AI usage.[3] Specifically, they have highlighted the rules below.
- Model Rule 1.1 (Competence). Lawyers should understand “the benefits and risks associated” with the technologies used to deliver legal services to clients.
- Model Rule 1.6 (Confidentiality of Information).
- Model Rule 1.4(a)(2) (Communications). Lawyers should “reasonably consult” with the client about the means by which the client’s objectives are to be accomplished.
- Model Rule 1.5 (Fees). Fees and expenses should be reasonable which means that if a lawyer uses an AI tool to draft a pleading and expends 15 minutes to input the relevant information into the program, the lawyer may charge for that time as well as for the time necessary to review the resulting draft for accuracy and completeness. However, in most circumstances, the lawyer cannot charge a client for learning how to work an AI tool.
These are principles borrowed from an earlier technological era and stretched to fit a new one. The rules assume a “tech-neutral” approach to regulating technology and lack the specificity needed to effectively oversee AI use, and thus, leave open significant questions like verification standards, vendor selection, and firm-level governance that the Model Rules alone were never built to resolve. This is precisely the gap legislation must fill: establishing clear standards to govern the use of this highly consequential and potentially high-risk technology.
B. Federal Prosecutors
The Department of Justice has been equally proactive in expanding AI capabilities across its components. The DOJ’s broader AI strategy reflects this commitment, emphasizing that AI can improve operational efficiency by accelerating information processing, supporting decision-making, and reducing administrative burdens, while also recognizing the need for governance structures to ensure responsible deployment.[4] DOJ’s Civil Division saw the second-highest increase in AI use between 2024 and 2025 and it has only continued to increase. The DOJ has focused on exploring how AI can be better used to synthesize records, summarize expert reports and depositions, and identify duplicate claims.[5] The Executive Office for United States Attorneys (EOUSA), which supports U.S. attorneys nationwide, has likewise integrated AI use by partnering with Palantir to integrate generative AI for analysis of case information and to update and maintain an accurate case management system.
These developments suggest that AI is increasingly being integrated into core enforcement functions. The ability to quickly analyze large volumes of documents, identify patterns across evidence, and synthesize complex factual records may enable prosecutors to investigate cases more efficiently and devote greater attention to litigation strategy. As government agencies continue adopting these technologies, AI has the capability to reshape how enforcement actions are investigated and litigated if harnessed properly.
At the same time, the government’s growing use of AI raises broader questions about parity in the enforcement process. While large corporations are increasingly investing in comparable AI capabilities for internal investigations and regulatory responses, smaller companies, individual defendants, and many public defenders may lack the resources to deploy similar tools. As AI becomes a competitive advantage in reviewing evidence and developing legal strategy, disparities in access could widen existing resource gaps between enforcement agencies and those responding to government investigations. More fundamentally, the government’s commitments to responsible AI adoption have not yet resolved many of these practical and institutional questions that widespread deployment will create.
The DOJ and other government bodies have recognized concerns surrounding privacy, transparency, bias training and human oversight, but acknowledging these concerns is not the same as establishing workable standards for addressing them.[6] For example, how will courts meaningfully scrutinize increasingly complex and opaque models when even experts may struggle to explain how a particular output was produced? What meaningful transparency can be required when AI systems rely on proprietary models or commercial vendors? And what are the financial realities of training federal, state, and local personnel to understand the systems government agencies increasingly plan to incorporate?
The need for clear guidance is particularly acute in high-stakes enforcement contexts, where vague commitments to “human oversight” provide little direction about when AI may appropriately be used, what degree of verification is required, or who bears responsibility when an AI-assisted decision causes harm. These unresolved questions suggest that the government’s current approach may be more focused on adoption than toward the development of the concrete governance structures necessary to ensure that such adoption is fair, transparent, and meaningfully accountable.
Jacob Hoffman-Andrews, a senior staff technologist at the Electronic Frontier Foundation, highlighted this concern in an interview with FedScoop:
“If the prosecuting side has access to tools that allow them to analyze a ton of evidence and come up with the most useful stuff for them, the question is, does the defense have the same ability to synthesize all the possible evidence and find all the possible best cases for the defendant to be exonerated?”
Whether AI ultimately promotes more effective enforcement or creates new asymmetries may depend less on the technology itself than on how broadly and uniformly it is adopted across the legal system. As both government agencies and private parties continue integrating AI into their workflows, ensuring that technological capabilities do not outpace procedural fairness will become an increasingly important consideration for the future of enforcement.
C. Judicial Use of AI
Judges are adopting AI as well, if unevenly. A March 2026 Northwestern University study conducted with the Sedona Conference and the New York City Bar was the first random-sample survey of federal judges on AI use.[7] The study found that more than 60% of the judges who responded had used at least one AI tool in their judicial work, mainly for legal research and document review, though only about 22% used AI weekly or daily, and nearly half reported that their court had provided no AI training at all.
Roughly 38% of respondents reported no AI use whatsoever, suggesting the technology has entered the courthouse without yet becoming a routine, embedded part of judicial decision-making.[8] Judges also reported far greater comfort with AI tools embedded in established research platforms, such as Westlaw’s AI-Assisted Research, than with general-purpose tools like ChatGPT, Copilot, or Gemini, a pattern suggesting that vendor familiarity, not technical sophistication, is driving adoption.[9] This highlights a limitation of judges’ AI adaptation focusing less on the AI’s technological ability which could include impact legal accuracy, recency of information, and efficiency. Furthermore, if the judges’ AI use and comfort levels has prioritized familiar platforms like Westlaw that aren’t pioneering the AI space rather than companies driving AI innovation such as ChatGPT and Gemini, will the same reluctance that applies to mass-adapting general AI vendors extend to tools such as Harvey and Legora?
Judicial sentiment on AI is nearly evenly split between optimism and concern, and in free-response answers, judges repeatedly flagged two distinct risks. First, “zombie cases,” which are citations to decisions that were once good law but have since been overturned, vacated, or superseded. Second, a rising volume of AI-drafted, facially sophisticated but frivolous filings from pro se litigants, which several judges said increases rather than decreases their workload.[10]
Individual judges are also experimenting on their own initiative. One Texas federal judge reportedly feeds case filings into an AI tool to generate a chronology and flag weaknesses in a party’s arguments before hearings work that previously consumed a law clerk thirty to sixty minutes per case.[11] These uses demonstrate AI’s potential to reduce administrative burdens while preserving judicial oversight, but they also underscore the importance of transparency and procedural safeguards when AI becomes integrated into litigation preparation. The emerging judicial approach appears to be one of cautious experimentation: embracing AI’s potential to improve efficiency while maintaining human responsibility for legal judgment.
A. Hallucinated cases
No AI-related litigation risk has drawn more judicial attention than the fabricated citation. The now-canonical example is Mata v. Avianca, Inc., in which two New York attorneys submitted an opposition brief citing cases that did not exist. When opposing counsel and the court could not locate the citations, the attorneys did not withdraw the filing; instead, one told the court he was on vacation, and the other ultimately submitted an affidavit attaching “decisions” that had been fabricated by ChatGPT. These decisions were riddled with gibberish reasoning and internally inconsistent procedural histories, one of which confused the District of Columbia with the state of Washington while purporting to cite itself as precedent.[12] Judge P. Kevin Castel found that both attorneys had acted in subjective bad faith and imposed a $5,000 sanction under Rule 11 on the attorneys and their firm, while emphasizing that “there is nothing inherently improper about using a reliable artificial intelligence tool for assistance” and that the failure was the attorneys’ abdication of their gatekeeping role, not their use of the tool itself.[13]
B. Attorney-Client Privilege and Discovery
AI also complicates two of litigation’s oldest doctrines. First, sharing client information with a third-party AI platform raises confidentiality concerns distinct from traditional cloud storage, since some consumer-grade tools may retain or train on submitted prompts absent contractual protections to the contrary.[14] Second, courts are increasingly being asked whether the AI tool a party or witness used to prepare for litigation is itself discoverable. In Morgan v. V2X, Inc., a federal court considered a motion to compel disclosure of the specific AI tool a pro se plaintiff used in litigation preparation and to restrict AI use of confidential material; the court held that Federal Rule of Civil Procedure 26(b)(3) applies to pro se litigants using AI tools and, thus, disclosure to an AI platform provider, without more, does not waive work-product protection because the disclosure is not made to an adversary or under circumstances substantially likely to reach one.[15] However, the court also ruled that this privilege was not absolute and the pro-se litigant was compelled to disclose which AI tool he used.[16] Separately, in the discovery context, at least one court has treated a generative AI review tool as a form of technology-assisted review governed by existing case law rather than a novel technology requiring new procedural rules, permitting a party to apply search terms before layering in generative AI review of the results. [17] Both threads point in the same direction: AI is being folded into existing doctrine, albeit imperfectly, and litigants should expect discovery disputes over AI tools, prompts, and outputs to become commonplace.
C. Bias
Generative AI models are trained on vast datasets drawn from human-generated information, which can encode and reproduce existing social biases. Research has demonstrated that large language models can produce different outputs based on demographic, linguistic, and identity-related signals even when the underlying information is substantively identical. For example, a study of large language models found that AI systems associated African American English with less prestigious occupations and assigned higher-status occupations, including lawyer and judge, to equivalent statements written in Standard American English.[18] Other studies have found that language models reproduce gender and racial stereotypes, including associating certain occupations, emotions, and social roles with particular demographic groups.[19] These concerns are especially significant in legal settings, where AI tools may influence research, compliance assessments, investigations, and litigation strategy. A biased output may not present as an obvious error; instead, it may subtly affect which information is prioritized, which risks are identified, or which conclusions appear most supported.[20] As organizations increasingly incorporate AI into legal and compliance functions, understanding and mitigating these forms of algorithmic bias will become a central component of responsible AI governance[21]
As of April 2025, only nine states had released informal guidance on lawyers’ responsibilities regarding awareness of bias in AI technologies.[22] However, while more states and firms continue to issue memorandums acknowledging the risk of bias and courts continue to tackle disparate impact of AI, it is a blind spot for litigation regulation and professional responsibility, especially regarding AI use in the legal profession, as there is no federal law in the US that specifically addresses fairness, bias, or other forms of algorithmic discrimination in AI systems.[23]
A. Sanctions
As courts are faced with the challenge of generative AI used in litigation both by government lawyers and private practitioners, there are a growing number of judges who have ordered sanctions against attorneys who have submitted incorrect information in their briefs due to reliance on AI tools. Sanctions for AI-related filing errors have ranged from $5,000 penalties to case dismissal to referral for disciplinary proceedings.[24] Recently, the Ninth Circuit sanctioned two attorneys after they filed briefs containing nonexistent cases, misattributed quotations, and materially misrepresented actual cases. The court said it was not sanctioning the attorneys simply for using AI.[25] The problem arose when the attorneys filed the AI-generated material without verifying it. Ultimately, the emerging rule suggests that AI does not change the lawyer’s existing obligations of accuracy, candor, and verification. Notably, in the internet era, the reputational consequences might be more impactful, tending to follow an attorney indefinitely.
B. Rule 11 Amendment
Federal judges are also considering amending Federal Rule 11 of the rules of civil procedure to address concerns posed by AI. Rule 11 governs representations attorneys make to the Federal Courts in pleadings, motions, and other writings. It requires that attorneys certify that, to the best of their knowledge, information and belief, the legal arguments contained within are “warranted by existing law or by a nonfrivolous argument for extending…existing law.” The proposed amendment would expand this certification requirement to include that the legal authorities exist and are accurately cited to directly address the issues with hallucinated cases and gaps in citations.[26]
C. AI Disclosure Certification
From Judge Brantley Starr’s certification requirement in the Northern District of Texas to Magistrate Judge Gabriel Fuentes’s disclosure order in the Northern District of Illinois to outright AI bans in other chambers, many courts have issued standing orders requiring parties to disclose AI use, certify human verification of AI-assisted content, or forgo AI altogether.[27] As one commentary has put it, courts are “shifting the burden to counsel at the outset. Disclose it, verify it, stand behind it.” This shift has consequences because a certification is itself a fact opposing counsel can probe to test the reliability of a filing and to lay the groundwork for a Rule 11 motion.[28] The absence of a uniform rule means practitioners appearing in multiple districts must track a fragmented landscape of local and even judge-specific requirements, with potential for real professional exposure for missing one.
Given this uncertain landscape, several steps have emerged as best practice for litigators and their firms:
- Address AI at the outset of discovery. In civil matters, raise AI usage at the Rule 26(f) conference. Discuss what tools are being used, how privilege will be protected, whether AI metadata is relevant, and what a proportional scope of any AI-related requests looks like. Narrow any requests you make to specific, case-linked materials rather than sweeping demands for a party’s internal drafting process.[29]
- Know your court’s disclosure obligations before you file. Because AI disclosure requirements vary by district and even by individual judges, confirm applicable standing orders. In any case, default toward voluntary disclosure in jurisdictions without a formal rule, since proactive disclosure carries essentially no downside.
- Verify before you file and correct promptly if you don’t. The Mata sanctions turned less on the attorneys’ initial reliance on ChatGPT than on their failure to acknowledge and correct the error once it surfaced. Build independent-verification steps into any AI-assisted workflow, and train attorneys to treat a suspected AI error the way they would treat any other mistake: disclosed and corrected immediately.
- Build AI governance into firm policy now, rather than after an incident. Formal training, careful tool vetting, particularly around the confidentiality protections available in proprietary platforms versus open consumer tools, and clear guidance on which tasks may be delegated to AI can help to prevent a Mata-style event.
AI is not going anywhere, and prosecutors, judges, and private practitioners are already using it without waiting for perfect rules to emerge. The professional responsibility framework developed for an earlier technological era is now being stretched to address AI, while courts are moving quickly through standing orders, sanctions, and proposed amendments to Rule 11. For practitioners, the lesson from the past several years is that lawyers remain responsible for checking the work they rely on, and AI has added a new and increasingly difficult set of facts, citations, and legal analysis that must be verified.
[1] Harvard Center on the Legal Profession, The Impact of Artificial Intelligence on Law Firm Business Models, https://clp.law.harvard.edu/knowledge-hub/insights/the-impact-of-artificial-intelligence-on-law-law-firms-business-models/.
[2] LawPay on ABA – AI for Law Firms: What the 8am Legal Industry Report Tells Us About AI Use. https://www.americanbar.org/groups/law_practice/resources/law-practicemagazine/2026/march-april-2026/8am-legal-industry-report/.
[3] Harvard Center on the Legal Profession, The Impact of Artificial Intelligence on Law Firm Business Models, https://clp.law.harvard.edu/knowledge-hub/insights/the-impact-of-artificial-intelligence-on-law-law-firms-business-models/.
[4] U.S. Dep’t of Justice, Artificial Intelligence Use Case Inventory (2025), https://www.justice.gov/ai.
[5] FedScoop, DOJ’s AI Inventory Shows Growing Use Across Divisions, https://fedscoop.com/justice-department-artificial-intelligence-ai-surveillance-inventory-predictive-technology-algorithm-bias/.
[6] Council on Criminal Justice, DOJ Report on AI in Criminal Justice: Key Takeaways (2024), https://counciloncj.org/doj-report-on-ai-in-criminal-justice-key-takeaways/.
[7] Northwestern Univ., Study Finds a Significant Number of Federal Judges Are Already Using AI Tools (Mar. 2026), https://news.northwestern.edu/stories/2026/03/northwestern-study-finds-a-significant-number-of-federal-judges-are-already-using-ai-tools.
[8] N.Y.C. Bar Ass’n & Sedona Conference, Artificial Intelligence in Federal Courts: A Random Sample Survey of Judges (2026), https://www.nycbar.org/reports/artificial-intelligence-in-federal-courts-a-random-sample-survey-of-judges/.
[9] Id.
[10] Id.
[11] The Washington Post, Judges Are Increasingly Using AI to Draft Rulings and Prepare for Hearings ( 2026), https://www.washingtonpost.com/nation/2026/04/02/judges-ai-hearings-rulings/.
[12] Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), 2023 WL 4114965 (S.D.N.Y. June 22, 2023); see also William A. Ryan, Allen Garrett & Brad Sears, Practical Lessons from Attorney AI Missteps: Mata v. Avianca, Ass’n of Corp. Counsel, https://www.acc.com/resource-library/practical-lessons-attorney-ai-missteps-mata-v-avianca.
[13] Id.
[14] Am. Bar Ass’n, AI Legal Issues and Concerns for Legal Practitioners, Law Tech. Today (2025), https://www.americanbar.org/groups/law_practice/resources/law-technology-today/2025/ai-legal-issues-and-concerns-for-legal-practitioners/.
[15] Morgan v. V2X, Inc., No. 25-cv-01991-SKC-MDB, 2026 WL 864223 (D. Colo. Mar. 30, 2026), discussed in Think Before You Prompt: What Recent Case Law Tells Us About Privilege, Work Product, and Your AI Interactions, Drug & Device Law Blog (Apr. 29, 2026), https://www.druganddevicelawblog.com/2026/04/guest-post-think-before-you-prompt-what-recent-case-law-tells-us-about-privilege-work-product-and-your-ai-interactions.html.
[16] Morgan v. V2X, Inc., No. 25-cv-01991-SKC-MDB, 2026 WL 864223 (D. Colo. Mar. 30, 2026), Should Protective Orders Expressly Restrict Using AI with Confidential Information? Lessons from Morgan v. V2X (Part II), Greenberg Traurig (May 19, 2026), https://www.gtlaw-ediscoverywatch.com/2026/05/should-protective-orders-expressly-restrict-using-ai-with-confidential-information-lessons-from-morgan-v-v2x-part-ii/.
[17] Schulte v. LinkedIn Corp., No. 22-cv-00237-HSG (LB), 2026 WL 1905851 (N.D. Cal. June 30, 2026), discussed in Generative AI Does Not Eliminate Discovery Burden: Key Lessons for White Collar Practitioners, Holland & Knight (2026), https://www.hklaw.com/en/insights/publications/2026/08/generative-ai-does-not-eliminate-discovery-burden.
[18] Valentin Hofmann et al., AI Generates Discriminatory Decisions About People Based on Their Dialect, Proc. Nat’l Acad. Sci. (2024), https://www.pnas.org/doi/10.1073/pnas.2404914121.
[19] Emily M. Bender et al., On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?, Proc. ACM Conf. on Fairness, Accountability, & Transparency (2021), https://doi.org/10.1145/3442188.3445922.
[20] Margaret Mitchell et al., Model Cards for Model Reporting, Proc. ACM Conf. on Fairness, Accountability, & Transparency (2019),https://dl.acm.org/doi/10.1145/3287560.3287596
[21] Nat’l Inst. of Standards & Tech., Artificial Intelligence Risk Management Framework (AI RMF 1.0) (2023), https://www.nist.gov/itl/ai-risk-management-framework.
[22] AI and Attorney Ethics Rules: 50-State Survey (April 2025), https://www.justia.com/trials-litigation/ai-and-attorney-ethics-rules-50-state-survey/.
[23] Fairness / unlawful bias in the United States, DLA Piper (July 29, 2026), https://intelligence.dlapiper.com/artificial-intelligence/?t=10-fairness-or-unlawful-bias&c=US.
[24] AI Court Disclosure Map: All Rules (2026), AI Vortex, https://www.aivortex.io/legal/guides/ai-court-disclosure-map-2026/.
[25] LNU v. Blanche, No. 24-4790 (9th Cir. June 3, 2026). https://law.justia.com/cases/federal/appellate-courts/ca9/24-4790/24-4790-2026-06-03.html?utm_source=chatgpt.com.
[26] Barnes & Thornburg LLP, Federal Judge Proposes Rule 11 Amendment to Address Generative AI in Court Filings (2026), https://btlaw.com/en/insights/alerts/2026/federal-judge-proposes-rule-11-amendment-to-address-generative-ai-in-court-filings.
[27] N.D. Tex. Judge Brantley Starr, Standing Order on Artificial Intelligence, discussed in What You Need to Know: AI Disclosure Rules in Legal Filings, https://www.eve.legal/blogs/what-you-need-to-know-ai-disclosure-rules-in-legal-filings; N.D. Ill. Magistrate Judge Gabriel A. Fuentes, Standing Order (May 31, 2023), https://guides.lib.uchicago.edu/AI/Practice.
[28] Courts Get Proactive on AI: Disclosure, Certification, and Consequences, Drug & Device Law Blog (Apr. 14, 2026), https://www.druganddevicelawblog.com/2026/04/courts-get-proactive-on-ai-disclosure-certification-and-consequences.html.
[29] Dentons, Landmark AI Rulings Impacting All (Mar. 3, 2026), https://www.dentons.com/en/insights/alerts/2026/march/3/landmark-ai-rulings-impacting-all.
Rahi Shah and Sanchita Teeka are PCCE Student Fellows and JD Candidates at NYU School of Law.
The views, opinions and positions expressed within all posts are those of the author(s) alone and do not represent those of the Program on Corporate Compliance and Enforcement (PCCE) or of the New York University School of Law. PCCE makes no representations as to the accuracy, completeness and validity or any statements made on this site and will not be liable any errors, omissions or representations. The copyright of this content belongs to the author(s) and any liability with regards to infringement of intellectual property rights remains with the author(s).
