Add an LLM policy for rust-lang/rust

[jyn514]

Summary

This document establishes a policy for how LLMs can be used when contributing to rust-lang/rust. Subtrees, submodules, and dependencies from crates.io are not in scope. Other repositories in the rust-lang organization are not in scope.

This policy is intended to live in Forge as a living document, not as a dead RFC. It will be linked from CONTRIBUTING.md in rust-lang/rust as well as from the rustc- and std-dev-guides.

Moderation guidelines

This PR is preceded by an enormous amount of discussion on Zulip. Almost every conceivable angle has been discussed to death; there have been upwards of 3000 messages, not even counting discussion on GitHub. We initially doubted whether we could reach consensus at all.

Therefore, we ask to bound the scope of this PR specifically to the policy itself. In particular, we mark several topics as out of scope below. We still consider these topics to be important, we simply do not believe this is the right place to discuss them.

No comment on this PR may mention the following topics:

• Long-term social or economic impact of LLMs
• The environmental impact of LLMs
• Anything to do with the copyright status of LLM output
• Moral judgements about people who use LLMs

We have asked the moderation team to help us enforce these rules.

Feedback guidelines

We are aware that parts of this policy will make some people very unhappy. As you are reading, we ask you to consider the following.

• Can you think of a concrete improvement to the policy that addresses your concern? Consider:
  • Whether your change will make the policy harder to moderate
  • Whether your change will make it harder to come to a consensus
• Does your concern need to be addressed before merging or can it be addressed in a follow-up?
  • Keep in mind the cost of not creating a policy.

If your concern is for yourself or for your team

• What are the specific parts of your workflow that will be disrupted?
  • In particular we are only interested in workflows involving rust-lang/rust. Other repositories are not affected by this policy and are therefore not in scope.
• Can you live with the disruption? Is it worth blocking the policy over?

---

Previous versions of this document were discussed on Zulip, and we have made edits in responses to suggestions there.

Motivation

• Many people find LLM-generated code and writing deeply unpleasant to read or review.
• Many people find LLMs to be a significant aid to learning and discovery.
• rust-lang/rust is currently dealing with a deluge of low-effort "slop" PRs primarily authored by LLMs.
  • Having a policy makes these easier to moderate, without having to take every single instance on a case-by-case basis.

This policy is not intended as a debate over whether LLMs are a good or bad idea, nor over the long-term impact of LLMs. It is only intended to set out the future policy of rust-lang/rust itself.

Drawbacks

• This bans some valid usages of LLMs. We intentionally err on the side of banning too much rather than too little in order to make the policy easy to understand and moderate.
• This intentionally does not address the moral, social, and environmental impacts of LLMs. These topics have been extensively discussed on Zulip without reaching consensus, but this policy is relevant regardless of the outcome of these discussions.
• This intentionally does not attempt to set a project-wide policy. We have attempted to come to a consensus for upwards of a month without significant progress. We are cutting our losses so we can have something rather than adhoc moderation decisions.
• This intentionally does not apply to subtrees of rust-lang/rust. We don't have the same moderation issues there, so we don't have time pressure to set a policy in the same way.

Rationale and alternatives

• We could create a project-wide policy, rather than scoping it to rust-lang/rust. This has the advantage that everyone knows what the policy is everywhere, and that it's easy to make things part of the mono-repo at a later date. It has the disadvantage that we think it is nigh-impossible to get everyone to agree. There are also reasons for teams to have different policies; for example, the standard for correctness is much higher within the compiler than within Clippy.
• We could have different standards for people in the Rust project than for new contributors. That would make moderation much easier, and allow us to experiment with additional LLM use. However, it reinforces existing power structures, creates more of a gap between authors and reviewers, and feels "unfriendly" to new contributors.
• We could have a more lenient policy that allows "responsible and appropriate" use of LLMs. This raises the question of what "responsible and appropriate" means. The usual suggestion is "self-review, and judging the change by the same standard as any other change"; but this neglects the reputational and social harm of work that "feels" LLM generated. It also makes our moderation policy much harder to understand, and increases the likelihood of re-litigating each moderation decision.
• We could have a more strict policy that removes the threshold of originality condition. This has the advantage that our policy becomes easier to moderate and understand. It has the disadvantage that it becomes easy for people to intend to follow the policy, but be put in a position where their only choices are to either discard the PR altogether, rewrite it from scratch, or tell "white lies" about whether an LLM was involved.
• We could have a more strict policy that bans LLMs altogether. It seems unlikely we will be able to agree on this, and we believe attempting it will cause many people to leave the project.

Prior art

This prior art section is taken almost entirely from Jane Lusby's summary of her research, although we have taken the liberty of moving the Rust project's prior art to the top. We thank her for her help.

Rust

• Moderation team's spam policy
• Compiler team's "burdensome PRs" policy

Other organizations

These are organized along a spectrum of AI friendliness, where top is least friendly, and bottom is most friendly.

• full ban
  • postmarketOS - also explicitly bans encouraging others to use AI for solving problems related to postmarketOS - multi point ethics based rational with citations included
  • zig
    • philosophical, cites Profession (novella)
    • rooted in concerns around the construction and origins of original thought
  • servo
    • more pragmatic, directly lists concerns around ai, fairly concise
  • qemu
    • pragmatic, focuses on copyright and licensing concerns
    • explicitly allows AI for exploring api, debugging, and other non generative assistance, other policies do not explicitly ban this or mention it in any way
• allowed with supervision, human is ultimately responsible
  • scipy
    • strict attribution policy including name of model
  • llvm
  • blender
  • linux kernel
    • quite concise but otherwise seems the same as many in this category
  • mesa
    • framed as a contribution policy not an AI policy, AI is listed as a tool that can be used but emphasizes same requirements that author must understand the code they contribute, seems to leave room for partial understanding from new contributors.

      Understand the code you write at least well enough to be able to explain why your changes are beneficial to the project.

  • forgejo
    • bans AI for review, does not explicitly require contributors to understand code generated by ai. One could interpret the "accountability for contribution lies with contributor even if AI is used" line as implying this requirement, though their version seems poorly worded imo.
  • firefox
  • ghostty
    • pro-AI but views "bad users" as the source of issues with it and the only reason for what ghostty considers a "strict AI policy"
  • fedora
    • clearly inspired and is cited by many of the above, but is definitely framed more pro-ai than the derived policies tend to be
• curl
  • does not explicitly require humans understand contributions, otherwise policy is similar to above policies
• linux foundation
  • encourages usage, focuses on legal liability, mentions that tooling exists to help automate managing legal liability, does not mention specific tools
• In progress
  • NixOS
    • How do we deal with AI-generated issues? NixOS/nixpkgs#410741

Unresolved questions

See the "Moderation guidelines" and "Drawbacks" section for a list of topics that are out of scope.

Rendered

[rustbot]

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Fallback group: @Mark-Simulacrum, internal-sites
• @Mark-Simulacrum, internal-sites expanded to Mark-Simulacrum, Urgau, ehuss, jieyouxu
• Random selection from Mark-Simulacrum, Urgau, ehuss, jieyouxu

[jyn514]

@rustbot label T-libs T-compiler T-rustdoc T-bootstrap

[jieyouxu]

I really like this version, and thanks a ton for working on it. Specifically:

• It doesn't try to dump entire walls of text, which is unfortunately a good way to be sure nobody reads it. Instead, it gives you concrete examples, and a guiding rule-of-thumb for uncovered scenarios, and acknowledges upfront that it surely cannot be exhaustive.
• I also like where it points out the nuance and recognizes the uncertainties.
• I like that it covers both "producers" and "consumers" (with nuance that reviewers can also technically use LLMs in ways that are frustrating to the PR authors!)

I left a few suggestions / nits, but even without them this is still a very good start IMO.

(Will not leave an explicit approval until we establish wider consensus, which likely will take the form of 4-team joint FCP.)

View changes since this review

[ChayimFriedman2]

The links to Zulip are project-private, FWIW.

[jyn514]

The links to Zulip are project-private, FWIW.

I'm aware. This PR is targeted towards Rust project members moreso than the broad community.

[davidtwco]

I'm happy with this as an initial policy for the rust-lang/rust repository.

View changes since this review

[alice-i-cecile]

Suggested change

	All contributions are your responsibility; you cannot place any blame on an LLM.
	Your contributions are your responsibility; you cannot place any blame on LLMs that you have used.

Clarity / wording.

View changes since the review

[joshtriplett]

Suggested change

	> LLMs work best when used as a tool to write *better*, not *faster*.
	> In `rust-lang/rust`, please do not use LLMs as a tool to write *faster*.

Having this as a high-level summary is offering a judgement on LLMs that feels like it isn't necessary for the policy, and makes consensus more difficult to reach. For anti-LLM folks it's saying that they work best when used to write "better", which is a point in dispute. I would also expect (but don't want to put words in people's mouths) that for pro-LLM folks the point that they don't work well when used to work faster may be in dispute.

I've tried to rephrase this in a fashion that, rather than expressing a general statement on when "LLMs work best", is instead expressing what is desired *for rust-lang/rust as that's the scope of this policy.

View changes since the review

[jyn514]

This is adapted from a quote by @ubiratansoares. This edit changes the quote beyond recognition, and I would rather remove it than edit this much.

[joshtriplett]

Then I think it would be best removed, on the basis that previous line covers similar territory and seems less controversial.

[Kobzol]

Tbh I don't actually understand what this quote is supposed to mean; if anything, I would phrase it the other way around (you can use LLMs to do [things you can already do] to get them done faster, but you shouldn't use them to do things you don't already know how to do you).

[joshtriplett]

Suggested change

	- ℹ️ This includes non-trivial source comments, such as doc-comments or multiple paragraphs of non-doc-comments.
	- ℹ️ This includes *any* doc comments, or non-trivial source comments.

Reordering this to make it clear first and foremost that "Documentation" includes any doc comments, moving "non-trivial source comments" second. This also drops the quantitative "multiple paragraphs"; some multi-paragraph comments may be trivial, and some one-sentence comments may not be.

View changes since the review

[jyn514]

If you are using an LLM to write a multi-paragraph comment that is trivial, IMO that should also be banned. If you have a load-bearing single-line comment, I think that falls under "code changes authored by an LLM", although I'm not sure how to say that concisely.

[joshtriplett]

Suggested change

	- ℹ️ This includes compiler diagnostics.
	- ℹ️ This includes compiler diagnostics or similar user-visible output.

View changes since the review

[jyn514]

We cannot be exhaustive in this policy and I think it hurts us to try.

[joshtriplett]

Suggested change

	However, you must still type out the changes yourself; you cannot give the LLM write access to your source code.
	However, you must still type out the changes yourself; you cannot give the LLM write access to your source code. This is not because we are trying to turn you into a typist, but because it is *very easy* to err in the direction of increasingly non-trivial changes if the LLM gets to directly write the code.

Trying to address potential reactions of "why are you making me re-type this?!".

View changes since the review

[jyn514]

I think if we want a rationale for each rule that should be a separate document, not part of the moderation and contributing guidelines.

[joshtriplett]

Suggested change

	- We do not accept PRs made up solely of trivial changes.
	- We do not in general accept PRs made up *solely* of trivial changes, such as non-user-visible typos.

Trying to make sure people get an idea of what "trivial" means here; for instance, typos in user-visible documentation aren't trivial.

View changes since the review

[GuillaumeGomez]

Very good start, thanks for writing it!

View changes since this review

[joshtriplett]

Suggested change

	but they may not have a policy that an LLM counts as a person.
	but they may not have a policy that an LLM review substitutes for a human review.

View changes since the review

[joshtriplett]

Suggested change

	No amount of editing can change authorship; authorship sets the initial style and it is very hard to change once it's set.
	In the manner the phrase is used in this policy, no amount of editing changes how something was "originally authored"; authorship sets the initial style and it is very hard to change once it's set.

Taking a different approach here, of narrowing the focus to the phrasing in this policy, rather than trying to get people to agree with the fully general statement.

View changes since the review

[joshtriplett]

Suggested change

	## Policy
	## Interim LLM Usage Policy

Adding a title that mentions LLM usage, and flagging this as interim to foreshadow the section at the end noting that policies may evolve.

I am hopeful that this is capturing a sentiment shared both by people who want the policy to be stricter and bypeople who want the policy to be less strict.

View changes since the review

[jyn514]

I stand by this policy. I would be happy for this to be a semi-permanent policy. We can of course edit it, but I consider "interim" to be a forward-looking statement and I don't want to make those in this policy.

[obi1kenobi]

I really like this! It does a masterful job of the tightrope walk that tricky decisions require in a large project with many stakeholders.

My hat's off to everyone involved here!

View changes since this review

[xtqqczze]

I’m not comfortable with the definition of "originally authored" as written here. Authorship is something that applies to a person, not tools; a LLM can generate text, but it isn’t an author.

View changes since the review

[nikomatsakis]

I've been thinking for a while @jyn514 about what I think about these policy proposals. I think the bottom line for me is that I feel like we are working on pretty incomplete data and I feel ungreat about it.

I would prefer if we established an explicitly temporary policy now and, at the same time, a process and a timeline for establishing a better one. In that circumstances, I would be ok with a restrictive policy like this one. As it is, I feel uncomfortable with it, because it feels like we are letting a handful of loud voices dictate the direction we are going (I could myself amongst the handful of people who've been speaking loudly, though I don't see this policy reflecting my opinions in particular).

View changes since this review

[nikomatsakis]

I would feel better if we made this policy explicitly time-limited or tied to a process of gathering more information.

[clarfonthey]

Niko, you're one of the loudest voices trying to dictate the direction we're going. I would argue that a majority of the pushback from sensible policies like this one have come from you; since you're effectively the project manager for the project, your voice carries further than a dozen people's, and it feels like you're genuinely oblivious to this. Plus, a lot of the arguments you've offered have been from the position that whatever you think is reasonable is canonically reasonable, which is perspective that resists all form of negotiation.

We all agree that this policy is not going to be permanent, but a large portion of the project seems to be in agreement that this should be the policy we adopt until a project-wide policy is adopted.

It's also worth noting, since it's been brought up multiple times, that we don't do policy by majority vote. This is even true for a policy like this one: if we did majority vote, we'd just ban all LLM usage, but we're not doing that because we're willing to compromise.

Right now, it seems pretty unsubstantiated that a handful of voices have dictated this position. While it's true that a small number of people have been active in the policy channel, a majority of the project have pointed out their desire for a total ban on LLM usage. This, being noticeably more lenient on that, is a compromise from us. You should consider whether you're willing to compromise at all on your stance, and what compromise would mean for you.

As I mentioned in one of the discussions, I do think it's a false equivalence that both sides need to concede something, but if you don't even know what it means to compromise, then negotiation is utterly impossible. I really am not convinced that you understand what a compromise of the pro-LLM position would be, based upon the utter confusion you've expressed when mentioning that some of the contributions you've done would not be acceptable under some of the proposed policies.

[clarfonthey]

Do not plan to actually engage in this conversation any further (I acknowledge my biases and when to step out), but I think it's worth pointing out to the at-least-5 people who gave a thumbs-down reaction to my comment that I personally have a rule when it comes to this.

If I ever decide to mark my dissent on a comment with the thumbs-down emoji, I always reply explaining why unless everything I wish to say has already been said. Many times, the result is far more critical to the poster than a simple emoji, but I do this because I genuinely want people to understand why I feel a particular way, rather than just saying "I don't like this and will not explain why." We don't improve if we don't know what's wrong.

My above comment, in my mind, is required to give Niko's a thumbs-down reaction, because otherwise I'm being insincere to him and everyone else reading. I do not say that I disagree with something without saying why; in that case, it's better to not say anything at all.

Again, I acknowledge that my explanation can be deeply hurtful. Disagreement is a painful but necessary process. I also know that there are plenty of times where I have been excessively hurtful without providing the relevant constructive feedback, and think it's worth calling me out for that.

I don't apply my standards to anyone else. Lots of people just don't have time to write up a full response. But I personally, in these cases, simply don't respond at all.

So, consider whether your simple thumbs-down emoji constitutes genuinely useful feedback, or whether you're just being excessively hurtful instead. And, if you would like to express your dissent in private, I'm open to DMs on Zulip too; this is an open invitation to just say what you feel without a filter. It would be hypocritical of me to be so blunt with my opinions and not accept the same in kind.

[Mark-Simulacrum]

I'll reply here since this is 'the thread', but I want to say first that I don't agree with much of what you wrote @clarfonthey. I believe Niko is raising a concern in good faith, though I'd like to understand it better.

I would feel better if we made this policy explicitly time-limited or tied to a process of gathering more information.

@nikomatsakis, can you elaborate on specifically what information you think we should be seeking, and what process you're imagining for iterating towards a better policy? Are you seeking commitment from folks who have engaged so far to keep engaging in discussion on Zulip? Something else? I'd be happy to chat offline (Zulip or more sync meeting if you'd prefer) if that makes more sense.

I see @jyn514 left a comment below with some more data on project opinions, but it's not clear to me if that's the kind of data you're seeking, or something else. Could you elaborate on what you're looking for and what kinds of process/timeline you would find better than the copious discussion and iteration that has landed us on this (and some other) proposals?

I personally think a policy like this one that is relatively restrictive, but scope-limited and leaving room for usage in other areas of the Project gives us a good balance for continued input on where the world is and leaving the door open for private usage for those comfortable with doing so. That combination seems guaranteed to ensure we're not going to stop discussing since everyone seems to want something different from this policy, even if we manage to get to consensus on landing this in the meantime.

[jyn514]

@nikomatsakis You've talked a few times about having more quantitative data. I have some new quantitative data for you: people have been talking to me in private since this policy was posted, including several people who I haven't heard from publicly. Here are statistics on their opinions:

for the curious: opinions from people who aren't in the Rust organization

The 3 people who are unhappy all want a less restrictive policy, and they've all spoken publicly: It's you, TC, and Tobias. Now, this isn't representative — I haven't heard from some people who I know weren't in favor of this policy, so this is probably leaning towards people who support it. But I still find it quite convincing as an indication that there isn't some kind of "silent majority" who oppose the policy.

---

I want to quote this bit from the PR description:

Does your concern need to be addressed before merging or can it be addressed in a follow-up?

• Keep in mind the cost of not creating a policy.

Can you live with the disruption? Is it worth blocking the policy over?

Do you think this policy is so restrictive that it's worse than not having a policy at all? If not, what do you suggest if your "timeline for establishing a better policy" elapses and we haven't come to a consensus?

[kennytm]

Maybe I'm OOTL but I find this section situationally strange — where did the "review bot" come from?

IME AI-powered review bots that directly participates in PR discussions (esp the "app" ones) are configured by repository owner, but AFAIK r-l/r (which this policy applies solely to) did not have any such bots. I highly doubt a contributor will bring in their own review bot in public. So practically this has to be either

• someone requested a review from Copilot, which may be we can opt-out?
• the reviewer outsourced the review work to a coding agent, which is already covered in the sections
• at least one team actually considered enabling such review bots in the future? as this is linked previously in that "Teams can have a policy that code can be merged without review" part, but I don't think this will ever happen given the the stance of this policy

View changes since the review

[Kobzol]

I highly doubt a contributor will bring in their own review bot in public.

I wish it worked like that :( People can just trigger GitHub copilot, or I suppose any other review bot, and let it comment on a r-l/r PR. Some people don't even do it willingly, but GH does it automatically for them, as GH copilot has a tendency to re-enable itself even if you sometimes disable it.

It is also not possible to opt-out of the PR author requesting a Copilot review, if I remember correctly.

[xtqqczze]

I’ve seen this behavior elsewhere on GitHub, where contributors effectively use a personal account as a kind of "review bot" to comment on PRs without approval from maintainers.

[kennytm]

It is also not possible to opt-out of the PR author requesting a Copilot review, if I remember correctly.

Yeah currently disabling review is a personal/license-owner setting, it is not possible to configure from the repository PoV 😞 but I think this is something that we may bring up to GitHub.

It may be possible to use content exclusion to blind Copilot, but I'm not sure if this hack is going to produce any overreaching effects (e.g. affecting private IDE usage too).

[apiraino]

someone requested a review from Copilot, which may be we can opt-out?

I think this is exactly the point of pointing that out in our policy. Some people trigger a "[at]copilot review" in our repos without asking us for consent. This is rude behaviour and we don't want that.

And, yes, as you point out opting out of this "trigger" is currently only a project-wide setting, not at a repository level so we are looking with GitHub if they could make this setting more fine-grained (here on Zulip a discussion with the Infra team)

[kennytm]

Well I wouldn't say it's "rude" because GitHub's UI makes it so easy to request a review from Copilot, even manually 😅

IMO this is caused by a decision from the upstream (GitHub), so should better be solved at the GitHub level. This policy about "review bot" is only stop-gap workaround, and placing the burden on the wrong parties. I think we should explore more about disabling Copilot review "statically" (e.g. through GitHub implementing such option; banning Copilot (user ID 175728472) from making comment; using the instruction files etc), rather than relying on a policy.

Not saying that this section should be removed, but maybe informatively explain that "requesting review from Copilot on a PR is typically not welcomed".

[apiraino]

yes I agree, it's a "UI decision" that ends up enabling rude or, if you prefer, unsettling behaviours. I'd like that this policy makes that clear.

ℹ️ Review bots that post without being approved by a maintainer will be banned.

I think this covers enough the "do not prompt LLM reviews on rust-lang/ without consent" part.

[jyn514]

@clarfonthey I understand you are frustrated but it doesn't help to take it out on the people we're working with. Can I ask you to take a break from commenting on this RFC for a bit? Feel free to DM me with any concerns you have about the policy itself.

[clarfonthey]

yeah, you're right; I deleted the comment