Add contribution policy for AI-generated work#3950
Add contribution policy for AI-generated work#3950traviscross wants to merge 9 commits intomasterfrom
Conversation
In RFC 3936, we had proposed a *be present* contribution policy. That framing may have been broader than necessary. Let's *mechanically monomorphize* that policy to instead speak only about AI-generated contributions.
traviscross
added
T-leadership-council
|
Is this meant to be an updated version of #3936? |
This is a monomorphized version of that proposal. Rather than speaking generally about what is expected of contributions, this proposal targets specifically and exclusively those AI-generated contributions that are prohibited. In doing that, I'm exploring whether we can find an intersection of agreement. |
|
|
||
| ### By not banning use of AI tools, does this RFC endorse them? | ||
|
|
||
| By not banning use of AI tools, this RFC does not endorse their use. People in the Project have diverse views on generative AI and on its use. This RFC takes no position — positive or negative — on the use of these tools beyond forbidding those things the policy prohibits. |
There was a problem hiding this comment.
| By not banning use of AI tools, this RFC does not endorse their use. People in the Project have diverse views on generative AI and on its use. This RFC takes no position — positive or negative — on the use of these tools beyond forbidding those things the policy prohibits. | |
| Although use of AI tools is not banned, this RFC does not endorse their use. People in the Project have diverse views on generative AI and on its use. This RFC takes no position — positive or negative — on the use of these tools beyond forbidding those things the policy prohibits. |
|
|
||
| ### What's it mean to be in the loop? | ||
|
|
||
| To be in the loop means to be part of the discussion — to be an integral part of the creative back and forth. You were in the loop if you were there, engaged, and contributing meaningfully when the creation happened. |
There was a problem hiding this comment.
does this unintentionally prohibit things that were created by someone other than the contributor and that someone else used AI tools but actually verified the output? Maybe we should add something saying that this should not be interpreted as prohibiting contributing code that you weren't directly involved in creating because some other person responsibly created it maybe with AI.
There was a problem hiding this comment.
Contributing someone else's code would seem weird to me. Why isn't that other person the one submitting the PR? Surely they have a better understanding of the code and would be better placed to respond to feedback?
There was a problem hiding this comment.
does this unintentionally prohibit things that were created by someone other than the contributor and that someone else used AI tools but actually verified the output? Maybe we should add something saying that this should not be interpreted as prohibiting contributing code that you weren't directly involved in creating because some other person responsibly created it maybe with AI.
That's a good point. In earlier proposals, I had been carefully trying to avoid tripping over disallowing things otherwise allowed by the DCO. Tripped over that here. Thanks for catching this.
I'm not immediately certain how to draft around this. How could one be sure, when one's contribution contains (appropriately-licensed) open source code written by a third party acting at arms-length that the third-party author created the code in a way that complies with the policy? Maybe we'll have to exempt that but include the arms-length restriction so that it's not just an easy loophole. But that's getting a bit legalistic. Will think about this. Let me know if you have ideas.
There was a problem hiding this comment.
Contributing someone else's code would seem weird to me. Why isn't that other person the one submitting the PR? Surely they have a better understanding of the code and would be better placed to respond to feedback?
well, you might be contributing code you copied from somewhere because the other person wrote it a long time ago and/or isn't interested in contributing themselves. a good example is if there's some handy method in a library somewhere that you need but you don't want a whole new dependency just for that method, e.g. promoting itertools methods to std
There was a problem hiding this comment.
Maybe we'll have to exempt that but include the arms-length restriction so that it's not just an easy loophole
to exploit that loophole would require multiple people collaborating which seems rare enough in combination with fully AI generated code that maybe we can just leave the loophole and do something if/when it becomes a problem?
maybe just add a sentence like so: This does not mean you can't contribute code written by other people if it has the proper open-source licenses.
There was a problem hiding this comment.
Contributing someone else's code would seem weird to me. Why isn't that other person the one submitting the PR?
This is fairly common when dealing with abandoned, salvaged PRs.
|
|
||
| ### Does this policy require disclosure of the use of generative AI tools? | ||
|
|
||
| This policy does not require disclosure of the use of generative AI tools. This is a complex question on which Project members have diverse views and where members are continuing to explore, investigate, learn, and discuss. Later policies may further address this. |
There was a problem hiding this comment.
I believe that disclosure of authorship should be required (which can go beyond AI, e.g. to acknowledge co-authors). When reviewing student work, I have found it very helpful to have a clear statement of whether AI was involved or not, since it reduces the guessing game in many cases. If someone falsely declares to have not used AI, it can also simplify moderation choices. While I would understand if a specific policy on disclosure is postponed so that the larger policy can be agreed upon more quickly, I do think disclosure should follow soon after.
There was a problem hiding this comment.
Authorship is something that applies to a person, not tools; a LLM can generate text, but it isn’t an author.
|
I feel like this should be better aligned with the other ongoing proposal we have for a similar purpose (rust-lang/rust-forge#1040). This RFC seems to propose a policy which errs on the side of allowing LLM use, while the other proposal proposes a policy which errs on the side of forbidding their use. Having both of these in place seems likely to lead to confusion on the part of contributors and team members alike. (Note: For what it's worth, I prefer the other policy's stance here) |
For the record, this is something that I explicitly pointed out in a private Zulip chat; by posting this RFC after the linked scoped policy was posted, this now effectively forces the two policies into conflict and potentially blocks the policy that was posted first from being merged, even though its exact purpose was to get a policy out sooner than an RFC. It's unlikely for this RFC to really make substantial process since it resulted from one team member deciding to write and submit it without being super vigilant about other policies and their states of progress. (This last is just an informed opinion.) Technically, I'm breaking my own decision to abstain from commenting on this RFC, although since this is mostly to clarify the situation for others who might not be in the private Zulip chat (for example, a majority of the community), I figured I'd make an exception. |
|
I skimmed through this proposal, trying to get a vibe. While I like the prose, I've put myself in the shoes of a contributor and I am left with a question: what can I actually do and can't do? The policy itself is 5 lines of text, the rest is meta (definitions, questions, and answers). I undestand the premise of "this policy describes only those items on which Project members agree" and while I agree with those 5 lines, I also think it needs to be more specific about do's and dont's, especially the dont's. I would abstract and generalize from past "incidents" (i.e. things that we don't want to happen) and present more practical suggestions. |
the other proposal is specifically only for the main https://github.com/rust-lang/rust repository and thus only asks for comments and approval for the relevant teams (that is why I've stayed out of it). I don't think that repository-specific proposal should dictate the general policy, because then the unrelated teams would've been excluded from the discussion.
IMHO this RFC proposes the minimum set of rules that everyone so far has agreed on. A policy "which errs on the side of forbidding their use" is unlikely to get consensus from the people in the project that do use AI, while the proposed policy at least has a chance, unless people block it because it doesn't go far enough in forbidding AI use. I'd personally rather have a policy that forbids the most problematic uses, than no policy at all. |
|
May I say the intention of this RFC is to list out the absolute forbidden use cases of generative AI, in Rust project, including any of it source codes, documentation, websites, bug reports, feature requests, RFCs, and comments? And then for more specific parts of aforementioned aspects of Rust project, the team could later impose stricter policy if needed? |
|
@clarfonthey -- I don't feel that's what happened here. I also do not think your prediction is correct. I'm in the private channel too, but I don't think we should be leveraging that here to advance our own positions. Both @jyn514's proposal and this RFC could go forward independently. From what I'm led to understand there are real challenges with r-l/r low-effort and AI-created contributions. But frankly speaking, I'd prefer we adopt this RFC. From what I've read it appears to be a careful compromise many of us can get behind. Adopting one policy for the whole Project would be more clear for contributors for the reasons @thomcc mentioned. This RFC seeks to provide that clarity. @jyn514's proposal is specific to one repository. Both ban the contributions that have been a problem for us recently. However, to @Turbo87's point: this RFC doesn't also ban things Project members such as @nikomatsakis are already successfully doing. |
|
For the record @PLeVasseur I both did clarify that whether the policy could go forward is just an informed opinion, not fact, and that I was intentionally planning to not comment on this RFC for now. So, please don't @ mention me to bypass the fact that I've muted this thread. |
|
We are straying from discussing technical merits of the RFC here, so I'll keep it brief. I do not appreciate having ill-intent assigned to me here on the @ mention as an intentional means to bypass the thread muting. I was genuinely unaware that @ mentioning someone bypasses a muted thread. Mea culpa on that. |
People have asked whether this policy bans vibecoding. It does. Let's articulate why and also answer why it bans slop and fully automated AI-generated contributions.
The RFC contains the policy items and a section of definitions, questions, and answers that together establish what the policy means and how it applies. We intend for most of these answers to be normative, but not all. Let's move the not-normative ones to a new section, then let's say explicitly which sections are normative.
Thanks for having a look and for this feedback. The definitions, questions, and answers are intended normatively as part of the policy: they define the meaning of the terms and describe how this policy applies. I've added a section to make this explicit. Based on hearing this and other feedback, I've now also added normative sections for:
Hopefully these help make the policy "more specific about do's and don'ts, especially the don'ts". |
The word *slop* has long meant a kind of putrid waste. This usage has recently seen a resurgence, and it's being applied more widely than to AI-generated content. In this context, we mean specifically *AI slop*, so let's say that.
|
|
||
| Other sections are not normative. | ||
|
|
||
| ## Contribution policy for AI-generated work |
There was a problem hiding this comment.
These all seem like good rules to follow, but are all nearly impossible for reviewers or moderators to enforce.
"is prohibited" feels like the wrong framing here as a result. "Do not X" is much more compatible with rules of this nature. If you would like to pursue a permissive, anti-slop AI policy, a "we will teach people to contribute effectively and pro-socially using these tools" is a much better fit than "we will ban you based on our vibes of your initial submission".
Obviously contributors who are learning-resistant or aggressively spamming will need moderation, but that was true well before LLMs.
|
|
||
| ### Is requiring that contributors take care an acceptable policy item? | ||
|
|
||
| To take care is to give something your full attention and treat its correctness as important to you. That's a meaningful distinction. As reviewers, we can tell when someone has taken care and when the person has not — there are many signs of this. |
There was a problem hiding this comment.
As a seasoned reviewer, I am very skeptical of the claim that reviewers can reliably tell when people have or have not taken care, especially in the context of LLM-assisted work.
|
|
||
| ## Motivation | ||
|
|
||
| In the Rust Project, we've seen an increase in unwanted and unhelpful contributions where contributors used generative AI. These are frustrating and costly to reviewers in the Project. We need to find ways to reduce the incidence of these and to lower the cost of handling them. |
There was a problem hiding this comment.
Will this policy meaningfully accomplish its goals? High-quality LLM-generated work, including from trusted contributors, still requires careful review. With an de-facto endorsement of LLM-generated contributions from trusted contributors, I worry that this will worsen review shortages on net.
|
|
||
| [Contribution policy for AI-generated work]: #contribution-policy-for-ai-generated-work | ||
|
|
||
| In all Rust Project spaces: |
There was a problem hiding this comment.
IMO it's important, for the avoidance of doubt, to explicitly say "AI-generated contributions that follow this guidance are allowed by default." I believe that's the intended effect of this policy, but you really have to read between the lines to get there.
Rather than prescribing an escalation ladder, this RFC leaves the handling of violations fully to the discretion of the moderators. Let's describe that explicitly.
We mean for this policy to apply to shared Project spaces and act as a Project-wide baseline. But we mean to allow teams to add prohibitions as they see fit. Let's say that more clearly.
Rejecting contributions for policy reasons can be hard on people. We need to offer a template to make this easier. Let's add one. Likely this will take further iteration to hit exactly the right notes.
|
Based on hearing helpful feedback from @kennytm and @alice-i-cecile (thanks for that), I've now added a template for reviewers to use when rejecting contributions under this policy. See: Having a template is necessary because rejecting contributions can be hard on people — hard on the reviewers and hard on those whose contributions are being rejected. We want the words we use to be thoughtful and well-crafted to make this difficult conversation as uneventful as possible. Probably it will take some iteration for this template to hit exactly the right notes. I'd welcome suggestions (leave review comments). Here's why the words there were chosen:
Unfortunately signals that we wish the outcome were different. Appears to be leaves room for the reviewer to be mistaken. The focus is on the work — this isn't a judgment of the person or the person's motives. And one or more means the reviewer doesn't have to pick a specific item; the contributor can self-assess.
Quoting the full list lets the contributor see what's being evaluated and reflect on where the contribution may have fallen short. Linking to the policy document gives this weight and answers questions.
This protects the reviewer. Without it, we would invite negotiation. The next steps further below do leave some room for reconsideration (if we got it wrong and the contributor can concisely explain why), but the contributor needs to click through for those details, and that's intentional. We want a considered response, not a snap one.
Both things are true: we trust the intent and the effect is harmful. We're separating the intent from the effects. The contributor's identity as a well-meaning person survives; the rejection is about the artifact and its effects on us.
This names the dynamic. It explains why we're doing this thing that is difficult for us. It's meant to be direct and not dance around our reasons.
This acknowledges the likely emotions of both sides. It's another aspect of naming the dynamic. We're emphasizing that we didn't make this decision lightly.
This provides paths for recovery. While we have slammed one door in the contributor's face, we want to point to the other doors that are open. |
In my model of tool use, the human is always the author. Where I'd used the word *authorship* here, I'd meant it in the sense of using tools to assist with human authorship. But it's easy to mistake this to mean something else. So let's avoid the word and make the phrasing more clear.
| In all Rust Project spaces: | ||
|
|
||
| - Submitting AI-generated work when you weren't in the loop is prohibited. | ||
| - Submitting AI-generated work when you haven't checked it with care is prohibited. | ||
| - Submitting AI-generated work when you don't have reason to believe you understand it is prohibited. | ||
| - Submitting AI-generated work when you can't explain it to a reviewer is prohibited. | ||
| - Feeding reviewer questions into an AI tool and proxying the output directly back is prohibited. |
There was a problem hiding this comment.
Can we simplify the wording, and since this is a normative section, and use RFC 2119 terms, something like.:
| In all Rust Project spaces, contributors MUST not: | |
| * Submit AI-generated work without meaningful involvement in its creation. | |
| * Submit AI-generated work that they have not carefully reviewed. | |
| * Submit AI-generated work that they do not reasonably believe they understand. | |
| * Submit AI-generated work that they cannot explain to a reviewer. | |
| * Submit AI-generated responses to reviewer feedback without independent understanding. |
There was a problem hiding this comment.
In all Rust Project spaces, contributors MUST not: * Submit AI-generated work without meaningful involvement in its creation.
the suggested wording falls in the trap I explained in this thread, it doesn't account for submitting code that someone else wrote by responsibly using AI.
There was a problem hiding this comment.
Context: https://github.com/rust-lang/rfcs/pull/3951#issuecomment-4286674950Do we assume this RFC 3950 is already in effect? 🤔
Additionally, assuming this RFC is merged eventually, can it be retroactively applied to all open PRs and issues etc?
There was a problem hiding this comment.
Most people would understand a new policy to apply prospectively, unless clearly stated otherwise.
One of the common concerns about accepting AI-generated work is copyright risk. Niko and the other Project Directors asked Foundation counsel about this. Let's record the result of that discussion.
|
Based on a discussion @nikomatsakis and the other Project Directors had with Foundation counsel, I've added a section addressing questions on the copyright situation with AI-generated work: |
| - Submitting AI-generated work when you can't explain it to a reviewer is prohibited. | ||
| - Feeding reviewer questions into an AI tool and proxying the output directly back is prohibited. | ||
|
|
||
| ## Definitions, questions, and answers |
There was a problem hiding this comment.
I think Q&A shouldn't be a normative section of the policy.
There was a problem hiding this comment.
Thanks for raising this item. There are two sections that have answers. One is focused on the rationale of the policy itself and is not normative.
The other contains definitions of the terms used in the policy items and specific guidance on how the policy items are to be interpreted. The policy comprises these definitions and this guidance, so this section is normative.
If there are specific items that you do not believe should be normative, I'd be curious to hear which and the reasons for that.
There was a problem hiding this comment.
In the current revision, the section "Definitions, questions, and answers" is marked as normative. This conflates distinct types of material that should be treated differently.
15 participants
View all comments
Summary
We adopt a Rust Project contribution policy for AI-generated work. This applies to all Project spaces.
Motivation
In the Rust Project, we've seen an increase in unwanted and unhelpful contributions where contributors used generative AI. These are frustrating and costly to reviewers in the Project. We need to find ways to reduce the incidence of these and to lower the cost of handling them.
We hope that by stating our expectations clearly that fewer contributors will send us unhelpful things and more contributors will send us helpful ones. We hope that this policy will make decisions and communication less costly for reviewers and moderators.
Policy design approach
People in the Rust Project have diverse — and in some cases, strongly opposed — views on generative AI and on its use. To address the problem in front of us, this policy describes only those items on which Project members agree.
Contribution policy for AI-generated work
In all Rust Project spaces:
Acknowledgments
Thanks to @jieyouxu for fruitful collaboration on earlier policy drafts. Thanks to @nikomatsakis, @ehuss, @tmandry, @oli-obk, @Kobzol, @lqd, @PLeVasseur, @eholk, @yoshuawuyts, @davidtwco, @jackh726, @Eh2406, and many others for thoughtful discussion.
All views and errors remain those of the author alone.
See the RFC text for definitions, questions, and answers.
cc @rust-lang/leadership-council
Rendered