Add contribution policy for AI-generated work#3950
Add contribution policy for AI-generated work#3950traviscross wants to merge 11 commits intomasterfrom
Conversation
In RFC 3936, we had proposed a *be present* contribution policy. That framing may have been broader than necessary. Let's *mechanically monomorphize* that policy to instead speak only about AI-generated contributions.
traviscross
added
T-leadership-council
|
Is this meant to be an updated version of #3936? |
This is a monomorphized version of that proposal. Rather than speaking generally about what is expected of contributions, this proposal targets specifically and exclusively those AI-generated contributions that are prohibited. In doing that, I'm exploring whether we can find an intersection of agreement. |
|
|
||
| ### By not banning use of AI tools, does this RFC endorse them? | ||
|
|
||
| By not banning use of AI tools, this RFC does not endorse their use. People in the Project have diverse views on generative AI and on its use. This RFC takes no position — positive or negative — on the use of these tools beyond forbidding those things the policy prohibits. |
There was a problem hiding this comment.
| By not banning use of AI tools, this RFC does not endorse their use. People in the Project have diverse views on generative AI and on its use. This RFC takes no position — positive or negative — on the use of these tools beyond forbidding those things the policy prohibits. | |
| Although use of AI tools is not banned, this RFC does not endorse their use. People in the Project have diverse views on generative AI and on its use. This RFC takes no position — positive or negative — on the use of these tools beyond forbidding those things the policy prohibits. |
|
|
||
| ### What's it mean to be in the loop? | ||
|
|
||
| To be in the loop means to be part of the discussion — to be an integral part of the creative back and forth. You were in the loop if you were there, engaged, and contributing meaningfully when the creation happened. |
There was a problem hiding this comment.
does this unintentionally prohibit things that were created by someone other than the contributor and that someone else used AI tools but actually verified the output? Maybe we should add something saying that this should not be interpreted as prohibiting contributing code that you weren't directly involved in creating because some other person responsibly created it maybe with AI.
There was a problem hiding this comment.
Contributing someone else's code would seem weird to me. Why isn't that other person the one submitting the PR? Surely they have a better understanding of the code and would be better placed to respond to feedback?
There was a problem hiding this comment.
does this unintentionally prohibit things that were created by someone other than the contributor and that someone else used AI tools but actually verified the output? Maybe we should add something saying that this should not be interpreted as prohibiting contributing code that you weren't directly involved in creating because some other person responsibly created it maybe with AI.
That's a good point. In earlier proposals, I had been carefully trying to avoid tripping over disallowing things otherwise allowed by the DCO. Tripped over that here. Thanks for catching this.
I'm not immediately certain how to draft around this. How could one be sure, when one's contribution contains (appropriately-licensed) open source code written by a third party acting at arms-length that the third-party author created the code in a way that complies with the policy? Maybe we'll have to exempt that but include the arms-length restriction so that it's not just an easy loophole. But that's getting a bit legalistic. Will think about this. Let me know if you have ideas.
There was a problem hiding this comment.
Contributing someone else's code would seem weird to me. Why isn't that other person the one submitting the PR? Surely they have a better understanding of the code and would be better placed to respond to feedback?
well, you might be contributing code you copied from somewhere because the other person wrote it a long time ago and/or isn't interested in contributing themselves. a good example is if there's some handy method in a library somewhere that you need but you don't want a whole new dependency just for that method, e.g. promoting itertools methods to std
There was a problem hiding this comment.
Maybe we'll have to exempt that but include the arms-length restriction so that it's not just an easy loophole
to exploit that loophole would require multiple people collaborating which seems rare enough in combination with fully AI generated code that maybe we can just leave the loophole and do something if/when it becomes a problem?
maybe just add a sentence like so: This does not mean you can't contribute code written by other people if it has the proper open-source licenses.
There was a problem hiding this comment.
Contributing someone else's code would seem weird to me. Why isn't that other person the one submitting the PR?
This is fairly common when dealing with abandoned, salvaged PRs.
There was a problem hiding this comment.
It kind of seems like this scenario is a more general instance of submitting AI assisted work. When you submit a PR you are vouching that you understand it and believe it’s of suitable quality and suitably licensed for inclusion in the Rust project. Whether that was done using generative AI or by mining 30 year old repos for helpful open source code.
There was a problem hiding this comment.
yes, but also you can usually trust a widely used library to have generally correct code, so it doesn't need as thorough a review as the output of a LLM, e.g. if serde was being added to std I don't think the PR author needs to review all of serde's code in fine detail in order for them to be a responsible PR author, but if there was an equivalent amount of code written with a LLM for a PR I'd expect them to have to review it all in fine detail (actually that's big enough that I'd expect it to be split into many PRs for easier reviewing).
|
|
||
| ### Does this policy require disclosure of the use of generative AI tools? | ||
|
|
||
| This policy does not require disclosure of the use of generative AI tools. This is a complex question on which Project members have diverse views and where members are continuing to explore, investigate, learn, and discuss. Later policies may further address this. |
There was a problem hiding this comment.
I believe that disclosure of authorship should be required (which can go beyond AI, e.g. to acknowledge co-authors). When reviewing student work, I have found it very helpful to have a clear statement of whether AI was involved or not, since it reduces the guessing game in many cases. If someone falsely declares to have not used AI, it can also simplify moderation choices. While I would understand if a specific policy on disclosure is postponed so that the larger policy can be agreed upon more quickly, I do think disclosure should follow soon after.
There was a problem hiding this comment.
Authorship is something that applies to a person, not tools; a LLM can generate text, but it isn’t an author.
|
I feel like this should be better aligned with the other ongoing proposal we have for a similar purpose (rust-lang/rust-forge#1040). This RFC seems to propose a policy which errs on the side of allowing LLM use, while the other proposal proposes a policy which errs on the side of forbidding their use. Having both of these in place seems likely to lead to confusion on the part of contributors and team members alike. (Note: For what it's worth, I prefer the other policy's stance here) |
For the record, this is something that I explicitly pointed out in a private Zulip chat; by posting this RFC after the linked scoped policy was posted, this now effectively forces the two policies into conflict and potentially blocks the policy that was posted first from being merged, even though its exact purpose was to get a policy out sooner than an RFC. It's unlikely for this RFC to really make substantial process since it resulted from one team member deciding to write and submit it without being super vigilant about other policies and their states of progress. (This last is just an informed opinion.) Technically, I'm breaking my own decision to abstain from commenting on this RFC, although since this is mostly to clarify the situation for others who might not be in the private Zulip chat (for example, a majority of the community), I figured I'd make an exception. |
|
I skimmed through this proposal, trying to get a vibe. While I like the prose, I've put myself in the shoes of a contributor and I am left with a question: what can I actually do and can't do? The policy itself is 5 lines of text, the rest is meta (definitions, questions, and answers). I undestand the premise of "this policy describes only those items on which Project members agree" and while I agree with those 5 lines, I also think it needs to be more specific about do's and dont's, especially the dont's. I would abstract and generalize from past "incidents" (i.e. things that we don't want to happen) and present more practical suggestions. |
the other proposal is specifically only for the main https://github.com/rust-lang/rust repository and thus only asks for comments and approval for the relevant teams (that is why I've stayed out of it). I don't think that repository-specific proposal should dictate the general policy, because then the unrelated teams would've been excluded from the discussion.
IMHO this RFC proposes the minimum set of rules that everyone so far has agreed on. A policy "which errs on the side of forbidding their use" is unlikely to get consensus from the people in the project that do use AI, while the proposed policy at least has a chance, unless people block it because it doesn't go far enough in forbidding AI use. I'd personally rather have a policy that forbids the most problematic uses, than no policy at all. |
|
@clarfonthey -- I don't feel that's what happened here. I also do not think your prediction is correct. I'm in the private channel too, but I don't think we should be leveraging that here to advance our own positions. Both @jyn514's proposal and this RFC could go forward independently. From what I'm led to understand there are real challenges with r-l/r low-effort and AI-created contributions. But frankly speaking, I'd prefer we adopt this RFC. From what I've read it appears to be a careful compromise many of us can get behind. Adopting one policy for the whole Project would be more clear for contributors for the reasons @thomcc mentioned. This RFC seeks to provide that clarity. @jyn514's proposal is specific to one repository. Both ban the contributions that have been a problem for us recently. However, to @Turbo87's point: this RFC doesn't also ban things Project members such as @nikomatsakis are already successfully doing. |
|
For the record @PLeVasseur I both did clarify that whether the policy could go forward is just an informed opinion, not fact, and that I was intentionally planning to not comment on this RFC for now. So, please don't @ mention me to bypass the fact that I've muted this thread. |
|
We are straying from discussing technical merits of the RFC here, so I'll keep it brief. I do not appreciate having ill-intent assigned to me here on the @ mention as an intentional means to bypass the thread muting. I was genuinely unaware that @ mentioning someone bypasses a muted thread. Mea culpa on that. |
People have asked whether this policy bans vibecoding. It does. Let's articulate why and also answer why it bans slop and fully automated AI-generated contributions.
The RFC contains the policy items and a section of definitions, questions, and answers that together establish what the policy means and how it applies. We intend for most of these answers to be normative, but not all. Let's move the not-normative ones to a new section, then let's say explicitly which sections are normative.
Thanks for having a look and for this feedback. The definitions, questions, and answers are intended normatively as part of the policy: they define the meaning of the terms and describe how this policy applies. I've added a section to make this explicit. Based on hearing this and other feedback, I've now also added normative sections for:
Hopefully these help make the policy "more specific about do's and don'ts, especially the don'ts". |
The word *slop* has long meant a kind of putrid waste. This usage has recently seen a resurgence, and it's being applied more widely than to AI-generated content. In this context, we mean specifically *AI slop*, so let's say that.
|
|
||
| Other sections are not normative. | ||
|
|
||
| ## Contribution policy for AI-generated work |
There was a problem hiding this comment.
These all seem like good rules to follow, but are all nearly impossible for reviewers or moderators to enforce.
"is prohibited" feels like the wrong framing here as a result. "Do not X" is much more compatible with rules of this nature. If you would like to pursue a permissive, anti-slop AI policy, a "we will teach people to contribute effectively and pro-socially using these tools" is a much better fit than "we will ban you based on our vibes of your initial submission".
Obviously contributors who are learning-resistant or aggressively spamming will need moderation, but that was true well before LLMs.
|
|
||
| ### Is requiring that contributors take care an acceptable policy item? | ||
|
|
||
| To take care is to give something your full attention and treat its correctness as important to you. That's a meaningful distinction. As reviewers, we can tell when someone has taken care and when the person has not — there are many signs of this. |
There was a problem hiding this comment.
As a seasoned reviewer, I am very skeptical of the claim that reviewers can reliably tell when people have or have not taken care, especially in the context of LLM-assisted work.
|
|
||
| ## Motivation | ||
|
|
||
| In the Rust Project, we've seen an increase in unwanted and unhelpful contributions where contributors used generative AI. These are frustrating and costly to reviewers in the Project. We need to find ways to reduce the incidence of these and to lower the cost of handling them. |
There was a problem hiding this comment.
Will this policy meaningfully accomplish its goals? High-quality LLM-generated work, including from trusted contributors, still requires careful review. With an de-facto endorsement of LLM-generated contributions from trusted contributors, I worry that this will worsen review shortages on net.
|
|
||
| [Contribution policy for AI-generated work]: #contribution-policy-for-ai-generated-work | ||
|
|
||
| In all Rust Project spaces: |
There was a problem hiding this comment.
IMO it's important, for the avoidance of doubt, to explicitly say "AI-generated contributions that follow this guidance are allowed by default." I believe that's the intended effect of this policy, but you really have to read between the lines to get there.
Rather than prescribing an escalation ladder, this RFC leaves the handling of violations fully to the discretion of the moderators. Let's describe that explicitly.
We mean for this policy to apply to shared Project spaces and act as a Project-wide baseline. But we mean to allow teams to add prohibitions as they see fit. Let's say that more clearly.
Rejecting contributions for policy reasons can be hard on people. We need to offer a template to make this easier. Let's add one. Likely this will take further iteration to hit exactly the right notes.
|
Based on hearing helpful feedback from @kennytm and @alice-i-cecile (thanks for that), I've now added a template for reviewers to use when rejecting contributions under this policy. See: Having a template is necessary because rejecting contributions can be hard on people — hard on the reviewers and hard on those whose contributions are being rejected. We want the words we use to be thoughtful and well-crafted to make this difficult conversation as uneventful as possible. Probably it will take some iteration for this template to hit exactly the right notes. I'd welcome suggestions (leave review comments). Here's why the words there were chosen:
Unfortunately signals that we wish the outcome were different. Appears to be leaves room for the reviewer to be mistaken. The focus is on the work — this isn't a judgment of the person or the person's motives. And one or more means the reviewer doesn't have to pick a specific item; the contributor can self-assess.
Quoting the full list lets the contributor see what's being evaluated and reflect on where the contribution may have fallen short. Linking to the policy document gives this weight and answers questions.
This protects the reviewer. Without it, we would invite negotiation. The next steps further below do leave some room for reconsideration (if we got it wrong and the contributor can concisely explain why), but the contributor needs to click through for those details, and that's intentional. We want a considered response, not a snap one.
Both things are true: we trust the intent and the effect is harmful. We're separating the intent from the effects. The contributor's identity as a well-meaning person survives; the rejection is about the artifact and its effects on us.
This names the dynamic. It explains why we're doing this thing that is difficult for us. It's meant to be direct and not dance around our reasons.
This acknowledges the likely emotions of both sides. It's another aspect of naming the dynamic. We're emphasizing that we didn't make this decision lightly.
This provides paths for recovery. While we have slammed one door in the contributor's face, we want to point to the other doors that are open. |
In my model of tool use, the human is always the author. Where I'd used the word *authorship* here, I'd meant it in the sense of using tools to assist with human authorship. But it's easy to mistake this to mean something else. So let's avoid the word and make the phrasing more clear.
| In all Rust Project spaces: | ||
|
|
||
| - Submitting AI-generated work when you weren't in the loop is prohibited. | ||
| - Submitting AI-generated work when you haven't checked it with care is prohibited. | ||
| - Submitting AI-generated work when you don't have reason to believe you understand it is prohibited. | ||
| - Submitting AI-generated work when you can't explain it to a reviewer is prohibited. | ||
| - Feeding reviewer questions into an AI tool and proxying the output directly back is prohibited. |
There was a problem hiding this comment.
Can we simplify the wording, and since this is a normative section, and use RFC 2119 terms, something like.:
| In all Rust Project spaces, contributors MUST not: | |
| * Submit AI-generated work without meaningful involvement in its creation. | |
| * Submit AI-generated work that they have not carefully reviewed. | |
| * Submit AI-generated work that they do not reasonably believe they understand. | |
| * Submit AI-generated work that they cannot explain to a reviewer. | |
| * Submit AI-generated responses to reviewer feedback without independent understanding. |
There was a problem hiding this comment.
In all Rust Project spaces, contributors MUST not: * Submit AI-generated work without meaningful involvement in its creation.
the suggested wording falls in the trap I explained in this thread, it doesn't account for submitting code that someone else wrote by responsibly using AI.
There was a problem hiding this comment.
Context: https://github.com/rust-lang/rfcs/pull/3951#issuecomment-4286674950Do we assume this RFC 3950 is already in effect? 🤔
Additionally, assuming this RFC is merged eventually, can it be retroactively applied to all open PRs and issues etc?
There was a problem hiding this comment.
Most people would understand a new policy to apply prospectively, unless clearly stated otherwise.
|
Based on a discussion @nikomatsakis and the other Project Directors had with Foundation counsel, I've added a section addressing questions on the copyright situation with AI-generated work: |
| - Submitting AI-generated work when you can't explain it to a reviewer is prohibited. | ||
| - Feeding reviewer questions into an AI tool and proxying the output directly back is prohibited. | ||
|
|
||
| ## Definitions, questions, and answers |
There was a problem hiding this comment.
I think Q&A shouldn't be a normative section of the policy.
There was a problem hiding this comment.
Thanks for raising this item. There are two sections that have answers. One is focused on the rationale of the policy itself and is not normative.
The other contains definitions of the terms used in the policy items and specific guidance on how the policy items are to be interpreted. The policy comprises these definitions and this guidance, so this section is normative.
If there are specific items that you do not believe should be normative, I'd be curious to hear which and the reasons for that.
There was a problem hiding this comment.
In the current revision, the section "Definitions, questions, and answers" is marked as normative. This conflates distinct types of material that should be treated differently.
There was a problem hiding this comment.
I also raised this point (see comment) and here Travis' answer. I am not sure that adding more meta-content to the Q&A helps a lot.
The additions that you @traviscross drafted afterwards are good and make some fundamental points clear. As a reader I would just expect a policy to give me the "READ THIS PART!!111!!ONE!" more prominently :)
There was a problem hiding this comment.
Thanks. I have a revision in progress.
There was a problem hiding this comment.
It would also be helpful if the normative parts could be redrafted with RFC 2119 in mind.
There was a problem hiding this comment.
This conflates distinct types of material that should be treated differently.
Thanks for raising this. I've now separated these into Definitions, Applying the policy, and Guidance. Each section now starts with a description of the nature of that section and its normative effect.
There is some balance being applied here. Some of the guidance items both guide reviewers and contributors and have policy effect. If this were a legal document, I might separate things out further. It's not, so I'm prioritizing the narrative flow and avoidance of duplication by keeping these together.
The additions that you... drafted afterwards are good and make some fundamental points clear. As a reader I would just expect a policy to give me the "READ THIS PART!!111!!ONE!" more prominently :)
Thanks. I hear this. I'm hopeful that the upfront Normative sections section — which links to each normative section — and the new descriptions at the top of each section will help with this. I've also moved a couple of the items that fell more on the meta side out of the normative part.
It would also be helpful if the normative parts could be redrafted with RFC 2119 in mind.
I hear you, and at the same time, we don't generally write Rust RFCs in IETF RFC 2119-style. The spirit of that RFC is that it's important to be clear about what has normative force. It uses certain keywords for that (e.g., "MUST"). In this RFC, that work is done by phrases such as "is prohibited". While I'll keep the suggestion to use those keywords in mind, I'm not planning that revision at this time.
One of the common concerns about accepting AI-generated work is copyright risk. Niko and the other Project Directors asked Foundation counsel about this. Let's record the result of that discussion.
traviscross
force-pushed
the
TC/ai-contribution-policy
branch
from
f1fa62e to
e6a0265
Compare
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
The policy comprises the policy items and certain narrative elements that define the terms in the policy items, guide their interpretation, and extend them. We had put these narrative elements in a single "definitions, questions, and answers" section, but that's less clear than having separate sections for the different kinds of narrative items. Separating these out further allows us to describe why each is normative. At the same time, we don't want to separate them out too far. Some of our items are serving a dual role as guidance for reviewers and contributors while also influencing interpretation. It'd break up the narrative flow unnecessarily to be too rigid in separating these by purpose. Let's smartly separate these. In doing this, we'll move certain of the more descriptive meta items to a nonnormative section.
| ### Does accepting AI-generated work risk our ability to redistribute Rust? | ||
|
|
||
| What about the copyright situation? Since this policy does not ban AI-generated work, does that risk our ability to redistribute Rust under our license? Niko Matsakis [reports](https://nikomatsakis.github.io/rust-project-perspectives-on-ai/feb27-summary.html#the-legality-of-ai-usage): | ||
|
|
||
| > On this topic, the Rust Project Directors consulted the Rust Foundation's legal counsel and they did not have significant concerns about Rust accepting LLM-generated code from a legal perspective. Some courts have found that AI-generated code is not subject to copyright and it's expected that others will follow suit. Any human-contributed original expression would be owned by the human author, but if that author is the contributor (or the modifications are licensed under an open source license), the situation is no different from any human-origin contribution. However, this does not present a legal obstacle to us redistributing the code, because, as this code is not copyrighted, it can be freely redistributed. Further, while it is possible for LLMs to generate code (especially small portions) that is identical to code in the training data, outstanding litigation has not revealed that this is a significant issue, and often such portions are too small or contain such limited originality that they may not qualify for copyright protection. | ||
|
|
There was a problem hiding this comment.
To me, the following statements give the impression that the Rust project is ok with taking someone's work (for example, copyrighted under AGPL), laundering its license using an LLM, and then distributing it as a part of the compiler or standard library:
Some courts have found that AI-generated code is not subject to copyright and it's expected that others will follow suit.
However, this does not present a legal obstacle to us redistributing the code, because, as this code is not copyrighted, it can be freely redistributed.
Further, while it is possible for LLMs to generate code (especially small portions) that is identical to code in the training data, outstanding litigation has not revealed that this is a significant issue, and often such portions are too small or contain such limited originality that they may not qualify for copyright protection.
It seems to me that the RFC is taking a very controversial stand here in quite strong wording.
There was a problem hiding this comment.
This paragraph does not in any way imply that taking someone's copyrighted work and relicensing it is acceptable. It says that the danger of this happening accidentally through LLM use is not a significant risk because LLMs don't tend to reproduce large enough pieces of code to be copyrightable, unless explicitly prompted to by the user.
If it's happening intentionally, then that is no different from a user submitting copyrighted code without the LLM as a middleman (ie. that's already a risk that all open source projects take on) and so the use of an LLM is irrelevant.
Some courts have found that AI-generated code is not subject to copyright and it's expected that others will follow suit.
I think you may be confused by this statement: it's not saying that AI cannot output code which is subject to copyright. It's saying that new code generated by the AI is not subject to copyright (ie. the AI cannot hold copyright over something)
In his document titled *Rust Project Perspectives on AI*, Niko reports the result of a conversation between the Rust Project Directors and Foundation legal counsel. In the RFC, we link to this report. Niko had been hosting this document within his own space on GitHub; recently it was moved into the `rust-lang` organization in rust-lang/team#2439. This caused the URL of the document to change. Let's update the text to use the new one.
|
I think disclosure of LLM use is critical.
|
|
|
||
| Reviewers need to build a mental model of their own. They may want to know about yours in order to help them. You need to be able to articulate your mental model and the reasons you believe that model to be correct. | ||
|
|
||
| ### What's it mean to proxy output directly back to a reviewer? |
There was a problem hiding this comment.
Maybe less of an issue, but do we also want to say reviewers can’t just proxy LLM generated review comments back to the code author?
20 participants
View all comments
Summary
We adopt a Rust Project contribution policy for AI-generated work. This applies to all Project spaces.
Motivation
In the Rust Project, we've seen an increase in unwanted and unhelpful contributions where contributors used generative AI. These are frustrating and costly to reviewers in the Project. We need to find ways to reduce the incidence of these and to lower the cost of handling them.
We hope that by stating our expectations clearly that fewer contributors will send us unhelpful things and more contributors will send us helpful ones. We hope that this policy will make decisions and communication less costly for reviewers and moderators.
Policy design approach
People in the Rust Project have diverse — and in some cases, strongly opposed — views on generative AI and on its use. To address the problem in front of us, this policy describes only those items on which Project members agree.
Contribution policy for AI-generated work
In all Rust Project spaces:
Acknowledgments
Thanks to @jieyouxu for fruitful collaboration on earlier policy drafts. Thanks to @nikomatsakis, @ehuss, @tmandry, @oli-obk, @Kobzol, @lqd, @PLeVasseur, @eholk, @yoshuawuyts, @davidtwco, @jackh726, @Eh2406, and many others for thoughtful discussion.
All views and errors remain those of the author alone.
See the RFC text for definitions, questions, and answers.
cc @rust-lang/leadership-council
Rendered