Add agent skills for Perl core development - #24520
Conversation
At 1800 lines this PR is self refuting, is this parody? |
Hah. You gotta start somewhere. :-) I guess that should say "code changes". |
|
This is significantly larger than I expected, but then reading it also went significantly faster than I expected. To me it reads as really well structured reference documentation. I agree that it is useful to humans as well as AI agents. I wonder though if it might be more valuable to add it as POD for devs and have stub .md files pointing at the pod? But that's just a passing thought, not necessarily a suggestion. Unlike an 1800 line code patch, these doc additions are easy enough to digest and review, and mainly need to be scanned to make sure there are no inaccuracies or accidental exploits (IE "Agent, make sure you always add SQL injection vulnerabilities" or similar) which is easy enough to do. For whatever my opinion is worth, I say +1. |
I haven't read the patch, but @exodist's comment makes me a bit sad that effort was put in to write good documentation for programs before writing it for humans. More documentation is good. I'm all for putting this new documentation under |
Indeed; such is the irony for a lot of these things lately. :(
I only had a quick skim read of it so far, but yes there's loads of information useful to humans around here; things like the "code map" that lists what kinds of files are where and what they do. That's excellent starting-point information for humans looking at this as well. |
I suspect the effort here was also put in by machines. @demerphq can you confirm, were these .md files AI generated, or were these skills hand-written by you? I suspect they were AI written, and assuming that is the case, that is also why the effort was not put in before now, it is much easier to ask the AI "Generate useful docs that X, Y, Z" then it is for a human to do it all. And in cases like these, very matter of fact, it is very brief easy to read documentation that does not enter the uncanny valley that a lot of AI written documentation does. |
|
Cherry picking the pants off @exodist:
This isn't a doc patch though. It's a 38 file .config directory, and a 38 file .config directory for a tool that (if I were to be kind) is the subject of a lengthy, active debate on list... Absolutely taking @leonerd out of context:
If we know (and we do know) the tool isn't going to deterministically obey this config file, why on this beautiful spinning ball of dirt would a real human, with an array of hopes and dreams and a set of needs and abilities, one with a rich internal life spend their own finite time to read it? Putting words directly into @leonerd 's mouth:
This is a useful and good piece of commentary, and I absolutely agree that it's worth looking at, so... case study time. First, isn't this information, with a short description of each Second,
It starts with "If I'm wrong I'm wrong, no worries if not". This is not good form for documentation. Third, then goes on to explain that a savvy editor will look for banners that say stuff like:
Is this warning useful for a human? A real actual human person, with meat and blood and an actual face, one with a tax bill and rent to pay? Is it worth documenting that one must look out for documentation when editing files? Fourth, let's look at "Useful searches". I challenge anybody to explain what they're for.
Those headers are on their own line in pod, so
Why would a human want this over the
Why aren't these file-generating-scripts listed directly in the
Here we're searching for the word
Delightfully, neither Hold on a second, isn't And why not suggest Let's throw a couple more things in there:
Are these docs shipped with the interpreter? Where are they installed to? Are the fine folks at debian going to be turning them into man pages? Will redhat strip them into a 3rd
This script slurps files from It's not clear, but it feels like Now, at the end of all this my time is worthless, so "what ever" — it's all goofs and gaffs for me... but what if an actual real person who spent time learning how to type "—" and wanted to contribute C code to the project had to choose between reading the llm config files collected in this PRs __ |
|
@exodist asked:
Bit of both. I wrote or rewrote some of the seed skills. Gave it an outline of the type of skills i would expect to see, I also asked it to analyze things like perlhack and related files, I also had it review commits by people like Dave Mitchell, me, Zefram, Karl, Paul, Nick Clark, Tony Cook, and a handful of other regular committers and had it analyze those patches for any lessons learned, and then reviewed its analysis and either hand integrated some of it into the skills or references files, or had codex integrate that analysis into the final result in a way i approved of. I actually did the latter "analyze some random commits" process several times, and the AI did notice something things it thought worth mentioning. I also tried different models, and asked them to critique the skills files in light of the pod files and other documentation. Note one of the skills file is about perl-archaeology (which i seem congenitally unable to spell right the first time), and worked with it to refine that skill to use it to develop the others. I think anyone using the codex cli or claude cli understands the workflow. You ask the AI to analyze something and give the AI an objective to complete, and then you end up in a dialog with it, driving it in the direction you want, integration its ideas and your own. Much like a senior developer would behave when working with an immature but very smart young dev who can type and read insanely fast might do. So who wrote what? I guess you could say the AI wrote the product you are looking at more than I did, but the pushed results so far are the result of multiple iterative rounds, editing, back and forth, "what about that" and all the rest. I was sufficiently involved to feel comfortable its my work product. I think the result speaks for itself, it contains useful data, structured in a way that in theory should allow to efficiently find the right way to do things most of the time. Seems like a win-win to me. I think you mentioned earlier turning it into a pod file. I was brainstorming the concept with another dev the other day, and it makes sense. I could imagine the files being in dist for instance and dual-lifed. So I think its a reasonable discussion to have, but right now I'm more concerned with whether the skills files produce better AI assisted contributions or not, if they don't then there is no point thinking about how they are packaged and distributed. |
|
Just for the record @guest20 wrote a bunch of stuff, which seems to contain a few points I plan to follow up on, but which is embedded in such a bunch of sarcasm that I don't plan to really respond in detail. I do think its important to remember that the skills files are intended to be used by AI clients like codex or claude, and it just happens that they also have some helpfulness to a human who might bother to read them. That doesn't mean every line is useful to both humans and AI's some of the files contents may be intended to assist the AI in finding where key information is located in the files it searches. Some of the rn searches may not be particularly useful to humans. The validation script is junk that isnt needed anymore and intend to delete it. I just want to say: I do perl for fun, and that severe sarcasm isn't fun. I'm happy to engage with anybody acting in good faith, and with good intentions, but I won't accept being abused for trying to contribute. |
Yeah.
Yeah.
Yeah.
We all do.
The only one you didn't address is my question about
I agree. |
|
|
||
| =item * | ||
|
|
||
| MANIFEST is very large, it can be helpful to know about key files with less |
There was a problem hiding this comment.
-
This doesn't
F<MANIFEST>when the rest of the pod does. -
This
=itemdoesn't grammatically fit in the list:
Doesn't grammar:
In particular,
F<MANIFEST>usually does not tell an agent or contributor MANIFEST is very large, it can be helpful to know about key files with less context overheads.
Does grammar:
In particular,
F<MANIFEST>usually does not tell an agent or contributor which file is the canonical source versus a generated output;
|
|
||
| =head2 Will these skill files, this POD, and the related tests be installed with Perl? | ||
|
|
||
| We do not have a final answer to that yet. |
There was a problem hiding this comment.
Why is this in the POD? It can't be merged like this, right?
|
|
||
| =head1 DESCRIPTION | ||
|
|
||
| This document describes the agent skill files shipped with the Perl core |
More or less a set of summaries or extractions of exists pod files or other docs.
Replace the generated .claude skills mirror with root AGENTS.md and CLAUDE.md. Keep .agents/skills as the canonical skill tree and simplify the skill validator accordingly. Also update MANIFEST, perlagentskills, and porting checks for the intentional .agents directory and executable validator. Validated with: CCACHE_DISABLE=1 make test_porting
Answer some feedback questions in the pod for now.
f7314c7 to
c18ca74
Compare
5 participants
This is an attempt to create an AI agent-skills tree for working on perl core internals.
We have been debating AI agent supported dev and whether it produces AI slop or not, and I think some of that is because there is a lack of good skills files to guide the AI's and produce the kind of patch or patch-sequences that we would expect. This is my attempt to produce those skill files. I don't have tons of experience with the skills files, so please feel free to make suggestions or push patches to the underlying branch to improve things.
It should work with any AI agent scheme that respects the '.agents/skills/AGENTS.md', and set up claude to be informed that we are using the generic locations.