feat(langfuse): add CI/CD experiment gate guidance#45
Conversation
|
@Lotte-Verheyden: @annabellscha suggested creating a skill for the new CI/CD action so here it is :-) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0afd6ff390
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…cli commands - Revert the top-level SKILL.md description change and remove the broad github.com allowed-tools entries (per agents.md skill-authoring guidelines). - Drop the routing sentence from the ci/cd reference body and de-pin SDK version numbers that would go stale. - Replace hardcoded langfuse-cli and gh secret command blocks with pointers to references/cli.md and the action README, matching the cli.md convention.
Lotte-Verheyden
left a comment
There was a problem hiding this comment.
Thanks @wochinge! I reviewed this and pushed one commit (1e1dde9). Reasoning below, plus a bigger question for the remaining part:
I made the following changes (I also added these as general guidelines in the agents.md file):
- Reverted the top-level description edit, because that field only controls invocation, and a CI/CD request likely already mentions Langfuse/eval so it would trigger anyway.
- Dropped the broad github.com allowed-tools. When an action isn't on it, the agent just gets a one-time “always allow” prompt the first time. We keep the list to read-only LF-specific commands so this list does not become a point of hesitation for people who want to install the skill.
- Removed the hardcoded CLI/gh blocks + de-pinned SDK versions so that the skill doesn't go stale when we update anything to our SDK/CLI. 2 of the hardcoded commands were in fact already wrong (scores list doesn’t exist; get-get-runs takes a positional, not --dataset-name).
For the part that remains, it looks very long for what it needs to do, knowing that the agent is also already fetching the docs. I didn't test it completely, did you try a test flow with vs. without the skill update on your side? I wanted to trim it to information that's not on the docs page, and what was left wasn’t very long.
--> If the workflow works as well without the skill update as with, I would say we don't include this as a specific use case. If it doesn't, we trim it to the necessary lines and add it to the skill :)
Summary
langfuse/experiment-action.langfuseskill and README, and bump both plugin manifests to1.1.0for the new published capability.Linear
Review Focus
1.0.1to1.1.0matches the repo's published-skill rules.