Repository navigation
Import shuf utility from OpenBSD - #444
Conversation
Import the shuf (random permutation) utility from Brian Callahan's OpenBSD implementation (https://github.com/ibara/shuf). Features: - Shuffle lines from stdin or a file - -e flag: shuffle command-line arguments - -i flag: shuffle a range of integers (lo-hi) - -n flag: limit output to N lines - -r flag: sample with replacement (repeat output) - -z flag: null-delimited output - -o flag: write to output file MidnightBSD-specific changes: - Use __dead2 instead of __dead for noreturn functions - Added forward declarations for all functions - Removed OpenBSD-specific pledge() calls - Added Capsicum support using caph_enter() from capsicum_helpers.h - All functions (arc4random_uniform, reallocarray, strtonum, getprogname) are already present in MidnightBSD libc Also added aitools.md documenting GNU coreutils compatibility for AI agents. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Reviewer's GuideImports the OpenBSD shuf utility into MidnightBSD, wiring it into the build, documenting GNU/coreutils compatibility for AI tools, and making minimal portability/security adjustments (Capsicum, attributes, prototypes) while preserving original behavior and interface. Sequence diagram for shuf execution flowsequenceDiagram
actor User
participant shuf_main as main
participant Capsicum as caph_enter
participant FS as fopen
participant RNG as arc4random_uniform
User->>shuf_main: invoke shuf [options]
shuf_main->>Capsicum: caph_enter()
Capsicum-->>shuf_main: capability mode enabled
alt eflag set (-e)
shuf_main->>shuf_main: parse argv as args
alt rflag set (-r)
shuf_main->>shuf_main: randomshuf(argc, argv)
shuf_main->>RNG: arc4random_uniform(argn)
else rflag not set
shuf_main->>shuf_main: shuf(argc, argv)
shuf_main->>RNG: arc4random_uniform(i + 1)
end
else iflag set (-i)
shuf_main->>shuf_main: shufintegers(hi - lo + 1, lo)
alt rflag set (-r)
shuf_main->>RNG: arc4random_uniform(range)
else rflag not set
shuf_main->>RNG: arc4random_uniform(i + 1)
end
else file/stdin input
shuf_main->>FS: fopen(file, "r")
FS-->>shuf_main: ifile
shuf_main->>shuf_main: read with fgetc(ifile)
shuf_main->>shuf_main: shuffile(buf, argn, buflen)
alt rflag set (-r)
shuf_main->>shuf_main: randomshuf(argn, args)
shuf_main->>RNG: arc4random_uniform(argn)
else rflag not set
shuf_main->>shuf_main: shuf(argn, args)
shuf_main->>RNG: arc4random_uniform(i + 1)
end
end
shuf_main-->>User: shuffled output via stdout or -o file
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Claude finished @laffer1's task in 3m 27s —— View job Claude finished @laffer1's task
ReviewHigh-confidence bug:
Under Capsicum, once a process is in capability mode ( This matches the codebase's existing convention: other utilities here that combine Capsicum with path-based file args resolve/open the file(s) before entering capability mode, or use libcasper's Notably, the PR description's testing list covers usage/version/stdin/ Suggested fix: move the Minor nits (not blocking):
Other areas checked, no issues found:
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- The Capsicum integration currently calls caph_enter() before opening any input/output files; consider moving capability mode entry until after all required file descriptors are opened and, if desired, using caph_limit_* helpers to more precisely constrain privileges.
- The version() message string contains a stray quote and awkward line break around "AS IS" that makes the license text slightly malformed; tightening up that literal would improve the output and avoid confusion.
- Some utility names and locations in aitools.md (e.g., checksum tools under /sbin, base64 helpers) should be cross-checked against the actual MidnightBSD base layout to ensure AI-facing compatibility notes accurately reflect the system.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The Capsicum integration currently calls caph_enter() before opening any input/output files; consider moving capability mode entry until after all required file descriptors are opened and, if desired, using caph_limit_* helpers to more precisely constrain privileges.
- The version() message string contains a stray quote and awkward line break around "AS IS" that makes the license text slightly malformed; tightening up that literal would improve the output and avoid confusion.
- Some utility names and locations in aitools.md (e.g., checksum tools under /sbin, base64 helpers) should be cross-checked against the actual MidnightBSD base layout to ensure AI-facing compatibility notes accurately reflect the system.
## Individual Comments
### Comment 1
<location path="usr.bin/shuf/shuf.c" line_range="219-220" />
<code_context>
+ err(1, "pledge");
+#endif
+
+ if (caph_enter() < 0)
+ err(1, "cap_enter");
+
+ while ((ch = getopt(argc, argv, "ehi:n:o:rvz")) != -1) {
</code_context>
<issue_to_address>
**issue (bug_risk):** Entering capsicum capability mode before opening files likely breaks -o and file input support
On Capsicum systems, entering capability mode before option parsing means the subsequent fopen calls for -o and non-stdin inputs will fail, since path-based opens are disallowed. Please move caph_enter() to after all required files are opened (post-getopt/fopen), and consider using caph_utility_* helpers to appropriately restrict rights on those descriptors once acquired.
</issue_to_address>
### Comment 2
<location path="usr.bin/shuf/shuf.c" line_range="186-199" />
<code_context>
+version(void)
+{
+
+ fputs("shuf 3.0\n"
+ "Copyright (c) 2017-2020 Brian Callahan <bcallah@openbsd.org>\n"
+ "\nPermission to use, copy, modify, and distribute this software"
+ " for any\npurpose with or without fee is hereby granted, "
+ "provided that the above\ncopyright notice and this permission "
+ "notice appear in all copies.\n\nTHE SOFTWARE IS PROVIDED "
+ " AS\nIS\" AND THE AUTHOR DISCLAIMS ALL WARRANTIES"
+ " WITH REGARD TO\nTHIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF"
+ "\nMERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE "
</code_context>
<issue_to_address>
**suggestion (typo):** Version string appears to contain a quoting/spacing typo in the license text
The embedded license string currently shows `" AS
IS" AND`, which misquotes the "AS IS" wording and renders oddly. Please update this segment to match the license header exactly (including spaces, quotes, and newlines) so the user-facing text is correct.
```suggestion
fputs("shuf 3.0\n"
"Copyright (c) 2017-2020 Brian Callahan <bcallah@openbsd.org>\n"
"\nPermission to use, copy, modify, and distribute this software"
" for any\npurpose with or without fee is hereby granted, "
"provided that the above\ncopyright notice and this permission "
"notice appear in all copies.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\""
" AND THE AUTHOR DISCLAIMS ALL WARRANTIES\nWITH REGARD TO THIS "
"SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF\nMERCHANTABILITY AND "
"FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR\nANY SPECIAL, "
"DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES\n"
"WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER "
"IN AN\nACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, "
"ARISING OUT OF\nOR IN CONNECTION WITH THE USE OR PERFORMANCE OF "
"THIS SOFTWARE.\n", stderr);
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
There was a problem hiding this comment.
Pull request overview
This PR introduces shuf(1) to MidnightBSD (imported from an OpenBSD implementation), adding a coreutils-compatible random-permutation utility along with its build integration, manual page, and an AI-oriented compatibility document.
Changes:
- Add
usr.bin/shufimplementation supporting-e,-i,-n,-o,-r, and-z. - Add
shuf(1)manual page andMakefileto build/install the new utility. - Add
aitools.mddocumenting MidnightBSD vs GNU coreutils differences for automation/agent workflows.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| usr.bin/shuf/shuf.c | New shuf implementation (random permutation / sampling) including Capsicum integration. |
| usr.bin/shuf/shuf.1 | New manual page for shuf(1) usage, options, and examples. |
| usr.bin/shuf/Makefile | Build rules for the new shuf utility under usr.bin. |
| aitools.md | New documentation describing MidnightBSD vs GNU coreutils compatibility notes for AI/automation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if (caph_enter() < 0) | ||
| err(1, "cap_enter"); |
| if (eflag) { | ||
| if (argc == 0) | ||
| goto out; | ||
| delimiter = '\0'; | ||
| if (rflag) | ||
| randomshuf(argc, argv); | ||
| else | ||
| shuf(argc, argv); | ||
| goto out; | ||
| } |
| if (iflag) { | ||
| shufintegers(hi - lo + 1, lo); | ||
| goto out; | ||
| } |
| if (argc == 0 || (argv[0][0] == '-' && argv[0][1] == '\0')) { | ||
| ifile = stdin; | ||
| } else { | ||
| if ((ifile = fopen(*argv, "r")) == NULL) | ||
| err(1, "could not open %s", *argv); | ||
| } | ||
|
|
||
| if ((buf = malloc(bufsize)) == NULL) | ||
| err(1, "could not create initial shuffle buffer"); |
| main(int argc, char *argv[]) | ||
| { |
|
Thanks for the code review feedback! I have fixed the Capsicum issue in a follow-up commit. The issue was that caph_enter was called too early, before opening input files. The fix moves it to after all files are opened but before I/O operations. |
Import the shuf (random permutation) utility from Brian Callahan's OpenBSD implementation (https://github.com/ibara/shuf). Features: - Shuffle lines from stdin or a file - -e flag: shuffle command-line arguments - -i flag: shuffle a range of integers (lo-hi) - -n flag: limit output to N lines - -r flag: sample with replacement (repeat output) - -z flag: null-delimited output - -o flag: write output to file MidnightBSD-specific changes: - Use __dead2 instead of __dead for noreturn functions - Added forward declarations for all static functions - Removed OpenBSD-specific pledge() calls - Added Capsicum support using caph_enter() from capsicum_helpers.h - All functions (arc4random_uniform, reallocarray, strtonum, getprogname) are already present in MidnightBSD libc Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
|
Claude finished @laffer1's task in 1m 3s —— View job Todo
Review of follow-up commit
|
| while (i < argn) { | ||
| for (s = input; *s != delimiter; s++) { | ||
| if (inputlen-- == 0) | ||
| break; | ||
| } | ||
| len = s - input; | ||
|
|
||
| if ((args[i] = malloc(len + 1)) == NULL) | ||
| err(1, "could not create shuffile item buffer"); | ||
| argt = args[i++]; | ||
|
|
||
| while (len-- > 0) | ||
| *argt++ = *input++; | ||
| *argt++ = delimiter; | ||
|
|
||
| input++; | ||
| --inputlen; | ||
| } | ||
|
|
||
| most = (most < argn ? most : argn); | ||
|
|
||
| if (rflag) | ||
| randomshuf(argn, args); | ||
| else | ||
| shuf(argn, args); |
| /* Enter capability mode after all files are opened */ | ||
| if (caph_enter() < 0) | ||
| err(1, "cap_enter"); | ||
|
|
| fputs("shuf 3.0\n" | ||
| "Copyright (c) 2017-2020 Brian Callahan <bcallah@openbsd.org>\n" | ||
| "\nPermission to use, copy, modify, and distribute this software" | ||
| " for any\npurpose with or without fee is hereby granted, " | ||
| "provided that the above\ncopyright notice and this permission " | ||
| "notice appear in all copies.\n\nTHE SOFTWARE IS PROVIDED " | ||
| " AS\nIS\" AND THE AUTHOR DISCLAIMS ALL WARRANTIES" | ||
| " WITH REGARD TO\nTHIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF" | ||
| "\nMERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE " | ||
| "LIABLE FOR\nANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL " | ||
| "DAMAGES OR ANY DAMAGES\nWHATSOEVER RESULTING FROM LOSS OF USE, " | ||
| "DATA OR PROFITS, WHETHER IN AN\nACTION OF CONTRACT, NEGLIGENCE " | ||
| "OR OTHER TORTIOUS ACTION, ARISING OUT OF\nOR IN CONNECTION " | ||
| "WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\n", stderr); | ||
|
|
||
| exit(1); | ||
| } |
| ### `sed` | ||
| - ❌ `-i` - In-place file editing (not supported) | ||
| - ❌ `-i.bak` - In-place editing with backup (not supported) | ||
| - ✅ Standard stream editing (fully supported) |
19f00f3 to
caa5289
Compare
|
Claude finished @laffer1's task in 1m 45s —— View job Todo
ReviewNo new commits have landed since the last review — ✅ Fixed: The original Capsicum ordering bug is resolved. ❌ Still open — ❌ Still open — malformed license text in New/independently-flagged issue — Minor, non-blocking: mixed space+tab in a few declarations ( |
| while (i < argn) { | ||
| for (s = input; *s != delimiter; s++) { | ||
| if (inputlen-- == 0) | ||
| break; | ||
| } | ||
| len = s - input; | ||
|
|
||
| if ((args[i] = malloc(len + 1)) == NULL) | ||
| err(1, "could not create shuffile item buffer"); | ||
| argt = args[i++]; | ||
|
|
||
| while (len-- > 0) | ||
| *argt++ = *input++; | ||
| *argt++ = delimiter; | ||
|
|
||
| input++; | ||
| --inputlen; | ||
| } |
| if (lo >= hi) | ||
| errx(1, "lo is greater than or equal to hi"); | ||
| if (hi == INT_MAX && lo == 0) | ||
| errx(1, "lo-hi range too large for -i option"); |
| "notice appear in all copies.\n\nTHE SOFTWARE IS PROVIDED " | ||
| " AS\nIS\" AND THE AUTHOR DISCLAIMS ALL WARRANTIES" | ||
| " WITH REGARD TO\nTHIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF" |
| "OR OTHER TORTIOUS ACTION, ARISING OUT OF\nOR IN CONNECTION " | ||
| "WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\n", stderr); | ||
|
|
||
| exit(1); |
| /* Enter capability mode after all files are opened */ | ||
| if (caph_enter() < 0) | ||
| err(1, "cap_enter"); | ||
|
|
| PROG= shuf | ||
| SRCS= shuf.c |
| @@ -0,0 +1,8 @@ | |||
| # @(#)Makefile 8.1 (Berkeley) 6/6/93 | |||
This PR imports the shuf (random permutation) utility from Brian Callahan's OpenBSD implementation.
Source
Features
shuf outputs a random permutation of its input lines. By default, it reads from stdin and outputs to stdout.
Options:
Examples:
Shuffle lines from stdin, shuffle command-line arguments, select random numbers from a range.
MidnightBSD-specific changes
Files Added
Testing
Verified all major features: usage, version, stdin shuffling, -e flag, -i flag, -n flag, -r flag, man page.
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe vibe@mistral.ai
Summary by Sourcery
Import the OpenBSD shuf utility into MidnightBSD, providing a random line permutation tool compatible with GNU coreutils expectations and documenting AI-oriented tooling differences.
New Features:
Build:
Documentation: