feat: add csv output support with customizable fields (#745) - #746
Merged
Conversation
* fix body read err * feat(agent): add daydaymap search engine support Add daydaymap as a new supported search engine with complete integration: - Add daydaymap agent implementation in sources/agent/daydaymap/ - Integrate daydaymap CLI option (-ddm/--daydaymap) in runner options - Add daydaymap API key configuration in provider and keys - Register daydaymap in available engines list - Support DAYDAYMAP_API_KEY environment variable The implementation follows the existing agent pattern and maintains consistency with other search engine integrations (greynoise, driftnet, etc). * adding docs * fix: thread ctx through sources.Agent (projectdiscovery#724) --------- Co-authored-by: Doğan Can Bakır <dogancanbakir@protonmail.com> Co-authored-by: Mzack9999 <mzack9999@protonmail.com> Co-authored-by: taielab <52345183+taielab@users.noreply.github.com> Co-authored-by: Dogan Can Bakir <65292895+dogancanbakir@users.noreply.github.com>
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds a ChangesCSV Output Support
Estimated code review effort: 2 (Simple) | ~10 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
runner/runner_test.go (1)
20-39: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winStrengthen this test to cover escaping and deduplication.
Right now this only proves that a simple row is present somewhere in the buffer. It does not catch regressions in the two CSV-specific contracts added in
runner/output_writer.go: RFC-4180 escaping and duplicate suppression inWriteCSVData. A second write of the same result plus a field containing a comma/quote would make this test cover both behaviors.Suggested test tightening
fields := []string{"ip", "port", "host"} writer.WriteCSVRow(fields) result := sources.Result{ - IP: "192.168.1.1", - Port: 80, - Host: "localhost", + IP: "192.168.1.1", + Port: 80, + Host: `local,"host"`, } writer.WriteCSVData(result, fields) + writer.WriteCSVData(result, fields) output := buf.String() - expectedHeader := "ip,port,host\n" - expectedRow := "192.168.1.1,80,localhost\n" - - if !strings.Contains(output, expectedHeader) { - t.Errorf("Expected output to contain header %q, got %q", expectedHeader, output) - } - if !strings.Contains(output, expectedRow) { - t.Errorf("Expected output to contain row %q, got %q", expectedRow, output) + expected := "ip,port,host\n192.168.1.1,80,\"local,\"\"host\"\"\"\n" + if output != expected { + t.Errorf("Expected output %q, got %q", expected, output) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@runner/runner_test.go` around lines 20 - 39, Strengthen the CSV writer test in runner_test by verifying the behaviors implemented in WriteCSVData and WriteCSVRow: add an input field value that requires RFC-4180 escaping (for example, a comma or quote) and assert the written output is escaped correctly, then call WriteCSVData twice with the same sources.Result and confirm the duplicate row is suppressed. Use the existing writer, buf, and output assertions to check the exact CSV output rather than only searching for a substring, so the test covers both escaping and deduplication.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@runner/options.go`:
- Around line 225-227: The CSV default normalization in configureOutput() is
only applied before loadConfigFrom(), so config-driven csv: true values can
leave OutputFields as ip:port instead of the expected ip,port,host. Update
ParseOptions() so the CSV OutputFields rewrite is applied again after config
merging, or move the existing options.CSV and options.OutputFields check to run
after loadConfigFrom() using the same configureOutput() logic.
In `@runner/output_writer.go`:
- Around line 94-103: The duplicate suppression in the CSV writer currently uses
the endpoint-derived key in output_writer.go instead of the projected row
content, so distinct rows can be dropped when the selected fields differ. Update
the duplicate check in the row-writing path that builds dupKey and calls
findDuplicate so it derives the key from the rendered values produced by
getFieldValues for the current fields set, rather than from
data.IP/data.Port/data.Host/data.Url. This should keep deduplication aligned
with the actual CSV output while preserving the existing duplicate tracking
logic in findDuplicate.
---
Nitpick comments:
In `@runner/runner_test.go`:
- Around line 20-39: Strengthen the CSV writer test in runner_test by verifying
the behaviors implemented in WriteCSVData and WriteCSVRow: add an input field
value that requires RFC-4180 escaping (for example, a comma or quote) and assert
the written output is escaped correctly, then call WriteCSVData twice with the
same sources.Result and confirm the duplicate row is suppressed. Use the
existing writer, buf, and output assertions to check the exact CSV output rather
than only searching for a substring, so the test covers both escaping and
deduplication.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 0448a451-2469-431a-9a07-a19a9f6fead3
📒 Files selected for processing (4)
runner/options.gorunner/output_writer.gorunner/runner.gorunner/runner_test.go
ThryLox
force-pushed
the
feat/csv-output
branch
2 times, most recently
from
June 30, 2026 05:28
17124fa to
5d8c8cc
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@runner/output_writer.go`:
- Around line 100-104: The parseFields helper is only treating a literal space
as whitespace, so tab- or newline-separated field lists are not split correctly.
Update the strings.FieldsFunc predicate in parseFields to use unicode.IsSpace
for the whitespace branch while keeping the existing comma, colon, and semicolon
separators, so output handling stays aligned with the intended whitespace
parsing.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: f091faca-7c80-4799-9554-92b1faa2e85d
📒 Files selected for processing (4)
runner/options.gorunner/output_writer.gorunner/runner.gorunner/runner_test.go
🚧 Files skipped from review as they are similar to previous changes (3)
- runner/runner.go
- runner/runner_test.go
- runner/options.go
Fixes projectdiscovery#745. Introduces a new -csv / -c flag to output results in CSV format, supporting customizable fields aligned with the -field flag (ip, port, host, url). Defaults to 'ip,port,host' if not overridden. Added unit tests for CSV serialization in the runner package.
dogancanbakir
approved these changes
Jun 30, 2026
dogancanbakir
left a comment
dogancanbakir
left a comment
Member
There was a problem hiding this comment.
LGTM -pls update README
Mzack9999
approved these changes
Jul 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #745. Introduces a new
-csv, -ccommand-line flag to output uncover search results in standard CSV format, integrating seamlessly into existing pipelines (httpx -csv,nuclei -csv).Key Changes
-csv / -cflag tooptions.goto format output as CSV.ip,port,hostwhen-csvis used without explicitly overriding the-fieldflag.encoding/csvpackage in theOutputWriterto ensure robust escaping and RFC-4180 compatibility.runner/runner_test.goverifying field parsing, CSV formatting, and duplicate checks.Summary by CodeRabbit
--csv(-c) CLI option to enable CSV-formatted output.ip,port, andhostby default.JSON,Raw, andCSVoutput options.