fix(fuzz): fall back to fs operator so fuzz targets produce real coverage#7723
Conversation
| log = { workspace = true } | ||
| logforth = { workspace = true } | ||
| opendal = { path = "..", features = ["tests"] } | ||
| opendal = { path = "..", features = ["tests", "services-fs"] } |
There was a problem hiding this comment.
Good find! Do we need a fallback since we could always exit 1 and warn the user in fuzzer test? Because a test failure will force us to check CI status.
There was a problem hiding this comment.
Thanks @erickguan! Good question. The fallback is deliberately scoped to the unconfigured case only:
init_test_service()returnsOk(None)whenOPENDAL_TESTis unset,Ok(Some(op))when a service is selected and inits, andErr(..)when a selected service fails to init.- So
init_test_service().expect("operator init must succeed")still panics loudly if someone explicitly sets e.g.OPENDAL_TEST=s3and it fails to init — a genuine misconfiguration does fail the run and shows up in CI, exactly as you want. Thefsfallback only kicks in for theOk(None)(nothing selected) case.
The reason we fall back instead of exit 1 in that unconfigured case is OSS-Fuzz: it runs these binaries continuously with no OPENDAL_TEST env set (that's the root cause of the zero coverage in #7722). If the target exited 1 / panicked when unconfigured, OSS-Fuzz would just record every run as an immediate crash and we'd still get no useful coverage — the same dead state we're fixing, only louder. Falling back to fs (which needs no external setup beyond a temp dir, and exercises OpenDAL's real path-handling code) gives the fuzzer something concrete to drive, while local/CI runs can still target any specific service via OPENDAL_TEST.
Happy to switch the unconfigured default to memory instead if you'd prefer to avoid filesystem side effects — just let me know which you'd like.
There was a problem hiding this comment.
Added a log::warn! on the unconfigured fallback per your suggestion (pushed in 1468565) so it's visible in logs. PTAL!
There was a problem hiding this comment.
Thanks for the context. I understand the problem much better.
If the target exited 1 / panicked when unconfigured, OSS-Fuzz would just record every run as an immediate crash and we'd still get no useful coverage — the same dead state we're fixing, only louder
Indeed true. Having a silent fallback to fs works to bring back fuzz coverage. This would be a good fix.
Now considering a maintenance perspective, if Ok(None) panics, any maintainers would be able to tell something requires for a fix. It might be stringent to understand fuzz setup but that would help someone else to jump in.
Nevertheless, I would be happy to see we are getting fuzz coverage. So your call.
|
For reviewers' context: this also unblocks the OSS-Fuzz side — google/oss-fuzz#15648 updates the build to pull these four targets and fixes the moved repo URL, and is just waiting on this to land. CI here is green; happy to address any feedback. Addresses #7722. |
erickguan
left a comment
There was a problem hiding this comment.
See inline comment. I would be happier to see a panic test without env. Also happy to see the ball gets rolling.
|
Also thanks for taking care of oss-fuzz! |
Which issue does this PR close?
Closes #7722.
Rationale for this change
All OSS-Fuzz targets exit immediately when
OPENDAL_TESTis unset becauseinit_test_service()returnsNone. OSS-Fuzz never sets this variable, so coverage has been stuck at 0.26% (harness startup only) since integration.What changes are included in this PR?
fuzz_reader.rs/fuzz_writer.rs: replace theif let Some(op)early-return with aLazyLock<Operator>that falls back to a localFsoperator in a temp directory when no test service is configured.fuzz/Cargo.toml: enableservices-fsfeature so the fallback compiles.fuzz_path.rstarget: feeds arbitrary strings as paths throughwrite/stat/list/copy/rename/delete— already found a real bug (fix(services/fs): preserve backslash in filenames on Unix during list #7721) within 2 minutes of running.Are there any user-facing changes?
No. Changes are limited to the
core/fuzz/directory which is only used bycargo fuzzand OSS-Fuzz.AI Usage Statement
Claude was used for assistance with fuzz target development.