Should we add more ways to parse raw files and sections?

[NikolajSchlej]

Both @platomav (in a recent discussion in #270) and @Ykidia (in his recent PR #423) expressed the need to parse contents of raw files and raw sections.

The problem with this approach for a generic tool like this one is that such parsing can only be done by trying multiple parsers in a row, and choosing something that generated less parsing errors.

Another possible issue is on trying to rebuild such a raw file (when we get to that point again), given that there are basically not spec-compliant ways to rebuild such a file.

I'm notionally OK with adding more parsers for the sake of increasing usability as a reverse-engineering tool, and we already got way off the original plan to only parse/edit UEFI PI images, so the only issue I currently have is "adding new code needs to be justified, and we will have to support that added code forever into the future".

CC @vit9696 for his opinion.

[platomav]

In my opinion, it is important to be able to detect Volumes, Files and/or Sections within non-expected areas (raw, padding, non-UEFI) because it's not rare for vendors to place things there. This is very useful when doing research or reverse engineering.

One important topic is that of re-building that you mentioned. The way I see it, we should not have to worry about that. At least not before a builder has been written for the expected/canonical UEFI structure. I suggest to "simply" not allow editing of FV/FILE/SECTION which were found in non-standard/expected areas (with some parser note/warning indicating that of course).

Regarding methodology, and based on what I've observed from another project which added such parsing (Chipsec), the method which works is to attempt FV first, otherwise fallback to Files, otherwise fallback to Sections. Yes, the last 2 don't have strong methods to verify, so it would be expected to get errors from such brute-force parsers. As an extra (more brute-forcy) option, there could be a way to also allow checking for FV + Files + Sections without stopping at first match. This may be useful for (even weirder) cases where an FV is included as part of the data of a PE/TE Section. These do sound (and may look) ugly, so maybe it could be optional instead of always being attempted.

This type of topic has been discussed at the Chipsec project, where I had also provided various samples which can be used for testing and/or proof that these brute-force parsing modes are needed:

• Fallback parsing of detected UEFI volumes within padding/non-UEFI sections chipsec/chipsec#1790
• Chipsec fails to detect Computrace EFI binaries within UEFI image chipsec/chipsec#2197
• Chipsec fails to detect VariableSmm EFI binaries within UEFI images chipsec/chipsec#2253
• Added environment variable for brute force EFI module scan at HAL chipsec/chipsec#2459