f_startswith

[mcenerney1]

When running diags, f_startswith is frequently used. For eaxmple f_startswith('HadISST'). The problem in this case is that each of 'HadISST_PD_1', 'HadISST_PI_1', 'HadISST_1' starts with HadISST. So this particular case may or may not be computed. The latter appears to be happening. There are other cases of this for other experiments. This is a significant issue.

[painter1]

f_startswith('foo') should be about the same as the shell expression "foo*". For this situation, we need a different filter. This particular case may work with things like f_startswith('HadISST_1'), but in general f_startswith cannot be a complete solution. We could add more filters, or support traditional regular expressions, or let the shell expand an expression and take a (lengthy) file list instead (that probably works already, but I'm not sure.) For very complicated situations, I think that the best solution is for the user to set up more than one directory.

[mcenerney1]

The kludge I'm pursuing is to rely on the name of the file to have some extra XXX as in the case of HadISST_PD_ and HadISST_CL_. The other cases such as CERES follow this pattern. This solution is easily defeated and we need to rethink it.

[painter1]

So you're thinking of a new filter. The filenames we encounter are (at present, for ACME and obs only) very standardized, so I wouldn't call it a kludge if you're only targeting our present users. Anyway, I'd give the new filter a new name.