AsciiGrid does not detect `-3.4e+38` as a noData value

[latwood]

Been playing around with no data filling methods. When I used bell_steep.tif converted to a new projection using the gdalwarp command line utility, the resulting noData values of -3.4e+38 wreaked havoc in the AsciiGrid fillNoDataValues() function, because if( get_cellValue(i,j) == get_noDataValue() ) was not seen as true. In addition, the call to if( input.dem.checkForNoDataValues() ) in readInputFile.cpp did not detect -3.4e+38 as a noData value, I had to swap to if( GDALHasNoData( poDataset, 1 ) ) to correctly detect the noData vals.

Note that the gui indirectly uses the GDALFillNodata() GDAL function, via calls to the gdal_util.cpp GDALFillBandNoData() function, which properly detects and fills -3.4e+38 as a noData value, but runs a heck of a lot slower than the AsciiGrid fillNoDataValues() function for larger datasets.

In the meantime, bell_steep.tif converted to a new projection using the gdalwarp command line utility works fine when specifying a no data value of 32767 for the destination band.

I did attempt a fix, similar to

windninja/src/ninja/ninjaMathUtility.cpp

Line 43 in 4fbac78

return std::fabs(a - b) < epsilonMultiplier*std::numeric_limits<double>::epsilon();

but not quite the same, with no luck. I used if( std::fabs(get_cellValue(i, j) - get_noDataValue()) <= tolerance ) where I just set tolerance to be a value of 0.000001. Printing the information, I found that the resulting comparison difference was 4.78556e+30, so it seems like some kind of overflow is potentially happening as well.

[nwagenbrenner]

@latwood is there still work that needs to be done for this issue?

[latwood]

I left it as it was, because we still haven't developed a strategy for how to deal with it yet.

Basically, all ascii grid if statement checks of, if it is a NO_DATA value, for certain values, will not be detected as NO_DATA values, causing all kinds of problems downstream. The GDAL equivalent functions seem to be able to handle these specific NO_DATA values though.

[nwagenbrenner]

It sounds like a precision problem comparing two huge doubles. I think you need to implement a tolerance check like you mentioned, but you might need to change the epsilon. You might trying scaling the epsilon to the nodata value we're checking. For example:

bool areDoublesEqual(double a, double b, double epsilon = std::numeric_limits<double>::epsilon()) {
    return std::abs(a - b) <= epsilon * std::max(1.0, std::max(std::abs(a), std::abs(b)));
}

[latwood]

Well, it's weird, because when I printed the raw values, there's definitely some kind of overflow going on.

std::cout << get_cellValue(i, j) - get_noDataValue() << std::endl;
prints
4.78556e+30

After experiencing something similar, stuffing the size_t npos output value of string.find() into an int resulted in a value of -1 rather than npos because of overflow, I've been wondering if the overflow for our NO_DATA values might be related to the type of the NO_DATA value we are using. But we still need to look more into it.