Handling Null Values

[cfortenbach]

The plugin requires all fields be assigned a value (i.e., cannot handle null values). This requires the user to either manually enter 0 or to use a layout with the areas with null values being omitted. It'd be great to have an option to treat nulls as 0 or to leave the areas with null values unchanged in area.

[christophfink]

This is a really good observation! Thank you for reporting it.

If you don’t mind, what’s your use case or data source, in which NULL values are frequent?

To be perfectly honest, at first, I was a bit hesitant to implement data mangling beyond a simple warning the user. There are simply too many different reasons why a data set might have NULL values, and - as you already wrote - no one-fits-all way of treating such features exists:

• for instance, in same cases, it might be good to set NULL values to zero (the Dougenik algorithm then sets them to 1), which shrinks features to practical invisibility.
• In other cases, it might be more appropriate to skip such features alltogether (and leave a whole in the output layer).
• In yet another variant, it might be advisable to assign a value that’s equivalent to the feature’s original areal share, in order to retain its approximate size (but that should be accompanied with appropriate styling to highlight the data deficiency).
• There are probably still other options that don’t come to me from the top of my head.

I’m still a bit divided on whether that should be something the plugin handles, or whether it should rather provide instructions on how to use QGIS’ existing data management tools to fill NULL values with a sensible value. The latter option would mean that users are necessarily aware of what they did to the input data; of course it comes at the price of a slight inconvenience.

I’m currently travelling, so this change will take at least a few weeks, which also gives me more time to think about how to implement it, and how to integrate the warning(s) and option(s) into the user interface in an unobstrusive and accessible way.

[yuletide]

Adding an error message at minimum would be super helpful here (or even better gracefully handle them somehow). I was about to submit a bug report before realizing my issue is actually caused by a null value. The tool just exited with a cryptic log entry TypeError: cannot pickle 'QVariant' object

[christophfink]

Indeed, that’s a cryptic error message.

I now added a warning if NULL values are contained in an input data set (in the selected field).

Could you test whether the new version (3.1.5rc, 93ecc3e) reliably prevents you from going forward with data sets containing NULL values? (download the archive using the link below, then install it using QGIS’ plugin manager->Install from ZIP)

cartogram3-3.1.5rc.zip

[yuletide]

It does seem to work with my test data.

[christophfink]

Great, thanks! I still tested it on Windows 10 (additionally to my own Gnome setup; both only QGIS 3.26), from the screenshot I assume you’re on MacOS.

Just submitted a new version to QGIS’ plugin repository, it will take a few days until it’s approved and available as an update.

I still plan to write a short discussion on the implications of the different ways of handling NULL values (omitting, filling with 0, filling with average, ...; see earlier comment in this thread), maybe as a wiki page on github, and would add a link pointing to that discussion to the error message. (Leaving this issue open until then)

[christophfink]

(note to self): The error message also still need to be translated! (re #30 )