Use comparable settings for benchmarks

[susilehtola]

The comparison between programs is not totally fair, because they use different algorithms.

For instance comparing geometry optimization with NWChem with that of ERKALE is not really meaningful, because NWChem uses a lot more sophisticated algorithms, namely optimization in internal coordinates, which results in a much smaller number of steps to reach the optimum.

Also, sophisticated programs can calculate analytic geometry Hessians, which further speed up the convergence. This will, though, increase the time per iteration.

To do a more just comparison, you should do the benchmarks in cartesian coordinates and without analytic hessians.

[baoilleach]

You misunderstand the purpose of the speed test. It's simply to measure the speed of typical calculations using different packages.

[langner]

I agree with @baoilleach in principle -- we are not comparing the code itself, and if some setting can speed up the timing for a task in package X, it should of course be used.

I do think, however, that some parameters of the calculation are so critical that they should be set, for example the basis set (already agreed on) but also the grid size used in DFT and similar things. In general, parameters that would change the result should be fixed, anything else (algorithm parameters) free to be changed.

[susilehtola]

Yes, the grid size is of paramount importance. If you use a small grid you will get a wrong result but fast. If you use a big grid, you'll get the right answer, but also spend some time getting it. Probably a fixed size grid should be used, e.g. 99 radial shells and a Lebedev mesh of 590 points.

[langner]

Those numbers sound reasonable. Ideally one would enforce some accuracy in the expected result, but that will not be possible for density functionals, since they are rarely identical in various implementations.