Be smarter about multithreading when predicting #4
Description
Currently booster::predict runs on all cores. When you predict one or few datapoints, this is much slower than single-threaded prediction due to syncronization overhead (not to mention starting all those threads).
The number of threads can be modified by setting it in the params variable in prediction like this let params = CString::new("num_threads=1").unwrap();. Currently this option is not exposed at all. An easy fix would be to add a prediction variant with a settings object (maybe with builder pattern to make it easier to add more options in later versions without breaking compatability). The more sophisticated version might be to look at how many data points were given to the predict method, and choose a reasonable number of threads based on that. It should be possible to do something reasonable by running a couple benchmarks. We would have to investigate if there's a difference between windows and unix-likes though due to different overhead of starting threads.