AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods

The original implementation of the experiments in the paper of AdaShift (See https://arxiv.org/abs/1810.00143)

In optimizer_all.py, we have implemented the optimizer AdaShift with Tensorflow. In every independent folder, there is additional README file to tell how to reporduce the experiments in the paper.