At the stage where I've got too much game data and am not able to train on all of it because my RAM is not big enough.
2018-06-04 06:42:29.484700: E tensorflow/core/common_runtime/direct_session.cc:170] Internal: failed initializing StreamExecutor for CUDA device ordinal 0: Internal: failed call to cuDevicePrimaryCtxRetain: CUDA_ERROR_OUT_OF_MEMORY; total memory reported: 2099904512
Should implement some data generator framework. At the very least, data should be read on the fly, and not inserted as a massive numpy array.
At the stage where I've got too much game data and am not able to train on all of it because my RAM is not big enough.
2018-06-04 06:42:29.484700: E tensorflow/core/common_runtime/direct_session.cc:170] Internal: failed initializing StreamExecutor for CUDA device ordinal 0: Internal: failed call to cuDevicePrimaryCtxRetain: CUDA_ERROR_OUT_OF_MEMORY; total memory reported: 2099904512Should implement some data generator framework. At the very least, data should be read on the fly, and not inserted as a massive numpy array.