Supplement-Pages

Neural networks and deep learning

• http://neuralnetworksanddeeplearning.com/ - Michael Neilsen

  • chapter1. Standard neural network : http://neuralnetworksanddeeplearning.com/chap1.html

  • chapter2. Backpropagation : http://neuralnetworksanddeeplearning.com/chap2.html

  • chapter3. Optimization : http://neuralnetworksanddeeplearning.com/chap3.html

  • chapter4. Universal function : http://neuralnetworksanddeeplearning.com/chap4.html

  • chapter5. Vanishing/Exploding gradient : http://neuralnetworksanddeeplearning.com/chap5.html

  • chapter6. Mainly about CNN : http://neuralnetworksanddeeplearning.com/chap6.html

Recurrent Neural Network

• Backpropagation Through Time(BPTT)

  • http://aikorea.org/blog/rnn-tutorial-3/
  • https://r2rt.com/styles-of-truncated-backpropagation.html

• Long Short Term Memory(LSTM)

  • http://colah.github.io/posts/2015-08-Understanding-LSTMs/
  • https://r2rt.com/written-memories-understanding-deriving-and-extending-the-lstm.html
  • tensorflow/tensorflow#15087

Speech Recognition

• http://andrew.gibiansky.com/blog/machine-learning/speech-recognition-neural-networks/

Softmax and Cross Entropy

• http://peterroelants.github.io/posts/neural_network_implementation_intermezzo02/

Batch Normalization, Layer Normalization

• https://theneuralperspective.com/2016/10/27/gradient-topics/
• http://sanghyukchun.github.io/88/
• https://shuuki4.wordpress.com/2016/01/13/batch-normalization-%EC%84%A4%EB%AA%85-%EB%B0%8F-%EA%B5%AC%ED%98%84/

Generative Adversarial Nets

• https://github.com/hindupuravinash/the-gan-zoo

• Wasserstein GAN

  • https://www.slideshare.net/ssuser7e10e4/wasserstein-gan-i
  • http://www.alexirpan.com/2017/02/22/wasserstein-gan.html
  • https://vincentherrmann.github.io/blog/wasserstein/

Variational Auto Encoder

• http://jaejunyoo.blogspot.com/2017/05/auto-encoding-variational-bayes-vae-3.html

Convolution

• Convolution Arithmetic : https://arxiv.org/abs/1603.07285
• https://github.com/vdumoulin/conv_arithmetic
• https://towardsdatascience.com/types-of-convolutions-in-deep-learning-717013397f4d
• Padding : https://stackoverflow.com/questions/37674306/what-is-the-difference-between-same-and-valid-padding-in-tf-nn-max-pool-of-t
• Deconvolution : https://www.quora.com/How-do-you-calculate-the-output-dimensions-of-a-deconvolution-network-layer

Optimzer

• http://ruder.io/optimizing-gradient-descent/index.html

• https://arxiv.org/pdf/1609.04747v2.pdf

Tensorflow

• Install
  • http://ejklike.github.io/2017/03/06/install-tensorflow1.0-on-ubuntu16.04-1.html
• TFRecord
  • http://www.machinelearninguru.com/deep_learning/tensorflow/basics/tfrecord/tfrecord.html
• Variable scope
  • https://stackoverflow.com/questions/35919020/whats-the-difference-of-name-scope-and-a-variable-scope-in-tensorflow
• Gradient clipping
  • https://stackoverflow.com/questions/36498127/how-to-effectively-apply-gradient-clipping-in-tensor-flow
• Custom gradient
  • https://stackoverflow.com/questions/39048984/tensorflow-how-to-write-op-with-gradient-in-python

Github

• https://nolboo.kim/blog/2013/10/06/github-for-beginner/

• https://wayhome25.github.io/git/2017/07/08/git-first-pull-request-story/

• https://git-scm.com/book/ko/v2/Git-%EB%B8%8C%EB%9E%9C%EC%B9%98-%EB%B8%8C%EB%9E%9C%EC%B9%98%EC%99%80-Merge-%EC%9D%98-%EA%B8%B0%EC%B4%88

Docker

• https://github.com/nsml-admin/ml/blob/master/tf-torch-cv/tf-1.11torch-0.4keras2.2cv3.4/Dockerfile

• https://blog.nacyot.com/articles/2014-01-27-easy-deploy-with-docker/

• http://pyrasis.com/Docker/Docker-HOWTO

• https://devyurim.github.io/python/tensorflow/development%20enviroment/docker/2018/05/25/tensorflow-3.html

  • To install nvidia-docker, https://nvidia.github.io/nvidia-docker/