evoGrad is a lightweight automatic differentiation engine implemented from scratch. It serves as an educational tool to understand the fundamental concepts behind autograd and neural network operations.
- Automatic Differentiation: Tracks operations to build a computational graph and performs backpropagation to compute gradients.
- Custom Value Class: Represents scalar values with support for various arithmetic operations and gradient tracking.
- Neural Network Components: Includes basic components like Neurons, Layers, and a Multi-Layer Perceptron (MLP) for building neural networks.
- Activation Functions: Implements key activation functions such as Tanh and ReLU.
The core class that represents a scalar value and its gradient. Supports arithmetic operations like addition, multiplication, power, and exponential functions while maintaining the computational graph for gradient calculation.
A base class for all neural network components, providing methods to reset gradients and collect parameters.
Represents a single neuron with configurable nonlinearity. Each neuron performs a weighted sum of inputs followed by an optional activation function.
A collection of neurons that forms a layer in the neural network. Supports forward passes through all neurons in the layer.
Stacks multiple layers to form a fully connected neural network. Supports forward passes through all layers.
evoGrad can be used to create and train simple neural networks. It is ideal for educational purposes and small-scale experiments to understand the mechanics of backpropagation and gradient-based optimization.
- Initialize an MLP: Create a multi-layer perceptron with specified input size and layer configurations.
- Forward Pass: Perform a forward pass with input data to get predictions.
- Backward Pass: Calculate gradients via backpropagation.
- Update Parameters: Adjust the network parameters based on gradients for training.
Clone the repository to get started with evoGrad. No external dependencies are required.
git clone https://github.com/kanavgoyal898/evograd.gitevoGrad is designed to provide an intuitive and clear understanding of the inner workings of automatic differentiation and neural network training. Happy learning!