Autograd Overview
The autograd module is the machinery that makes training possible. Every call to autograd: it computes the output value (the forward pass), and it records a backward function that knows how to compute the gradient of that output with respect to its inputs. When you call autograd::backward(arena, loss), GradCore-Tensor replays those recorded functions in reverse to propagate gradients all the way back to the leaf parameters.
Variables & the Graph
This page covers the two foundational functions of the autograd engine: create_leaf, which wraps a raw tensor into a graph-trackable Variable, and backward, which traverses the computation graph in reverse and accumulates gradients. Everything else in the autograd module builds on these two.
Arithmetic Operations
These are the differentiable arithmetic operations in the autograd layer. Each one calls the corresponding tensor* function for the forward computation and registers a backwardfn that computes the local gradient and accumulates it into parent grad tensors.
Activation Operations
Every activation function in GradCore-Tensor has an autograd-layer wrapper. Each wrapper computes the activation in the forward pass, saves whatever tensors its backward function needs, and registers a backwardfn that calls the corresponding tensor*_grad function.
Loss Operations
The autograd loss functions are differentiable wrappers around the tensor-level loss functions. Each one computes the scalar loss in the forward pass, saves the tensors needed for the gradient, and registers a backward_fn that computes the gradient of the loss with respect to the model's predictions and accumulates it into the prediction's grad tensor.
Tensors in the Autograd Engine
The tensor module provides the data. The autograd engine provides the memory of what you did with that data — so it can run the chain rule backwards. Here's how the two connect.