Pytorch Autograd - Grad
The grad runction is used to compute the gradient of an inputted with respect to one of its downstream parameters. The parameters to the function must be specified as torch tensors. In addition, when the tensor is instantated, the "requires_grad" parameter to the tensor must be set to true. Any tensors where the "requires_grad" parameter is not set, will not be able to have a gradient against it computed.Example 1
import torch
from torch.autograd import grad
x = torch.tensor([1.0])
y = torch.tensor([1.0], requires_grad=True)
f = x + 2*y + 5
grad2 = grad(f, y)
Example 2
import torch
from torch.autograd import grad
def func(x, y):
return x+2*y + 5
args = torch.tensor([1.0,2.0], requires_grad=True)
loss1 = func(*args)
test3 = grad(loss1, args)