monitor – Track Intermediate Results

neuralnet_pytorch.monitor.track(name, x, direction=None)

An identity function that registers hooks to track the value and gradient of the specified tensor.

Here is an example of how to track an intermediate output

input = ...
conv1 = nnt.track('op', nnt.Conv2d(shape, 4, 3), 'all')
conv2 = nnt.Conv2d(conv1.output_shape, 5, 3)
intermediate = conv1(input)
output = nnt.track('conv2_output', conv2(intermediate), 'all')
loss = T.sum(output ** 2)
loss.backward(retain_graph=True)
d_inter = T.autograd.grad(loss, intermediate, retain_graph=True)
d_out = T.autograd.grad(loss, output)
tracked = nnt.eval_tracked_variables()

testing.assert_allclose(tracked['conv2_output'], nnt.utils.to_numpy(output))
testing.assert_allclose(np.stack(tracked['grad_conv2_output']), nnt.utils.to_numpy(d_out[0]))
testing.assert_allclose(tracked['op'], nnt.utils.to_numpy(intermediate))
for d_inter_, tracked_d_inter_ in zip(d_inter, tracked['grad_op_output']):
    testing.assert_allclose(tracked_d_inter_, nnt.utils.to_numpy(d_inter_))

Parameters:

• name – name of the tracked tensor.
• x – tensor or module to be tracked. If module, the output of the module will be tracked.
• direction –

  there are 4 options

  None: tracks only value.

  'forward': tracks only value.

  'backward': tracks only gradient.

  'all': tracks both value and gradient.

  Default: None.

Returns:

x.

neuralnet_pytorch.monitor.get_tracked_variables(name=None, return_name=False)

Gets tracked variable given name.

Parameters:

• name – name of the tracked variable. can be str or``list``/tuple of str``s. If ``None, all the tracked variables will be returned.
• return_name – whether to return the names of the tracked variables.

Returns:

the tracked variables.

neuralnet_pytorch.monitor.eval_tracked_variables()

Retrieves the values of tracked variables.

Returns:a dictionary containing the values of tracked variables associated with the given names.