monitor – Track Intermediate Results

neuralnet_pytorch.monitor.track(name, x, direction=None)[source]

An identity function that registers hooks to track the value and gradient of the specified tensor.

Here is an example of how to track an intermediate output

input = ...
conv1 = nnt.track('op', nnt.Conv2d(shape, 4, 3), 'all')
conv2 = nnt.Conv2d(conv1.output_shape, 5, 3)
intermediate = conv1(input)
output = nnt.track('conv2_output', conv2(intermediate), 'all')
loss = T.sum(output ** 2)
d_inter = T.autograd.grad(loss, intermediate, retain_graph=True)
d_out = T.autograd.grad(loss, output)
tracked = nnt.eval_tracked_variables()

testing.assert_allclose(tracked['conv2_output'], nnt.utils.to_numpy(output))
testing.assert_allclose(np.stack(tracked['grad_conv2_output']), nnt.utils.to_numpy(d_out[0]))
testing.assert_allclose(tracked['op'], nnt.utils.to_numpy(intermediate))
for d_inter_, tracked_d_inter_ in zip(d_inter, tracked['grad_op_output']):
    testing.assert_allclose(tracked_d_inter_, nnt.utils.to_numpy(d_inter_))
  • name – name of the tracked tensor.
  • x – tensor or module to be tracked. If module, the output of the module will be tracked.
  • direction

    there are 4 options

    None: tracks only value.

    'forward': tracks only value.

    'backward': tracks only gradient.

    'all': tracks both value and gradient.

    Default: None.



neuralnet_pytorch.monitor.get_tracked_variables(name=None, return_name=False)[source]

Gets tracked variable given name.

  • name – name of the tracked variable. can be str or``list``/tuple of str``s. If ``None, all the tracked variables will be returned.
  • return_name – whether to return the names of the tracked variables.

the tracked variables.


Retrieves the values of tracked variables.

Returns:a dictionary containing the values of tracked variables associated with the given names.