zennit.attribution

Attributors are convenience objects to compute attributions, optionally using composites.

Functions

constant	Wrapper function to create a function which returns a constant object regardless of arguments.
identity	Identity function.
occlude_independent	Given a mask, occlude pixels of input independently given a function fill_fn.

Classes

Attributor	Base Attributor Class.
Gradient	The Gradient Attributor.
IntegratedGradients	This implements Integrated Gradients [2].
Occlusion	This implements attribution by occlusion.
SmoothGrad	This implements SmoothGrad [1].

class zennit.attribution.Attributor(model, composite=None, attr_output=None)

Bases: object

Base Attributor Class.

Attributors are convenience objects with an optional composite and when called, compute an attribution, e.g., the gradient or anything that is the result of computing the gradient when using the provided composite. Attributors also provide a context to be used in a with statement, similar to CompositeContext`s. If the forward function (or `self.__call__) is called and the composite has not been registered (i.e. composite.handles is empty), the composite will be temporarily registered to the model.

Parameters:

• model (torch.nn.Module) – The model for which the attribution will be computed. If composite is provided, this will also be the model to which the composite will be registered within with statements, or when calling the Attributor instance.

• composite (zennit.core.Composite, optional) – The optional composite to, if provided, be registered to the model within with statements or when calling the Attributor instance.

• attr_output (torch.Tensor or callable, optional) – The default output attribution to be used when calling the Attributor instance, which is either a Tensor compatible with any input used, or a function of the model’s output. If None (default), the value will be the identity function.

abstract forward(input, attr_output_fn)

Abstract method. Compute the attribution of the model wrt. input, by using attr_output_fn as the function of the model output to provide the output attribution. This function will not register the composite, and is wrapped in the __call__ of Attributor.

Parameters:

• input (torch.Tensor) – Input for the model, and wrt. compute the attribution

• attr_output (torch.Tensor or callable, optional) – The output attribution function of the model’s output.

property inactive

Return the attributor’s composite’s .inactive context.

class zennit.attribution.Gradient(model, composite=None, attr_output=None, create_graph=False, retain_graph=None)

Bases: Attributor

The Gradient Attributor. The result is the product of the attribution output and the (possibly modified) jacobian. With a composite, i.e. EpsilonGammaBox, this will compute the Layer-wise Relevance Propagation attribution values.

Parameters:

• model (torch.nn.Module) – The model for which the attribution will be computed. If composite is provided, this will also be the model to which the composite will be registered within with statements, or when calling the Attributor instance.

• composite (zennit.core.Composite, optional) – The optional composite to, if provided, be registered to the model within with statements or when calling the Attributor instance.

• attr_output (torch.Tensor or callable, optional) – The default output attribution to be used when calling the Attributor instance, which is either a Tensor compatible with any input used, or a function of the model’s output. If None (default), the value will be the identity function.

• create_graph (bool, optional) – Specify whether to use create_graph=True (default is False) to compute the gradient with torch.autograd.grad. This needs to be True to compute higher order gradients.

• retain_graph (bool, optional) – Specify whether to use retain_graph=True (default is the value of create_graph) to compute the gradient with torch.autograd.grad.

forward(input, attr_output_fn)

Compute the gradient of the model wrt. input, by using attr_output_fn as the function of the model output to provide the vector for the vector jacobian product. This function will not register the composite, and is wrapped in the __call__ of Attributor.

Parameters:

• input (torch.Tensor) – Input for the model, and wrt. compute the attribution.

• attr_output_fn (torch.Tensor or callable) – The output attribution function of the model’s output.

Returns:

• output (torch.Tensor) – Output of the model given input.

• attribution (torch.Tensor) – Attribution of the model wrt. to input, with the same shape as input.

grad(input, attr_output_fn)

Compute the gradient of the model wrt. input, by using attr_output_fn as the function of the model output to provide the vector for the vector jacobian product. This function is used by subclasses to compute the gradient with the same parameters.

Parameters:

• input (torch.Tensor) – Input for the model.

• attr_output (torch.Tensor or callable, optional) – The output attribution function of the model’s output.

Returns:

• output (torch.Tensor) – Output of the model given input.

• gradient (torch.Tensor) – Gradient of the model wrt. to input, with the same shape as input.

class zennit.attribution.IntegratedGradients(model, composite=None, attr_output=None, create_graph=False, retain_graph=None, baseline_fn=None, n_iter=20)

Bases: Gradient

This implements Integrated Gradients [2]. The result is the path integral of the gradients, estimated over multiple discrete iterations. Supplying a composite will result instead in the path integral over the modified gradient.

Parameters:

• model (torch.nn.Module) – The model for which the attribution will be computed. If composite is provided, this will also be the model to which the composite will be registered within with statements, or when calling the Attributor instance.

• composite (zennit.core.Composite, optional) – The optional composite to, if provided, be registered to the model within with statements or when calling the Attributor instance.

• attr_output (torch.Tensor or callable, optional) – The default output attribution to be used when calling the Attributor instance, which is either a Tensor compatible with any input used, or a function of the model’s output. If None (default), the value will be the identity function.

• create_graph (bool, optional) – Specify whether to use create_graph=True (default is False) to compute the gradient with torch.autograd.grad. This needs to be True to compute higher order gradients.

• retain_graph (bool, optional) – Specify whether to use retain_graph=True (default is the value of create_graph) to compute the gradient with torch.autograd.grad.

• baseline_fn (callable, optional) – The baseline for which the model output is zero, supplied as a function of the input. Defaults to torch.zeros_like.

• n_iter (int, optional) – The number of iterations used to estimate the integral, defaults to 20.

References

[2] (1,2)

M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, ser. Proceedings of Machine Learning Research, D. Precup and Y. W. Teh, Eds., vol. 70. PMLR, 2017, pp. 3319–3328.

forward(input, attr_output_fn)

Compute the Integrated Gradients of the model wrt. input, by using attr_output_fn as the function of the model output to provide the vector for the vector jacobian product used to compute the gradient. This function will not register the composite, and is wrapped in the __call__ of Attributor.

Parameters:

• input (torch.Tensor) – Input for the model, and wrt. compute the attribution.

• attr_output (torch.Tensor or callable, optional) – The output attribution function of the model’s output.

Returns:

• output (torch.Tensor) – Output of the model given input.

• attribution (torch.Tensor) – Attribution of the model wrt. to input, with the same shape as input.

class zennit.attribution.Occlusion(model, composite=None, attr_output=None, occlusion_fn=None, window=8, stride=8)

Bases: Attributor

This implements attribution by occlusion. Supplying a composite will have no effect on the result, as the gradient is not used.

Parameters:

• model (torch.nn.Module) – The model for which the attribution will be computed. If composite is provided, this will also be the model to which the composite will be registered within with statements, or when calling the Attributor instance.

• composite (zennit.core.Composite, optional) – The optional composite to, if provided, be registered to the model within with statements or when calling the Attributor instance. Note that for Occlusion, this has no effect on the result.

• attr_output (torch.Tensor or callable, optional) – The default output attribution to be used when calling the Attributor instance, which is either a Tensor compatible with any input used, or a function of the model’s output. If None (default), the value will be the identity function.

• occlusion_fn (callable, optional) – The occluded function, called with occlusion_fn(input, mask), where mask is 1 inside the sliding window, and 0 outside. Either values inside or outside the sliding window may be occluded for different effects. By default, all values except inside the sliding window will be occluded.

• window (int or tuple of ints, optional) – The size of the sliding window to occlude over the input for each dimension. Defaults to 8. If a single integer is provided, the sliding window will slide with the same size over all dimensions except the first, which is assumed as the batch-dimension. If a tuple is provided, the window will only slide over the n-last dimensions, where n is the length of the tuple, e.g., if the data has shape (3, 32, 32) and window=(8, 8), the resulting mask will have a block of shape (3, 8, 8) set to True. window must have the same length as stride.

• stride (int or tuple of ints, optional) – The strides used for the sliding window to occlude over the input for each dimension. Defaults to 8. If a single integer is provided, the strides will be the same size for all dimensions. If a tuple is provided, the window will only stride over the n-last dimensions, where n is the length of the tuple. stride must have the same length as window.

forward(input, attr_output_fn)

Compute the occlusion analysis of the model wrt. input, by using attr_output_fn as function of the output, to return a weighting, which when multiplied again with the output, results in the classification score. This function will not register the composite, and is wrapped in the __call__ of Attributor.

Parameters:

• input (torch.Tensor) – Input for the model, and wrt. compute the attribution.

• attr_output (torch.Tensor or callable, optional) – The output attribution function of the model’s output.

Returns:

• output (torch.Tensor) – Output of the model given input.

• attribution (torch.Tensor) – Attribution of the model wrt. to input, with the same shape as input.

class zennit.attribution.SmoothGrad(model, composite=None, attr_output=None, create_graph=False, retain_graph=None, noise_level=0.1, n_iter=20)

Bases: Gradient

This implements SmoothGrad [1]. The result is the average over the gradient of multiple iterations where some normal distributed noise was added to the input. Supplying a composite will result instead in averaging over the modified gradient.

Parameters:

• model (torch.nn.Module) – The model for which the attribution will be computed. If composite is provided, this will also be the model to which the composite will be registered within with statements, or when calling the Attributor instance.

• composite (zennit.core.Composite, optional) – The optional composite to, if provided, be registered to the model within with statements or when calling the Attributor instance.

• attr_output (torch.Tensor or callable, optional) – The default output attribution to be used when calling the Attributor instance, which is either a Tensor compatible with any input used, or a function of the model’s output. If None (default), the value will be the identity function.

• create_graph (bool, optional) – Specify whether to use create_graph=True (default is False) to compute the gradient with torch.autograd.grad. This needs to be True to compute higher order gradients.

• retain_graph (bool, optional) – Specify whether to use retain_graph=True (default is the value of create_graph) to compute the gradient with torch.autograd.grad.

• noise_level (float, optional) – The noise level, which is \(\frac{\sigma}{x_{max} - x_{min}}\) and defaults to 0.1.

• n_iter (int, optional) – The number of iterations over which to average, defaults to 20.

References

[1] (1,2)

D. Smilkov, N. Thorat, B. Kim, F. B. Viégas, and M. Wattenberg: “SmoothGrad: removing noise by adding noise,” CoRR, vol. abs/1706.03825, 2017.

forward(input, attr_output_fn)

Compute the SmoothGrad of the model wrt. input, by using attr_output_fn as the function of the model output to provide the vector for the vector jacobian product used to compute the gradient. This function will not register the composite, and is wrapped in the __call__ of Attributor.

Parameters:

• input (torch.Tensor) – Input for the model, and wrt. compute the attribution.

• attr_output (torch.Tensor or callable, optional) – The output attribution function of the model’s output.

Returns:

• output (torch.Tensor) – Output of the model given input.

• attribution (torch.Tensor) – Attribution of the model wrt. to input, with the same shape as input.

zennit.attribution.constant(obj)

Wrapper function to create a function which returns a constant object regardless of arguments.

Parameters:

obj (object) – Constant object which the returned wrapper function will return on call.

Returns:

wrapped_const – Function which when called with any arguments will return obj.

Return type:

function

zennit.attribution.identity(obj)

Identity function.

Parameters:

• obj (object) – Any object which will be returned.

• Result –

• ------ –

• obj – The original input argument obj.

zennit.attribution.occlude_independent(input, mask, fill_fn=<built-in method zeros_like of type object>, invert=False)

Given a mask, occlude pixels of input independently given a function fill_fn.

Parameters:

• input (torch.Tensor) – The input tensor which will be occluded in the pixels where mask is non-zero, or, if invert is True, where mask is zero, using function fill_fn.

• mask (torch.Tensor) – Boolean mask, at which non-zero or zero (given invert) elements will be occluded in input.

• fill_fn (function, optional) – Function used to occlude pixels with signature (input : torch.Tensor) -> torch.Tensor, where input is the same shape as input, and the output shall leave the shape unchanged. Default is torch.zeros_like, which will replace occluded pixels with zero.

• invert (bool, optional) – If True, inverts the supplied mask. Default is False, i.e. not to invert.

Returns:

The occluded tensor.

Return type:

torch.Tensor