zennit.attribution

Attributors are convenience objects to compute attributions, optionally using composites.

Functions

`constant`
`identity`
`occlude_independent`

Classes

`Attributor`	Base Attributor Class.
`Gradient`	The Gradient Attributor.
`IntegratedGradients`	This implements Integrated Gradients 2.
`Occlusion`	This implements attribution by occlusion.
`SmoothGrad`	This implements SmoothGrad 1.

class zennit.attribution.Attributor(model, composite=None, attr_output=None)[source]

Bases: object

Base Attributor Class.

Attributors are convenience objects with an optional composite and when called, compute an attribution, e.g., the gradient or anything that is the result of computing the gradient when using the provided composite. Attributors also provide a context to be used in a with statement, similar to CompositeContext`s. If the forward function (or `self.__call__) is called and the composite has not been registered (i.e. composite.handles is empty), the composite will be temporarily registered to the model.

Parameters

model (obj:torch.nn.Module) – The model for which the attribution will be computed. If composite is provided, this will also be the model to which the composite will be registered within with statements, or when calling the Attributor instance.
composite (obj:zennit.core.Composite, optional) – The optional composite to, if provided, be registered to the model within with statements or when calling the Attributor instance.
attr_output (obj:torch.Tensor or callable, optional) – The default output attribution to be used when calling the Attributor instance, which is either a Tensor compatible with any input used, or a function of the model’s output. If None (default), the value will be the identity function.

abstract forward(input, attr_output_fn)[source]

Abstract method. Compute the attribution of the model wrt. input, by using attr_output_fn as the function of the model output to provide the output attribution. This function will not register the composite, and is wrapped in the __call__ of Attributor.

Parameters

input (obj:torch.Tensor) – Input for the model, and wrt. compute the attribution
attr_output (obj:torch.Tensor or callable, optional) – The output attribution function of the model’s output.

class zennit.attribution.Gradient(model, composite=None, attr_output=None)[source]

Bases: Attributor

The Gradient Attributor. The result is the product of the attribution output and the (possibly modified) jacobian. With a composite, i.e. EpsilonGammaBox, this will compute the Layerwise Relevance Propagation attribution values.

forward(input, attr_output_fn)[source]

Compute the gradient of the model wrt. input, by using attr_output_fn as the function of the model output to provide the vector for the vector jacobian product. This function will not register the composite, and is wrapped in the __call__ of Attributor.

Parameters

input (obj:torch.Tensor) – Input for the model, and wrt. compute the attribution.
attr_output (obj:torch.Tensor or callable, optional) – The output attribution function of the model’s output.

Returns

output (obj:torch.Tensor) – Output of the model given input.
attribution (obj:torch.Tensor) – Attribution of the model wrt. to input, with the same shape as input.

class zennit.attribution.IntegratedGradients(model, composite=None, attr_output=None, baseline_fn=None, n_iter=20)[source]

Bases: Attributor

This implements Integrated Gradients 2. The result is the path integral of the gradients, estimated over multiple discrete iterations. Supplying a composite will result instead in the path integral over the modified gradient.

Parameters

model (obj:torch.nn.Module) – The model for which the attribution will be computed. If composite is provided, this will also be the model to which the composite will be registered within with statements, or when calling the Attributor instance.
composite (obj:zennit.core.Composite, optional) – The optional composite to, if provided, be registered to the model within with statements or when calling the Attributor instance.
attr_output (obj:torch.Tensor or callable, optional) – The default output attribution to be used when calling the Attributor instance, which is either a Tensor compatible with any input used, or a function of the model’s output. If None (default), the value will be the identity function.
baseline_fn (callable, optional) – The baseline for which the model output is zero, supplied as a function of the input. Defaults to torch.zeros_like.
n_iter (int, optional) – The number of iterations used to estimate the integral, defaults to 20.

References

2(1,2): M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, ser. Proceedings of Machine Learning Research, D. Precup and Y. W. Teh, Eds., vol. 70. PMLR, 2017, pp. 3319–3328.

forward(input, attr_output_fn)[source]

Compute the Integrated Gradients of the model wrt. input, by using attr_output_fn as the function of the model output to provide the vector for the vector jacobian product used to compute the gradient. This function will not register the composite, and is wrapped in the __call__ of Attributor.

Parameters

input (obj:torch.Tensor) – Input for the model, and wrt. compute the attribution.
attr_output (obj:torch.Tensor or callable, optional) – The output attribution function of the model’s output.

Returns

output (obj:torch.Tensor) – Output of the model given input.
attribution (obj:torch.Tensor) – Attribution of the model wrt. to input, with the same shape as input.

class zennit.attribution.Occlusion(model, composite=None, attr_output=None, occlusion_fn=None, window=8, stride=8)[source]

Bases: Attributor

This implements attribution by occlusion. Supplying a composite will have no effect on the result, as the gradient is not used.

Parameters

model (obj:torch.nn.Module) – The model for which the attribution will be computed. If composite is provided, this will also be the model to which the composite will be registered within with statements, or when calling the Attributor instance.
composite (obj:zennit.core.Composite, optional) – The optional composite to, if provided, be registered to the model within with statements or when calling the Attributor instance. Note that for Occlusion, this has no effect on the result.
attr_output (obj:torch.Tensor or callable, optional) – The default output attribution to be used when calling the Attributor instance, which is either a Tensor compatible with any input used, or a function of the model’s output. If None (default), the value will be the identity function.
occlusion_fn (callable, optional) – The occluded function, called with occlusion_fn(input, mask), where mask is 1 inside the sliding window, and 0 outside. Either values inside or outside the sliding window may be occluded for different effects. By default, all values except inside the sliding window will be occluded.
window (int or tuple of ints, optional) – The size of the sliding window to occlude over the input for each dimension. Defaults to 8. If a single integer is provided, the sliding window will slide with the same size over all dimensions except the first, which is assumed as the batch-dimension. If a tuple is provided, the window will only slide over the n-last dimensions, where n is the length of the tuple, e.g., if the data has shape (3, 32, 32) and window=(8, 8), the resulting mask will have a block of shape (3, 8, 8) set to True. window must have the same length as stride.
stride (int or tuple of ints, optional) – The strides used for the sliding window to occlude over the input for each dimension. Defaults to 8. If a single integer is provided, the strides will be the same size for all dimensions. If a tuple is provided, the window will only stride over the n-last dimensions, where n is the length of the tuple. stride must have the same length as window.

forward(input, attr_output_fn)[source]

Compute the occlusion analysis of the model wrt. input, by using attr_output_fn as function of the output, to return a weighting, which when multiplied again with the output, results in the classification score. This function will not register the composite, and is wrapped in the __call__ of Attributor.

Parameters

input (obj:torch.Tensor) – Input for the model, and wrt. compute the attribution.
attr_output (obj:torch.Tensor or callable, optional) – The output attribution function of the model’s output.

Returns

output (obj:torch.Tensor) – Output of the model given input.
attribution (obj:torch.Tensor) – Attribution of the model wrt. to input, with the same shape as input.

class zennit.attribution.SmoothGrad(model, composite=None, attr_output=None, noise_level=0.1, n_iter=20)[source]

Bases: Attributor

This implements SmoothGrad 1. The result is the average over the gradient of multiple iterations where some normal distributed noise was added to the input. Supplying a composite will result instead in averaging over the modified gradient.

Parameters

model (obj:torch.nn.Module) – The model for which the attribution will be computed. If composite is provided, this will also be the model to which the composite will be registered within with statements, or when calling the Attributor instance.
composite (obj:zennit.core.Composite, optional) – The optional composite to, if provided, be registered to the model within with statements or when calling the Attributor instance.
attr_output (obj:torch.Tensor or callable, optional) – The default output attribution to be used when calling the Attributor instance, which is either a Tensor compatible with any input used, or a function of the model’s output. If None (default), the value will be the identity function.
noise_level (float, optional) – The noise level, which is $\frac{\sigma}{x_{max} - x_{min}}$ and defaults to 0.1.
n_iter (int, optional) – The number of iterations over which to average, defaults to 20.

References

1(1,2): D. Smilkov, N. Thorat, B. Kim, F. B. Viégas, and M. Wattenberg: “SmoothGrad: removing noise by adding noise,” CoRR, vol. abs/1706.03825, 2017.

forward(input, attr_output_fn)[source]

Compute the SmoothGrad of the model wrt. input, by using attr_output_fn as the function of the model output to provide the vector for the vector jacobian product used to compute the gradient. This function will not register the composite, and is wrapped in the __call__ of Attributor.

Parameters

input (obj:torch.Tensor) – Input for the model, and wrt. compute the attribution.
attr_output (obj:torch.Tensor or callable, optional) – The output attribution function of the model’s output.

Returns

output (obj:torch.Tensor) – Output of the model given input.
attribution (obj:torch.Tensor) – Attribution of the model wrt. to input, with the same shape as input.