Observation noise#

What do you do if your function evaluations are noisy? Plain finite difference schemes struggle in this setup.

[1]:
import jax.numpy as jnp
from jax import random

from probfindiff import central, differentiate

key = random.PRNGKey(seed=1)
WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)

Noisy observations (even with very little noise) tend to mess up finite difference approximations.

[2]:
scheme, xs = central(dx=0.01)
noise = 1e-2 * random.normal(key, shape=xs.shape)

df, _ = differentiate(jnp.sin(xs) + noise, scheme=scheme)
print(df, jnp.cos(0.0))
1.5257977 1.0

Once we know which kind of error we are dealing with (noisy function evaluations), we can tune the FD scheme accordingly.

[3]:
scheme, xs = central(dx=0.01, noise_variance=1e-4)
df, _ = differentiate(jnp.sin(xs) + noise, scheme=scheme)
print(df, jnp.cos(0.0))
1.0176924 1.0

Mathematically, this is due to the fact that PN finite differences do some clever Gaussian process regression, which deals well with noise. The takeaway is: don’t overfit your finite difference scheme. Adapt it to incorporate noise.