Observation noise#
What do you do if your function evaluations are noisy? Plain finite difference schemes struggle in this setup.
[1]:
import jax.numpy as jnp
from jax import random
from probfindiff import central, differentiate
key = random.PRNGKey(seed=1)
WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
Noisy observations (even with very little noise) tend to mess up finite difference approximations.
[2]:
scheme, xs = central(dx=0.01)
noise = 1e-2 * random.normal(key, shape=xs.shape)
df, _ = differentiate(jnp.sin(xs) + noise, scheme=scheme)
print(df, jnp.cos(0.0))
1.5257977 1.0
Once we know which kind of error we are dealing with (noisy function evaluations), we can tune the FD scheme accordingly.
[3]:
scheme, xs = central(dx=0.01, noise_variance=1e-4)
df, _ = differentiate(jnp.sin(xs) + noise, scheme=scheme)
print(df, jnp.cos(0.0))
1.0176924 1.0
Mathematically, this is due to the fact that PN finite differences do some clever Gaussian process regression, which deals well with noise. The takeaway is: don’t overfit your finite difference scheme. Adapt it to incorporate noise.