ksg2

Use second Kraskov algorithm to compute mutual information

Specification

• Alias: None

• Arguments: None

Description

This algorithm is derived in [KStogbauerG04] . The mutual information between $m$ random variables is approximated by

$$I_2(X_1,X_2,\dots ,X_m)=\psi (k)+(m-1)\psi (N)-(m-1)/k-<\psi (n_{x_1})+\psi (n_{x_2})+\dots +\psi (n_{x_m})>,$$

where $\psi$ is the digamma function, $k$ is the number of nearest neighbors being used, and $N$ is the number of samples available for the joint distribution of the random variables. For each point $z_i=(x_{1,i},x_{2,i},\dots ,x_{m,i})$ in the joint distribution, $z_i$ and its $k$ nearest neighbors are projected into each marginal subpsace. For each subspace $j=1,\dots ,m$ , ${ϵ}_{j,i}$ is defined as the radius of the $l_{\mathrm{\infty }}$ -ball containing all $k+1$ points. Then, $n_{x_{j,i}}$ is the number of points in the $j$ -th subspace within a distance of ${ϵ}_{j,i}$ from the point $x_{j,i}$ . The angular brackets denote that the average of $\psi (n_{x_{j,i}})$ is taken over all points $i=1,\dots ,N$ .

Examples

method
 bayes_calibration queso
   dram
   seed = 34785
   chain_samples = 1000
   posterior_stats mutual_info
  ksg2

method
 bayes_calibration
   queso
   dram
   chain_samples = 1000 seed = 348
  experimental_design
   initial_samples = 5
   num_candidates = 10
   max_hifi_evaluations = 3
   ksg2