Playing roulette for intractable likelihoods

This webpage provides information on our exact-approximate MCMC work for sampling doubly intractable distributions. Further detail on experiments and links to code can be found on the left.

The term 'doubly intractable' applies to posterior distributions where the likelihood has an intractable normalising constant which is a function of the likelihood parameters. In these cases, standard sampling techniques such as the Metropolis-Hastings algorithm cannot be used as the normalising constant does not cancel in the acceptance ratio.

doubly intractable posterior

Our solution requires only the ability to produce an unbiased estimate of the likelihood normalising constant and avoids the requirement of a perfect sample from the model. The three key steps are:

  1. Write the intractable likelihood or doubly-intractable posterior as an infinite series in which each term can be estimated unbiasedly.
  2. Truncate the infinite series unbiasedly so as to make the scheme computationally feasible.
  3. Make use of potentially negative truncations so as to obtain estimates of expectations with respect to the required posterior.
Ising and Bingham

Examples in the paper include Ising models and the Fisher-Bingham distribution on a sphere, as well as a very large Gaussian model.