The Gibbs sampler, proposed by Geman and Geman in 1984 in the image-processing context, became popular among statisticians thanks to a seminal work of Gelfand and Smith (1990) who suggested it as a sampling-based approach to calculating marginal densities. This work had an extraordinary impact in the development and applications of the Bayesian methodology. However, the Gibbs sampler is nothing but a technique for generating samples from complex multivariate distributions, indirectly, without having to calculate the joint distribution. It uses the fact that, under some conditions, the knowledge of the full conditionals completely specifies the joint distribution. Hence, the Gibbs sampling algorithm works by generating an ergodic Markov chain that has the joint distribution as the equilibrium distribution. Although straightforward to describe, it is very important to have a knowledge of the theory behind the mechanism that drives it, in order to avoid falling in pitfalls due to violation of the conditions under which the chain is ergodic. The objective of this work is to explain and explore, through examples, when and why the algorithm works.

keywords: Gibbs sampler; Markov chains; full conditionals.