Different between PDFs

Given two PDFs over the same random variable, we might want to quantify how ‘far’ they are.

Kullback-Leibler Divergence

$$KH(p_2||p_1)=-\int p_2(\mathbf{x})\ln \left(\frac{p_1(\mathbf{x})}{p_2(\mathbf{x})}\right)d\mathbf{x}\ge 0$$

Only equals to 0 when $p_1=p_2$

This is not a very visual way of seeing things. Something for more visual understanding would be

Quantile-Quantile Plot

Graphs the CDF of two probability functions against each other. #probability