Probability for AI

Here is an attempt to add material on probability and statistic subjects affecting the field of Artificial Intelligence.

1. Introduction slides
   • The Basics and intuition on counting
   • Frequentist views of probability
   • Introduction to random variables
2. Families of Distributions
   • Discrete Distributions
   • Continuous Distributions
   • Exponential Families
   • Location and Scale Families
3. Functions of Random Variable
   • Transformations
   • Expectations
   • Moments
   • Method of Moments
   • Bivariate Transformations
4. Bayesian Estimation slides
   • The likelihood principle, $\ell \left( \theta \right)$.

   • General A Posteriori methods

     $$P\left( \theta \vert x \right) \approx P \left( x \vert \theta \right) P\left( \theta \right)$$
5. Priors and Estimation
   • A Problem with subjectivity
   • Priors
   • Exponential Priors
   • Conjugate Priors
   • Local/Scale Family
   • Improper Priors
   • The Great Jeffrey
6. Hierarchical Bayes
   • The Posterior
   • A Graphical View
   • Plate Notation
   • Why Hierarchical Models?
   • Examples
7. Introduction to Markov Chain Monte Carlo
   • The History of the Monte Carlo
   • The Basics
   • Monte Carlo Integration
   • Rejection Sampling
   • Slice Sampler
   • Importance Sampling
   • Metropolis-Hasting
   • Gibbs Sampler
8. More Advanced Methods in Markov Chain Monte Carlo
   • Bayesian Estimation
   • Simulated Annealing and Beyond
   • Population-Based MCMC Methods
   • Hamiltonian Monte Carlo
   • Rao-Blackwellisation
   • Divide & Conquer strategies
   • Pseudo-marginal strategies
9. Theory of Point Estimation
   • The main problem
   • Unbiasdness Estimators
   • Equivariance
   • Average Risk Optimality
   • Minimaxity and Admissibility
   • Asymptotic Optimality

UNDER CONSTRUCTION