Introduction to Probabilistic programming – Data Science Central

Last week, I saw a nice presentation on Probabilistic Programming from a student in Iran (link below).  

I am interested in this subject for my teaching at the #universityofoxford. In this post, I provide a brief introduction to Probabilistic programming. Probabilistic programming is a programming paradigm designed to implement and solve probabilistic models. They unite probabilistic modeling and traditional general-purpose programming.

Probabilistic programming techniques aim to create systems that help make decisions in the face of uncertainty. There are already a number of existing statistical techniques that handle uncertainty ex Latent Variable Models  and Probabilistic Graphical Models.

There are several tools and libraries  for Probabilistic Programming: PyMC3 (Python, Backend: Theano) , Pyro (Python, Backend: PyTorch), Edward (Python, Backend TensorFlow) Turing (Julia) and TensorFlow Probability

While Probabilistic Programming techniques are powerful, they are relatively complex for traditional developers. Because variables are assigned a probability distribution, Bayesian techniques are a key element of probabilistic programming. But because the mathematics of Bayesian inference is intractable, we use other techniques that build on Bayesian strategies such as  Markov Chain Monte Carlo, Variational  Inference and Expectation  Propagation

The diagram above explains this idea visually..

Image source:  Tensorflow probability

Link for presentation on Probabilistic programming:

Other references: https://towardsdatascience.com/intro-to-probabilistic-programming-b…