Introducing Gen, a new PPL language by MIT

Hi everybody, today I’ll introduce you a new PPL, “Gen”.

In an article presented at the conference Programming Language Design and Implementation at the end of June/19 , the researchers of MIT described a new probabilistic programming system called “Gen.”

Probabilistic modeling and inference are core tools in diverse fields including statistics, machine learning, computer vision, cognitive science, robotics, natural language processing, and artificial intelligence. To meet the functional requirements of applications, practitioners use a broad range of modeling techniques and approximate inference algorithms. However, implementing inference algorithms is often difficult and error prone. Gen simplifies the use of probabilistic modeling and inference, by providing modeling languages in which users express models, and high-level programming constructs that automate aspects of inference.

Like some probabilistic programming research languages, Gen includes universal modeling languages that can represent any model, including models with stochastic structure, discrete and continuous random variables, and simulators. However, Gen is distinguished by the flexibility that it affords to users for customizing their inference algorithm. It is possible to use built-in algorithms that require only a couple lines of code, as well as develop custom algorithms that are more able to meet scalability and efficiency requirements.

Gen’s flexible modeling and inference programming capabilities unify symbolic, neural, probabilistic, and simulation-based approaches to modeling and inference, including causal modeling, symbolic programming, deep learning, hierarchical Bayesian modeling, graphics and physics engines, and planning and reinforcement learning.

Gen is a package for the Julia programming language. Gen consists of multiple modeling languages that are implemented as DSLs in Julia and a Julia library for inference programming.

“Gen is the first system to be flexible, automated and efficient enough to cover these kinds of very different examples in computer vision and data science and offers state-of-the-art performance,” says Vikash K. Mansinghka ’05, MEng ’09. , PhD ’09, a researcher in the Department of Brain and Cognitive Sciences who runs the Probabilistic Computing Project.

More information can be found in:

First, download Julia 1.0 or later.

Then, install the Gen package with the Julia package manager. From the Julia REPL, type ] to enter the Pkg REPL mode and then run:

In [ ]:

pkg> add https://github.com/probcomp/Gen

Let’s write a short Gen program that does Bayesian linear regression: given a set of points in the (x, y) plane, we want to find a line that fits them well.

There are three main components to a typical Gen program.

First, we define a generative model: a Julia function, extended with some extra syntax, that, conceptually, simulates a fake dataset. The model below samples slope and intercept parameters, and then for each of the x-coordinates that it accepts as input, samples a corresponding y-coordinate. We name the random choices we make with @trace, so we can refer to them in our inference program.

In [ ]:

using Gen@gen function my_model(xs::Vector{Float64})slope = @trace(normal(0, 2), :slope)intercept = @trace(normal(0, 10), :intercept)for (i, x) in enumerate(xs)@trace(normal(slope * x + intercept, 1), "y-$i")endend

Second, we write an inference program that implements an algorithm for manipulating the execution traces of the model. Inference programs are regular Julia code, and make use of Gen’s standard inference library.

The inference program below takes in a data set, and runs an iterative MCMC algorithm to fit slope and intercept parameters:

In [ ]:

function my_inference_program(xs::Vector{Float64}, ys::Vector{Float64}, num_iters::Int)# Create a set of constraints fixing the# y coordinates to the observed y valuesconstraints = choicemap()for (i, y) in enumerate(ys)constraints["y-$i"] = yend# Run the model, constrained by `constraints`,# to get an initial execution trace(trace, _) = generate(my_model, (xs,), constraints)# Iteratively update the slope then the intercept,# using Gen's metropolis_hastings operator.for iter=1:num_iters(trace, _) = metropolis_hastings(trace, select(:slope))(trace, _) = metropolis_hastings(trace, select(:intercept))end# From the final trace, read out the slope and# the intercept.choices = get_choices(trace)return (choices[:slope], choices[:intercept])end

Finally, we run the inference program on some data, and get the results: