Using a complex likelihood in PyMC3

Question

pymc.__version__ = '3.0'  
theano.__version__ = '0.6.0.dev-RELEASE'

I'm trying to use PyMC3 with a complex likelihood function:

First question: Is this possible?

Here's my attempt using Thomas Wiecki's post as a guide:

import numpy as np
import theano as th
import pymc as pm
import scipy as sp

# Actual data I'm trying to fit
x = np.array([52.08, 58.44, 60.0, 65.0, 65.10, 66.0, 70.0, 87.5, 110.0, 126.0])
y = np.array([0.522, 0.659, 0.462, 0.720, 0.609, 0.696, 0.667, 0.870, 0.889,  0.919])
yerr = np.array([0.104,  0.071,  0.138,  0.035,  0.102, 0.096,  0.136,  0.031, 0.024, 0.035])

th.config.compute_test_value = 'off'
a = th.tensor.dscalar('a')

with pm.Model() as model:
    # Priors
    alpha = pm.Normal('alpha', mu=0.3, sd=5)
    sig_alpha = pm.Normal('sig_alpha', mu=0.03, sd=5)
    t_double = pm.Normal('t_double', mu=4, sd=20)
    t_delay = pm.Normal('t_delay', mu=21, sd=20)
    nu = pm.Uniform('nu', lower=0, upper=20)

    # Some functions needed for calculation of the y estimator
    def T(eqd):
        doses = np.array([52.08, 58.44, 60.0, 65.0, 65.10, 
                          66.0, 70.0, 87.5, 110.0, 126.0])
        tmt_times = np.array([29,29,43,29,36,48,22,11,7,8])
        return np.interp(eqd, doses, tmt_times)

    def TCP(a):
        time = T(x)
        BCP = pm.exp(-1E7*pm.exp(-alpha*x*1.2 + 0.69315/t_delay(time-t_double)))
        return pm.prod(BCP)

    def normpdf(a, alpha, sig_alpha):
        return 1./(sig_alpha*pm.sqrt(2.*np.pi))*pm.exp(-pm.sqr(a-alpha)/(2*pm.sqr(sig_alpha)))

    def normcdf(a, alpha, sig_alpha):
        return 1./2.*(1+pm.erf((a-alpha)/(sig_alpha*pm.sqrt(2))))

    def integrand(a):
        return normpdf(a,alpha,sig_alpha)/(1.-normcdf(0,alpha,sig_alpha))*TCP(a)

    func = th.function([a,alpha,sig_alpha,t_double,t_delay], integrand(a))

    y_est = sp.integrate.quad(func(a, alpha, sig_alpha,t_double,t_delay), 0, np.inf)[0]

    likelihood = pm.T('TCP', mu=y_est, nu=nu, observed=y_tcp)

    start = pm.find_MAP()
    step = pm.NUTS(state=start)
    trace = pm.sample(2000, step, start=start, progressbar=True) 

which produces the following message regarding the expression for y_est:

TypeError: ('Bad input argument to theano function with name ":42" at index 0(0-based)', 'Expected an array-like object, but found a Variable: maybe you are trying to call a function on a (possibly shared) variable instead of a numeric array?')

I've overcome various other hurdles to get this far, and this is where I'm stuck. So, provided the answer to my first question is 'yes', then am I on the right track? Any guidance would be helpful!

N.B. Here is a similar question I found, and another.

Disclaimer: I'm very new at this. My only previous experience is successfully reproducing the linear regression example in Thomas' post. I've also successfully run the Theano test suite, so I know it works.

Answer 1

Yes, its possible to make something with a complex or arbitrary likelihood. Though that doesn't seem like what you're doing here. It looks like you have a complex transformation of one variable into another, the integration step.

Your particular exception is that integrate.quad is expecting a numpy array, not a pymc Variable. If you want to do quad within pymc, you'll have to make a custom theano Op (with derivative) for it.