PyMC: Getting zero or close-to-zero Categorical likelihood

Question

I am trying to estimate rates from a stochastic Petri Net model. I don't understand why, but I get a ZeroProbability Error, even when making up data data to correspond exactly to the expected number of observations given the initial values I define for the rates.

For example, the following rates [0.01, 2, 10, 1] correpond to a probability of 3 different outcomes of [0.33, 0.66, 0.01]. If I observed, 100 outcomes, I would expect to observe that [33, 66, 1] fall within each of the outcomes.

Yet if I run the following model, I get a ZeroProbability Error (I'm simplifying the prob function, which connects to a much larger piece of code):

data=[33,66,1]
rates=pymc.Uniform('rates',0,100,size=4,value=[0.01,2,10,1])

@pymc.deterministic
def prob(rates=rates):
    return np.array([0.33,0.66,0.01])

likelihood=pymc.Categorical('likelihood',p=prob,value=data,observed=True)

Calling pymc.categorical_like(data,prob.value) returns -1.8 e308...

What am I missing?

Answer 1

I figured out the problem was the difference between Categorical distribution and Multinomial distribution. I had struggled on finding the actual difference between the two and finally found it here:

The categorical distribution is equivalent to a multinomial distribution with the number of trials equal to one.

Therefore, the Categorical likelihood has only probabilities of the different outcomes as a parameter, and takes frequencies as observed data.

On the other hand, the Multinomial distribution takes the probabilities of the different outcomes AND the number of trials as parameters, and takes number of observations per outcome as data.

The following code works as I expected:

data=np.array([33,66,1])
rates=pymc.Uniform('rates',0,100,size=4,value=[0.01,2,10,1])

@pymc.deterministic
def prob(rates=rates):
    return np.array([0.33,0.66,0.01])

likelihood=pymc.Multinomial('likelihood',n=sum(data),p=prob,value=data,observed=True)

And the two following probabilities are very similar:

pymc.categorical_like(data/data.sum(),prob.value)
pymc.multinomial_like(data,sum(data),prob.value)