Create Bayesian Network and learn parameters with Python3.x

Question

I'm searching for the most appropriate tool for python3.x on Windows to create a Bayesian Network, learn its parameters from data and perform the inference.

The network structure I want to define myself as follows:

It is taken from this paper.

All the variables are discrete (and can take only 2 possible states) except "Size" and "GraspPose", which are continuous and should be modeled as Mixture of Gaussians.

Authors use Expectation-Maximization algorithm to learn the parameters for conditional probability tables and Junction-Tree algorithm to compute the exact inference.

As I understand all is realised in MatLab with Bayes Net Toolbox by Murphy.

I tried to search something similar in python and here are my results:

1. Python Bayesian Network Toolbox http://sourceforge.net/projects/pbnt.berlios/ (http://pbnt.berlios.de/). Web-site doesn't work, project doesn't seem to be supported.
2. BayesPy https://github.com/bayespy/bayespy I think this is what I actually need, but I fail to find some examples similar to my case, to understand how to approach construction of the network structure.

3. PyMC seems to be a powerful module, but I have problems with importing it on Windows 64, python 3.3. I get error when I install development version

   WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.

UPDATE:

4. libpgm (http://pythonhosted.org/libpgm/). Exactly what I need, unfortunately not supported by python 3.x
5. Very interesting actively developing library: PGMPY. Unfortunately continuous variables and learning from data is not supported yet. https://github.com/pgmpy/pgmpy/

Any advices and concrete examples will be highly appreciated.

Answer 1

It looks like pomegranate was recently updated to include Bayesian Networks. I haven't tried it myself, but the interface looks nice and sklearn-ish.

Answer 2

For pymc's g++ problem, I highly recommend to get g++ installation done, it would hugely boost the sampling process, otherwise you will have to live with this warning and sit there for 1 hour for a 2000 sampling process.

The way to get the warning fixed is: 1. get g++ installed, download cywing and get g++ install, you can google that. To check this, just go to "cmd" and type "g++", if it says "require input file", great, you got g++ installed. 2. install python package: mingw, libpython 3. install python package: theano

this should get this problem fixed.

I am currently working on the same problem with you, good luck!

Answer 3

Late to the party, as always, but I've wrapped up the BayesServer Java API using JPype; it might not have all the functionality that you need but you would create the above network using something like:

from bayesianpy.network import Builder as builder
import bayesianpy.network

nt = bayesianpy.network.create_network()

# where df is your dataframe
task = builder.create_discrete_variable(nt, df, 'task')

size = builder.create_continuous_variable(nt, 'size')
grasp_pose = builder.create_continuous_variable(nt, 'GraspPose')

builder.create_link(nt, size, grasp_pose)
builder.create_link(nt, task, grasp_pose)

for v in ['fill level', 'object shape', 'side graspable']:
    va = builder.create_discrete_variable(nt, df, v)
    builder.create_link(nt, va, grasp_pose)
    builder.create_link(nt, task, va)

# write df to data store
with bayesianpy.data.DataSet(df, bayesianpy.utils.get_path_to_parent_dir(__file__), logger) as dataset:
    model = bayesianpy.model.NetworkModel(nt, logger)
    model.train(dataset)

    # to query model multi-threaded
    results = model.batch_query(dataset, [bayesianpy.model.QueryModelStatistics()], append_to_df=False)

I'm not affiliated with Bayes Server - and the Python wrapper is not 'official' (you can use the Java API via Python directly). My wrapper makes some assumptions and places limitations on functions that I don't use very much. The repo is here: github.com/morganics/bayesianpy

Answer 4

I was looking for a similar library, and I found that the pomegranate is a good one. Thanks James Atwood

Here is an example how to use it.

from pomegranate import *
import numpy as np

mydb=np.array([[1,2,3],[1,2,4],[1,2,5],[1,2,6],[1,3,8],[2,3,8],[1,2,4]])

bnet = BayesianNetwork.from_samples(mydb)

print(bnet.node_count())

print(bnet.probability([[1,2,3]]))
print (bnet.probability([[1,2,8]]))