How can I set up MongoDB on a Node.js server using node-mongodb-native in an EC2 environment?

Question

I got help from many people here, and now I want to contribute back. For those who are having trouble making a Node.js server work with MongoDB, here is what I\'ve done.

Answer 1

It looks like there might be a bug. It won't allow me to use a var for the first argument in a:b inside collection.insert({

It is treating first agument as 'a' or a, hard coded, either way.

I will look into this and post a fix on github

Answer 2

This was originally posted by the question asker. A mod asked him in the comments to post it as an answer, but got no response. So, I cleaned it up and am posting it myself.

When you look at the code, you will notice that the createServer code is inside db.open. It won't work if you reverse it. Also, do not close the db connection. Otherwise, after the first time, the db connection will not be opened again. (Of course, db.open is declared outside of createServer.) I have no clue why createServer is inside db.open. I guess it may have to do with not opening too many db connections?

Also, one problem I face is that when I run it via SSH, even if I run the server in the background (e.g. $ node server.js &), after 2.5 hours, the server dies (not the instance though). I am not sure if it is because of terminal connection or what.

Here is the procedure & code

Environment: EC2, AMS-Linux-AMI

Purpose: Take an HTTP request and log the query, IP and timestamp into MongoDB.

Steps

1) After creating the instance (server), install gcc.

$ yum install gcc-c++

2) Download Node.js files and unzip them. (I used version 2.6.)

$ curl -O http://nodejs.org/dist/node-v0.2.6.tar.gz
$ tar -xzf node-v0.2.6.tar.gz

I renamed the unzipped folder to just "nodejs"

$ cd nodejs
$ sudo ./configure --without-ssl
$ sudo make
$ sudo make install

make takes a long while.... After that you can try running the sample in nodejs.org

3) Install MongoDB. I installed version 1.6.5, not 1.7.

$ curl -O http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-1.6.5.tgz
$ tar -xzf mongodb-linux-x86_64-1.6.5.tgz
$ sudo mkdir /data/db/r01/

I renamed the folder to "mongodb"

Run the db process:

$ ./mongodb/bin/mongod --dbpath /data/db/r01/

Then if you like, you can run and try out the command line. Refer to MongoDB's website.

4) I recommend that you create your own AIM based on your instance. It will take 20 minutes. Then, recreate the install and run MongoDB again.

5) Install node-mongodb-native

$ curl -O https://download.github.com/christkv-node-mongodb-native-V0.8.1-91-g54525d8.tar.gz
$ tar -xzf christkv-node-mongodb-native-V0.8.1-91-g54525d8.tar.gz

I renamed the folder to node-mongodb-native

$ cd node-mongodb-native
$ make

6) Here is the code for the server:

GLOBAL.DEBUG = true;
global.inData = '';
var http = require('http');
sys = require("sys");

/* set up DB */

var Db = require('./node-mongodb-native/lib/mongodb').Db,
  Connection = require('./node-mongodb-native/lib/mongodb').Connection,
  Server = require('./node-mongodb-native/lib/mongodb').Server,
  BSON = require('./node-mongodb-native/lib/mongodb').BSONNative;

var host = process.env['MONGO_NODE_DRIVER_HOST'] != null ? process.env['MONGO_NODE_DRIVER_HOST'] : 'localhost';
var port = process.env['MONGO_NODE_DRIVER_PORT'] != null ? process.env['MONGO_NODE_DRIVER_PORT'] : Connection.DEFAULT_PORT;
var db = new Db('test01', new Server(host, port, {}), {native_parser:true});

db.open(function(err, db) { 
    http.createServer(function (req, res) {
        res.writeHead(200, {'Content-Type': 'text/plain'});

        global.inData = {'p':'', 'url':''};

        // get IP address
        var ipAddress = req.connection.remoteAddress;
        global.inData.ip = ipAddress;

        // date time
        var d = new Date();
        var ts = d.valueOf();
        global.inData.ts = ts;

        // get the http query
        var qs = {};
        qs = require('url').parse(req.url, true);
        if (qs.query !== null) {
            for (var key in qs.query) {
                if (key == 'p') {
                    global.inData.p = qs.query[key];
                }
                if (key == 'url') {
                    global.inData.url = qs.query[key];
                }
            }
        }

        if (global.inData.p == '' && global.inData.url == '') {
            res.end("");
        } else {
            db.collection('clickCount', function(err, collection) { 
                if (err) {
                    console.log('is error \n' + err);
                }

                collection.insert({'p':global.inData.p, 
                  'url':global.inData.url,
                  'ip':global.inData.ip, 
                  'ts':global.inData.ts});
                res.end("");
                //db.close();  // DO NOT CLOSE THE CONNECTION
            }); 
        }
    }).listen(8080); 
});

console.log('Server running at whatever host :8080');

This may not be perfect code, but it runs. I'm still not used to the "nested" or LISP kind of coding style. That's why I cheated and used global.inData to pass data along. :)

Don't forget to put res.end("") in the appropriate location (where you think the HTTP request call should be ended).

Answer 3

By the way, the answer I posted above works for CentOS and Fedora.

For people who have Ubuntu, here it is:

# for Gcc
$ sudo apt-get install build-essential

# for SSL
$ sudo apt-get install libssl-dev

Then just install node.js and mongodb as described above.

---

Also, after few months of development, I find out using "npm", "express" and "mongoose" can bring my life much easier. Also, I have installed other tools, like debugger.

# Install Node Package Manager
$ sudo curl http://npmjs.org/install.sh | sh

# for debugging
$ sudo npm install node-inspector

# for Profiling
$ sudo npm install profile

# Install Express, the Node.js framework
$ sudo npm install express

# Install Template Engines (Now, let’s install Jade, jQuery Templates and EJS. You can pick the one you want)
$ sudo npm install jade jqtpl ejs

# XML related, install node-expat and then node-xml2js-expat
$ sudo apt-get install -y libexpat1-dev
$ sudo npm install node-xml2js
$ sudo npm install xml2js-expat

# Install Mongoose, (Mongo Driver)
$ sudo npm install mongoose

Reference: http://npmjs.org

http://expressjs.com

http://mongoosejs.com