How to config Meteor on AWS/EBS using METEOR_SETTINGS environment variable

Question

Trying to set up a Meteor on an AWS/EBS (Amazon Web Services, Elastic Beanstalk) environment.

A Meteor dev-run can be passed a command line flag: --settings s

Answer 1

Tested another workaround.

after meteor build --directory edit main.js as following

process.argv.splice(2, 0, 'program.json');
var settingfilename = './settings.json';
if (process.env.METEOR_SETTING_FILE)
  settingfilename = process.env.METEOR_SETTING_FILE;
var settings = require(settingfilename);
if (settings) {
  try {
    process.env.METEOR_SETTINGS = JSON.stringify(settings);
  } catch (e) {
    console.error(e);
  }
}
process.chdir(require('path').join(__dirname, 'programs', 'server'));
require('./programs/server/boot.js');

and copy settings.json into bundle/ and eb init and eb deploy.

you can set the other settings file with adding METEOR_SETTING_FILE at Environment Properties in Configuration tab from EB Console.

editing file is needed after every build.

---

added the patch file to use in the build script like ed - ../build/bundle/main.js < main.js.patch

main.js.patch

8a
var settingfilename = './settings.json';
if (process.env.METEOR_SETTING_FILE)
  settingfilename = process.env.METEOR_SETTING_FILE;
var settings = require(settingfilename);
if (settings) {
  try {
    process.env.METEOR_SETTINGS = JSON.stringify(settings);
  } catch (e) {
    console.error(e);
  }
}
// console.log (JSON.stringify(process.env));
.
w

Answer 2

After discussing this issue with AWS support I realized that AWS/EBS does not support storing JSON in environment variables. This is because the environment variables are stored as key/value strings in unencoded JSON (apparently, in CloudFormation). The bottom line here a bit disappointing:

METEOR_SETTINGS cannot be used in the AWS/EBS console

This is indeed unfortunate, however there are a couple of workarounds.

1st Workaround

Move the json configs into an s3 bucket and place the following content in a .ebextensions/app.config file:

container_commands: 
  01_setvariable: 
    command: "aws s3 cp s3://<bucket-name>/nodejs.conf /tmp/deployment/config/#etc#init#nodejs.conf 

This will entirely override /etc/init/nodejs.conf with content retrieved from your s3 bucket. Naturally there's an opportunity to set/override individual settings using fine-tuned/fancy bash scripting.

I ended up not choosing this method, because it involves another entity (an S3 bucket) and the dev iteration requires a new version deploy, which isn't terribly fast.

2nd Workaround

Note: this is a simple code-hack I came up with. It seems to put all this mess behind while not requiring much effort.

My original need was to propagate AWS/EBS env vars to the client, so I decided to bypass the METEOR_SETTINGS variable and populate Meteor.settings.public directly with env vars from node's process.env space. The whitelisting is managed by a simple list. Add a server/lib/config.js file with:

Meteor.startup(function () {
    // public settings that need to be exposed to the client can be added here
    var publicEnvs = {
        S3_PATH: 's3path'
    };
    var modified;
    _.each(publicEnvs, (value, key) => {
        let envValue = process.env[key];
        if (envValue) {
            Meteor.settings.public[value] = envValue;
            modified = true;
        }
    });
    if (modified) {
        __meteor_runtime_config__.PUBLIC_SETTINGS = Meteor.settings.public;
    }
});

Hurray, your client can access the env vars of your choice!

For example with this change, an S3_PATH environment variable defined in the EBS console can be accessed as Meteor.settings.public.s3path on the client. Quite simple, and without many moving parts :)

Answer 3

NEW SOLUTION

This is thet easier way I'm able to come with. I've made a bash script that automatically generates a compressed build of your project and integrates the settings file so you don't really have to do anything.

BEFORE TO RUN THIS SCRIPT

In your meteor project create ./lib/beanstalk-settings-fix.js

/*==============================================================================
 * Globals
 *============================================================================*/
/*global process*/
/*global Meteor*/
/*global Npm*/


if (Meteor.isProduction){
  var meteorFile = process.env.METEOR_SETTINGS_FILE;
  if(meteorFile == undefined) throw new Error(
    'METEOR_SETTINGS_FILE env variable must be defined in production.')
  var fs = Npm.require('fs');
  var pjsonBuf = fs.readFileSync( meteorFile );
  Meteor.settings = JSON.parse( pjsonBuf.toString().trim());
}

HOW TO USE IT

1. Copy the next code to a text file and save it as build.sh
2. Edit the constants.
3. Give it execution permissions and run it.

You'll end with something like project-name.zip ready to upload it to your beanstalk environment. I hope you find it useful!

This solution is based on AWS forums. If you want to check the old solutions, please check the edit history.

#!/bin/bash

#===============================================================================
# DESCRIPTION:
#===============================================================================
# This script creates a build of the project ready to be uploaded to beanstalk.
# Requires pyton 2.7.x

#===============================================================================
# COMMON ISSUES:
#===============================================================================
# -If you upload the output to a sample application, it will fail.
# -Version format must be 0.0.0

#===============================================================================
# CONSTANTS
#===============================================================================
CURRENT_VERSION="1.0.0"
OUTPUT_NAME="file-name-without-extension"
PRODUCTION_SETTINGS_JSON="./your-project-directory/settings-prod.json"
PROJECT_DIRECTORY="./your-project-directory"
OUTPUT_DIRECTORY="./the-output-directory"
ROOT_URL="http://www.SOMEENVIRONMENT-env.us-west-2.elasticbeanstalk.com"
MONGO_URL="none"

#===============================================================================
# SAY HELLO
#===============================================================================
initial_directory=$(pwd)  #  This file's local path
clear
echo "COOKING OUTPUT"
echo "========================================================="

#===============================================================================
# RAW PROJECT BUILD
#===============================================================================
echo "> BUILDING RAW PROJECT"
cd $initial_directory
cd $PROJECT_DIRECTORY
rm -f -R "../build/bundle"
meteor build --directory ../build/

#===============================================================================
# SET PRODUCTION ENVIRONMENT VARIABLES
#===============================================================================
cd $initial_directory
json=`cat $PRODUCTION_SETTINGS_JSON`
cd $OUTPUT_DIRECTORY/bundle
mkdir -p .ebextensions
echo "option_settings:" >> .ebextensions/environment.config
echo "  - option_name: MONGO_URL" >> .ebextensions/environment.config
echo "    value: $MONGO_URL" >> .ebextensions/environment.config
echo "option_settings:" >> .ebextensions/environment.config
echo "  - option_name: ROOT_URL" >> .ebextensions/environment.config
echo "    value: "$ROOT_URL"" >> .ebextensions/environment.config
echo "files:" >> .ebextensions/environment.config
echo " '/tmp/settings.json':" >> .ebextensions/environment.config
echo "    content : |" >> .ebextensions/environment.config
echo "        "$json >> .ebextensions/environment.config
echo "option_settings:" >> .ebextensions/environment.config
echo "  - namespace:  aws:elasticbeanstalk:application:environment" >> .ebextensions/environment.config
echo "    option_name: METEOR_SETTINGS_FILE" >> .ebextensions/environment.config
echo "    value: '/tmp/settings.json'" >> .ebextensions/environment.config
chmod 444 .ebextensions/environment.config
echo "> ADDING 'settings.json' AS ENV VAR"

#===============================================================================
# CREATE PACKAGE.JSON
#===============================================================================
cd $initial_directory
cd $OUTPUT_DIRECTORY/bundle

# Write base package.json
echo '{
  "name": "'$OUTPUT_NAME'",
  "version": "'$CURRENT_VERSION'",
  "scripts": {
    "start": "node main.js"
  },
  "dependencies": {
  }
}' > ./package.json

# Add dependencies from meteor in packages.json
# Then add extra dependencies defined by us.
EXTRA_DEPENDENCIES='{"forever": "*"}'
meteor_packages=$(cat ./programs/server/package.json)
packages=$(cat ./package.json)
packages_updated=`python <<END
import json;
# We cannot operate directly bash variables, so we make a copy.
a = $packages
b = $meteor_packages
a['dependencies'] = b['dependencies']
for key, value in $EXTRA_DEPENDENCIES.iteritems():
    a['dependencies'].update({key: value})
print json.dumps(a, sort_keys=False, indent=4, separators=(',', ': '));
END`
echo "$packages_updated" > ./package.json
chmod 444 ./package.json
echo "> ADDING 'package.json'"

#===============================================================================
# ZIP OUTPUT
#===============================================================================
cd $initial_directory
cd $OUTPUT_DIRECTORY/bundle
zip -FSrq "../$OUTPUT_NAME-$CURRENT_VERSION.zip" .
echo "> ZIP THE OUTPUT"

#===============================================================================
# CLEAN THE HOUSE
#===============================================================================
cd $initial_directory
cd $OUTPUT_DIRECTORY
rm -R -f ./bundle
echo "> CLEAN THE HOUSE"

#===============================================================================
# SAY GOODBYE
#===============================================================================
echo "========================================================="
echo "YOU CAN UPLOAD THE PROJECT TO A BEANSTALK ENVIRONMENT NOW"

Some extra help: In case you want to check that everything went fine, you can find your final settings in your zipped output, under /.ebextensions/environment.config and your packages file under /package.json