Packaging and shipping a python library and scripts, the professional way

Question

I have the task of packaging and shipping a commercial application bundle, which will include:

1. a python library (developed by us)
2. some python programs dep

Answer 1

This is not a complete answer but just a bunch of ideas. I wrote an installer for a client that incorporated some ideas that might be useful to you.

It was Linux only so I focussed on just that. We needed to ship specific custom versions of mySQL, lighttpd, python, memcached, a few 3rd party Python modules and some custom scripts. We needed to launch all these services without any problems and let the user control them using regular initscripts. It should work fine on a bunch of popular distros and therefore shouldn't rely on distro specific stuff.

What I did was as follows.

1. Created a 500MB (I'm don't recollect the size) file and formatted it as an ext3fs file system.
2. Mounted it at a point using a loopback device.
3. Ran deb-bootstrap on the mountpoint to create a custom Debian install.
4. Chrooted inside the partition and then ran a bunch of scripts which did an apt-get install on all our dependencies, installed all the eggs and other packages which were necessary for the app, installed the app itself in /opt (inside the chroot), installed supervisord (to do process management) and set things up. Now, this partition was a completely self contained Linux filesystem that contained the application and everything needed to run it. You could dump it anywhere, chroot inside it and launch the app. The only dependency it had with the outside world were the ports it would use for its services and the supervisord control socket. This was the main point. We were able to include exactly what we needed (compiled files, .pycs only etc.) for a few of the applications and didn't have to bother with any limitations in standard installation tools.
5. After this, we packaged a few extra scripts that would go into the external operating system. These were custom made for each distro that we would have to support. This part was distro specific. There were scripts that would go into /etc/init.d and some scripts that would setup the database and stuff at the beginning.
6. We then created an archive of the entire filesystem using makeself. It would checksum stuff and all that and provide a self extracting archive which if run would untar the whole thing into /opt on the host machine, chroot inside the directory and run a setup script that would ask the user a few questions like db username/password etc. and set things up. After that, it would fetch the scripts I mentioned in step 5 and put them on the host OS.

The initscripts would simply chroot into the partition and start supervisord. It would then take care of launching all the services we cared about. Shutting down the application was simply a matter of connecting to running supervisord and running a command. We wrapped this in the initscript so that the user experience was UNIX like.

Now, we'd give clients the self extracting .run file. They'd run it, get asked a few questions and it would create a directory under /opt which contained our app and all it's dependencies. The init scripts would be modified to start our app on bootup and things would work as expected.

I think step 4 gives you the freedom to install whatever you want, however you want so that things would work fine.