Bizzarre issue trying to make Rpy2 2.1.9 work with R 2.12.1, using Python 2.6 under Windows xp - Rpy can't find the R.dll?

Question

I've been having a real issue trying to make Rpy2 play nice with my R install. I first tried installing the rpy2 MSI package, and this didn't appear to work. When I ran the recommended tests, it was giving me an error saying that it couldn't find the R.dll, because the new R installs (post 2.11) install the DLLs into an i386 folder, where rpy2 can't find them because its looking in the bin folder instead of the bin/i386 folder.

Then I tried to build the install from scratch myself using the command line tools (distutils) included with python. This didn't work, because setup.py claimed to be unable to find the R_home location. But I did work out that editing an environment variable (PATH) might show the rpy2 setup where to find the R installation. I then made a couple of edits to the environment, adding the "R_home" variable pointing to the bin/i386 directory, and made a new entry under the PATH variable, pointing to the same spot.

Unfortunately, when it found the R path, I got this issue instead:

running build
running build_py
running build_ext
Traceback (most recent call last):
  File "setup.py", line 372, in <module>
    [os.path.join('doc', 'source', 'rpy2_logo.png')])]
  File "C:\Python26\lib\distutils\core.py", line 152, in setup
    dist.run_commands()
  File "C:\Python26\lib\distutils\dist.py", line 975, in run_commands
    self.run_command(cmd)
  File "C:\Python26\lib\distutils\dist.py", line 995, in run_command
    cmd_obj.run()
  File "C:\Python26\lib\distutils\command\build.py", line 134, in run
    self.run_command(cmd_name)
  File "C:\Python26\lib\distutils\cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "C:\Python26\lib\distutils\dist.py", line 994, in run_command
    cmd_obj.ensure_finalized()
  File "C:\Python26\lib\distutils\cmd.py", line 117, in ensure_finalized
    self.finalize_options()
  File "setup.py", line 111, in finalize_options
    config += get_rconfig(r_home, about)
  File "setup.py", line 264, in get_rconfig
    rc = RConfig.from_string(rconfig)
  File "setup.py", line 252, in from_string
    + '\nin string\n' + string)
ValueError: Invalid substring in string

So I went back to trying to use the premade install, thinking that maybe the new edits to the environment might work but got this issue here

 Traceback (most recent call last):
      File "<string>", line 245, in run_nodebug
      File "C:\Documents and Settings\User\Desktop\rpy2-2.1.9\rpy\tests.py", line 3, in <module>
        import rpy2.robjects.tests
      File "C:\Python26\lib\site-packages\rpy2\robjects\__init__.py", line 12, in <module>
        import rpy2.rinterface as rinterface
      File "C:\Python26\lib\site-packages\rpy2\rinterface\__init__.py", line 56, in <module>
        raise RuntimeError("Unable to locate R.dll within %s" % R_HOME)
    RuntimeError: Unable to locate R.dll within C:\Program Files\R\R-2.12.1\bin\i386

This is REALLY weird, because (as anyone can check on their own install) R installs R.dll into "C:\Program Files\R\R-2.12.1\bin\i386" and I've checked and verified that its in there, and I've pointed rpy2 to this directory in the windows default PATH! I know for a fact that rpy2 is looking in the right place, but can't understand why its not seeing R.dll.

So why can't rpy2 find it? And does anyone know a way to get rpy2 working with R 2.12? Perhaps I should try the newer rpy2 2.2.0 version? Its still in development though, and 1.9 is supposed to be able to handle R 2.12 according to this website so I don't know what to do...

Thanks to anyone who can help out...

[EDIT] I've also tried these instructions over here but they return the same "can't find DLL" error... Unless you change the environment variable "R_home" to point straight at the c/program files/R/R 2.12 directory instead of into the i386 subdirectory.

If it points at the right place, you get these errors back. This looks a bit more promising... But its still pretty bad!

.......................F....................................F.FFF...F....................................................................F..................................
======================================================================
FAIL: testNewWithoutInit (rpy2.rinterface.tests.test_SexpVector.SexpVectorTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python26\lib\site-packages\rpy2\rinterface\tests\test_SexpVector.py", line 43, in testNewWithoutInit
    self.assertTrue(False) # worked when tested, but calling endEmbeddedR causes trouble
AssertionError

======================================================================
FAIL: testCallErrorWhenEndedR (rpy2.rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python26\lib\site-packages\rpy2\rinterface\tests\test_EmbeddedR.py", line 122, in testCallErrorWhenEndedR
    self.assertTrue(False) # worked when tested, but calling endEmbeddedR causes trouble
AssertionError

======================================================================
FAIL: testReadConsoleWithError (rpy2.rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python26\lib\site-packages\rpy2\rinterface\tests\test_EmbeddedR.py", line 117, in testReadConsoleWithError
    self.assertTrue(errorstring.startswith('Traceback'))
AssertionError

======================================================================
FAIL: testSetReadConsole (rpy2.rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python26\lib\site-packages\rpy2\rinterface\tests\test_EmbeddedR.py", line 97, in testSetReadConsole
    self.assertEquals(yes.strip(), res[0])
AssertionError: 'yes' != ''

======================================================================
FAIL: testSetWriteConsole (rpy2.rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python26\lib\site-packages\rpy2\rinterface\tests\test_EmbeddedR.py", line 36, in testSetWriteConsole
    self.assertEquals('[1] "3"\n', str.join('', buf))
AssertionError: '[1] "3"\n' != ''

======================================================================
FAIL: testWriteConsoleWithError (rpy2.rinterface.tests.test_EmbeddedR.EmbeddedRTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python26\lib\site-packages\rpy2\rinterface\tests\test_EmbeddedR.py", line 55, in testWriteConsoleWithError
    self.assertTrue(errorstring.startswith('Traceback'))
AssertionError

======================================================================
FAIL: testVectorUnicodeCharacter (rpy2.robjects.tests.testNumpyConversions.NumpyConversionsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Python26\lib\site-packages\rpy2\robjects\tests\testNumpyConversions.py", line 54, in testVectorUnicodeCharacter
    self.assertTrue(False) # arrays of unicode characters causing segfault
AssertionError

----------------------------------------------------------------------
Ran 172 tests in 0.407s

FAILED (failures=7)
Exit code:  True

Answer 1

After many hours of searching on the web and trying out many different things, amongst others encountering the same issues as above, I finally got Rpy2 working on my windows 7 computer!

Basically, the crucial help came from this thread: http://www.mail-archive.com/rpy-list@lists.sourceforge.net/msg03348.html.

Summarized, these were the steps to get rpy2 up and running on Windows7:

1. Install rpy2 from this link: https://bitbucket.org/breisfeld/rpy2_w32_fix/issue/1/binary-installer-for-win32
2. Add the path to R.dll to the environment variable PATH (C:\Program Files\R\R-2.12.1\bin\i386 in my case)
3. Add an environment variable R_HOME (C:\Program Files\R\R-2.12.1 in my case)
4. Add an environment variable R_USER (simply my username in Windows)
5. Restart your Python IDE (otherwise the environment variables are not reloaded!)

Answer 2

RPy2 isn't tested on Windows. You can try using an older version (2.0.8) with a Windows installer, but that may have trouble with newer versions of R.

The author doesn't use Windows. If you've got the knowledge to get a newer version working on Windows, I'm sure he'd welcome contributions.

Answer 3

I'm not sure where to write this, as I can't comment (no reputation points), but feel it is useful info on this problem. The reason for the aggravating "Unable to locate R.dll within..." error message, even when you know that R.dll is located exactly where it says, is that the program isn't actually looking in that directory. I think the relevant action happens in rpy2's "init.py" module in these lines:

import win32api
os.environ['PATH'] += ';' + os.path.join(R_HOME, 'bin', _win_bindir)
os.environ['PATH'] += ';' + os.path.join(R_HOME, 'modules', _win_bindir)
os.environ['PATH'] += ';' + os.path.join(R_HOME, 'lib')

# Load the R dll using the explicit path
R_DLL_DIRS = ('bin', 'lib')
# Try dirs from R_DLL_DIRS
for r_dir in R_DLL_DIRS:
    Rlib = os.path.join(R_HOME, r_dir, _win_bindir, 'R.dll')
    if not os.path.exists(Rlib):
        continue
    win32api.LoadLibrary( Rlib )
    _win_ok = True
    break
# Otherwise fail out!
if not _win_ok:
    raise RuntimeError("Unable to locate R.dll within %s" % R_HOME)

As you can see, the error message will always say that it is looking in whatever directory you have set as R_HOME, but it will actually be looking in the directory "R_HOME\ r_dir\win_bindir". So when you see a message like "Unable to locate R.dll within C:\Program Files\R\R-2.12.1\bin\i386", that's because it is actually looking for a directory named C:\Program Files\R\R-2.12.1\bin\i386\R-2.12.1\bin\i386\, or something like that.

Point 3 in Kadee's answer fixes this by leaving the path specified only down to immediately above the \bin level.