My R project is structured like a package with directories /R, /vignettes, /data etc. In one of my Rmd docs in /vignettes
As Yihui has pointed out in his answer the mistake was simply that I used opts_chunk$set() instead of opts_knit$set().
However it might be worth noting that the change of the working directory affects not the current but only the next chunk. So e. g. if you want to load data relative to the new working directory do that in the following chunk.