Why does xts shift a date one day back when creating an xts object from a data frame when TZ is not specified?

Question

Let me start by saying that I took a look at ?xts, realised that this is a timezone related problem and seem to have resolved it, but I don't understand why it was happening. So: I have a simple data frame of price data. When I convert it to an xts object the first date of the xts object is a day earlier than the first date in the data frame. If I specify the time zone the dates match problem disappears. I thought at first it might be because xts() assumes that an order.by date without TZ specified is UMT, and Sys.timezone() gives "JST" for me but I don't see why that would lead to a date that is a full day earlier...?

Q. Why is this happening?

require(xts)
aa <- structure(list(Date = structure(c(6822, 6823, 6824, 6825, 6826,
6829), class = "Date"), Open = c(2145, 2126, 2130, 2148, 2144,
2137), High = c(2148, 2131, 2141, 2152, 2146, 2151), Low = c(2124,
2111, 2128, 2140, 2135, 2136), Close = c(2124, 2120, 2141, 2140,
2140, 2149), Volume = c(0L, 0L, 0L, 0L, 0L, 0L)), .Names = c("Date",
"Open", "High", "Low", "Close", "Volume"), row.names = c(NA,
6L), class = "data.frame")

str(aa)
aa

bb <- xts(aa[5], order.by = aa$Date)
str(bb)
bb ## first date is a day earlier than the first day of the data frame

bb <- xts(aa[5], order.by = aa$Date, tzone = Sys.getenv("TZ"))
str(bb)
bb ## first dates in xts object and data frame match...

This is on:

sessionInfo():
R version 2.15.1 (2012-06-22)

Platform: i386-pc-mingw32/i386 (32-bit)

locale:
[1] LC_COLLATE=English_United Kingdom.1252
[2] LC_CTYPE=English_United Kingdom.1252   
[3] LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] grid      stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
 [1] gridExtra_0.9.1 scales_0.2.2    plyr_1.7.1      ggplot2_0.9.2.1
 [5] lubridate_1.2.0 quantmod_0.3-17 TTR_0.21-1      xts_0.8-8      
 [9] zoo_1.7-9       Defaults_1.1-1 

loaded via a namespace (and not attached):
 [1] colorspace_1.2-0   dichromat_1.2-4    digest_0.5.2       gtable_0.1.1      
 [5] labeling_0.1       lattice_0.20-10    MASS_7.3-22        memoise_0.1       
 [9] munsell_0.4        proto_0.3-9.2      RColorBrewer_1.0-5 reshape2_1.2.1    
[13] stringr_0.6.1     

Answer 1

I do not know, and I cannot exactly reproduce your problem, but I believe it has to do with the Dates being coerced to POSIXct and back.

This line is in the code for the xts function:

if (inherits(order.by, "Date") && !missing(tzone)) 
    order.by <- .POSIXct(unclass(order.by) * 86400, tz = tzone)

In your first call, tzone is missing, so this code is not executed. In your second call, tzone is not missing, so it is executed.

If you step through the code in xts.R, you can see that (if tzone is missing) the Dates in aa$Date are coerced to POSIXct.

index <- as.numeric(as.POSIXct(order.by))

---

I think the issue is that as.Date.POSIXct has a default of tz="UTC", so that is used unless you specify a different one.

x <- structure(1290125760, 
               tzone = structure("America/Chicago", .Names = "TZ"), 
               tclass = c("POSIXt", "POSIXct"), 
               class = c("POSIXct", "POSIXt"))
x
#[1] "2010-11-18 18:16:00 CST"
str(x)
#POSIXct[1:1], format: "2010-11-18 18:16:00"
as.Date(x)
#[1] "2010-11-19"
as.Date(x, origin=as.Date("1970-01-01"), tz="America/Chicago")
#[1] "2010-11-18"