An xts R Inferno-ism

Another of the all ye entering here.

Issue

When subscripting an xts object, columns that don’t exist in the object are silently ignored.

Example

First, create an xts object:

xtx <- xts(cbind(a=1:4, b=11:14, c=21:24), order=Sys.Date() + 1:4)

which looks like:

> xtx
           a  b  c
2014-02-07 1 11 21
2014-02-08 2 12 22
2014-02-09 3 13 23
2014-02-10 4 14 24

Using this, we see that there is no error and no warning when asking for a non-existent column:

> xtx[, c("b", "notHere", "a")]
            b a
2014-02-07 11 1
2014-02-08 12 2
2014-02-09 13 3
2014-02-10 14 4

In contrast, an error is thrown when this is done with a matrix:

> as.matrix(xtx)[, c("b", "notHere", "a")]
Error in as.matrix(xtx)[, c("b", "notHere", "a")] : 
  subscript out of bounds

Big danger

I discovered this when testing some new code.  I observed non-missing values in a column that should have had all missing values.  It would have been easy to miss this if that hadn’t have been the case.

Consider:

newmat <- array(NA, c(4, 4), list(NULL, c("b", "notHere", "a", "Z")))
newmat[] <- xtx[, c("b", "notHere", "a", "Z")]

And the result is:

> newmat
      b notHere  a Z
[1,] 11       1 11 1
[2,] 12       2 12 2
[3,] 13       3 13 3
[4,] 14       4 14 4

No error, no warning (in this case), but wrong.

The very worst possibility in software is getting wrong results with no indication of a problem (Chapter 50 of Tao Te Programming).

Desiderata

I’d like to lobby for a warning if columns are requested that don’t exist.

There’s the question of whether an error should be thrown or not.  At this point throwing an error would break backward compatibility (Chapter 72 of Tao Te Programming).  I doubt that there’s much code that depends on the current behavior, but presumably there was some reason not to throw an error.

A compromise would be to add an argument to subscripting that would force an error.  For example:

xtx[, c("b", "notHere", "a"), strict=TRUE]

The default value for strict would be FALSE in order to preserve the current behavior.  However, there is an argument for making the default TRUE — this forces people to truly decide that the current behavior is what they want.

See also

There are some related items in The R Inferno.

The closest in spirit is probably Circle 8.2.13.

The  most troublesome is Circle 8.1.44 — not saying drop=FALSE in subscripting a matrix when it is appropriate.

Circle 8.1.46 is also related, but my favorite in this genre is Circle 8.3.25.

Epilogue

Written by an Italian poet
From the thirteenth century
And every one of them words rang true
And glowed like burning coal

from ”Tangled Up in Blue” by Bob Dylan