According to Underscore.js documentation, you should simply write your own iterator for that cause. Something like this:
_.sortBy(dateGroups, function(arrayElement) {
//element will be each array, so we just return a date from first element in it
return arrayElement[0].date.getTime();
});