The reason why an epoch in the past was picked, I think, is that the designers wanted date stamps to represent all dates in common use, e.g. birth dates (of course, that did not cover >80-year-olds in 1984).

Pretty sure that the reason to pick 1904 in particular was to simplify leap year calculation to !(year & 3). 1900 didn't have a leap day, and by picking 1904, all leap years within the range of the time stamp had one.