“The ideal timeframe to use for the reward function” is an instance of an age old question about what’s really important in life. Most people agree that taking lots of heroin and dying in a matter of months is suboptimal. Most people also agree that living a life of utter sacrifice in pursuit of some higher goal might not be good either. You might be working on something pointless. Some self interest is actually beneficial to society in general. In order to validate that you’re doing something worthwhile. It’s also debatable how to value the opportunity cost of missed fun now, against future fun. Especially if the future fun is experienced by someone else.
Conscience, selfless love for our children, culture, and religion are varying levels of patch on this simple human brain chemistry. I don’t think technology adds anything fundamentally new.
In addition to everything you said, there is also optimization issues of your goals. Eg, I like to think of myself as a very long term focused person (over the last 10 years, at least), but my ambitions outweigh my energy. So I have to balance in short term "play" as a way to destress and enable long term progress.
The fact that we're not robots in that sense makes this even more complicated as we can't just choose our long/short goals and stick to them, we often have to set them up within the context of what we can actually achieve.. which I think is often less than people think.
If robots had a complicated feedback loop that would force them such complicated requirements such as taking care of them selves too, they would face the same challenges.
"Especially if the future fun is experienced by someone else."
I'm always reminded of the commercial where the young man saves up his money and works hard so his much older self can fly first class somewhere. I always think the same, this could be some other version of himself, does it make sense to give this your future self?
That reminds me of a quote from Machiavelli that I recently saw on HN:
"Severities should be dealt out all at once, so that their suddenness may give less offense;
benefits should be conferred gradually, and in that way they will taste better."
This advice may also apply to one's own spending and activity choices.
Reminds me of the story in Invisible Cities - a young man dreams of the perfect city that contains all of his favorite things. By the time he arrives at the city of his dreams, he is an old man and sits off to the side with the other old men, who have moved on from the dreams of their youth.
“The ideal timeframe to use for the reward function”
One can punt on this. One can live for today under the constraint of no heroin, etc. Actually I think devs have a lot of exp with these types of problems as managing tech debt is a very similar problem---weighing (mutually interactive) actions at different timescales.
Conscience, selfless love for our children, culture, and religion are varying levels of patch on this simple human brain chemistry. I don’t think technology adds anything fundamentally new.