As the society advances and problems are solved for us, we are left with the most central core problem, ourselves.
Related to concepts like 'agency', 'impulse control', 'willpower', 'restraint', 'free will', 'delayed gratification' are connected to our status and self image.
If you think humans as AI systems, we are facing issues with our reward function. Short term reward seems to overrule long term planning and goals if the outside pressure is not there and we have a choice.
Is this something without technical solution or can we learn to reprogram our reward function with biofeedback, brain implants, meds or application in Apple watch?
I think the subject deserves a Black Mirror episode and few startups.
“The ideal timeframe to use for the reward function” is an instance of an age old question about what’s really important in life. Most people agree that taking lots of heroin and dying in a matter of months is suboptimal. Most people also agree that living a life of utter sacrifice in pursuit of some higher goal might not be good either. You might be working on something pointless. Some self interest is actually beneficial to society in general. In order to validate that you’re doing something worthwhile. It’s also debatable how to value the opportunity cost of missed fun now, against future fun. Especially if the future fun is experienced by someone else.
Conscience, selfless love for our children, culture, and religion are varying levels of patch on this simple human brain chemistry. I don’t think technology adds anything fundamentally new.
In addition to everything you said, there is also optimization issues of your goals. Eg, I like to think of myself as a very long term focused person (over the last 10 years, at least), but my ambitions outweigh my energy. So I have to balance in short term "play" as a way to destress and enable long term progress.
The fact that we're not robots in that sense makes this even more complicated as we can't just choose our long/short goals and stick to them, we often have to set them up within the context of what we can actually achieve.. which I think is often less than people think.
If robots had a complicated feedback loop that would force them such complicated requirements such as taking care of them selves too, they would face the same challenges.
"Especially if the future fun is experienced by someone else."
I'm always reminded of the commercial where the young man saves up his money and works hard so his much older self can fly first class somewhere. I always think the same, this could be some other version of himself, does it make sense to give this your future self?
That reminds me of a quote from Machiavelli that I recently saw on HN:
"Severities should be dealt out all at once, so that their suddenness may give less offense;
benefits should be conferred gradually, and in that way they will taste better."
This advice may also apply to one's own spending and activity choices.
Reminds me of the story in Invisible Cities - a young man dreams of the perfect city that contains all of his favorite things. By the time he arrives at the city of his dreams, he is an old man and sits off to the side with the other old men, who have moved on from the dreams of their youth.
“The ideal timeframe to use for the reward function”
One can punt on this. One can live for today under the constraint of no heroin, etc. Actually I think devs have a lot of exp with these types of problems as managing tech debt is a very similar problem---weighing (mutually interactive) actions at different timescales.
I would say, we are facing issues with our CULTURAL reward function.
For very long now our culture have deified those few people that can overcome, achieve and excel. What if the point to our existence isn't to overcome our obstacles to productivity in "getting things done" - but instead to go out, explore, create, drink, eat & be kind to each other within a relatively small community of people. I'm not saying that I know what that looks like, but I know I've wanted to run into the woods and never come back, a good few times now.
What if, for some of us, it's neither? Not just getting arbitrary things done, with $things being a free variable. And not just living a life in maintenance/standby mode - the "drink, eat and be kind" part. To me, the latter sounds empty, and the former sounds pointless - because the whole point is that free variable. The things we do.
One of the nice and addicting things about videogames is that they tend to set a goal in front of you that you emotionally care about, and then let you achieve it step by step through wits and skill. The goal is relevant, achievable; the steps and the outcomes have clear impact on your progress towards that goal, and the feedback is pretty much immediate.
This is in contrast to our modern lives. There are almost no large, social-scale goals that are achievable by individuals or small groups. There are goals in the smaller communities, but smaller communities these days are less present and more fluid. And even among the goals affecting you and those close to you, achieving them tends to feel either like following a tutorial (no point in having agency), or like a game of chance (work, love). The feedback is distant, failure and success both turn your "gameplay" upside down.
I'm not totally convinced of this, but I have this feeling that the problem of our cultural reward function is that today, it's mostly defined as "do whatever you like, be whoever you want to be". Maximum freedom, but minimum direction.
>set a goal in front of you that you emotionally care about, and then let you achieve it step by step through wits and skill.
This reminds me of the concept of achieving a flow state. Find a meaningful goal that forces you to be focused while on the cusp of exceeding your skill level. One that is both creative and allows you to progressively increase your abilities. That seems to be an integral part of finding purposeful "work" that helps one find purpose. I also think this is why people are drawn to "craftsman" type hobbies like woodwork, brewing, and programming.
>I'm not totally convinced of this, but I have this feeling that the problem of our cultural reward function is that today, it's mostly defined as "do whatever you like, be whoever you want to be". Maximum freedom, but minimum direction.
This is right on the money as far as I'm concerned, see the throngs of men flocking to people like Jordan Peterson.
You don't have to imagine. You can do a real cultural economic comparison to other countries:
Italy, Greece, Russia, Spain (and others) are countries that index more on your proposed value system: socialization, family, quality of life over hard work.
China, Japan and the US index very highly on the productivity side of the spectrum, China probably significantly more than the US.
Some of those QOL-indexing countries also have high unemployment rates and relatively lower GDP per capita.
Unemployment + GDP are probably the wrong things to optimize for. Personally I believe Unemployment is overloaded as our optimization function for economic health, where median wage increase is more representative of livability.
One question I have (and genuinely can't guess at): if you are unemployed in Greece, on average, what exactly is your quality of life?
All that said, there are countries that strike a healthier balance with strong economies: Germany, Canada, UK.
Zooming way out, my take on productivity generally is: human pressure to push our productivity is inevitable because we still need to solve some very hard, expensive problems:
(1) Our history of industrialization has led to
unsustainable environmental impact. This may require 10-100x the engineering focus & coordination we've ever had before.
(2) Disease still has significant impact.
(3) We still need human labor to harvest and transform resources (agriculture, metals)
(4) We still need human labor to generate more "life necessities" supply: electricity, housing, water / plumbing.
One could argue that for (3) & (4) we can be more like Germany and we'll still do great. Also one could argue that if we increased taxes / shifted government spend to subsidize housing development (vs. home ownership) we might also be fine.
>Unemployment + GDP are probably the wrong things to optimize for. Personally I believe Unemployment is overloaded as our optimization function for economic health, where median wage increase is more representative of livability.
This is likely well known, it'd just be awful if the official measure of QOL put pressure on employers to raise wages. Better to measure something more abstractly valuable.
The fact is that society benefits and advance because some humans do extraordinary things, for example the Guthemberg's printer, crossing the Ocean,antibiotics, diodes, transistors, computers, going to the moon...
Society rewards you because you give the society. And it is a good thing, because the cultures that do that improve, and the cultures that not collapse.
I am in Spain right now which has a more socialistic culture, "subsidizing everything" and punishing the productive(progressive taxing). In lots of ways the society is collapsing because of that,everybody wanting to be subsidized,lots of unemployment, debt skyrocketing and productive people leaving to other countries.
Part of the problem is that making the "right" decision requires knowing the future.
Sometimes procrastination is actually valuable. I've had many situations where I've been putting something off for a long time, and then eventually it turns out I didn't need to do that thing after all. Doing it would have ended up being a waste of time.
The problem is that you can't predict which things are important to do early and which ones you should wait on.
What you're describing is more like general uncertainty of the future. Procrastination is when those uncertainties have collapsed into pressing realities that need to be handled. And then you don't do them anyways.
Putting off learning a new framework because you're not sure if it will be useful isn't really procrastination. Not preparing for your interview tomorrow is.
In my procrastination in 95 percent of cases this is definitely not it. I usually will not find out that I did not have to do this unless I do it. I am talking about entrepreneurship mostly. So just do it to find out asap.
> Short term reward seems to overrule long term planning and goals if the outside pressure is not there and we have a choice.
Short term reward overrules long term planning on the opposite extreme of situation also: when you have to struggle to make it through the next day, which is a situation that many still face today. And operating on a short term reward basis for long enough ends up cementing that way of thinking, whether you are doing it as a result of comfortable boredom or persistent struggle.
IMO, there is a Goldilocks zone of balanced security and still unfulfilled needs wherein people are incentivized to think longer term with respect to both themselves and society.
Don't forget to add the "Facebook Effect" where everyone else does everything just perfectly, and most everyone is scared of experimenting by "trying" something that might not work out.
is it real ? by watching youtube videos of people with various skill levels (including grads, phds, old gurus) I am quite motivated to try and have no notion of failure or fear of it in my mind (and I'm the kind to be afraid to fail)
it's surprising how two platforms can breed two opposite sentiments. Culture I guess.
It's one thing to watch experts you don't know excel at things.
It's a very different thing to get the notion that your peers/friends on social media have everything worked out while you're struggling due to their selective sharing.
>Short term reward seems to overrule long term planning and goals if the outside pressure is not there and we have a choice.
I think the emphasis is wrong here. Short term rewards overrule long term planning when there isn't room for long term planning. In modern society, it's not that long-term planning isn't emphasized enough, nor that internal will towards it isn't strong enough. To the contrary, we're bombarded with messages encouraging instant gratification. You'll notice that many long-term initiatives are bolstered by offloading either the physical effort or the cognitive load onto others: construction and farming are highly-mechanized and specialized jobs; retirement planning is done by experts, as is healthcare; education is a matter of enrolling in a program that figures out timing and curriculum for you.
Of course, this paradigm favors individuals who have wealth.
The idea of community is neither a panacea nor a platitude; it's a structural schema for offloading personal effort onto people who are concerned but dispassionate, routing around both issues of money and of direct biological impetus, thereby reducing the individual and collective load. Any workable solution to a systemic problem such as this is going to include other people, so we need to stop looking at it as a per-person engineering problem.
I'd be happy to go back to when we didn't have solutions. Call me crazy but I take some pleasure in maintaining good old ways that have balancing benefits for us all in the group. Granted I don't mean having to walk 10 miles with a 50kg bucket of water on my back .. but something that enforces some long time walk, physical exercise, simple team work. Even if that's just to make a shed and not a new k8s cluster to do whatever.
Were people in the old times happy and satisfied, because I would argue it is in human nature to never be satisfied because it makes us evolve. We can only counter this by hacking us chemically.
I for one appreciate modern medicine and the internet. I also practice hand tool woodworking as a hobby, and while it scratches an itch to be more physical and present I’m in no way a Luddite.
I would much rather have extreme physical exertion be an optional thing that I do to make myself feel good than a thing that must be done else my family and I starve. Let's not lose sight of the fact that being able to take "past good stuff" comes with the incredible luxury of leaving a lot of past bad stuff in the past.
Well, the article states that a solution to procrastination is therapy (cbt, act, or whatever flavor you choose). Procrastination is a manifestation of general anxiety and your brain can be reprogrammed by a combination of reframing your thought process and medication if needed.
Supervised exposure therapy is probably the best biofeedback you can get.
Humans are not "AI" systems but infinitely more complex. Which is why, among other things, humans are capable of driving a car safely while "AI" is not.
I for one would feel disgusted with myself if I were to discover that I'm nothing more than the output of a reward function. YMMV.
The human "reward function" is likely more complex than current AI - and not always particularly logical (though vaguely correlated to survival of your own genes!) - but it's probably naïve to assume that we're anything more complicated than a very advanced automaton trying to optimise a reward function...
It seems logical we would be exactly that. This is how we would evolve during natural selection. It all makes sense. What reason would there be to be otherwise? It seems like human, wishful thinking to me that we are more than this. Since people want to think they are more than advanced automaton it is likely that this theory is based on imaginary foundation.
The system is complex, but I would argue against it being unknowable. We do not know all the details right now, but it is possible we will know them in the future.
> What we don't know is infinitely larger than what we do know.
Yes, but that unknown, by your own description is infinitely larger than us as individuals, who have a limited time and space of existence.
Unless you want to argue for a fictional contract between human existence and a "universal" or divine purpose (as most spiritual or religious systems do), the infinite unknowns don't pertain at all to the question of whether we are just very advanced automatons.
I didn't encounter that concept, so I don't know. That said, I'd argue these are unknown unknowns, but they're not unknowable in principle (except perhaps in areas where incompleteness theorem applies).
> How do you even add them.. it's like asking what is the utility of the statue of liberty plus the utility of NAFTA?
You add them by asking how much each part of the utility function makes you want to do a thing.
As TeMPOraL says, for your specific example you could do that in dollars, but perhaps a more emotionally affective example than USA icons would be asking how many oranges someone would need to offer you to convince you to have a hand amputation without anaesthetic: if you’re well fed, it’s ridiculous to even ask as even a lifetime supply of citrus fruit won’t come close, but if you’re literally starving to death you might well choose to lose a hand to gain anything edible.
If you agree that starting from the assumption that a utility function exists leads to being always able to define one, then you can’t simultaneously take the position that there can’t be two incommensurate things.
You could argue that utility functions are “too powerful” on the grounds that being able to explain anything is equivalent to being able to explain nothing.
> Why don't we just create utility functions to solve politics?
What do you mean by “solve”? I reckon the utility function of politics is approximately “democracy” in many cases.
> it could very well be that human intelligence is [unsolvable in terms of well-defined mathematization] too
That’s equivalent to saying “whatever human intelligence depends on isn’t limited to the laws of physics” as the laws of physics are written in maths, and as we invent new maths for new understanding of physics, that new understanding is also available for modelling our own minds, as they are physical objects. A similar argument also applies if we have an immortal soul. ;)
> If you agree that starting from the assumption that a utility function exists leads to being always able to define one, then you can’t simultaneously take the position that there can’t be two incommensurate things.
> What do you mean by “solve”? I reckon the utility function of politics is approximately “democracy” in many cases.
I don't understand what you're getting at here and I feel like it's missing the broader point I'm making anyways.
Of course there's no one set of dollar values for most items. They very much depend on a context. But in a particular, well-defined context, you can very much convert both Statue of Liberty and NAFTA to dollars and sum them up to something that makes sense in that context.
> Why did the drunken man look under the streetlight for his keys? "Didn't loose them here, but that's where the light is"
Well, drunk people tend to have problems calculating expected value.
Not a single reward function, but multiple. Although for each decision/action you could in theory calculate a single value that represents the weight toward making that decision. This value comes from multiple systems.
What makes you think we are anything more than the output of a reward function? Dopamine and other hormones function as our reward system and are so fundamental and drive effectively all of our behaviours and our learning from the moment we were born.
The fact that it disgusts him, and that you in turn might appreciate the irony of the reward system actually hating itself is pretty strong evidence that there's more to the human experience than biologically reinforced operant conditioning.
> is pretty strong evidence that there's more to the human experience than biologically reinforced operant conditioning.
I'm not sure that follows. The reward system can't produce output by itself. It requires an input. Any input. It requires an environment, a context. And one of the possible outputs is, well, his point of view. But that doesn't invalidate the reward system. It just shows how unpredictable it is.
There are layers upon layers in that reward system, although everything should be explainable. So how did the reward system come to a point that it does not want to believe it will die at some point and for that reason it had to create several imaginary stories such as religion etc? I wonder if it maybe has to do with group dynamics? In order for some conscious systems to be controlled by more powerful entities they would have had to been fed delusions in order for their reward systems to be happy and not feel cheated upon. Eg reward system wants to maximize survivability and when it is controlled by other systems in power it has to be led lies that after death it will be better so all in all the reward system gets so skewed that eventually it is likely to believe any other type of delusions as well. Such as for example that itself is not a reward system and if it was it would hate itself.
So in the end are all delusions defense mechanisms for that reward system to accept that things are unfair, this reward system might not be able to achieve the desired goals so it has to adapt itself? But why can't this reward system just hibernate without delusions?
Edit: actually I think coming to hate itself as a reward system is part of societal pressures. Society pressures each reward system to do common good for that society and creates morals. These morals have been created in a manner that one has to be something more than a selfish creature or otherwise gets shunned so reward system will create all those thoughts in order to avoid having to face selfish desires vs societal pressures head on?
Why? That only suggests that the reward system can hate itself. It's not out of scope, much like a computer being able to self-diagnose some of its workings isn't a weird thing.
> I for one would feel disgusted with myself if I were to discover that I'm nothing more than the output of a reward function. YMMV.
Well we use different words for it, but I’m not sure I understand the alternative? The brain is a physical thing, and it’s (in the broader physiological/hormonal environment) making the decisions. “Reward function” sounds reductive, but obviously it’s a much more complex phenomena than all current human created AI. We’re not just minimizing a sum of squares, if that’s what you mean.
I sympathise with your feelings of disgust, as I hated the idea of hypnosis and psychological research since I was a teenager. Perhaps “disgust” is the wrong word for me, as I felt understanding works take away the magic and wonder and mystery of what it means to be able to feel like I exist.
I still don’t like the idea of being able to modify my own low-level preferences in a meaningful way, but I have come to accept that pretending it’s not a thing can make it unreal.
(Also: AI is about as good as humans at driving cars, despite the computers in self driving cars being nowhere near as complicated as even just the bits of your brain that are involved in driving).
Humans are capable of driving cars safely. Many people go their whole lives without ever being at fault for a single accident. Some drivers, especially professionals, practice defensive driving at avoid or preempt accidents where they would not have been at fault. My own father, for example, has been the "n+1" car in an n-car pile up on more than one occasion, as he was the only one leaving a safe distance. He avoids all distractions and pays attention to the behavior of other cars on the road. He gives advice like, "don't just look at the brake lights - look for the car to pitch forward because that means it's braking hard and you should be too." Another time I saw him avoid a serious accident even though he was in the passengers seat simply by telling the driver, who had just taken their foot off the brake after the life turned green and was about to enter the intersection, "wait, stop, that guy's going to run the red light." Note that this was before the other car had actually done so - he just noticed it was slowing down. If everyone drove like that, cars would be safer than airplanes.
So it's not a cognitive issue. Almost anyone can learn to drive a car very safely, they just choose not to.
Related to concepts like 'agency', 'impulse control', 'willpower', 'restraint', 'free will', 'delayed gratification' are connected to our status and self image. If you think humans as AI systems, we are facing issues with our reward function. Short term reward seems to overrule long term planning and goals if the outside pressure is not there and we have a choice.
Is this something without technical solution or can we learn to reprogram our reward function with biofeedback, brain implants, meds or application in Apple watch?
I think the subject deserves a Black Mirror episode and few startups.