The best undergraduate project I ever did was one where we had to design and simulate a 4 bit microprocessor (this was 1991). Whilst it pulled together strands of our analogue and digital electronics classes the biggest impact was on my software development. It brought the programmers model of a CPU completely to life and stuff that I knew I now Knew. I became a pretty mean C programmer (even if say so myself) because of this.
How far you go down the rabbit hole depends of how much of this supporting knowledge you need to work at the level you work. Given the layers of software abstraction that exist in systems these days is it really necessary for a programmer to have a detailed knowledge of how a transistor works? Not at all. Your much better spending the time building a mental model of how your VM or OS layer is working and going to interpret the code you write. Or even looking up and understanding the layers that will use what you write - like users and customers (now there is a real challenge!) For sure take an interest in all things, but as you go down the layers your thinking can be increasingly abstract and like any model, plain wrong and it won't really matter.
How far you go down the rabbit hole depends of how much of this supporting knowledge you need to work at the level you work. Given the layers of software abstraction that exist in systems these days is it really necessary for a programmer to have a detailed knowledge of how a transistor works? Not at all. Your much better spending the time building a mental model of how your VM or OS layer is working and going to interpret the code you write. Or even looking up and understanding the layers that will use what you write - like users and customers (now there is a real challenge!) For sure take an interest in all things, but as you go down the layers your thinking can be increasingly abstract and like any model, plain wrong and it won't really matter.