And academically the prestige is poor. One is not granted 'research time' to develop software, but to 'get things done' (see u/Danso's comment). As such there's no one to take the first step in actually making software that would benefit anyone. And in the generous circumstance that one can be allotted time to write the software, the result is a pat on the head - 'good job' - for reducing everyone's workflow from weeks to minutes. Sometimes you get an acknowledgement. And no one will ever support/read your software when you leave - it will be used ritualistically until the lab's last computer's OS no longer supports it.
On the topic I see two other significant problems:
1) In basic research there is often a need for 'Every Option' style software - you're doing something that's never been done before and you need to be able to tweak it exactly how you need (but also be able to 'just hit run' for a first pass when coming from your native field). And those types of software are inherently a mess to design and build (ie. photoshop, CAD, 3D, programming languages).
2) Some of this software can only be written by those who directly do the research - or someone who very closely collaborates with them. Scientific software contains scientific assumptions in it that are very hard to evaluate if you're not part of the field. Deciding to go right-way-round, or rounding up, or leaving off the last element in an array, or any other of those programming tricks can really mess up scientific work. Or conversely, using the entire array, using a non-weighted, 'avg' or treating the red channel mathematically the same as the blue channel is a very different way of designing software than other industries - and is not common and rarely given much thought.
There's a modest living to be made at the interface between "scientific programming" and "commercial implementation of programs scientists write". This is an under-appreciated niche because it takes a lot of work to get into it: you need to be a good developer and a good scientist with diverse experience. My way in was through experimental and computational physics, but there are certainly other avenues these days.
Modern statistics is the biggest piece of the picture that every interesting area has in common. If you're interested in scientific programming you need to understand Bayes as well as algorithms etc. I have friends in psychology, biology, etc and we can communicate surprisingly well because we all speak the same statistical language.
But more importantly you need to understand how scientists think. They are amazingly hard to pin down to the kind of specs developers need.
For example, a guy on my team once said after talking to one of the scientists we were working with for a couple of days, "I now have a much better grasp of the problem, but I still don't know what the default value of this parameter should be." I spent ten minutes talking to the scientist and came back and told the developer "5", because I could tell from the way the scientist was talking that he had no clue if the number should be 3 or 10, but seemed to be favouring the lower values. I didn't need to understand the problem domain in detail to make that judgement, but to have a reasonable grasp of the psychology of working scientists. So far as I know, there's no way to get that without working as a scientist yourself.
true I was talking more about doing tech/scientific programing as a discipline - I started as an associate professional what's some times called a professional apprentice though at the time if you had called me or my pears apprentices we would have told you where to go in no uncertain terms
I recall I was considered a bit flash because I used Mixed Case in my Hollerith statements for input prompts.
On the topic I see two other significant problems:
1) In basic research there is often a need for 'Every Option' style software - you're doing something that's never been done before and you need to be able to tweak it exactly how you need (but also be able to 'just hit run' for a first pass when coming from your native field). And those types of software are inherently a mess to design and build (ie. photoshop, CAD, 3D, programming languages).
2) Some of this software can only be written by those who directly do the research - or someone who very closely collaborates with them. Scientific software contains scientific assumptions in it that are very hard to evaluate if you're not part of the field. Deciding to go right-way-round, or rounding up, or leaving off the last element in an array, or any other of those programming tricks can really mess up scientific work. Or conversely, using the entire array, using a non-weighted, 'avg' or treating the red channel mathematically the same as the blue channel is a very different way of designing software than other industries - and is not common and rarely given much thought.