Just a small datapoint when it comes interdisciplinarity, at least when it comes to Physics.
I'm faculty in a Physics department working in an interdisciplinary field (Social Networks/Human Dynamics with applications to Sociology and Epidemiology). Over the years I've seen reactions spanning the entire range from "how is that physics" to "that's awesome". Reactions tend to be correlated with "age". Younger faculty and students tend to love it and find it extremely interesting why Older (and more entrenched) faculty tend to dismiss it completely.
Institutionally, life is not always easy. Physics departments don't really know what to do with publications in WWW, KDD, BMC Medicine or EPJ Data Science and CS departments don't know what to do with a PRL or a BMC Infectious Diseases. Not all of these publications (CS ones in particular) are indexed by ISI so you keep having to direct people to your Google Scholar page if you don't want them to miss a large fraction of your publications.
On the bright side, being interdisciplinary means that it's easier for you to reach out the outside world and working on applications to "real life problems" means that you're more likely to actually make an impact in the world. It also means that you get a bit more of media coverage than your collegues which both helps and hurts you (hell hath no fury like a scientists ego scorned).
I have my list of complaints as many others do, but at least when it comes to research I can't really imagine myself doing anything else. Less teaching or more research funding would be awesome but (at least so far) they haven't been deal breakers.
He is so right on the 'interdisciplinary' thing: as long as you try to learn something new, you are wasting your time (not only from the financial standpoint but also from the 'status' one). Even more: those interdisciplinary studies will most likely be seen as 'not interesting' by each of the 'departments' they link. So... less ranking for your possible publications.
And on and on and on. It takes guts not to grind the same mill as everybody. And of course, it requires a permanent position to begin with! Otherwise, you will not get it.
There are reasons that departments exist, and that they are primary. For better or worse, you trade off interdisciplinary studies (and organizational flexibility) when you have a tenure driven department system. My impression is that some schools (MIT & Stanford?) seem to get it right, but most don't. Of course getting interdisciplinary right could just be the job of the real world.
Oh, certainly, I am not the one to disagree on then necessity of departments.
The problem is that Rectors seem to push for "interdisciplinarity" but without any reward for it in any way. So, it is just grandiloquent speech as far as it goes on like this.
My impression of (tenured) academics is a lot like my impression of Microsoft: they're making out like bandits in the short-term, while seriously head-in-sand about fundamental changes necessary to tackle the long-term.
This comment sounds flippant - but it has some truth... I've had a lot of experience with many academics from many different disciplines, from science to media studies, to philosophy.
The sense in which they are 'making out like bandits' in the short term is simply the kind of isolated freedom that tenured academics have been able to enjoy for so long. They only have to talk to one another and get the esteem of their most immediate peers - but beyond that the world leaves them alone.
The sense in which they have their head in the sand is that the world is increasingly asking them to justify their existence more and more - which is forcing academics to engage in various ways more with the external world. And naturally, they don't enjoy doing this. But rather than bite the bullet and tackle full on the problem of demonstrating their value - they just become somewhat histrionic at the suggestion that they have to justify themselves... and in the end - and this is the important bit - FAIL to deliver a coherent defense.
And before you say it: "Academics need to be left alone to their own devices otherwise you won't get innovations borne of long terms motivation and incomprehensible (to the common folk) abstract thinking."
This doesn't help - even if true. Because what is needed is a cost-benefit analysis of all the wasted resources going into the research community as a result of tenured academic life vs the the innovations that this system alone could produce. I for one have no idea how to a) identify the innovations the definitively WOULDN'T have been generated by an alternative system, nor do I - b) know how to measure the costs.
However, the one group of people that probably are smart enough to figure out an answer to this are the people who don't want to even consider the question.
(If anyone knows of actual research on this question - I'd be very interested).
"I for one have no idea how to a) identify the innovations the definitively WOULDN'T have been generated by an alternative system, nor do I - b) know how to measure the costs."
Exactly. It can't be measured.
The situation is political, and the academics are right not to engage in a dialog that uses the language of market economics and 'value'. The private sector market experts are going to want to use their models and supply-demand curves to justify resource allocation to obvious industry-relevant subjects. The academics are going to use history, ethics, etc to defend breadth, freedom, and security of non-obvious inquiry.
The faculties need to take back control of the university. They're being bent over a barrel by their administrations.
If the benefits can't be measured, why are we wasting money on them? I can also give you oodles of unmeasureable and unquantifiable benefits. Just send me a big check and then don't try to box me in with oversight or accountability.
I would put the burden of proof on the 'radicals' in this case: if you believe that the traditional scientific research system can be cut in half, and/or replaced by startups and other endeavors, and still produce the same or more advances in science, I'd like to see evidence supporting that hypothesis.
Put in Bayesian terms, I put a reasonably strong prior on the current system successfully generating science, so radical changes to it, like significantly gutting universities, need evidence to justify them. If we had metrics that fully explained where scientific advances come from and how they feed back into engineering and other areas, in at least the medium-term, we could use those as a guide for changes. But the metrics so far are about as good as lines-of-code metrics: not completely noise, but relatively low explanatory value. Metrics are only good if they can be validated: if you can come up with metrics for scientific productivity and also show that your metrics explain a large portion of scientific advances and their later use, then they might be worth taking seriously. So far, though, the metrics are relatively poor. But that does not prove the opposite: that scientific advances don't happen, and don't get later commercialized.
The fact that one doesn't fully understand why scientific advances happen, and have not successfully found a way to attach lines-of-code-style metrics to measuring them, is not actual evidence that the current system for scientific research doesn't work, or that any particular proposed change will be neutral to positive.
If there's a validated metric on the 'benefit' side, sure. What I'm skeptical of is making up some unrigorous numbers to do quantitative cost-benefit analysis when a system isn't well understood enough to be quantified with any degree of confidence in the numbers, at least at present. Assuming anything we can't measure well yet is ~0 is also not a very rigorous approach.
One cautious approach could be to try to come up with lower and upper bounds on benefits, e.g. that with high confidence, we can conclude that academic physics research, or academic geography research, or whatever, has had at least $N but not more than $M impact on the U.S. economy, taking into account direct and indirect effects over a period of, say, 30 years. I suspect the space between those bounds would be large, but we would at least be able to say for sure (for certain values of "sure") that quantifiable benefit lies within a certain range. I'd be interested in that research, if there is any, but these kinds of estimates are notoriously difficult to establish, given that history isn't an experimental science.
Assuming anything we can't measure well yet is ~0 is also not a very rigorous approach.
At the end of the day, you need a number - the amount of dollars to fund physics with. This number will have uncertainty attached to it, just like every other number.
I agree that such numbers are difficult to establish, they are likely to be strongly dependent on priors, and credible intervals will be large. So what? It's still a better approach than simply throwing arbitrary politically determined amounts of money at endeavors that sound cool to a rationally innumerate and ignorant electorate.
No. Similarly, there is no non-circular evidence that statistics works, that double blind clinical trials work, or that any of our basic scientific machinery works. If you adopt a different epistemiology, you will arrive at different conclusions.
Of course there is! If statistics didn't work, casinos wouldn't work. If our basic scientific machinery didn't work, computers wouldn't work.
If cost-benefit analyses didn't work, government projects would often go massively over budget producing nothing, while other areas that were underfunded would occasionally have startling breakthroughs....
On a more specific level, it's possible to test formalized versions of cost-benefit analysis as a planning methodology. In some setting (company? education? research?) have the experimental groups make decisions according to strict guidelines for how to quantify costs and benefits and weigh the two, and the control groups make decisions based on traditional "holistic management" where subject-matter experts have wide latitude to make recommendations, after perusing data, but not according to a strictly quantified methodology. See how their performance compares over some suitable period.
What I think this would mainly be testing is not really the philosophical question of whether we ought to weigh costs and benefits, but whether the erroneously-captured and un-captured information in quantification is a bigger or smaller problem, from the perspective of decision-making, than the potential bias and unrepeatability of a subject-matter expert's case-by-case judgment. Some of my skepticism of quantitative cost-benefit analysis is that it's basically GIGO: many quantifications are so inaccurate as to actually make quantitative cost-benefit analysis worse than "expert judgment" at decision-making, which is one reason we can't just replace all decision-makers with computers applying mechanical decision-theoretic rules yet, except in specific areas (like some kinds of medical diagnosis) where we actually do have very good data. But it's not impossible to test in a particular setting whether that's true.
And here we are discussing on Y Combinator's website, who are in a perfect position to run such an experiment! They could choose some companies to fund via the traditional method, "Paul Graham & friends' expert judgment on how to allocate money, based on their history and experience", and another portion of the pool on a more quantitative basis. Then see which pool is more successful.
I think that you're quite correct about GIGO for CBAs, and I think the issue is that it's almost impossible to not get GI.
But certainly such an experiment could be run. The issue there is: what's the best alternative? Randomly assign money? Go with "gut feeling"? Assign it evenly? It looks like trying to outguess the stock market...
No, I know this because I know a) casinos use statistics and b) casinos actually exist.
Look at the desperate efforts you're having to make to try and justify a cost-benefit analysis. You're questioning the validity of tried and tested statistical methods! Why? Shouldn't a CBA be easier to justify if it's so clearly a good idea that should be applied to everything as you stated earlier?
You previously asserted a counterfactual: "If statistics didn't work, casinos wouldn't work."
This is not the same claim as "a) casinos use statistics and b) casinos actually exist." If you want invalid post-hoc justifications of that nature, then here you go: profitable corporations use cost/benefit analysis to determine which projects to run, and they remain profitable.
But the fact of the matter is that you can't prove that statistics works. How would you do it? Compare a sample of casinos that use statistics to a sample who don't? Then run a statistical test to measure the difference, thereby engaging in circular reasoning?
Like it or not, this is a fundamental problem of epistemiology. Even if you want to compare two epistemiologies empirically, you still need to accept an underlying meta-epistemiology to have a method of comparison.
No, it's not. I'm quite happy to use a recursive definition for a truth value for statistics because it reduces to counting. I'll accept counting as an axiom.
Show me one casino that ignores statistics and remains profitable. Try setting one up yourself.
And, in fact, if you'd said that someone had run a CBA on CBAs, I'd have accepted that as your answer. You couldn't even do that though. You seem determined to turn it into a philosophy of science debate.
So, in short, you like CBAs and that's reason enough to insist on them for everything. No evidence required. And if someone asks for evidence, look to the sky and ask "what is truth?" I see.
And, in fact, if you'd said that someone had run a CBA on CBAs, I'd have accepted that as your answer.
I'm not "determined to turn it into a philosophy of science debate". It became one the moment you questioned computing (benefits - costs) and making decisions based on that.
And we will never come to agreement, given the fact that your meta-epistemiology accepts circular reasoning and mine doesn't. I'm sure if you can find someone else out there who has considered the issue and accepts circular reasoning, you can probably find some justification that you'll accept for CBAs, statistics, controlled experiments and all the rest of it. I suspect you'll have trouble finding this because your meta-epistemiology is rather unique, to say the least.
As for your hypothetical casino, I've never rejected statistics. It's circular reasoning (which you have repeatedly appealed to) that I reject. I don't actually have a meta-epistemiology at all, I simply accept the standard scientific epistemiology and move on with my life.
(Incidentally, if you accept subtraction in addition to counting, together with the idea that costs and benefits are commensurate, you've already justified cost/benefit analysis.)
Recursive definitions are hardly unique. If you have a problem with them, perhaps you don't understand them properly.
Your last statement is false. There doesn't seem much point in continuing this. You've already stated you have no evidence that CBAs work and that's clear enough.
Excellent, alongside the Humanities, then, we can also close down physics, because, clearly, if benefits can't be measured, why waste money?
Many of the "impact" arguments assume that the only victims will be the humanities, a sacrifice lots of people are very willing to make. (The original article talks, albeit not quite as strongly, of a hemorrhaging that is slowly happening in terms of funding, jobs and perspective.) However, there is some other, less obvious, collateral damage: physics, mathematics (except for applied maths), economics all share similar problems.
And here are some choice quotes about the UK's REF and its "Impact" pilot study, with "Impact" being an integral part of the future score (20%, which is going to go up to 25% for the next one) that decides about research spending from Research Councils over the next couple years, due in 2014:
> "Physicists had some particular concerns about the challenge of assessing impact in the manner foreseen by HEFCE.
1. The difficulty in attributing UoA-specific ‘ownership’ of impact that arises from the work of a large collaboration, either experimental (in our case, a ten-institute international collaboration working on experiments at CERN) or theoretical (an eight-institute collaboration using High Performance Computing facilities).
2. The blurry distinction between the ‘use of’ new high performance computers and the ‘stimulation of’ the development of these machines. At what point does close collaboration with a company such as IBM in the development of teraflop or petaflop computers become ‘impact’?
4. Commercial confidentiality became an issue. Even in this pilot exercise, and making use of the ‘Confidential’ tag on the submissions, we still had to remove some of the strongest material concerning cooperation with industrial companies because of confidentiality agreements. In REF itself, this would be a huge problem. It should be noted that this also contributed to the Institute of Physics giving up on a recent exercise to try to quantify the impact of Physics. It is impossible to quantify the impact of public understanding of science initiatives such as TV and radio appearances, popular lectures, books, etc at the scale of individuals or groups. Across the community, such work clearly has an important impact, but this cannot be sensibly quantified." (p. 29)
> "Several respondents noted that research impact was most readily observed and measured where the distance between the two realms, research and impact, was relatively short and the pathways rather direct: for example, applied scientists working in collaboration with a private business seeking expressly to translate the generic insights into commercial advantage.
The socio-economic impact of more fundamental research might go unnoticed and unremarked most of the time, reflecting the diffuse, cumulative and rather unpredictable nature of intellectual advances. Indeed, one might imagine that the occasion where a fundamental breakthrough produces a relatively immediate and direct social improvement is rare, even anomalous. In practical terms, this means the impact case studies track back to particular types of people and types of work. It might also mean that some of the most consequential socio-economic contributions will be excluded by virtue of their rather indirect link with cutting-edge research.
There was a suggestion that institutions had taken the low hanging fruit, and that in some sense the portfolio of case studies was not a good representation of the breadth of research undertaken. This sentiment appears to be rooted in a general sense that impact is most obvious in some narrow areas, within subjects rather than between them, and largely unknowable, at least in practical terms, in most instances. There were also suggestions made that a proportion of the more significant, evident impacts was linked to people and activities outside the very best academic research, albeit dependent upon that work, and which would be overlooked as a result. There is always going to be an over-reliance on a small number of sub-groups, even individuals, whose research happens to lend itself to non-academic impact. The impact submission is not necessarily representative of the UoA as a whole." (p. 30)
> "The biggest challenge was the need to acquire evidence to reveal the reach and significance of a given impact. There were many practical issues that stood in the way, and the response of the majority, for the pilot exercise at least, was to firstly focus on the more obvious cases and, secondly, to use whatever narratives, references and statistics that came to hand readily." (p. 31)
I'm quite happy to cut unproductive parts of physics, math and economics. Cutting the entirety of these fields seems counterproductive, since the benefits of the fields as a whole are quite clear. Semiconductor physics alone has certainly paid for all of physics and operations research alone has certainly paid for all of math.
But if we can identify useless segments of these fields, I favor cutting them. I see no good reason any particular endeavor should be exempt from a cost/benefit analysis.
Right, the whole problem is that most of the subfields look probably-useless until someone goes all the way down the path and discovers a use. So the benefit side of the cost-benefit looks like huge benefit times small probability, and that's a calculation that's very sensitive to error.
To continue your example: one of my ancestors worked on investigating the semiconducting properties of germanium in the '30s, but he couldn't think of anything it might be useful for. Oddly enough, he's not mentioned in the history books.
As I said, "if we can identify useless segments of these fields". I didn't say we could actually do that. I'm not a strong proponent of cutting scientific funding. Science contains a lot of stories about random seemingly useless research coming in handy in 10-30 years.
Medieval literature has far fewer stories, however.
(Incidentally, in case you think we don't currently attempt to predict the future uses of assorted subfields, you've clearly never looked at the NSF budget.)
But look - if your default position on a topic is X, and it happens to be the case that there is no rational way to explain why you are wrong about X, then you can hardly be blamed for continuing to carry on believing X.
So if you're right that there is nothing that can be said on this issue - then the powers that be will just steam-roll academia. How can they even play political? They won't be able to convince anyone to support them!
My point is that while my intuition is that academics ARE valuable and should be given large amounts of freedom - still we don't have a choice but to believe that we can find some justification for this. We certainly won't figure it out by being defeatist about it.
My position is the only rationale one on this topic - but lordy convincing a single academic of its reason is beyond me. The irony of this is painful.
The sense in which they have their head in the sand is that the world is increasingly asking them to justify their existence more and more - which is forcing academics to engage in various ways more with the external world. And naturally, they don't enjoy doing this.
At top-tier research universities in the sciences, this ship long ago sailed, and many academics these days spend the majority of their time on the external world, whether it's running a biotech company on the side, or the CNN/TED/etc. interview/lecture circuit. The university is increasingly just becoming a base of operations for people's larger business/media empires. I think this generally helps funding and demonstrating impact in the short term, and deans love it. But I'm not sure it improves the production of science research; professors in this mold no longer do the classic style of blue-skies research, but have very full schedules, carefully managed labs, and an eye for low-hanging fruit (especially monetizable fruit).
In the short term the pay is not so great and getting worse, in the long term they still can't get fired. Your analogy does not work. Unless you are talking about their administrators. In which case yes, quite a bit like Microsoft.
I don't think "making out like bandits" is accurate at all. Tenured academics don't seem like they make all that much money and I think most would make more in corporate.
The irony is that tenure exists to protect the future from short term politics, since you can't fire truth tellers just because you disagree with them.
In case anyone else sees an almost-empty page, it looks like Ghostery ends up blocking all the content. Or should I say, the owner of the website has inextricably and inexplicably tied the basic functionality of their blog to an advertising script (Disqus). Very lame.
It's not but if you configure Ghostery's filters strictly, it blocks a lot of third party elements that are widespread on the web and as such can be used to track you. Ghostery in general doesn't target ads in particular. That's what AdBlock is for.
It's hard to imagine questioning a tenure decision on your wife can ever end well. Not that you shouldn't stand by your spouse, but anything you say will appear biased. I suspect that Mizzou knew they would lose both of them.
As a recovering Academic myself I really enjoyed this post. And as good as it is, that first comment really moved it from good to legendary: "Like many career academics your pompous prose distracts from the content of your argument."
What I took from this story was a first hand account of resistance to change. This does not bode well for the ability of specific academic departments to maintain relevance in the years ahead.
Experimentation in the best ways to educate has been embraced by the private sector. It will be interesting to watch the level of competition that universities can provide.
>I should begin by acknowledging that I've had some major and sometimes quite public conflicts with my home department and administration, especially about their treatment of my spouse, which I strongly believe to be the result of highly sexist attitudes.
If you go back through his archives (which took more time than I thought it would) you find a FAQ post about an essay concerning alleged sexual discrimination in his wife's tenure hearing [1] (it's referenced here [2]). Both claim to have links to the original article, but they're dead.
EDIT:
Concretely? It seems like his wife was up for tenure and was denied it on account of what the author claims was sexism, based on past actions of the administrators in his department.
I'm faculty in a Physics department working in an interdisciplinary field (Social Networks/Human Dynamics with applications to Sociology and Epidemiology). Over the years I've seen reactions spanning the entire range from "how is that physics" to "that's awesome". Reactions tend to be correlated with "age". Younger faculty and students tend to love it and find it extremely interesting why Older (and more entrenched) faculty tend to dismiss it completely.
Institutionally, life is not always easy. Physics departments don't really know what to do with publications in WWW, KDD, BMC Medicine or EPJ Data Science and CS departments don't know what to do with a PRL or a BMC Infectious Diseases. Not all of these publications (CS ones in particular) are indexed by ISI so you keep having to direct people to your Google Scholar page if you don't want them to miss a large fraction of your publications.
On the bright side, being interdisciplinary means that it's easier for you to reach out the outside world and working on applications to "real life problems" means that you're more likely to actually make an impact in the world. It also means that you get a bit more of media coverage than your collegues which both helps and hurts you (hell hath no fury like a scientists ego scorned).
I have my list of complaints as many others do, but at least when it comes to research I can't really imagine myself doing anything else. Less teaching or more research funding would be awesome but (at least so far) they haven't been deal breakers.