There's nobody here but us chickens.

Friday, June 25, 2004

some stuff

Some quick clinks:

Watch this video, if you can, and try and figure out what is going on. Then go here to find out the full fun facts. My childhood future is coming faster than I thought.

If you want to identify with 70% of North American Students then have a browse of this here map, or traverse it with this index if that won't load. It's a place you should recognise - you've been there many a time...

Apologies for coyness. I'm in a coy mood!

Tuesday, June 22, 2004

Publish it all, I say.

Tom at Idiolect ruminates on this piece of research on awareness of being stared at. We’re talking ‘6th Sense’ here, people. Hairs bristling on the back of necks. Mind to mind communication. Bruce Willis, meandering with his best haunted expression (do you see what I did there? Perversely fitting, no?)

He’s generally unhappy about the fact that this was published, given that the authors do not present any possible ways of connecting the findings with our understanding of how reality works. I’m more optimistic, and think that, absent outright falsification or methodological error, it’s a good thing it was published, small effect or no.

It calls to mind the Maharishi effect, which I was vaguely aware of for some time but got a thorough schooling in through Robert P Abelson's excellent “Statistics as Rhetorical Argument”. This is my favorite book I’ve read this year, and I’m heading for a re-reading sooner rather than later. Abelson recounts this remarkable finding, wherein
The number of people performing transcendental meditation correlates with reduction in car accidents, lower crime rates, and incidence of fire. He takes up this example a number of times, in which the major study1 got complex but significant results and appeared to follow its own methodology to the letter.

Of course, a result like this is going to be controversial, and rightly so: there are huge implications for science as well as national policies (mandatory meditation anyone?), so its damn important that this be right. As the research was published, every Tom Dick and Hermione with an inquiring mind was free to pore over what was done, and see if anything was wrong.

Sure enough, an explanation was found, and you will be unsurprised to hear it did not require the rejection of materialism, nor a consequent global rise in guru-republics. The problem lay with the non-random distribution of number of meditators across weeks used to match against the meditation periods. Higher numbers of meditators were available at holidays, weekends and the like and these calendar effects were trackable and produced an alternative explanation for the correlations.

It’s a good thing the Maharishi effect research was published, even though it turned out to be wrong. Why? Because it wasn’t obviously wrong: the confound they uncovered was not considered by the journal, and so presumably was not standard practice in the field of (hmm- psychological epidemiology?). It was a problem that had not been clearly classified as a problem.

When deeply unusual and, lets say it, unwelcome (from the point of view of the scientific community) research comes along, it galvanizes people to focus ever-harder on methodology, in a mass-game of spot the confound. Once the confound is spotted, it raises the quality of future published work, as it shouldn’t get by without recognizing and overcoming the issue. It has a hint of working with paradoxes in philosophy, but with an overtly empirical bent: “This can’t possibly mean what it says it means – so which of the inferences [or methodological steps] are unsound?”

Of course, if a confound really can't be spotted, and the result proves to be replicable, then this is an equally important reason to consider it. It might suggest that we need to be prepared for a paradigm shift in that scientific domain, as current theory simply cannot account for the genuine empirical findings. Or, perhaps more worryingly for the scientist, we might need to consider that our tools are not right for the job: either current scientific method or the human capacity to undertake and appraise it simply can't cope with everything we throw at it. Both would be radical results, but that doesn’t make them taboo. I don’t know of anything out there that seems likely to be taking us down that road though.

1 Orme-Johnson, D.W., Alexander, C.N., Davies, J.L., Chandler, H.M. & Larimore, W.E. (1988). International peace project in the middle east: The effects of Maharishi technology of the unified field. Journal of Conflict Resolution, 32, 776-812

UPDATE: More here in the comments thread, and a followup post further up the page.

Friday, June 18, 2004

A big hello any xoverboard readers who thought to check me out. a lot of current stuff is kinda science heavy - if you're not so into that then you might want to check out "Specialisation is for insects" pts 1 & 2, the beginnings of my ruminations on whether doing one thing alone is system we've been shoehorned into, without regard for quality of life or progress in general.

Other than that, there an introduction posted (2 parts) and hell, this place isn't that big, stick around, and ignore the chicken jokes; I try to.

UPDATE: Oh, maybe this would suit some - a blog post about blog posts is just what everyone needs, no?

Thursday, June 17, 2004

A few links:

Zombie Reagan for Vice President? That's a strong ticket! Check especially some of the genius slogans: "What do we want? BRAINS! When do we want it? BRAINS!" and the letters from the US-cousins of 'Disgusted, Tunbridge Wells'.

New liberal blog notable for a) simple, systematic bases for liberal arguments and b) looking spookily like this blog in format.

Guess your sitcom/dictator - this is just too damn good. Seriously.

Tuesday, June 15, 2004

Knowledge Communism

Some working in academia have a bone to grind about publication. Others could wave at least a couple at you, and there are those trying hard to fit skeletons into outsized mortars. The publication system sucks.

One major concern is the peer-review system. Many feel that it is vulnerable to poisoned attacks by referees who have an interest in work not being published; that it allows work to be assessed by those who consider themselves above such things, don’t apply themselves to the task, so filter like a one-holed fishing net. Moreover, factors such as sex and national language of author affect publication likelihood, different referees show agreement “little greater than chance”, and the whole process is extremely conservative and causes great problems for unconventional projects to ever see the light of day. You can see more of this, and one suggestion for an alternative, here (2 page pdf, quote taken from page 269, may not be available on all servers).

Another concern, one that I prioritise over the previous, is that of the whole ‘publish or perish’ culture of academia. This certainly affects some fields more than others, but mine – psychology – is steeped in this attitude, which rattles from the hot dogs with hefty grants down to students like me. This paper highlights the concern that the everyday student has. Note it was written in 1983 – to me, this is the good old days. Suffice to say that the problems have become ever more systematic. It’s still pretty timely, though. See this:

This pressure to publish--and to publish as often as possible--directly interferes with the kind of research advocated by Maddi, McGuire, and the others: research that is multi-dimensional, longitudinal, collaborative, and relevant to life in the real world. Indeed, the Publication Manual points out how "we become content with rapid, mediocre investigations where longer and more careful work is possible" (1974, p. 22),
Publish-or- perish as a guideline for untenured faculty has now become something of a mania even for graduate students. How else, we are told by our professors, will you find a job? It's a jungle out there, and a list of publications to flesh out the curriculum vitae is supposed to be our first line of defense. It doesn't much matter what we publish, or whether we actually write anything original or useful or thoughtful: what matters is how long a list of publications we can present to the Search Committee of whatever institution we hope to work at.

From everyone I’ve spoken to on the subject, this still appears to be the state of play. I’m lucky in having a relaxed and supportive supervisor, in that while he’d e very happy for me to accrue papers (as would I), it’s not considered the primary aim of what I’m doing – I’m here to complete a set of research and write a thorough thesis (and learn a few things along the way). For many other students I know, there is a pressure to publish as much as possible; I’ve been introduced to the concept of salami slicing your thesis – working out how many discrete components you can reduce your work to, in order to maximise your publications. This is undoubtedly detrimental to the ease of access to academic information: five studies essentially establishing one phenomena in various ways are far better presented together, allowing continuity of writing and far less redundancy, than as five different papers that have to prehash, hash, and rehash the same preambles, and may take slightly different tacks in their articulation, impeding a reader from getting their teeth into whatever story needs to be told. Moreover, as mentioned, it’s an impediment to certain types of research, which simply can’t be reduced to free-standing components.

It’s a disheartening climate, as it privileges publishing over all other pursuits. Teaching is what they make you do while you’re trying to get that paper in to Nature. Research groups and collaborations are a means to an immediate end, rather than opportunities to wrangle with assumptions and open conceptual horizons. It’s quite appropriate to have indexes of performance based on visible outcomes – and clans of researchers navel-gazing interminably without any goal is obviously of no consequence to anyone, except the odd piece of lint – but the focus entirely misses other ambitions the universities should keep in mind – furthering knowledge as well as creating it; providing an environment in which research students, who are effectively apprentices – can learn, make mistakes and explore, rather than serve as a publication and citation engine for the upper echelons.

Worst of all, it produces work that is just good enough. Who would polish their work beyond what is asked of them, when they could be moving on to the next paper? Well, luckily, I think a lot of researchers do, as certainly there are other pressures and incentives than the publication imperative. Certainly, as the Fox paper began with, those who are entrenched and secure, with tenure or masses of funding, are freed to explore other avenues at the paces they can specify. But certainly, the young researcher is in a squalid trap, not least because the candidates far outstrip the places available at every rung of the career ladder.
[EXTRA: this little baby on academic fraud, interesting in its own right, throws up something which is not hugely rare – the “stamp of approval” authorships where a big hitter puts themselves on a paper they’ve had no real contact with – only presumably to massage upwards their publication rates. So I guess its a problem which envelopes everyone on the ladder. For a less serious, although I hesitate to call it fun, situation then you can look at the whole Erdos obsession amongst the biggies.]

So what are the alternatives, if any?
We could prune away at the bad points of the publication system, and in this there are no shortage of ideas. The Della Salla paper ( pdf above) suggests the use of sponsors, who assess your work and decide whether they’d be happy to attach their name (and reputation to it), at which point it then goes to committee; there is an idea of referees taking on prosecutorial and defense roles, and possibly add incentive to performing these roles successfully (‘acquitting your client’ gets you brownie points in some way). The allocation of refereeing has been criticized, so perhaps a system that engaged in a bit of reciprocity, where researchers can build up a bank of review-me credits by themselves reviewing others, would get everyone involved in the process with a fair incentive (a bit like some of the P2P file-sharing systems, where you have to keep your upload and download ratio within some set of boundaries).

Many of these possibilities seem worth pursuing to me; the system as it stands is no-where near optimal and these are fairly pragmatic solutions. Most don’t require a universal decree to get rolling: you just need one or two respectable journals modify their submissions criteria, and then crucially for the system to work (and be seen to work) to observe the practise spreading. I’m turning my head now, so someone silly can shout ‘Meme!’

There. But I’d like to engage in some fanciful discussion of long-term alternatives. Alterations to the peer-review process such as those above don’t really seem to make much of a dent in the problem of publish-or-perish, although perhaps revisions to employment criteria, made in parallel to these, might help somewhat. I’m interested if we can imagine a system, say five, ten or thirty years on from now, that would allow the positive aspects of the publication system to continue (access to scientific findings, filtration of poorly produced science, rewarding of research activities) while minimising the negative (quantity of output as a measure, limited selection of individuals acting as a filter, biases against unconventional work).

I’m looking for answers, so I guess I should give my take on what might be a future direction.

Increasingly, the web is being used to publish scientific information. I’m convinced that in years to come we will be seeing this as the preeminent way to get hold of all academic information. My hope is that we will see journals eventually dispensed with altogether, with all information freely available and hosted either by individuals, university sites or research groups / special interests. I would argue that all scientific information, irrespective of quality, should be available within the same milieu, with no sharp division between journal-published, good science, and non-journal, questionable science.

Without filters operating, this would leave us in the unenviable position of being trapped in the morass of innumerable studies, with no way of navigating between them and no way of checking the science was done correctly. To avoid this, a peer review system would still exist, but one that is far more open and nuanced than the current one. Work to be published should be sent to research communities, in a manner akin to posting to a newsgroup: the suggested article, alongside the raw data and possibly copies of the program themselves would be posted there. The material can then be reviewed by everyone – probably having to go through some sort of gating system to ensure it is actually adequate to be considered (to avoid the wastage of many people looking over unacceptable work) , and following this being available for comments and ranking by their peers – a genuine peer community rather than one or two anonymous figures. A key element of this is that the work gets printed anyway (as work is hosted personally, not by the research community), but the ratings the work accrues can be used in literature searches, such that good work is preferentially selected over poor ones.

The bottom line with the ratings would be how useful the article was to the researcher. Usefulness can come in different forms: an article might put a persuasive case for something already fairly well established, or could turn common understanding on its head; it may progress theory very little but contain an ingenious methodology that could be turned to other methods, or carefully reveal lacuna in dominant models, whilst failing to offer an alternative. The ratings systems could represent this, meaning that a student looking for innovative techniques or one trying to get a sense of the theoretical landscape have those tools at their disposal, as adjudged by the entirety of the relevant scientific community. The way in which these communities should be composed isn’t entirely clear to me: should it be essentially self-selecting, decided by consensus (like a club), or through a board with applications? Certain of these options lean this system closer to the closed methods it attempts to displace. Nonetheless, it still would provide judgment by a far broader range of individuals.

I suppose one might why the community would bother to do this at all? I think this would be an odd criticism, given that the academic community is based on progress via understanding what other people are doing, and then doing it better (or contrary). If you are presented work by someone in your area, it would be strange indeed not to react by reading through it, sifting the useful from the useless and inspiring your own work. This, after all, is why seminars and conference presentations are not played to empty chairs: scientists want to know more about their area, so they can do it better. This would simply add a formalised feedback component to the back of these existing practise – feedback that could be later reviewed according to the individual paper’s “impact factor” (ie the number of citations that this particular paper has accrued) thus giving a prestige incentive to be fair and not miserly. If everything is transparent, it is very dangerous to be biased.

I should add that I think that basing the rating of the article on how useful it seems to the peer group seems a good way to make sure that basic/non-applied research is still of some definite service. It meets the criticism that certain types of research are privileged due to their fashionability rather than any real contribution to understanding (as numerous critics of imagin work would put – I’ll cover this soon) - if no-one can say that it’s helped them progress their own ideas or techniques, then the research will sink to the bottom.
That’s it in a nutshell – I might have had more to say but its gone with my now certainly lamented journal…. anyone care to raise me on this one?

CV, the drug of the nation.

Attempting an update of my CV, particularly for Public Engagement with Science type jobs. My first stab at a profile is a bit clunky. Any thoughts?

I am keenly interested in promoting science to a wide audience. For modern progress to continue, we need to attract new people into the sciences, and to ensure we are equipped to deal with the novel situations such progress always throws up we must provide the public both with accessible knowledge of the scientific facts on the ground, and, equally, the ability to bring scientific thinking to novel situations. Through my teaching, I spend a lot of getting students to apply systematic scientific reasoning to unexpected experimental results; via scientific training and public presentations I have learned to tell a story over outputting facts. The scientific community continues to teach me its processes, and I hope to promote them internally and externally.

Sentence 2 is particularly nasty, methinks. I've had to ruthlessly compress to fit with my 1-page criteria, so any advice should bear in mind I can't unpack these sentences at will - they can't be much longer than they are. Doesn't mean they have to be there at all, of course...

Thursday, June 10, 2004

Shit! Phew. Shit!

I discovered this week that my work computer is defective – it’s been compromised from the outside by nefarious folk with nebulous intentions. Luckily I had the opportunity and presence of mind to back everything up when I began noticing abnormalities. I actually did this with my iRiver, and it’s worth noting that for this purposes it’s definitely superior to the brand leader: disa went thru a similar spat recently and backed all her music up on her office-mates iPOD, only to find that the critter is unwilling to drop it back again. An anti-theft strategy that really punches holes into its utility (for an illuminating discussion on this face spited by needless nose cutting, see here).
Today I moseyed to work with my bag half-open – the zip pops when it’s too full – and I have a strong worry that along the way it ejected my journal, full of piping hot knowledge. I’m hoping to add a second Phew to the post, in the event I return home and find it staring at me from my bedroom floor, but damn! I really have lost my memory – it’s as much that externalized in scripts, notes and mnemonic aids as the stuff inside the noggin, an argument elucidated in Being There, Andy Clark’s great book about the concept of the embodied mind that entered cognitive science sometime in the late 90s (I think; others – notably Eldan – know much more about this than me, having only incompletely read it as an undergrad). Obviously I study the noggin somewhat more, but the truth of it isn’t lost on me. Unlike, I fear, my journal.

So I’m internet/email crippled (I can get basic messages now via my hotmail, but no big attachments, and for that I’ve got to go to a shared room on another floor. The indignity!) and without all my productive notes made this year. So expect extreme erraticism.

Tuesday, June 08, 2004

The merry Whys/Whynots of Windsor

We had our postgrad departmental conference at Cumberland Lodge a few weeks ago. It’s a highlight of the year, based in scenic Windsor Great Park, and offering good cooked meals, huge baths, and table tennis. I co-organized the trip for 2003, where we shook up the format with a series of contentious debates: Is any phenomena truly psychogenic?; Is permission always a prerequisite of involvement in research? [mainly animal ethics]; What bearing does evolutionary theory really have on psychology?; Does science have all the answers?
Oh, the shouting! Oh, the finger-pointing! Oh, the sacrificing of hens on the prince regent’s bed. Love it.

This year the charming Disa and the also-charming David put it together. The topics?
    Psychological education – is Psychology a single unified discipline?
    What is a psychological law of nature?
    Peer-review – the best possible option?
    Who sets the scientific agenda?

Good meaty stuff, as I’m sure you’ll agree. I’ll be posting on a few of these – I made a few notes at the time and I’m trying to translate them into a human language. Quaking with anticipation? Oh stop it, you.

Wait – that was sarcasm? Bitch.

Thursday, June 03, 2004

Nu left

At one time bemused at punks being politically active, the big man at Harrys place finds the whole festival thing all a bit too much - especially the notion that liberal, Guardian readers might be there. According to comments, he’d prefer a “smokey basement punk club”; it’s unclear what music preference the politically minded can have without risking a derogatory bashing at some point or other. The reason this is worth bringing up is that from the description I’ve painted here, you might think said blog was right-leaning in question: bash the Guardian, act baffled that punks are anything but a joke minority, etc. But no! This is the new left, formally distinguished from the old by their support of the Iraq war; equally easy to spot by their obsession with criticizing, deriding and generally hating on anyone else who is considered left-wing. – see these two blog topics: Trots (sounds like a trip to the toilet eh? I guess that’s the point) on the bankruptcy of socialism and this on The Left, more of a mixed bag but still largely negative stuff. There is no such topic on the Right, (update - there is one on Anti Fascism but it's extremely thin - 4 posts, the earliest in May, suggesting an afterthought) which seems surprising for a liberal UK site at a time when we are seeing the rise of the BNP and the emergence of the UKIP. These parties do get a mention, but generally in terms of how they are akin to the ‘extreme left’: they are a handy way to smear and criticize other people you really have a problem with.

It’s a shame because I like the site well enough in several respects – there are not enough UK sites I find readable or relevant (Matt T’s blog is an exception, just added to my blog-roll – deftly written, and the guy can handle statistics in a way that conjures envy in me); I like group blogs, which is why the Coop has fleshed out the flock, at least nominally; and I do think some aspects of what they are focussing on – extremes of the left wing being dogmatic and unreasoning, and potentially harmful – is a good thing. But my god, if you’re the policeman of the left must you model yourself on early 90s LAPD? If the response to left-wing knee-jerks is to jerk right back, nothing is achieved, except a damn good Cossack dance.

One theme that was strutting in these circles for a while that really, really got on my tits was the one about how concerns about attempting global democratisation were basically racism in a woolly jumper. I wrote something about this just before Japan, so it’s a bit late to be posting, but this is still being bandied about – as recently as yesterday, where a program on Radio 4 (the Moral Maze) saw these notions being raised again. So here it is:

It’s amazing how certain camps are prepared to trot out the remarkably flimsy straw man that the anti-war crowd have a distinctly colonial sheen about them, with the suspicion that ‘these sorts of people’ can’t be expected to govern themselves in the same way we can. In one sense, the thing has legs, but the rest is on all back to front.

Among those with misgivings about the current state of Iraq there is genuine concern about the transformation attempted. This is not a distrust of the Arab capacity to find democracy. It’s an awareness of the difficulty of sweeping revolutionary change. It’s something any good progressive-minded person should have cut their teeth on in the progression from naïve idealist to pragmatist with ideals. Looking at major 20th Century sweeping changes to the cultural landscape, you can see that the successful ones were remarkably circumscribed vs the big ones. Leftists can be proud of civil rights, extensions in the franchises, child welfare laws. They should be ashamed of the seizures of total power and attempts to redraw society utterly that has coloured and twisted the notion of progressive action. This should be bread and butter to the self-styled ‘new left’, as they revile the adherence to Marxist principle and starry-eyed idealism seen in the big left-wing institutions of the UK today. If they could turn it in toward themselves, they might recognise that that their zealous commitment to the Iraq project, with all its embroiled ambitions (from domino theories to transforming the entire middle east, to flypaper strategies for transforming the fight against terrorism into a conventional and finite land war) reeks of the same simplification and dogmatic commitment to project.

We all, if we call ourselves left wing, want to change the world for the better (not to imply this is exclusive to the left, but it is a prerequisite of being left-wing). However, not all of us are naïve enough to assume the world is going to skip along with our aspirations. What is being attempted in Iraq is a radical change. Perhaps the prowar left would argue that it is one that will always be radical, and so is wholly necessary. To which the follow-up, and critical step, would be to ascertain whether the proposed plan accounts for the scope of the problem in question. Will there be intricate planning for multiple, complex contingencies? Do we know what the short, mid and long term goals are and have we weighed a cost-benefit analysis? Could we do more good with the same resources elsewhere, or at less risk?
Some people did the job, and they found the proposals wanting. That’s the issue at hand, not Saddam-loving or Arab hating. I hope such claims will die down once people realise their ridiculousness. But I’m not holding my breath.