The other day we interviewed a couple of PhD scholarship candidates. Good applicants, with interesting and unusual research proposals. However, I was saddened when one of them started talking about ‘impact’. So, she’d found out about the institutional regime of evaluation that now governs research and learned the lingo, but is this the game that applicants ought to be playing?

I’ve been thinking about this because I’ve had to write a case-study self-evaluation for the university’s rehearsal for the REF, the administrative-bureaucratic exercise replacing the RAE, that governs the disbursement of £1.6b of research funds in England annually. Under the new rules, 20% of the final score is being awarded for the ‘impact’ of research in the wider world, for which the rubric is ‘an effect/change/benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia’. The figure of 20% is of course quite arbitrary, but the real problem is the requirement to quantify what cannot necessarily be quantified, especially in the arts and humanities.

To put it baldly, the requirement for research to demonstrate impact is a crude tool of bureaucratic manipulation that bends and distorts the multiple functions of research towards its own utilitarian purposes. These purposes are not just administrative but also political convenience—the funding body must satisfy the political paymasters. That’s how bureaucracy works, although it mustn’t be too blatant. When the Arts and Humanities Research Council announced almost a year ago that it would spend a considerable part of its £102m funding pot on research into ‘localism’ and the ‘big society’, widespread protests against subjection to a political agenda eventually led to a climbdown [here and here].

According to a leader in the THES in October [here] the current focus on ‘excellence’ stems from Business Secretary Vince Cable’s line that the UK had to stop funding ‘mediocre research’. But the problem here isn’t just what he said and how it gets interpreted. It’s the fact that the minister in charge of higher education is the Business Secretary. How come? It happened on New Labour’s watch, when the first Business Secretary to take charge of universities was Peter Mandelson, the come-back man who was comfortable with the filthy rich as long as they pay their taxes (which many of them don’t). We’re speaking of a managerialist ideology in the interests of commercial and business criteria—criteria essentially alien to the concept and practice of academic freedom. The same ideology that’s intent on ‘reforming’ the whole university system. Anything that can’t be value-costed doesn’t count. Humanist values of inquiry are old hat unless they can be commoditised, or at least conveniently packaged for purposes of cost analysis. If I remain attached to such values, it’s not because I’m an intellectual Luddite, it’s because ideas, arguments, debate, are not commodities, or not reducible to commodities, the material form in which they circulate within the market, but mental or psychological or aesthetic forms whose life is lived outside and beyond the market.

From the perspective of the bureaucracy, this is exactly what’s problematic about the traditions of disinterested learning of the university, whose freedom and autonomy is an essential element of the pursuit of knowledge for its own sake. Hence the new emphasis on ‘robust’ evidence, that is, hard data, that is, figures and numbers. This process ineluctably produces gobbledygook like the warning of a Guardian blogger that ‘There’s still a lot we don’t know about the relative quality profile of the output weightings.’ [here]   There is now even an official sub-branch of the academic research industry devoted to the issue of how to produce robust evidence. It’s called the National Co-ordinating Centre for Public Engagement [here] and it’s funded by JISC, the higher education technology and IT body. Not only that. Universities are spending money on employing specialised administrators to handle the complex process of compiling the required documentation. Academics themselves, meanwhile, as a reader commented on the aforementioned THES leader, are ‘having to waste their time filling in forms and inventing narratives to justify research which has already undergone a peer review process at point of funding’.

However, according to the NCCPE’s deputy director, some academics ‘are struggling to find any evidence, as they just didn’t think to capture it at the time…some will use anecdotal evidence, whereas others will use a more sophisticated way of evidencing their impact, and they will stand shoulders above those who don’t.’ ‘Sophisticated’ here is a euphemism. It means flashy data processing, whose results, when displayed in the form of animated bar graphs and pie graphs and what-have-you, however pretty, is a total abstraction.

From where I sit—the zone of what’s termed research-as-practice or practice-as-research, I don’t know the difference—this seems the height of absurdity, even a kind of mild insanity. You now not only have to do research but you have to do research into your research. Actually, I already do. I take the films I make to screenings and engage in Q&A sessions with the audience, following the practice of the independent film movement back in the 1970s. Then I write it up in a reception diary of some kind which gets published in an academic journal. This is what we academics are supposed to undertake to do to get the funding in the first place, and I always do it. But not in a way likely to please the bureaucrats, because what interests and concerns me is the interpretation, always provisional, of the different, often ambiguous, but felt responses to the film that you encounter in this way.

These are not the kind of questions that can be answered through a list of  quantifiable criteria, because they involve factors like aesthetic and moral judgement, where quantitative evaluation is always reductive and anyway irrelevant. Maybe the findings of the social sciences can sometimes be tracked in the required way, at any rate when they’re taken up by non-academic bodies in the context of policy debates. It is very rare for works of creative practice like video or the performance arts to produce a direct and concrete impact on, say, policy making. But they can and do contribute at the level of communities, both local and virtual, and are often linked to alternative social initiatives and campaigning. The kind of impact they achieve, however, is like all aesthetic experience, of an experiential and existential kind that necessarily falls beneath the bureaucrats’ radar.

In short, the impact of an aesthetic work begins by enhancing and intensifying the beholder’s awareness, and you don’t need a neuroscientist to tell you that (though they might have interesting things to say about it). In an aesthetic sense, the primary moment of impact is the moment of reception, and a video or performance is created for the purpose of arousing the beholder’s immediate response as such, because without it there cannot be any longer-lasting impact. If I’m wrong about this, then I wasted all the time I spent as a student of philosophy and aesthetics.

Then there’s the question of engagement in the digital sphere, which applies to all scholars in the humanities and is much encouraged—only for the evaluators to turn round and say that blogs don’t count (perhaps it’s because they aren’t peer reviewed). Part of the problem is in the mythologies of the digital domain, like the unexamined notion of the viral as a measure of impact when numbers alone tell you nothing and the viral is in fact entirely relative. You can use social media and blogging (like this) to extend the reach of your work, but this doesn’t produce a measurable indicator of impact. As another blogger puts it, ‘social media metrics are often poorly calculated and even more poorly understood’ [here]. Which doesn’t mean your blogging isn’t worth the effort (I hope).

The digital arts face a particular problem, because most of the works produced within the academy inevitably remain marginal, simply because the web is hugely dominated by commercial operators and popular trivia. Nevertheless, it is also very good at letting a thousand flowers bloom in its interstices. Appraising the impact of one these flowers, however, is far from straightforward. For example, such figures as provided by the web platforms that carry the videos are only rough-and-ready guides to dissemination. They may tell you about the distribution of viewers across different countries, but not what kind of viewer they are, even though the video in question is clearly reaching out into the wider world. The reception of streaming video, despite the web’s much touted interactivity, remains anonymous.

This is connected with sociological and conceptual questions about identity which again place the administrative categories in question. The idea that impact outside the academy can be neatly separated out and measured ignores what has been taught within the academy for many years, that everyone’s identity is multiple and shared with others in different groupings. A student shares a video through a social network with family or friends outside it: is she doing so as a student, or as a sister, or as a drinking companion? A university lecturer sends a link to members of his cricket club, some of whom are also academics but in other disciplines: is he doing so as an academic colleague, or a sporting pal, or as a concerned citizen?

The problem is not that targeted funding would improve results but a structural one, in which both intellectual production and social critique are disadvantaged and marginalised. The public sphere is not a homogenous arena but a network of dispersed, unequal and overlapping spaces of communication dominated by the centralised mass media which are now also embedded in the virtual domain of the web. (The web is dominated in turn by gigantic new players like Google, Facebook, Twitter, Amazon and the rest, which serve to extend the reach of pre-existing media while also forcing corporate consolidation, like the recent merger of Penguin and Random House.) But the web, notwithstanding its open and borderless flows, is highly fragmented, and most of the time, rather than overcoming cultural compartmentalisation, it amplifies it.  And while it has indeed opened up new spaces for ideas to circulate more or less freely, which is a good thing, it also reproduces the entrenched anti-intellectual populism of the centralised media.

All of which would mean that ‘impact’ is a specious and entirely ideological concept. In the words of another comment on the THES leader mentioned earlier, a weird dogma to apply to the workings of the intellect, and ‘an excrescence of an ideology that has temporarily established its domination not only in Britain but in the Western world generally’.

By the way, if you’re a non-academic reader, please make yourself known to me, so I can include you in my returns.