Measuring and Evaluating as Rhetorical Management

I’m struggling to write a chapter about the measurement and evaluation of school buildings in use, what’s called “Post-Occupancy Evaluation”. Writing this is an attempt to clarify some thoughts about how the selection and measurement of values in a process of evaluating buildings is necessarily political and involves the communication and projection of particular values (rather than a mere recording of them). I argue that evaluation, what is valued and their promulgation can be thought of as a form of rhetorics.

I take rhetorics in the first sense from the online Oxford English Dictionary: “The art of effective or persuasive speaking or writing, especially the exploitation of figures of speech and other compositional techniques” and not necessarily the second: “lacking in sincerity or meaningful content.”

Rhetorical management could therefore be thought of as how people use particular techniques to compose (i.e. organise, arrange, in/exclude, order, hierarchise, flag up, play down etc) the constituent parts of their message. A few of the writers below have helped me to think through what that means in relation to values and measuring generally and then in relation to school buildings in operation.

 

1) What gets to be composed in the act of evaluating? That is, which values and whose values get to be included in any evaluation and which not? What processes (managed by whom) selected those values and not others?

One way to explore these questions is through the work of Amartya Sen, an economist and philosopher working on the evaluation of poverty. Sen refers to the “evaluative space” which is specified by “the identification of the objects of value” (1995:43). A lot of his work can be seen as a challenge to classical economic approaches, effectively calling attention to the questions of: who does that identification work, which objects should go into the evaluative space and with what effects: “the selection of space can also have a good deal of discriminating power, both because of what it includes as potentially valuable and because of what it excludes from the list of objects to be weighted as intrinsically important.” (ibid) At this point, Sen is writing in general terms i.e. these claims about evaluation apply to any process of evaluation.

Who holds “discriminating power” over the evaluative space? That evaluation can be somehow apolitical is, on this reading, untenable. Applied to the Post-Occupancy Evaluation of schools, we could ask: what/where are the values of children? Of teachers? Who gets to compose the evaluative space and so forth. Ultimately, a further question can or should be asked, and that I explore below: whose rhetorics is this, and what is it trying to communicate?

 

2) Measuring does not and cannot simply assign a number to something (a value for example). Nor can it produce knowledge about something in a neutral way. Nor can measures report only – they feed back and flag up, emphasise, give visibility to what is to be measured and are knowingly used as such. Measures are a form of communication.

Take Andreas Schleicher (head of the OECD’s PISA programme), for example, in an interview: “If we want to bring it on the radar screen, we need to measure it” (2016: online). Making certain kinds of education (or learning) more visible, by measuring them, is a way of making them more important so that the use of measures is also the use of a selective, power-conferring, way to see the world and an attempt to communicate particular kinds of world into being (at the expense of others, as Sen shows). It is the rhetorical management of education itself.

Andrea Mubi Brighenti draws on a long history of thought in this area and writes:

A measure is an inherently relational device, one that defines relations of value and assembles disparate beings by bringing them into given configured relations within a defined environment … measures immediately entail a whole politics of visibility rather than simply epistemic constructs, measures are a domain of practical action (original emphasis, 2017:6-7 link to journal article)

That visibility (but less the politics) comes out clearly in Schleicher’s use of “radar” – the metaphor hiding, perhaps, the work of who constructed the radar in the first place, who calibrated it, what it sees and how, in short the nature of the evaluative space it provides. And UNICEF’s report on child poverty shows how Brighenti’s “domain of practical action” relies on measures as tools of policy management:

What is to be gained by measuring and comparing child well-being in different countries? The answer lies in the maxim ‘to improve something, first measure it’. Even the decision to measure helps set directions and priorities … Over the long-term, measurement serves as the handrail of policy, keeping efforts on track towards goals, encouraging sustained attention, giving early warning of failure or success, fuelling advocacy, sharpening accountability, and helping to allocate resources more effectively. (UNICEF, 2007:3)

 

What does all of this have to do with the Post-Occupancy Evaluation of schools?

It’s the repeated silence (at best) and misrepresentation of or burying (at worst) of the politics involved in measuring and evaluating that confuses and disturbs me. For example, two of the earliest developers of formal Post-Occupancy Evaluation and its successor, Building Performance Evaluation (BPE), celebrate “the independent eye of a BPE on the [building’s] design” (Preiser and Nasar, 2008:96). How is the “eye” constructed, by whom? How is it “independent”? Any sense of the “perspectival character of knowledge and experience” (Sayer, 2000:30) is lost.

While Preiser and Nasar do discuss the evolution of POE and BPE, the politics of how these tools and approaches are constructed, and by whom, is lost in the claim for “a consumer-oriented democratic approach” (2008:88) which I can understand only as a contradiction in terms.

Measuring and evaluation can’t escape being political because they involve a necessary step towards domesticating, then representing, communicating and establishing certain values as the values that count. Perhaps there is also a political art to this, a rhetorics of managing and composing systems of evaluation and measurement that is also the art of making their message seem natural, scientific, apolitical.

One way to acknowledge some of the artifice is to do more questioning before measurement and evaluation approaches are set up, something I’m trying (and so far failing) to think of in relation to POE. Robert Frost’s poem perhaps has something to say about how walls and evaluative spaces get built:

 

“Before I built a wall I’d ask to know

What I was walling in or walling out,

And to whom I was like to give offense.”

Mending Wall, Robert Frost

 

 

As a sort of postscript, something I need to think about more is the tendency to equate knowing about with measuring, and so knowledge with quantified information, and how this applies to buildings too. Again, this is potentially another way of silencing some values with respect to others since not all are so amenable to being quantified. The difficulty of finding out about children’s and adults’ experiences of school and then questions about how to represent those findings, the extent to which they can be generalised to other school buildings is problematic but something that needs to be dealt with if evaluative models are not to have an inherent political bias of excluding those who spend a lot of time in the buildings we want to find out about.