New website
The world according to Potter Part (1) Going Metric
By Terry Freedman
Tue, 30 Oct 2007, 07:46

In this brief series I'd like to see how the writings of Stephen Potter might be applicable in the world of education -- and, in particular, educational technology -- today. Writing predominantly in the 1940s and 50s, Potter codified the art and science of "one-upmanship". In so doing, he not only inspired a generation of undergraduates to put his theories to the test and invent new "ploys" and "gambits", but inspired the making of a film ("School for Scoundrels") and, perhaps more importantly, was taken seriously enough for the term "one-upmanship" to be cited in academic books.

His books are concerned with the study of how to be "one up" on other people. Although they are written very much tongue-in-cheek, they are clearly based on real-life observation. I first came across them 40 years ago, and have read and re-read them over the years for their humour. However, I find myself more and more discovering that a number of aspects of modern life may be found in these books, despite the elapsing of half a century, a fact which I believe puts them on a par with other classics such as Parkinson's Law and The Peter Principle.

So, with no further ado, let's see what Potter has to offer the educational ICT (Information & Communications Technology) subject leader in a school, Local Authority or School District. For this to make as much sense as possible, imagine yourself to be of a certain mindset: that of regarding every waking moment as an opportunity to place yourself, or appear to be, in a superior position to those around you. It may all sound too far-fetched, but as you read on I think you will start to recognise people you know....

I have already written about this in the context of getting ICT embedded in a school. In today's article, I should like to explore the wonderful world of statistics.

I don't know if you have noticed, but every presentation by a Government spokesperson consists of at least 5 minutes (and often much more), of statistics. Whatever the topic under consideration, there is always a section that goes something like this:

"Since we were elected X years ago we have more than doubled the number of Y, and over the next 3 years we will increase this by a further Z percent".

We hear it all the time in presentations about educational ICT in the UK, but it appears in every other branch of public affairs too.

The thing about statistics, though, is that so much depends on context, even if the figures themselves are (a) accurate and (b) not subject to interpretation -- both of which assumptions are highly dubious for a start. For example, if an educational spokesperson were to announce that the Government will spend an extra £10m on in-service training for teachers over the next 3 years, that sounds impressive until you work out that, in the UK, that amounts to just over £22 per head, or around £7.50 (approximately 15 USD) per teacher per year. (See http://www.statistics.gov.uk/cci/nugget.asp?id=1765 for the statistics on which I have based this calculation.)

Now, there is nothing startling about this per se, but what is interesting is the fact that it is completely disarming unless you (a) happen to know the underlying stats and (b) can do lightning fast calculations. The point is, by the time you have even had a chance to think about it, the moment is gone, and the speaker is on to yet another "fact".

Interestingly enough, Stephen Potter recognised the power of this sort of approach. In "One-upmanship" there is a chapter called "Doctorship", in which Potter discusses the important topics of medical studentship, doctorship, patientship and healthmanship. (I will be examining Potter's use of terminology in a future article.) In a footnote he says:

"An effective statement in the right context can sometimes be: 'I have had 140 days' illness in my life.' Listeners are unable, without a lame pause for calculation, to know whether to commiserate or admire."

So how does this apply in the context of educational technology leadership? The answer is that in today's world, metrics are all-important. I personally believe that that is how it should be, but it's easy to be fooled by statistics which sound good but which, on closer examination turn out to be less than desirable or even meaningless.

For example, I have no idea why any teacher would want their students to receive thousands of comments about their work, because not only is that volume of comments unhelpful, it is completely unmanageable, as I have already said recently (see, for example, the October 2007 edition of Computers in Classrooms, which is available via a free subscription).

I also think most RSS feed readership statistics raise more questions than answers, and that (for my website at least) Technorati's statistics are fictional. More importantly, the metrics given out by companies need further scrutiny.

For example, a technical support company that claims that 99% of its call-outs are rated excellent inspires no confidence in me whatsoever. If the company has 10 employees, each doing one job per day, it means that over a two week period one of those jobs or days will not be rated excellent. That sounds quite a lot to me.

Similarly, a web hosting company that promises 99% "up time" may actually be promising that you may have to put up with the site being "down" for 15 minutes a week, assuming a 25 hour school week. Even if we leave school out of it, given the global nature of communications, I don't want my website to ever be down, not even for 5 minutes a week -- and even then, I want it to be planned for so that I can put a notice up and warn people. Yes, I know I am asking for the impossible, but my point is that statistics like "99% up-time" are meaningless unless we understand the context in which they are cited.

As an educational technology leader, you should at least know some important statistics. When evaluating the quality of a school's ICT provision (at the request of the Head of ICT or the Principal), I make a point of asking a number of questions which involve facts and figures and which the Head of ICT should either know or have immediate access to. It is astonishing how many don't. For example, do you know if there is a difference in attainment in ICT between girls and boys in your school? If so, is it significant? Do you know the cause? What are you doing about it?

As well as knowing some basic figures, you should also know what they mean. Statistics are often given a spurious veneer of credibility by the addition of a graph. I recall one teacher showing me "before" and "after" charts to illustrate how much his students had progressed over the last term.

"But what were the tests actually measuring?", I asked.

"I don't know", came the reply. "But the point is that whatever it is, it has gone up."

Getting back to Stephen Potter, he was making a wry observation about the cavalier use of statistics. Although he wrapped it up in a humorous, not to say unlikely, package, he was alerting us all to be on our guard.

© Terry Freedman Tue, 30 Oct 2007