Peter Cochrane's Hard Drive 1997
Homepage > Publication & Opinion > Hard Drive

The mismeasure of machines
Many years ago I worked on radio system design when the distance between transmitter sites was measured in miles, the tower height was in feet, parabolic dish diameter in centimetres, frequency in cycles per second, field strength in mV/m, and wind force in pounds per square inch. What a nightmare of units, manipulation and potential for error.

Before the introduction of the metric (SI) system technologists faced a major hurdle of communication with the arbitrary use of feet and metres, ounces and grams, Celsius and Fahrenheit, slugs, dynes and more. Progress and thinking were fundamentally limited by the need to convert everything into an understandable form.

Today, disparate groups from different disciplines can comprehend the Newton, Kelvin and Watt. This unified language of time, space, action and energy has been fundamentally responsible for much technological progress over the past 30 years.

Now we face a new and more important challenge. How can we meaningfully rate computer power? We currently cite millions of instructions per second (MIPS) or floating point operations per second (FLOPS), with raw, formatted, and compressed information. There is no general way to make measurements or accurate comparisons. Extending this to non-serial, non-digital computers (animals and humans) we seem even worse off, and reduced to making the best estimates we can based on problem-solving parallels.

Probably the most commonly used measure for carbon systems is the Intelligence Quotient test based upon the time to solve a range of problems under test conditions. In reality this only gives an aggregated measure of the ability to solve a class of problems relative to a test population. It is on this basis that our species looks pretty smart, at the top of the class, because we thought up the problems in the first place. In a way this is an unfair and incomplete and unfair technique. If rats and monkeys set IQ tests we might find ourselves further down the league table for agility, finding food, a mate, and reproducing for example.

Compared with machines we are obviously outclassed on production lines, navigation, numerical computation, chess and much more. And yet we only have the conversational Turing Test as a notional measure of human-machine equivalence. Worse still, we are instinctively dismissive of "intelligent machines" - something we never do to our own species, no matter how seriously damaged.

Perhaps it would be more sensible to derive a comprehensive measure from problem-class evaluations. We could then say machines assemble camcorders a thousand times faster than us; sort letters a hundred times faster; are better than the best of us at chess; calculate a billion times faster; but recognise a face, pen, or a cup of coffee with only 0.001 of our precision. On such a basis we could have a running comparison of silicon and carbon systems, and most important, a true measure of their relative abilities.

This would not only give a clearer picture of machine progress, it might also help to allay techno-fear and provide a pointer for future investment and development.

Peter Cochrane holds the Collier Chair for the Public Understanding of Science & Technology at the University of Bristol. His home page is:
http://cochrane.org.uk

? Copyright Telegraph Group Limited 2000.

Telegraph Group Limited endeavours to ensure that the information is correct but does not accept any liability for error or omission.

Users are permitted to copy some material for their personal use, but may not republish any substantial part of the data either on another website or as part of any commercial service without the prior written permission of Telegraph Group Limited.