Prologue
The basis of all human understanding is fundamentally linked to our sensory abilities and totally dominated by visualisation. Conceptualisation and (and sometimes transitory) understanding usually follow a process of internalisation and mental modelling. The major difficulty we now face as a species is a world of increasing complexity and non-linearity - a world of counter intuitive results and outcomes. In this world our ability to visualise is increasingly vital to understanding and progress, but it is also becoming increasingly difficult. Interestingly, in 1945, Richard Feynman, whilst working on the Manhattan Project, stated;

"The single biggest problem we face is that of visualisation."

This is still true today and can increasingly be seen as a prime limitation to our progress in a range of sciences and technologies. In this paper, we examine the importance of visualisation by considering a few specific examples relating mathematics, science and engineering. The need to visualise complex non-linear systems in the information world is highlighted before we consider some of the network, computing and interface technology constraints. Many of these are not fundamental, but self imposed by the limited thinking vested in a largely copper mind-set, two dimensional, past.

The choice of title for this paper was purposeful as we are actually on a progressive journey through the three lines of an ancient Chinese proverb! Following the information age and visualisation comes the experience age and full sensory immersion - and as a result a progressively fuller and complete understanding of the world in which we live, and the systems we build. We started with words, progressed to static pictures, moving pictures and now multi-media - so we might view the technological progression as:

P1k = A picture is worth a 1000 words

P1M = A moving picture is worth a 1,000,000 words

P1Bn = An interactive pictographic world is worth a 1,000,000,000 words

Our basic hypothesis is thus: if we could see, touch and feel the objects generated by, and for, information systems, and the new abstractions we have to deal with, then our understanding would increase dramatically. Although we have all the technologies to hand, we seem to artificially constrain ourselves with networks, architectures, protocols, interfaces and display systems vested in the past. For a real breakthrough we need to shed the CRT and raster scanning, limited bandwidth communications and massive data compression that all detract from the ultimate goal of embedding ourselves in the electronic environment. We can have a far more `realistic' artificial world if we so choose; by and large - we are the limiter, not the technology!

Experience
Imagine being told that when you throw a cricket ball the trajectory is straight for a given distance followed by an abrupt fall to the ground. You would contest this immediately on the basis of your earliest of childhood recollections that the trajectory is actually an arc. Unfortunately, much of our education and understanding is not based on such fundamental and easily assimilated practical experience. For much of our work in engineering and science we have to rely on mathematical abstraction, which often takes some believing and a lot of understanding!

The micro-world of Quantum Mechanics represents one of the most difficult areas, and least understood. In fact, we can safely assume that no one understands Quantum Mechanics! The notion that an atom should change size on the basis of the speed of approach of a proton is something we can never experience directly. Yet this is well founded experimentally and lies at the very core of nuclear fission. A key feature of this example is the counter intuitive nature of the predominantly non-linear world in which we have to design and build systems. Visualisation technologies offer a new window into this world that surpasses all our established methods and techniques.

The nature of the universe
Throughout our formal education we are fed a diet of problems that can be solved. From our earliest days at school through to degree level, we are thus led to believe that the universe appears as Fig 1(a). Here we see largely solvable problems and things we can analyse and understand within a very small area of uncertainty and lack of knowledge. In later life, we find that this is not the case; we see the universe is not dominantly linear, and easily explained, but as shown in Fig 1(b) - grossly non-linear.

Although our universe is grossly non-linear, we have had the historically fortunate experience of being able to develop the majority of our technologies with nominally linear techniques. Perhaps the most common example of the development of non linear systems is software. Our experience prior to the arrival of computers is depicted in Fig 2 where we see a very strong linkage between our physical world experience, physical models and mathematics. The arrival of software however, introduces a realm that is wholly artificial and is not linked to any physical entity or world that we have previously experienced. We therefore see a new universe, with an abstraction beyond that of mathematics and one that is extremely difficult to understand. The sheer scale of software running to millions of lines of code, with of thousands of feed back loops and decision points, is something way beyond naked human ability to understand - so far!

Our fundamental need
During the past 10,000 years the power of the human brain has changed little and we expect very little change in the next 10,000. In fact our `wet wear" (brain) is in stasis and is fundamentally limited - it just cannot grow in capacity by more that ~ 15% of it's present size. We are in an evolutionary box canyon! In contrast, a mere 100 years have seen computers evolve from mechanical to electromechanical, thermionic valve, transistor, and integrated circuit based machines. An electronic wrist watch now has more processing power than a computer the size of a domestic washing machine 20 years ago (Fig 3). Today's machines have a staggering ability that will continue to double every 12 - 18 months for at least another two decades, and probably three.

So, we now face something of a dichotomy as the human mind is fundamentally incapable of formulating many complex problems, processing them and providing answers using paper and pen. Whilst computing power can analyse, synthesise, model, and solve problems of staggering complexity, in so doing they very often mystify even the well educated. All too often though true understanding is lost in the sophistication, and in short, we are often data rich and information poor! Data visualisation has thus a vital role to play bridging the gap between man and machine, the complex and the simple.

At the present rate of progress computers will be 1000 times more powerful in 10 years, 1,000,000 times in 20 and might just be 1,000,000,000 time more powerful in 30 years time. Around 2015 we will see super computers = humans from a processing and storage of information point of view. They may also exhibit true intelligence by this time. Our only means of communicating with machines of such power, without a direct link to our cortex, is through visualisation. The use of visual images, sound, and touch in ever increasing degrees of immersion and sophistication is key to this potential symbiosis.

Abstraction
An analogous situation to that of writing software, and understanding a hugely complex system, is that of trying to build a bridge on the basis of the binding energies in the atomic nucleus of the iron atom instead of using Young's Modulus. This abstraction allows us to stand back from the deep physical understanding of the material with sufficient accuracy to construct an entity that is fit for purpose. The difficulty with software is that at present we do not have such a level, or suitable form, of abstraction. So far we have migrated from (abstracted) machine code to high level languages, and often in a graphical form, but we still do not have a total systems nomenclature that allows us to grasp the full picture. Perhaps visualisation will allow us to enter this realm and get a new view, and hence a better understanding of non-linear systems.

The same is true of mathematics. Even as recently as the 13th Century the general use of pictures in mathematics was decried by many, and yet, visualisation is probably the single most powerful tool available for understanding and teaching the topic. But a parallel situation also exists today with those who decry the use of IT. And yet without computers the world of complexity would remain hidden from view - chaos and fractals cannot be explored without the screen! Turning the complex into animated, 3D, colour, pictographic forms gives a new means of understanding often invisible physical and abstract situations. It is almost certain that visualisation technology holds the vital key to creating a generalised understanding of almost everything. Without visualisation we would know virtually nothing about turbulent flow in gases and fluids, or evolutionary theory - we would be flying blind!

A mind experiment
Suppose you were a grain of salt inside a pot. What would your experience be? If the salt pot is tipped upside down you would experience a change in orientation in a largely stable world. Gradually you would feel and see slippage and movement around you with a sudden acceleration, falling sensation, abrupt deceleration and stability. Some time later, you would be involved in a series of land slides and earthquakes that would come and go, ultimately giving way to total stability. It would be extremely difficult for you to imagine the form of your universe. However, if you were lifted out with a pair of tweezers and placed at a distance so you could see the pile of salt, then you would have an instant understanding of the system that governs your world. In essence, this is how visualisation can aid our basic understanding, by allowing us to observe from a new vantage point. That vantage point can be as abstract a place as an electron, a proton, red dwarf, or a point in a data set, electromagnetic field or as mundane as a designer kitchen, architectural drawing, or inside the human body.

Real experiments
A large percentage of the population have difficulty with 3-D visualisation from 2-D drawings. The designer kitchen is one example of how this technology can help people to realise what they are about to purchase, or what they have just designed. Perhaps more challenging would be the ability to reach out and feel an electron that is a rather uncertain jelly like entity easily removed from the outer fringes of the atom. Whilst in contrast, the neutron is hard and round and solidly locked into the nucleus to the point where we have extreme difficulty removing it. Perhaps we could even see and feel how molecules latch together as we construct a new form of drug. How much better to learn about atomic physics from this standpoint, than from a series of abstract numbers that are so small and large that we have difficulty comprehending. Our real world experience of gravitational force is difficult to relate to the binding energy in the nucleus, which is ~10E39 times greater. Such numbers tend to be meaningless, even to the well educated!

Frog dissections, operations on humans, flying schools and tractor driving lessons are all now available in virtual environments. Some of these are on line, some on disc and some in specialised installations. All of them allow us to more in less time, but most importantly gain experience in a totally safe environment before we have top do the real thing!

Data worlds
A growing difficulty for many of us is the explosion in the amount of information available and the format in which it is presented. It is not uncommon for today's executives to receive a 3cm thick report only hours before a meeting. They are then expected to have done all the necessary reading and be in a position to make an educated set of decisions in almost zero time. This is fundamentally impossible and an irrational expectation. An alternative approach is to put information into a visual and acoustic form that recognises our inherent ability for correlation, pattern and event recognition.

There is a progressively greater human ability when moving from monochromatic, monocular, mono-audio 2D space, through to a full colour 3D surround sound animated environment. Experiments have confirmed our innate ability to digest and understand up to 2Gbits of information in less than half a minute. This same information set in paper form would represent more than one volume of the Encyclopaedia Britannica as raw numbers and several inches of paper in terms of 2D graphs. Our ability to visually correlate, and recognise patterns, can be enhanced further by allowing observers and participants to move about inside the data field to view the information from new and very different perspectives. The remaining challenge is one of defining a reasonably common set of formats for regular data encounters, whilst finding a low cost solution to the visualisation of raw data sets and the technology interface.

Software visualisation
In a recent R&D programme the thorny problem of understanding a large software programme of 1.6M lines of code was tackled. By mapping areas of activity on a plane (Fig 4) it became obvious that vast tracts of code were seldom if ever used. A combination of colour and height were used to indicate activity, connection and number of operations/intensity. Ideally, this should have also been animated and in real time as similar situations have shown the value of strobing and synchronised activity recognition. But the sheer complexity of this case precluded this as a near term possibility. Non the less, the results achieved were impressive, and totally changed our view of this regime.

The suspicion was that successive teams of software writers had invoked the `if it works, don't touch it' maxim beloved of the industry. In a second mapping (Fig 5), the amount of activity was represented as rotating coloured 3D spheres, connected by pipes of interaction. This revealed for the first time, and has subsequently been confirmed, that large software suites strongly resemble neural networks. So the problem seems to be that software and neural networks are strongly related. In fact, they may even be the same - both are inherently non-linear, and lack any general theory. A powerful result through simple visualisation!

Unfortunately, neither result advanced our total understanding very far! The reality is no one understands neural networks either - yet! However, to have the two domains linked in this way is perhaps a useful step on the road to a full understanding. As an investigatory tool the technique does allow the identification of superfluous lines and blocks of code that can be stripped out. Fundamentally however, we ought to be writing tight code in the first place - but without a full picture, and understanding, this remains impossible for the present.

On a simpler note my introduction to Maxwell's equations as a student, and my entire understanding, was dictated by the limited artistry of a mathematician. This involved the coloured chalks and sketches representing 3D and 4D conditions on a 2D blackboard! Today this can be animated computer screen the difference is stunning - but would be no more than planar on this passive paper! With such tools understanding is achieved far faster, and more effectively. The same is true of turbulent flow, transformations, chaos, catastrophe and much more that is conceptually difficult to grasp. A static picture in black and white, or indeed an equation, are insufficient for us to visualise the subtleties of interaction encountered. We can no more understand the underlying mathematical and physical processes than we could have predicted the big bang without a telescope!

Full interaction
Today we use the telephone to extend our ears over vast distances. The same is true for visual communication, with the television screen and camera extending the reach of the eye. Extending our sense of touch is also feasible! The time for information to travel from our fingertips to our brain (~10 ms) is of the same order as the time taken for a single photon to travel from the UK to the USA (~ 30 ms) on an optical fibre. Given that this delay is dominated by synaptic processing (~30 ms), and our innate ability to accommodate delays of 100ms or more (e.g. riding a bicycle), then an extended hand across the Atlantic is feasible. The "feelie gloves" and "prosthetic arms" currently being developed will soon allow the VR participant to reach out, and not only grab, but feel. They will then be able to react to the tensions and forces, the tactile qualities, size and viscosity of entities. This is also true of information. Early experiments with graphical data and emotional icons in this context have revealed a number of interesting advantages when linked to Artificial Intelligence (AI). Reaching out for information that reacts in a humanised way as being friendly or aggressive, defensive or nervous, allows participants to be steered through a decision making process with realistic cues abstracted from real life.

Screens
Breaking away from the convenience and universally accepted TV screen appears to be a major limitation. Whilst this window to other worlds has served us well since the 1930s, it is a very constraining portal. Attempts to head mount screens lead to eye strain and complaints about the weight and uncomfortable nature of the technology. I-MAX and 3D cinemas are appreciated for their novelty, but have not ventured beyond the theme park - so far - and seem unlikely in the home. Holgraphic projections are also possible, but prohibitively expensive to date.

For a true advance we need a fundamental breakthrough in display technology for public use, and not just a smaller and lighter/brighter replacement for the CRT. Perhaps an interactive I-MAX with real time computer images, as used by the military et al, economic holograms, or the total illumination of the retina by a laser projector. Until something of this nature arrives professional users will probably be stuck with VR and Multi-Media on a flat screen, that might just grow to living room wall size!

Split senses
In many walks of life human beings operate with sensors diversely focused on a number of activities. Perhaps the most common example is that of the pilot who will talk to ground control with the use of one ear whilst listening to his co-pilot in the other. At the same time he may be viewing activity on the runway and controlling the aircraft with his hands. A recent realisation has been that the split ear operation can be extended to the eye. It is possible for human beings to work in two visual worlds at once, the real, and the virtual.

A commercial manifestation of such a system has been named CamNet, where the operator utilises a conventional audio headset augmented with head mounted TV camera and television screen. It is therefore possible for expertise to be teleported from one part of the world to another. Specifically, this can be used for the maintenance of complex equipment, surgery, medical examinations, remote visits to new locations, loss adjusters for car repairs and many others. The principle is quite simple. By projecting the view from someone's head back to a site many thousands of miles away, it is possible to perceive the world of another. Instructions, schematics, fixed and moving images can then be transmitted back to the wearer, who can receive the latest information about the equipment or problem he is working on. Expertise on demand, anywhere, any time!

An obvious extension is to place two cameras at eye level on the wearer and at some distance provide a virtual reality headset. It is then possible for thousands of people to sit behind the eyes of the wearer of the cameras and enjoy a full binocular (3D) view. This surrogate head has interesting implications for the instruction surgeons, students and other forms of participation.

The information war
In the main we can see the general direction of hardware developments for the next 10 to 30 years. Since 1960 we have seen the electronic packing density double in micro-electronics. Similarly, our ability to transport information in telecommunications continues to double each year. So storage, processing and telecommunication costs can be expected to continue their exponential decline - effectively becoming free! To this extent we can consider the hardware war to be over (for the present). Software is also on a reasonably well defined trajectory. New developments may see some real understanding and engineering arrive within the decade with; artificial life, intelligence, genetic algorithms and evolutionary principles creating a new era for software. Again, to this extent, the software war will soon be over. Software can thus be expected to go the same way as hardware, it will effectively become free, and will be written, configured and shared by everyone.

The next technology wave will be content. This war is just about to commence. In the near future, if not now, it will be imperative that we are able to access information anywhere, at any time and in any form. Information is all around us, it is abundant, it is diverse and rich. For example: there are 6M pictures of church widows alone; new books are published at the rate of 3.5km of book shelving per year; 500 television channels will soon be available in the USA; and video on demand could offer an instant choice from 8k programmes. How are we going to interface with such systems and stand any chance of being able to find our choice in a reasonable amount of time? Now one knows!

Multi-media interfaces & data
Looking at the potential complexity of the information world we are now building there is an interesting contest between the immersive and non-immersive systems. Flat screen multi-media is generally constrained to the standard search tree approach, whilst immersive VR offers the potential of the walk through. When coupled with AI techniques, this latter approach becomes formidable as it matches our intuitive behaviour of walking through a building, and natural inclinations to reach out and touch. It also goes some way to making the technology instantly available to the vast proportion of the population.

It is interesting to reflect that the forms of information we now enjoy were dictated by the means of storage and transmission devised before biblical times. We have become locked into a narrow range of realisation dictated by our past history and limited senses. VR and multi-media offer the prospect of entirely new forms of information and interaction. Because of the density of our visual cortex, and it's dominance as a means of entry to the brain, we might anticipate that vision will play the key role.

Copper mind sets
Systems and networks are still being designed as if the constraints of the copper past were still with us: compress the signal; save bandwidth; continue with the old protocols although they are now the new bottle-neck; use the established design rules despite the disappearance of the original technology! The wholesale reductions in the hardware content of networks and machines relative to the massive expansion of software is a prime example, as is the poor human interface suffered on many services.

Today fibre optic technology is removing the bandwidth bottle neck and distance related costs in telecommunications. Integrated electronics is making information processing and storage limitless and effectively free, whilst software has introduced the potential for unexplained system and network failures on a grand scale. What should we do, or be doing? There would appear to be only a small step required to create a better electronic world. We just need to start thinking and engineering systems and not hardware, software, networks and applications as if they were in some way disconnected. Unless we do, the multi-media future could be a frustratingly slow one!

New networks
The longer transmission distances afforded by optical fibre systems over copper and radio is predicating a reduction in the number of switching nodes and repeater stations. This realises improvements across a broad range of parameters including, reliability, power and raw material usage, capacity and utility. Today we only access < 0.01% of the fibre bandwidth available, and we might be able to approach 1% with currently available technologies. But further, everything we have seen optical fibre do so far may be totally eclipsed by moving from the linear to the non-linear regime. We already see experiments at >100Gbits over 100km of amplifying fibre - the switching function for Soliton ATM is also being built into fibre - further negating the need for electronics in the transmission path.

In complete contrast the computer industry, and some telecommunication operators, are moving in a direction of concatenating ever more random boxes of network electronics with layer upon layer of interfaces and protocols. We now see LANs, MANs & WANs, plus Internet, operating over telecommunications bearers layered with PSTN, ISDN, PDH, SDH, ATM, SMDS, FDDI, etc. A veritable alphabet soup of complexity, delay and potential unreliability. Rationalisation is needed - fast!!

Mobility
Whilst cellular radio technology is now well developed, and satellite mobile will no doubt follow, we suffer an apparent lack of spectrum. However, we have yet to exploit microwave frequencies up at 60, 90, 180GHz and higher. We also have the ability to take radio off air, up-covert it to the optical regime, feed it down fibre that is transparent through optical amplification, and get that signal to emerge in a distant cell undistorted. So, for multi-site working we could have access to the same facilities in all the buildings throughout an organisation, regardless of its geography/demography We would effectively be sharing the same work space, thereby giving all participants the illusion of being at the same location.

Another exciting prospect is optical wireless. The performance of free space optics in the home and office is very similar to microwaves, but with the advantage of a vastly greater bandwidth. Research systems are already providing pico-cellular illumination of the desk, personal computer, staff member, and offer the potential of active badges/communicators and inter-desk/computer links.

Software size and complexity
By any comparison of man's efforts, software is becoming a cause for significant concern in terms of sheer scale. For example; the complete works of William Shakespeare take up about 450m of paper; the line code for a small telephone switch is about 1km and a central office is in the 4-6 km range; network control centres are in the 6-10 km region; the Encyclopaedia Britannica is about 4.3km. A full stop in the wrong place and the spacecraft misses the planet! In the software domain very minor things pose a considerable risk, which appears to be growing exponentially as we look to the future. We have to find new ways of solving this increasing burden of risk as the present trajectory looks unsustainable in the long term. Quite perversely the unreliability of hardware is coming down rapidly. It is becoming more and more reliable while software is becoming more and more unreliable. From any perspective this growing imbalance needs to be addressed - otherwise we face a progressive slow down as software size outstrips hardware speed.

A selection of limitations & irritants

• Brittle software in robust hardware, systems, networks and machines
• Coding systems that distort the voice and grossly distort moving pictures
• Trying to communicate with visual images of people and things that are the wrong size and colour, and are distorted when they move
• Sterile teleconferencing environments that lack basic meeting room facilities
• Having to send discs and tapes via surface mail (Frisbee Net!) because it takes hours to send the data over the phone line
• Time delays in communication, control, computing and associated interfaces
• Internet - it is so slow and chaotic - a near 100% serendipity world
• A PC with a 125MHz clock and a PSTN only able to deliver 64kbit/s (or low multiples thereof) as a switched service
• A PC screen that displays only 75% of one page in an 'N' page document
• Inappropriate technologies surviving, and/or having their lives extended
• The growth of conservatism and short termism driven by competition
• E-Mail addresses and URLs that are meaningless and long enough to define the position of every atom in the known universe
• Fighting through cascaded windows to achieve elementary objectives
• Voice command on car phones, but not on office machinery and computers
• Poor acoustic representation/reproduction

Final thoughts
In some respects we are currently suffering from technological indigestion in telecommunications and computing, and it is likely to worsen as we approach the new millennium. We now have so much `ripe technology' that we are spoilt for choice. What is required is a new approach to IT that breaks the bonds with the past and reaches out for the new. Multi-Media and VR have received considerable media attention, and probably for the wrong reasons - but they do offer the prospect of a new paradigm. However, they need an injection of new technology to more fully extend our sensory system into the electronic world. Without advancement in this area our progress could stall do to our primary `wet ware' limitations. Visualisation is one of the keys to our future, but our progress in the creation of suitable screens, projections and interfaces is remarkably slow.

About the author
Peter Cochrane graduated from Trent Polytechnic with a BSc Hons in Electrical Engineering in 1973 and gained an MSc, PhD and DSc in telecommunications systems from the University of Essex in 1976, 1979 and 1993 respectively. He is a fellow of Royal Academy of Engineering, the IEE and IEEE, an honorary professor at the University of Kent, and a visiting professor to UCL and Essex Universities. He joined BT Laboratories in 1973 and has been a consultant to numerous international companies. In 1991 he was appointed to head the Systems Research Division which is concerned with future advanced media, computing and communications developments. During 1993 he was promoted to head the Research Department at BT laboratories with 660 staff dedicated to the study of future technologies, systems, networks, services and society.

Sites to visit
http://www.conceptlabs.net
http://wearables.www.media.mit.edu/projects/wearables/
http://www.mathsoft.com
http://bambi.ccs.fau.edu/ccs.html
http://wwwmaths.anu.edu.au
http://blanche.polytechnique.fr/lactamme/Mosaic/descripteurs/demo_14.html

Bibliography
MacDonald L & Vince J: Interacting with Virtual Environments, Wiley, Chichester, 1994
Earnshaw R A & Vince J A: Computer Graphics - Developments in Virtual Environments, Academic Press, London, 1995
Cochrane P: Desperate Race to Keep Up With Children, The Times Educational Supplement, 23rd June 1995, p 25
Milne R & Montgomery A: Proceedings of Expert Systems 94, British Computer Society, Information Press, Oxford, 1994
Cochrane P: The Virtual University. Business of Education, Issue 5, March 1995
Ravitch D: When School comes to you. The coming transformation of education, and its underside, The Economist, September 11th, 1995 pp 53-55
Pagels H R: The Dreams of Reason - The Computer and the Rise of the Sciences of Complexity, Bantam New Age Books, London, 1988
Kruger, M W: Artificial Reality II, Addison Wesley, New York, 1991
Cochrane, P et al: CAMNET, Interlink 2000, Aug 92
Ishii, H & Kobayashi, M: ClearBoard, CHI-92 Conference May 92
Mason, C: "Software problem cripples AT&T long-distance network", Telephony, 218/4, 22 Jan 90, p 10
Cochrane P & Westall F: It Would be Good to Talk, 2nd Language Eng Convention, QE II Conf Centre, Westminster, London, 16 - 18 Oct 1995.
Cochrane P, Heatley D J T, Pearson, I D: Who Cares?, British Telecom Engineering, Vol 14, Pt 3, October 95, pp 225 - 232.
Jewell et al: 3-D Imaging Systems for Video Comms Applics. Proc.SPIE. Vol 2409. Stereo Displays and VR Systems II, San Jose CA. Feb 5-10, 95
Cochrane P & Heatley D J T H: Modelling Future Telecommunication Systems, Chapman Hall, London, September 1995.
Cochrane P: A Three Click One Second World, IEE Electronics & Communication Engineering Journal, Vol 7/4, August 95, pp 138 -139
O'brien T M, et al: Three Dimensional Remote Imaging of Surgery, Jnl of Telemedicine and Telecare, Vol 1, 1995.