12. Testing the Solution, using literature

12.1 Shneiderman

In 1980 Shneiderman published Software psychology. This is now an old book, but it was seminal. The very idea that psychologists had anything to offer was new at that time and the book has since been followed and superseded by many books. Nevertheless, it remains a classic in its field and can be regarded as setting the benchmark, so it is therefore worth analysing using the approach developed here.

This title suggests information on applied cognitive psychology; the contents can be seen, certainly in hindsight, to have determined how generations of programmers and interface designers thought about the humans they were supposed to be helping. One can test the Solution by comparing the contents and the structure of this book and the Solution of this book. See Annex 3 for a table of contents of Shneidermans book.

12.1.1 Testing the human component

There is little compatibility between the table of contents of Software psychology and the human component of the Solution presented here. There is only one chapter directly related to the human components of the Solution of this book. Chapter nine discusses Natural Language. However, this chapter does not discuss natural language of humans (and the implications for the interface) but the understanding of natural language by computer systems. Another indication for incompatibility is the concepts referred to in the index of Software psychology. The index includes 1400 items of which 1.6% refers in some way to human memory and 0.0% refers in any way to human perception. More or less the same figures apply to the successor volume, Designing the User Interface, that Shneiderman (1993) published thirteen years later, having 7500 index entries of which 0.2% refer to human memory and now 0.1% refer to human perception (See Annex 4 for table of contents). It can be rapidly concluded that there is a large discrepancy between the Solution presented here, in which human memory and human perception are main dimensions, and the thinking behind the books of Shneiderman, suggesting that human memory and human perception are rather irrelevant for interface designers.

Software psychology includes chapters on database query, manipulation languages and interactive interface issues. These are all system components and of the same type. However, these chapters reflect an interface technological approach that was rejected in Part I of this book.

For the system component a count was made for the frequency of the best-known visual element properties (form, luminance, colour, and flash). Of all 7500 index entries of Software psychology 0.1% referred to colour. No references were found for form, luminance and flashing. Again, more or less the same applies to Designing the User Interface the only difference is a 100% increase for colour, from 0.1% to 0.2%.As for the human component there is a considerable discrepancy between the Solution presented, focusing on properties of elements of interfaces and the system component of the books of Shneiderman focusing on interface technologies. One of the problems with such approaches that concentrate on the technologies can be seen from the fact that almost all the technologies have been superseded, leaving little of principle for an interface designer. The book had, in hindsight, been better titled Interface technologies for programmers.

12.2 A web-based usability handbook

Software psychology, discussed in the previous section, was published in 1980l. For other books published in that period more or less the same conclusion can be drawn (Shneiderman, 1987; Willemse & Lindijer 1988). How is the situation ten years later? A web-based handbook (Neerincx et al, 2001) has recently been developed for the European Space Agency. The handbook should assist the payload development teams in realizing usability and consistency based on experience with previous missions and recent cognitive engineering approaches. The handbook has been carefully designed in a scientific environment and is based on several cognitive studies and evaluations by psychologists (Neerincx et al., 2001). In that handbook the guidelines for communications include seven chapters: compatibility, consistency, memory, structure, integration, feedback and interaction load (See Annex 5 for more details). The web-based handbook and this book have several concepts in common. However, the way the concepts are related in a structure is different.

The handbook chapters are not of the same type. Some refer to properties of the system (e.g. compatibility and consistency), one refers to a human function (memory), one refers to a measurement of human performance (interaction load) and one is difficult to classify, referring directly to a process (feedback). Consequently, there is no clear distinction between man and system and applying the test on normality and ordinality to the handbook, will result in its failing, or at least being hard to use and a source of complaints.

There seems to be little consistency between the structure of that handbook and the structure suggested in this book. Comparing the two structures the following questions arise.
The handbook and the book both have memory as one of the main components. Why does the handbook not have motion, perception and thinking as main component as well? Astronauts are not sightless, handless and brainless.

For Memory the handbook recommends to Minimize the amount of information that the user must maintain in working memory. This book also includes the property of number for memory. So far there is no difference. However, in this book, on the same level as number there are also the memory properties distance and structure. The handbook seems to apply, a property approach, but why are the properties for memory different for the handbook and the book? The property structure in the book includes compatibility and consistency as type of structures which in the handbook are main components located on the same level as human functions. The property structure is distinguished in the book, but not in the handbook.

In this book it is concluded that a structure should be minimally of the nominal type. The structure the book presents is as far as possible of the nominal type. Does this requirement not apply for the structure of a usability handbook?

12.3 A foundation of knowledge

Another foundation of knowledge was published by Karwowski (2001,p.xxv). (See Annex 6 for table of contents).
In Part 1 of that Encyclopaedia of Ergonomics and Human Factors includes General Ergonomics. In that chapter the element Task Analysis is followed by the element Taiwan: Ergonomics Society of Taiwan. This foundation of knowledge paradoxically fails the test of having the minimal requirements for fundamental knowledge, no elements of the same type and no elements being nominal.

The test can also be performed using from Part 2 Human Characteristics. This part starts with Alternative controls and ends with Workload and electro-encephalography dynamics. Somewhere in between information processing is preceded by human muscles and followed by lifting strategies.

All these elements refer to human processes and consequently, are of the same type. The structure for the organization of these processes is alphabetical, an ordinal one. However, this only applies to the formal structure and is not conceptual as one could expect in a foundation of knowledge. The underlying structure for the alphabetic structure is the list of authors. Part 2 of the foundation of knowledge, focusing on the same content as this book, fails the test.

13 Testing the Solution, using the Solution

The Problem this book investigates is how to find fundamental (psychological) concepts for interface design and then how to arrange these concepts in a way that allows both an elementary approach and a holistic approach. The fundamental concepts should be generalizable i.e. time, domain and interface technology independent. At the same time the fundamental concepts should allow reliable predictions on human performance of any time, any interface technology and any domain, …. including applied cognitive psychology itself. Applying applied cognitive psychology to applied cognitive psychology, for cognitive psychologists, looks like pulling a rabbit out of a hat. This might be true and useful as discussed in section 14.1. The scientific label for that magic is recursive knowledge and involves a bootstrap procedure. The model can be applied to the model itself for the following reasons.

Train passengers try to find the train, appropriate for their trip, in a cognitive structure (a timetable). Designers and (applied) cognitive psychologists try to find an answer for their design problems in a cognitive structure as well, i.e. the structure of their knowledge about cognitive psychology or, alternatively, a book or other reference work on cognitive psychology. The task for both the train passenger and the (applied) psychologist is to find useful information in a cognitive structure. Both have to be sure that the system they use will bring them to the information they need. The structure of both, a timetable and the available cognitive knowledge is complex, having many concepts, structures and relations between structures.

The search process is, in both such cases, a process of traversing the possible pathways in efficient ways (Luger & Stubblefield, 1998l; Winston, 1992) in order to arrive at an effective solution to the Problem being posed. The conclusion in chapter 11 was that cognitive structure is a fundamental cognitive property, no restrictions were made and the conclusion proved to be generalizable to several domains and interface technologies. In addition, it is unlikely that humans have special physiological structures for timetable information and for cognitive knowledge. Therefore, it makes sense for cognitive psychologists to deal with cognitive psychology in much the same way as they do with train, boat or even plane timetables.

The system, the task for the user and the similarities between finding a train and finding cognitive psychological information suggests that testing the Solution using the Solution is not a Von Munchhausens method, but rather a matter of generalizability towards several domains reducing the need to perform empirical studies for each and every question and a matter of recursively being a positive characteristic of a theory. As such, it makes sense to test the Solution using the Solution.

When Baron Von Munchhausen was drowning in a swamp, he saved himself by grasping his own hair and thereby pulling himself out of the morass. This is the origin of the scientific bootstrap procedures.

In Chapter 11 it was concluded that cognitive structure is a fundamental property of a field with cognitive elements. After an experimental comparison of two types of structures, it was concluded that there is an effect on human performance of the way elements were structured in a cognitive field. There are several structures for cognitive structure.

13.1 Is the human component nominal?

The human component of the Solution is subdivided into a motor, a perceptual and three mental sub components. All of these elements are human function and consequently, of the same type.

Having elements of the same type, then the next question is: are the elements mutually exclusive (nominal)? The motor, the perceptual and the mental sub-components are mutually exclusive on a biological level. Fortunately over time and over place this distinction is rather stable. Most experimental psychological libraries have separate sections for the five functions of the human component. Apparently these five functions are the best psychology can offer, they are certainly a very stable set.

Within the mental component it is more difficult to answer such a question. The Whorfian discussion has shown that it is difficult to separate language and thinking. In Bests (1996) book Cognitive Psychology, as in many other texts, perception is a part of cognition. The human brain is a network, and the existence of multiple connections between such mental functions as language, memory and thinking obscures which one or more of those functions will have been used to arrive at a specific result.

13.3 Is the system component nominal?

The system component of the Solution consists of the properties of elements and fields. They all are properties and consequently, by definition of the same type. The visual element properties (size, luminance, colour and flash) are defined by different biological and physical structures and processes. Consequently, there is little freedom in defining them and the designer of a model for applied cognitive psychology only can discover them. Fortunately, as with many fundamental physical concepts, the visual element properties are nominal and independent. Having chosen visual form one still can choose any luminance, colour or flashing.

There can be some discussion on the element property form. It is difficult to define and to measure (Leeuwenberg 1971l, Uttal 1988). There might be confusion for instance between the properties visual form and visual structure. For the properties of elements the Solution does not provide new information. Usually that information is presented in a rather elementary way. This Solution provides a holistic structure for the element properties. On the level of the definition and elaboration of the field properties there is more diversity in applied psychology and more freedom for the designer of a model for applied cognitive psychology, who can design the concept. The field properties of the Solution in this book, as such are not new. Every field concept can be related to several concepts in use by applied psychologists. New in the Solution is the arrangement of the field properties within one human function and the compatibility of the field properties for all human functions. For all human functions there is a field property of number, distance and structure. There is also compatibility with the structure of the element properties. As discussed in Part I many concepts that have been constructed by applied psychologists can be positioned somewhere in the Solution. Because of these characteristics the Solution is a holistic one.

This book uses the concepts number, distance and structure. Are these concepts nominal? Number does not imply any distance. Number also does not imply any structure other than being the same. The same applies for distance at one hand and number and structure at the other hand. Structure, again, does not imply any number or distance. One can design a structure without having one element in it. Number and distance can easily be defined physically. There might be some difficulty with the field property structure. However, when structure is defined as a repetition of any other property, a physical definition is possible when the other property can be measure physically. Fortunately most of the properties can be used to structure elements. In Chapter 8 visual size was used to structure train destinations in groups of destinations having the same first letter.

13.4 Is the system component ordinal?

The Solution suggests that human performance is better when components of a structure are arranged in an ordered or ordinal way than using just a nominal arrangement of components. Assuming that the field properties are mutually exclusive, the next question is: Are they ordinal? Is ordinal lining up possible within a property of an element, between element properties, and between field properties?

The properties of elements can be put in an ordinal scale, for instance, using human performance. For motor and psychophysiological visual properties of elements this is rather obvious and has been elaborated in detail by psychologists working in the field of ergonomics. The relation between human performance and a position on the scale of visual size, luminance, colour and blinking has been well established.
The list of properties itself is an ordinal scale as well. In a visual field, the property form is far less effective for a vigilance task than the property flashing. At the other hand for identification in a complex system, human performance is better using form than using flashing.

There is a quantitative difference in complexity between the field properties. For number one only needs a definition of a unit to be able to count. For distance one needs the position for the element and the position for the user. For a structure one needs a number of elements organized using one or more properties. Such a complexity metric enables us to predict the order used to define ordinality independently.

13.5 The structure of the system components

This book focuses on the specification of components for applied cognitive psychology and not on how to arrange the human, system and task. However, a speculative preview of how these system components of the Solution can be arranged has its value for estimating the potential of the Solution.

There are three independent components in the Solution; man, properties of the system and the task. How is it possible to create a synthesis between these three quite different components? There are several options available to us.
A common way to organize information today is using a hierarchy. This method of organizing information is common for information presented in books and in computers. However, in those cases the organization of the knowledge is imposed by technology and not by the content or applied psychological principles. Designers writing books on how to structure applied cognitive information might select a hierarchical model, not because that is the best fit for the information but because that is the best fit for the paper technology used.

All users of computer menus will confirm that organizing information in this way makes users get lost. One of the advantages of the World Wide Web has been to show people that there are other, non-hierarchical ways of organizing knowledge. The WWW shows that the hierarchical approach, no matter how attractive to technology, is far too inflexible as a structure for an interface of a complex system.

Navigating in structures with ordinal orthogonal dimensions is easier than navigating in systems not having orthogonal dimensions and not having dimensions that are either ordinal or nominal, as was argued in Chapter 11.

The earlier mentioned Mendeleyev Periodic Table for chemical elements does not have a hierarchical structure but an orthogonal structure having nominal and ordinal dimensions.
All sailors and pilots, using the Global Position System to establish their position on Earth will confirm this argument.
Any tourist who has roamed in an old European town and has also found themselves in the ordinally and orthogonally organized streets in Manhattan will confirm this as well.

The structure of the Solution presented in this book might seem hierarchical because the paper technology used to present the Solution imposes a hierarchical structure. However, a more dimensional orthogonal structure is a more appropriate one. In Part II the man and the system components are presented in a table; a two dimensional orthogonal structure. The properties in the cells of this table each have values forming a dimension of quantifiable human performance. The third main component, the task, briefly discussed in Chapter 6, can be interpreted as a third independent and orthogonal dimension. These characteristics of the Solution suggest that the Solution has a potential for accurate navigation in a complex cognitive domain. Multidimensionality in interface design and interface design models has been elaborated in Verhoef (2009).

14 Epilogue

The main problem was, in my opinion, not engineering good designs and giving them a theoretical basis. This was accomplished for four properties (visual size, visual distance, cognitive quantity and cognitive structure). It turned out easy to match practical problems to theoretical topics (1). Experimental results are as predicted (2) and in most cases substantial (3). This work is fundamental and practical at the same time. There is a theory (4) and the variables investigated are fundamental (5), the designs are everyday interfaces and the experiments were in real live (6) or close to that. Now, armed with more than 20 years hindsight, an evaluation can be made of the designs based on this book. The interface for ETCS started in 1993 and is now the European standard enabling drivers from Lapland to drive with 500 km/h to Sicily with little education and regardless of all sophisticated satellite-based or pre-Word War II mechanical safety systems he will meet. It is now implemented for the Betuwe line; a low speed freight line. Twenty years later the dynamic pallet based train indicators are replaced by dynamic screens. In Schiphol the indicator does not present trains but destinations alphabetically as suggested by research reported in this book.

Now, 20 years after the first psychological evaluation (Verhoef, 1986) and two generations ticket vending machines later, there is hindsight to evaluate the approach started in 1986 and presented in this book in 2007. Every ticket vending machine generation was based on the approach presented in this book and every change carefully elaborated by cognitive psychologists familiar with that approach. Whereas in Belgium one can easily find train ticket vending machines that not have been used for several days, NS touch screen ticket vending machines sell 130 000 tickets per day. These tickets can be for passengers, children, dogs and bikes, they can be single, return, whole day valid, departing from another station, having a detour or circle wise, and, of course the usual parameters such as class, reduction, no date, the destination can be a station or a concert, inland or abroad. Combination of parameters and combination of tickets is possible. Most problems reported are with payment. By law these interfaces are the domain of banks and Netherlands Railways is not allowed to touch them. The basic design was tested with subjects ninety years of age and the tests stopped when these subjects did not make any errors using the interface (Verhoef, 2000). All prerequisites for these successes were present: good management, good technology, good design and … cognitive psychology was there and listen to.

In the three designs mentioned (ticket vending, train indicator and train control) the knowledge obtained, later proved to be generalizable (7) to interfaces using technologies unforeseen at the time of the experiments. In addition, in all cases the knowledge obtained proved to be generalizable (8) to other domains. The knowledge obtained on cognitive structure of train indicators could even be applied to a domain rather far from passengers catching a train: applied cognitive psychologists itself (9). This last application was not foreseen at the time the experiments were carried out. Although the designs suggested by the theory, in all cases were rather uncommon at that time and the comments today are that it is all common sense. An overview of common design practice at that time is neglected (10).

These ten validation indicators are usually not present at the same time in scientific and practical work of this kind. They show that a scientific approach to interface design is no problem. Solving the interface problem is no problem as well. It is not the validity nor the theory that is commented when the theory, the designs and the results are presented. The comments are: Why change?, Solve the problem using new technology and We already know. These comments do not solve the problem: What is closer to the truth: current practice and theory or practice and theory suggested in this book.

14.1 Conservative domain

Information technology has an advanced, high tech, fast, innovative and creative image. This book, and other work, not mentioned in this book, suggests that these properties of information technology do not apply to the interfaces used to operate information technology. Today's human interface technology is rather conservative (Norman, 1998, p.276).

Graphical User Interfaces, which are common practice today, were developed in the sixties of the previous century. The desktop has been in use for more than one decade. The only thing that has changed is that colours, depth and animations have been introduced. The cognitive structure of the desktop is still undefined.

Although, menus are very user-unfriendly many do not know any better way of communicating with computers. The hierarchical structure and the fixed number of options is an efficient way for technology to present its possibilities. The hierarchical structures are rather random, and as discussed in this book, will lead to problems with navigation. The options are labeled using one word selected by the programmer. Not having at least a verb and a noun and not anticipating the use of synonyms will increase problems with navigation.

14.2 Why bother, use new technology

A common comment made by audiences supposed to be professional is: Why so much ado about a train ticket vending machine? Why not just have voice input, that is just like today's practice at ticket windows? We would never make that comment when this kind of information is presented. There are some practical reasons.

As anyone can establish it is very difficult to control auditory information in a station hall. Humans, who are better able to interpret announcements than machines, still have difficulty in interpreting auditory announcements in stations that are presented with context and are rehearsed. Human ticket clerks often require customers to repeat what they have just said. For such reasons voice input might fail.

How should we prevent the machine from taking as a destination that is mentioned by a passenger operating a machine adjacent to the current one or a destination mentioned in an auditory announcement?

A new technology, e.g. a machine instead of a ticket window clerk, should not use the old interface, e.g. voice input. Examples taken from history show that copying the old interface for a new technology is a design error. Driving a car sitting in a saddle and steering with reins is not appropriate, as we all know now.

Another example is the desktop interface to command computers. It is rather obvious that a model with physical objects positioned according to anthropometric rules is not appropriate for interfacing with the abstract objects humans are communicating with (Verhoef, 2001a).

Solving a problem by changing technology is walking away from the problem and might provide an alibi for unprofessional professionals. Changing technology is legitimate when there is no solution within the interface technological options given. In the case of the ticket vending machine it has been demonstrated that a good, cheap, user-friendly solution is possible.

There are some psychological reasons.

Humans are built to perform functions in non sequential and parallel ways. Voice interface technology imposes a sequential procedure and does not allow parallel processes unless you are performing another task at the same time, such as buying your ticket using a mobile phone while searching the indicator information for the platform. In the touch screen ticket vending machine presented in this book there are several examples of non-sequential and parallel options. In the last phase of pressing the button with the finger, the passenger can visually inspect the next step. The passenger has an overview of all steps taken.

In most cases, as in this book, the problem is not in the input but in the knowledge to be presented. You have to understand the fare system, the ticket types and why there is a difference between tickets with and without a date. Talking to a machine instead of pressing buttons does not solve any of these problems. As such, the voice input comment is at least out of focus until more fundamental issues have been confronted.

But, even if one should decide on using voice input, the information is relevant. The information obtained by using non-voice input vending machines predicts routing problems. When using voice input for ticket vending machine users will skip steps, as they did on the non-voice input machine, but probably even more so when offered the opportunity to talk. They will forget a step, assume the listener already knows what they want, or they will not expect the information is relevant and then will ask for options that are not available, being constrained by their own knowledge and not at all by the interface. This was observed for information on class, fare type and number of tickets. It remains difficult to solve such problems just by using a voice interface.

14.3 The results and conclusions are obvious and not new

The rules of experimental psychology were, as far as possible, followed carefully in this book. Consequently, there should be no discussion on the conclusions. Presenting this knowledge and designs consistent with this knowledge, the comment often is that the knowledge and designs are obvious.

This comment is inconsistent with the power, not reported in this book, necessary to implement the interfaces. The comment is inconsistent as well with the fact that, in almost all cases, interfaces consistent with the Solution were the uncommon ones.

The lists of the opposite, common but ineffective designs being inconsistent with the Solution in all cases rather long. That was twenty years ago and today (2009).

14.4 The truth

In this book we have tried to find the fundamental variables for applied cognitive psychological knowledge. A methodological question for applied cognitive psychology is: what are the fundamental variables for establishing the truth?

14.4.1 Finding the truth with fora

Doing experiments is one method for finding applied psychological truth suggested by the father of Dutch experimental psychological methodology (De Groot, 1961). He also suggested evaluating new knowledge using the subjective opinions of colleague scientists. In fact that is the methodology of humans and consequently of science. Informal relationships determine positions, publication and funding. This is a rather dangerous method today. It is compatible with the suggestion threatening to emerge in Chapter 12 that structures of science did not reflect the structure of the content but of the social network of the scientists.

In this informal methodology, knowledge that is possessed by one person is called magic. When several persons have that same knowledge it is called science. When finally that knowledge has become available to every body, that same knowledge is called common sense. The knowledge is the same but the evaluation of the messenger is different. What makes knowledge change from magic to truth?

How should such a fundamental change take place when scientists approaching the borders too close, in order to shift borders, might be labeled as a magician at one border while at the other border their contribution is labeled as popular science? Having procedures for keeping scientists carefully between the two borders prevents them from shifting borders. Developing a mental model not being bounded by rules of the formal community, such a person will unconsciously cross borders and might find knowledge so badly needed in a science like applied cognitive psychology. History shows that in that foreigners save the world, people whose mental model of knowledge has no relationships with the informal social network of that science.

14.4.2 Finding the truth with experiments

Of course, one fundamental variable is the outcome of experiments. This method is the main method of experimental psychology and applied cognitive psychology. That is the reason this method has been used in this book. The problem is that one has to do the right experiments and it is a very weak stance to have no underlying reason for predicting what will and will not work.

The purely empirical consumer research approach is the refuge of those who can not solve problems by understanding. Experimental tests should test the understanding, not the solutions. In this book, reflection on practice was not passive observing as the reflective practitioner of Schön (1983).

He suggests at the end of his book … researchers and practitioners enter into modes of collaboration very different from the forms of exchange envisaged under the model of applied science. In this book science and practice actively engineered interfaces that were evaluated using traditional passive, observing methodology. Is that a valid methodology? There are more problems when evaluating engineered interfaces.

14.4.3 Finding the truth with history

Experimentation is one method for finding the truth, other methods are available as well. For instance the method of studying history. With hindsight, today we are better able to understand lightning and thunder than did our ancestors two millennia ago. Historical hindsight is easy to achieve.

Contemporary hindsight, however, is more difficult to achieve. Our grandchildren are probably better able to identify changes in obviousness, and will notice that today's evaluators of designs did not realize that striking and obvious conclusions look trivially true only with hindsight (Fischhoff, 1975; 1986; Hudson 2001). As time is a human construction, not having so many laws of gravity as usually is thought, it can be reconstructed. The same human mind that has constructed time for technological and administrative purposes can also travel in time. This capability can be used for science fiction. It also can be used as a method for finding truth.

Well-known examples of historical foresight can be found in Cianchi (no date) describing the inventions of Leonardo da Vinci. In this book empirical tested theories proved to be valid for unforeseen next generations interface technologies. Lets apply the method of historical foresight some steps further and travel to our grandchildren evaluating today's applied cognitive psychology in the same way as we evaluated the explanation of our ancestors for lightning and thunder.

Studying the education of today's psychologists they will establish that all psychologists were taught that knowledge should preferably have at least the level of nominal categories.

Studying today's designs, including those used in experiments of applied cognitive psychologists, they will find that this basic knowledge has not been applied. A lot of work of applied experimental psychologists has been done on menus having non-mutually exclusive lists of options. In the discussion of that work the concept of nominal categories is not even mentioned (see Chapter 11) and most of the results can be discarded.

Studying today's theories on applied cognitive psychology they will draw the same conclusion. As concluded in Chapter 12 that work does not meet this basic requirement.

Is applied cognitive psychology dead?

14.4.4 The truth found?

It is no wonder that applied cognitive psychology has lost all of its territory to technology and design. It is no wonder that many of its soldiers defect to technology, thinking they still are psychologists.

One can blame technology for the interface Problem as Norman (1990, 1998) does. One can blame management as Cooper does (1999).

So far the smoke screen of science has been effective in not blaming psychology for the interface Problem.

When time has blown away the smoke screen, I aim afraid our grandchildren may well blame today's psychology for designers not understanding their users.