Expert System Design Shells: A Critical Analysis
Susan M. Land
Over the past five years or so, articles about artificial intelligence have filled our education journals with the hopes of simulating human, "expert" decision processes. The success of such expert systems could potentially put the expert of our choice literally at our fingertips. These types of computer programs have been used by doctors to help diagnose blood diseases and by accountants to determine people's taxes (Knox-Quinn, 1988). Sener (1991) explains that expert systems programming allows symbolic representation of articles and heuristic knowledge that model the way people solve problems in real life. He states further that, in the past, expert system applications have mostly remained in the domain of those highly skilled in programming. But now, development of what are c alled rule-based shells have put the capabilities of the expert system technology in the reach of all interested in using them.
Expert system shells provide methods of building expert systems without extensive knowledge of programming through mechanisms that (1) input the decisions, questions and rules that are followed (2) construct a knowledge database that can be manipulated by subsequent parts of the system (3) verifies possible violations of surface validity and (4) operates the "inference engine" that operates on the rules, poses the questions to the users, and determines whether a particular decision is valid (Trollip and Lippert, 1987). Most expert systems also allow the user to halt the processing at any time to query the system why a question was asked, or how a decision was reached (Trollip and Lippert, 1987).
Most expert system shells can now run easily on most current micro-computers and are able to handle the manipulation of a relatively large number of rules and associated questions.
Expert system shells a re expert system development tools consisting essentially of the expert system without the knowledge base, embodying the inference engine, working memory, and the user interface (Sener, 1991). An example of the inference engine part of an expert system that deduces new conclusions from known facts is illustrated below (Sener, 1991):
IF liquid limit=known
Expert systems give advice or solve problems by drawing upon this knowledge stored in the IF/THEN rules.
Several problems in education today can be alleviated through the use of expert system design shells. Two of these problems discussed in this paper fall into two rather divergent categories: (1) inefficiencies in current CAI development and (2) under-development of thinking and problem solving skills in the classrooms. In reference to the first problem, O'Connor (1991) expounds on this issue in eloquent detail:
Growth of microcomputer technology in the 1980's has provided teachers with opportunities to develop effective self-study materials for his or her students. However, most teachers do not make effective use of computer-based instruction for three major reasons: It is too difficult,too time-consuming, and too expensive. Accordingly, most developmental efforts of CAI occur in large industries and government agencies that can afford to devote substantial resources to the design and development of computer-based instruction.
An appropriate analogy of this problem is the fact that photography existed in the United States for about fifty years without popular appeal or use. Photographs were relatively easy to take with a camera, but the "developing" process was too difficult, expensive, and time-consuming for most people. This old problem is not much different from the types of problems we now face in terms of CAI development.
O'Connor (1991) goes on to acknowledge that there are cur rently two possible ways for teachers to develop CAI: (1) independent use of an authoring system or general purpose programming language or (2) paid contractors. Neither approach is satisfactory for most teachers: authoring systems are expensive and time consuming to learn, and contracting for CAI programming is prohibitively expensive.
Nicolson and Scott (1986) expound further on this problem by addressing that the use of the computer as a teaching tool in today's classrooms is threatened by a failure of instructional designers to produce quality CAI programs in larger volumes. As designers, we realize that the production of good software is difficult, and the demand for quality software vastly exceeds its supply. This problem is complicated by the fact that life expectancies of quality software are short, given the rapid pace of hardware development (Nicolson &Scott, 1986). Additionally, various authorities report courseware development time ranging from 50-500 hour s of preparation to produce one hour of student CAI contact time at a terminal. (Chambers and Sprecher, 1984). Part of this large amount of production time is due to the team production strategies utilized in most software development endeavors. Most pieces of educational software utilize a content expert, instructional designer, and programmer in its design and development. These team production strategies are excellent for producing innovative high-quality software, but only in very small quantities (Nicolson & Scott, 1986). Authoring languages have helped to reduce the amount of programming knowledge required for software development, but still do not eliminate the need to learn programming skills completely.
Finally, Li and Merrill (1990) elaborate on this problem by stating that existing authoring systems allow an instructor to develop instructional materials with less effort than programming languages, but do not provide instructional design assistance and still require significant effort. Existing courseware is often cr iticized for its lack of adequate instructional design, for being difficult to integrate with existing curricula, and for being expensive to develop and maintain (Merrill, 1985).
In reference to the second problem, Trollip and Lippert (1987) describe that one of the basic problems for a teacher today is fostering productive thought capabilities in students. They state further that educational practice still tends to be more concerned with WHAT answers are given than with HOW they are produced. Instead of teaching students how to think, it often emphasizes teaching them what to think. Students often learn to solve problems without adopting the conceptualizations underlying them. New technology and the need to learn quickly and effectively require that learners become deliberately instrumental in their own learning. Today's instructional methods of emphasizing the training aspect rather than the learning aspect is not meeting the knowledge requirements of our rapidly expan ding technological society.
The purpose of this critical analysis is to examine the current literature on expert system shells and to glean an integrated, analytical perspective on the topic. The primary focus is in the application of expert systems shells to learning situations. Implicit within this technological topic are numerous pedagogical and psychological issues that hold tremendous impact for our field. This analysis will focus on using expert system shells to create knowledge databases as an instructional strategy to promote problem solving skills and hypothesis generation in students.
Critical Review and Analysis of Related Background Research/Theory
The background research reviewed for this paper will be presented in terms of the technological, psychological, and pedagogical perspectives for each of the two above-listed problems being addressed.
Available literature on the first problem (inefficiencies in current CAI development) clearly address the technological issues involved with expert system shells. Expert system shells provide high technological advantages while eliminating the programming component when creating small expert systems (fewer than 100 rules). By putting new rules into a knowledge base, the inference engine "shell" and question-asking programs can be re-used (Knox-Quinn, 1988). The release of these types of inexpensive expert system-building programs for IBM and Macintosh computers makes the technology usable by educators who have no programming experience. This technological advance can lead to important pedagogical and psychological perspectives now being included in the design of instruction.
For instance, Li and Merrill (1990) address the potential pedagogical influences that are now available as a result of expert systems shell technology. Li and Merrill present a transaction shell approach to courseware development. A transaction shell provides courseware design and development environment for generating instructions for a specific type of instructional objectives (Li and Merrill, 1990). This type of approach has the potential to decrease courseware development time, increases instructional software quality, and provides flexibility for integrating courseware with existing curriculum of teachers.
The transaction shell model consists of two systems: an authoring and delivery system. In order for the assumptions of the transaction shell to be effective, two strategies must be apparent: (1) being able to separate content from its underlying instructional design structure and (2) being able to separate content from the instructional strategies necessary to teach it. Basically, the knowledge structure itself is independent of the subject matter domain. The major pedagogical benefit of using the transaction shell approach to courseware development lies in the notion of creating flexible and reusable instructional software that is both instructional strategy reusable and content reusable. If the shells are designed using effective design prin ciples, presumably all instruction developed from these shells will also be built with effective design principles. The major flaw with this assumption lies in the difficulty of separating instructional strategies and content. It is questionnable whether basic design strategies can be transferred across different content within a learning domain; the nature of the content often drives the more interesting and creative design strategies.
The major criticism of the transaction shell approach is that it attempts to apply old instructional design principles to a new technological medium. Inflexible, externally-determined instructional approaches potentially limit the extent of learning that is possible on the part of the learner. The key for effecitve design of shells, then, lies in the ability of the designer to create an environment conducive to learner construction of understanding. In this regard, learners are provided with the capability to adapt the knowledge and feat ures provided by the system in order to meet their own learning needs.
With NewBook Editor (1991b), readers, in the act of reading, would create and eventually write another kind of small "book" that would be a record of their reading and their own decision-making about what is or is not important. The overall goals of NewBook as an instructional tool is to help develop the critical thinking and interpretive strategies of students in the Humanities.
The shell set up by NewBook could potentially allow a teacher to enter and modify instructional content with ease and without extensive software development. Conceivably then, any type of content could be put into the NewBook shell and be taught with the same underlying design strategies for critical thinking. This type of shell differs from Li and Merrills (1990) transaction shell because of the non-traditional design assumptions implicit within the NewBook Editor (1991b). A description of Warsaw 1939 (1991c) illustrates many of these assumptions.
Warsaw 1939 has three major levels that are developed from the CBT shell. Level 1 is a hypertext environment . This type of environment does not place inflexible design requirements on its developers. The hypertext linkages will depend on what the instructor deems important for a particular topic or skill. The shell, however, provides enough structure for developing this level of the courseware, so that a non-experienced user can develop an effectively-linked lesson. Level 2 is a role playing environment in which the reader is asked to make decisions about what s/he might do in that particular situation; Level 3 involves the development of an interactive essay through a linking of the student's writing with the hypertext environment. All of these levels put much of the learning responsibility on the student: The student is provided with loosely-structured opportunities to make decisions and see the results of his/her decisions.
The second problem (emphasizing problem solving and critical th inking skills in students) can be analyzed in a similar manner. The recent availability of expert system shells allows use of expert systems with students in a totally novel instructional purpose. These "user-friendly" shells allow novice computer users to structure their own expert systems by means of menu-driven and natural-language computer interfaces (Trollip and Lippert, 1987) and be used as an instructional strategy. Expert system shells appear to offer many benefits when used in the classroom as an instructional tool.
Knox-Quinn (1988) describes an educational use of expert system shells that involve students actually building an expert system. Knox-Quinn postulates that by having students build expert systems (using an expert system shell), several pedagogical benefits occur: students' internal hypotheses and intuitive ideas can be discovered, students decide the facts and rules that lead to logical conclusions and isolate factors that influence other factors, and students can externalize their ideas in their own lang uage. In this manner, students can spend most of their time analyzing the expertise they want to simulate rather than mastering commands. Most importantly, by using these shells to create expert systems, we can attempt to engage learners in the construction of their own conceptual "micro worlds" of a subject matter (Knox-Quinn, 1988).
Trollip and Lippert (1987) contend that educational practice still tends to be more concerned with what answers students provide than with how they are produced. They describe an instructional technique that utilizes expert system shells to enable students to exercise control over how they think in problem-solving situations by the construction of knowledge bases. They state that the knowledge bases can be implemented using a simple expert system shell as a way of testing the consistency of the characteristics and relationships depicted in the knowledge bases. They further state that the technological requirements are inexpensive and easy t o use.
In a study illustrating the use of these shells to promote problem solving, Trollip and Lippert (1987) asked students to construct simple knowledge bases on difficult topics as a means of forcing them to think deeply about the intrinsic relationships and characteristics of the topic. The knowledge bases were then implemented using a simple expert system shell as a way of testing their hypothesized relationships. The pedagogical purpose of this assignment was to help the students develop an understanding of the major factors underlying screen design. Each group had to identify its own experts and resources, create its own knowledge base, and implement it using the shell. The study's conclusions illustrated three main positive characteristics: (1) Peer interaction. In order to use the shell, the students had to debate on how the gathered information could be summarized into useful rules; (2) Dealing with ambiguity. The task forced the students to develop ways of extracting information from experts and incorporate any conflicting opinions into a knowledge base. The rules were not always "clear-cut." (3) Demonstration. The groups demonstrated their expert systems to the class. These demonstrations led to group discussions that proved to be insightful.
Trollip and Lippert (1987) concluded that this type of instructional strategy of using an expert system design shell could be easily implemented in today's classroom. This type of strategy is ideally suited for the teaching of complex concepts, procedures, or ideas, and it concentrates on teaching about the conditions and constraints surrounding a topic rather than just the isolated facts.
In a similar study, Wideman and Owston (1988), conducted a study that analyzed the development of knowledge bases by groups of seventh grade science students. They sought to have students use expert system shells to create a small expert system in order to have them develop thinking and problem-solving skills. They found that despite the relati ve complexity of their task, students were still able to complete the development of their knowledge bases and did so with enthusiasm. The students developed expert systems with the goal of classifying any type of living matter into one of a large number of exclusive categories, following standard biological classificatory criteria. ("Classification of Living Things"). The groups were able to complete tasks of greater cognitive complexity than is typically demanded of them in the curricula for their age level and were also enthusiastic about the new learning method.
Use of expert system shells as an instructional strategy for developing problem solving skills is an pedagogical improvement over previous attempts typified in programs like LOGO computer-based applications. Wideman and Owston identified two possible reasons for the LOGO shortcomings: (1) problem solving skills learned in one domain are not automatically transferred to another. Expertise in a new area usually requires considerable domain-specific knowledge (2) stud ents working with LOGO frequently fail to use any higher-level planning or debugging strategies and instead use trial and error programming (partly due to the students' need to focus on the details of syntax in order to get their programs to work). Expert system shells foster both of these issues.
Several other examples of the use of expert system shells as a pedagogical tool are prevalent throughout the literature. Sener (1991), describes a pilot involving the use of a knowledge-based expert system, developed using an expert system shell, for classification of soils for engineering purposes in the Soils Testing Lab in a Construction Technology curriculum. He found several benefits of using such a tool as a teaching aid as well as being a time saving expert advisor for students. Similarly, Welsh and Wilson (1987) describe the use of an expert system shell to represent job aids. These expert system job aids are developed with the intention of solving a particular perfor mance problem. Welsh and Wilson stress the underlying similarities between job aids in the form of flowcharts or decision tables to the types of rule systems set up for expert system "inference engines." Welsh and Wilson state further that more complex problems can be represented in the form of an expert system than in the form of a traditional job aid in several ways. First, an expert system can engage in dialogue with the user. In these situations, the user sees only the questions being asked, not the complex paths that are designed into the software. Second, expert system job aids can solve ambiguous problems with heuristic rules instead of typical algorithmic approaches. Third, expert system job aids have the capability of explaining choices, decisions, or procedures. The path for reviewing the rules of a particular decision can be provided to the student in order to examine the premises and decisions logic.
In conclusion, expert system shells are flexible technological innovations that obviously hold extremely interestin g and valuable potential for use in instructional systems. Some of the major issues addressed in these readings are discussed below.
Use as an Instructional Strategy:
There may be several potential educational benefits to be gained by students who develop small systems as class projects. It appears from the literature reviewed that using expert system shells for students to develop their own expert systems is an excellent strategy for promoting problem solving and higher reasoning skills (Knox-Quinn, 1988; Trollip & Lippert, 1987; Wideman & Owson, 1988; Sener, 1991; Thurber, Macy, & Pope, 1991a;). All findings suggested that these types of project activities forced students to employ rigorous, systematic thinking in order to succeed. However, these benefits do not come without tradeoffs. Several of these tradeoffs are addressed below.
The most frequently mentioned disadvantage of using expert system shells to develop transaction shells or to use as an instructional strategy was time. It is time consuming for students to construct an expert system (approximately 12-15 rules = 25-60 hours of work). A suggested alleviation to this problem involves less complex assignments and using the technique sparingly (e.g. only with important, rule-based concepts).
Wideman and Owsten (1988) cited that the complexity of developing these expert systems was high enough that students would often be unable to conceptually integrate the various levels of classification in a logical way. They also found several problems common to novices inherent in the task: lack of goal-related planning, a lack of interconnectedness of output, and a tendency to be influenced by the concrete content of problems and overlook their abstract structure. Without adults to assist in guidance, Wideman and Owsten state that the students would would have failed to make active searches for relationships amongst components of the problem. Students in Trollip and Lippert (1987) also stated that they thought the activity was too difficult for young students to accomplish. It could be concluded that proper up-front and continual guidance is necessary in order for young children to succeed at such a task. The issue of motivation was mentioned several times throughout all of the articles. The young children (and adults) were motivated to solve the problems. It is likely that young children would not be frustrated by the complexity as long as they were provided adequate guidance when necessary.
Expert system shells allow enough design flexibility to encourage the expression of creativity and initiative at various levels of complexity. Using expert system shells to create a new system requires the user to make explicit the classification schemes, multivariate reasoning, means-ends analysis, and production rules employed on the content domain to reach conclu sions (Wideman & Owston, 1988). Whereas LOGO projects typically focus on a limited range of mathematical concepts, expert systems projects can deal with any domain that can be subject to classification or analyzed for its component production rules (Wideman & Owston, 1988). Flexibility seems to be one key attribute that is recognized in all uses of expert system shells. Li and Merrill (1990) specify that their transaction shells can provide content flexibility and well as content flexibility.
Another frequently-stated issue involved with expert system shells is reusability. Reusability refers to whether the courseware can be used over and over. It has a direct effect on the cost effectiveness of a CAI application. A major benefit of using these reusable shells is that teachers can make slight modifications to the shells and use the same software over and over for the same students or for different students. Li and Merrill (1990) also discuss how shells can be content reusable and instructional strat egy reusable.
Implications for Learning Systems design
There is no question that expert system shells have implications for learning systems design. The question is where to begin talking about them. Trollip and Lippert (1987) talk of an instructional benefit of being able to attach reasons to both questions and rules to enable the user to ask why a particular question was asked or why a rule resulted in a given decision. These types of "production systems" provide a powerful model for human thought because they are discrete, simple and flexible (Trollip & Lippet, 1987). One of the major implications of this issue is that expert systems shells can communicate the underlying cognitive assumptions and rules involved in determining a problem solving solution. In this way, the student can learn about problem solving by pinpointing the questionable areas and seeing how "an expert" solved the problem.
A second implication involves the issue that a constructed k nowledge base in the form of decisions, questions, and rules in groups of learners is the result of a great deal of arguing, discussing, and compromising between members of the informal group. Although the final constructed knowledge base may seem to be over-simplistic in its form, it represents the synthesized knowledge of a number of now very knowledgable people. Although its use as a functioning expert system may, in fact, be limited by its simplicity, the students involved will obtain a better understanding of the critical elements involved in decisions concerning the topic.
The building of a knowledge based by students can be a powerful technique to assist students grasp the fundamental structures of difficult concepts and allows the funneling of diverse facts and opinions into a well-formulated knowledge base with sensible rules. Building knowledge bases with expert system shells also requires students to formulate good questions to ask of the expert;. Students also learn to distinguish between relevant and irrelevant info rmation.
As stated in the previous section, the notion of being able to reuse instructional strategies and/or content can open new doors for instructional designers. With applications such as NewBook and transaction shells, a bridge between teachers and designers can be established and new instructional software can be developed at much faster rates. Rule-based expert system shells make the power of the expert system technology and artificial intelligence readily available to teachers who may want to use them without getting involved with the complexity of programming or bogged down by the high costs of most expert systems.
Finally, the use of expert system shells as an instructional strategy bears resemblance to a vague conception of constructivism. Perkins (1991) states that constructivism involves the notion of the organism being "active" -- not just responding to stimuli but "engaging, grappling, and seeking to make sense of things." Learners in a constructi vist philosophy make tentative interpretations of experience, elaborate and test those interpretations, and work through their understanding in ways that they themselves construct. The issues discussed in this paper indeed bear resemblance to these basic tenets of constructivism. It's interesting to see that using expert system shells as instructional strategies complement the old adage -- the best way to learn something is to teach it.
A Final Note
As we incorporate technology into schools, we must not lose sight of what education is all about. How technology is incorporated and used may significantly impact what education becomes. Harrington (1991) has addressed the role of technology in education that have as one of their goals preparing teachers to prepare their students for a rapidly changing society. If we only prepare students to use technology without considering how it can be used to expand their reasoning, we are only focusing on training and not on learning. Using technology to help expand students' reas oning and problem solving skills can help us shift the focus of education from the mere teaching of skills to providing opportunities for growth in sociability, intelligence, creativity, and autonomy. Using technology in ways that tightly structure how and what students know will only help us to maintain the status quo. As designers of emerging educational technologies, we have the opportunity and the skills to potentially influence which choice will be implemented in the schools. Which one will we choose to influence?
Chambers, J. & Sprecher, (1984). Computer-assisted instruction: Current trends and critical issues. In Walker, D. and Hess, R. (Eds.), Instructional Software, Wadsworth.
Harrington, H. (1991). Normal styletechnology in teacher education: Technology and the education of teachers. Computers in the Schools, 8 (1), 49-57.
Knox-Quinn, C. (1988). A simple application and a powerful idea: Using expert syst em shells in the classroom. The Computing Teacher, 12 (1), 12-15.
Li, Z., & Merrill, M. D. (1990). Transaction shells: A new approach to courseware authoring. Journal of Research on Computing in Education, 23(1), 72-86.
Merrill, M. D. (1985). Where is the authoring in authoring systems? Journal of Computer-Based Instruction, 12, 90-96.
Nicolson, R. & Scott, P. (1986). Computers and education: The software production problem. British Journal of Educational Technology, 1(17), 26-34.
O'Connor, R. (1991). Facilitating CAI development via an authoring template. Computers in the Schools, 8(1), 249-251.
Perkins, D. (1991). Technology meets constructivism: Do they make a marriage? Educational Technology, May, 18-23.
Sener, E. (1991). Using microcomputers and an expert system shell for soil classification. Collegiate Microcomputer, 9 (1), 7-12.
Thurber, B., Macy, G., & Pope, J. (1991a). The Book, the computer and the humanities. Technological Horizons in Education Journal, Fall 1991, 57 -61.
Thurber, B., Macy G., & Pope, J. (1991b). The NewBook Editor [Computer program]. Chariot Software Group, San Diego, CA.
Thurber, B., Macy G., & Pope, J. (1991c). Warsaw 1939 [Computer program]. Chariot Software Group, San Diego, CA.
Trollip, S. & Lippert, R. (1987). Constructing knowledge bases: A promising instructional tool. Journal of Computer-based Instruction 14(2), 44-48.
Welsh, J. & Wilson, B. (1987). Expert system shells: Tools to aid human performance. Journal of Instructional Development, 10(2), 15-19.
Wideman, H. & Owston, R. (1988). Student development of an expert system: A case study. Journal of Computer-Based Instruction, 15 (3), 88-94.
This work has been published by Instructional Technology
Research Online and is available on the World-Wide Web at http://18.104.22.168/InTRO.html