Instructional Technology Research Online
 

Performance Support Systems for Education and Training:

Could This be the Next Generation?

Stephen Desrosiers

University of Hoston-Clear Lake

Stephen W. Harmon

Georgia State University


Were this industrial age approach working as it should, no adjustments to the system would be needed. But the growing call for reform (e.g. Banathy,1991; David, 1991; Morgan, 1994 ) clearly indicates that changes are inorder. One approach to answering this call is a collection of tools methodologies and information known as a Performance Support System (PSS).
 

The goal of a PSS is "to provide whatever is necessary to generateperformance and learning at the moment of need" (Gery, 1991, p. 34).Gery states that people have been provided with some of the help to accomplishthis goal with powerful tools such as job aids and CBT. However, these toolsare not a PSS by themselves, although they can be part of a PSS. "Thecommon denominator that differentiates an electronic performance supportsystem from other types of systems or interactive resources is the degreeto which it integrates information, tools, and methodology for the user" (Gery, 1991, p. 34). This paper will examine the definition of a PSS, review PSS design considerations, discuss advantages and drawbacksof using PSSs in education and training, and present examples of PSSs currently in use.

What is a Performance Support System

The integration of different tools to help the user perform a task is thekey feature of performance support systems and is part of most definitions.Laffey (1994) defined performance support systems as "systems [that]are built to integrate resources and tools and to facilitate working oncomplex tasks" (p. 1). Raybould (1990b) also includes the conc ept ofintegration in his definition: "An electronic Performance Support System(PSS) is a computer-based system that improves worker productivity by providingon-the-job access to integrated information, advice and learning experiences"(p. 4). McGraw (1994) defines a PSS as "an integrated tool suite thatsupports the user of a complex system by providing embedded assistance withinthe system itself" (p. 1).

In summary, a PSS is a system that provides the user with information, guidance,and learning experiences where ever and whenever a user needs it.

The Components of a Performance Support System

Most PSSs consists of four components: (a) an advisory component, (b) aninformation component, (c) a training component, and (d) the user interfacecomponent

Advisory Component

The advisory component is there to provide help whenever the user needsit. The advisory component usually consists o f a job aid in electronic form.The user does not have to have a deep understanding of the task to use theadvice. If the user desires a deeper understanding of the task, he can lookthe information up in the information component, or can get training inthe training component.

Information Component

The information component of a PSS is there to provide all the informationthe users require to do their job. The information component consists ofdata and the tools that the users can use to access the data. The data canbe procedures, regulations, company policies, specifications, digitizedphotographs, digitized movies, animation, digitized sounds, and drawings.There are many technologies that can be used by the information componentincluding data bases, hypertext, and on-line help systems.

Training Component

The training component is provided so that the user can get training ondemand. This component usually uses traditional CBT technology.

User Interface Component

Most authors agree that the single most important aspect of the PSS is theuser interface. It is the interface between components and the links tothe user which provide the employee with the tools to display information,advice, etc. Without appropriate integration, the system will not be ableto quickly or easily provide the required assistance to the person requestingsystem support (Milheim, 1992, p. 249).

The user interface must provide a seamless way to navigate from componentto component inside the PSS. It must present a consistent look and feelfor all the components in the PSS. It must be centered around the user because:"Ultimately the user will use or reject the system on the basis ofwhether it helps achieve success and whether it fits the way work is done."(Laffey, 1994, p. 11).

Designing Performance Support Systems

There are no published detailed systematic design guidelines for developinga PSS. Gery (1991) in her book Electronic Performance Support Systemsprovides the most detailed design guidelines. Courseware/Anderson Consulting(1990) presents a high level PSS model in a white paper. Several other authors(Carr, 1991; Milheim, 1992; Laffey, 1994; Raybould, 1990b; McGraw, 1994)also provide high level guidance. All these sources together only providethe questions the developers should ask themselves. There is no one placea developer can go to get the answers to these questions. As Gery wrote"This ride is not for the faint of heart or those who need absolutecertainty about what to do, how to do it, and what will happen before theybegin." (Gery, 1991, p. 284).

While help for designing the overall structure of a PSS is limited, thereis more specific help for the individual components of the PSS. This isbecause PSSs utilize many technologies that are usually used to build standalone, single purpose tools.

Advisory Component
The advisory component usually consists of o ne or more job aids. Carlisle& Coulter (1990), Knirk & Montague (1994), and Lawson (1986) giveexamples for the design of job aids . There is a common approach to designingjob aids. The process consists of: (a) identifying the task for which ajob aid will be developed, (b) determining what type of job aid best fitsthe desired performance, (c) designing and build the job aid, and (d) evaluatingthe job aid.

The first step in building a job aid is to do a task analysis of the jobthat is the target of the job aid. Information processing analysis (Gagne',1991) is one technique to do a task analysis. The purpose of the informationprocessing analysis is to determine the mental operations a person makeswhen performing a task. The analysis results in the mental operations beingdescribed in a flow chart format.

The second step involves an analysis of these mental operations. The firstquestion to answer is whether or not a job aid is an appropriate interv ention.A list of the types of tasks that are appropriate for job aids was presentedabove. After determining that a job aid is an appropriate intervention,the type of job aid is determined. If the task is a simple procedural taskwith no decisions, then a checklist type job aid might be warranted. Ifthe tasks asks the performer to make decisions based on data that is availableto the performer, then a decision tree, decision matrix, or expert systemmight be more appropriate.

The third step is to design and build the job aid. After the job aid isbuilt, it is important that the job aid be tested with real users. The jobaid should be modified to correct any deficiencies with the design.

Information Component
The information component can use any combination of databases, hypertext,and on-line help systems.

Databases.

Database systems are computer systems designed for the relatively permanentstorage and retrieval of data. Database systems usually provide methodsfor a dding, deleting, and changing the data in the database. Database systemsusually provide methods for querying the database to gather subsets of thedata. An introduction to database systems: Volume I (Date, 1990)is an example of a good relational database text book. There are many commerciallyavailable databases that can be used for the information component of aPSS.

Hypertext.

Hypertext is a non-linear way to go through information. With hypertextyou go through the information topic by topic, only viewing the informationthat is relevant to your purpose.

In a hypertext document, the information can be structured so that overviewinformation is encountered first. If the users wants more details, theycan get to the details by using the hyperlinks. Using this scheme allowsone audience to read the overview sections of a document, while a differentaudience can read the detailed information. Because of the hyperlinks, itis possibl e for both audiences to see only the information they desired.

Hypertext can reduce writing costs. Because of the modularity of hypertextdocuments, a piece of text only has to be written once. This piece can thenbe shared through multiple documents. Raybould (1990a) discusses the appropriateuses and benefits of hypertext in a PSS. Raybould states that hypertextis a good way to structure the information component, especially when theusers will have a need to go from one information topic to another.

Hypertext can also be used to build the advisory component. The developercan use hyperlinks to build the branching of decision trees (Raybould, 1990a).Using the same hypertext system to build the information component and theadvisory component is a good technique for insuring that the user interfacebe consistent throughout the system.

Raybould (1990a) lists several benefits of hypertext for use in a PSS: (a)users see only relevant information, (b) one document serves many audiences,(c) writing costs are lo wer, and (d) CBT development costs are lower.

On-line help systems.

On-line help systems should be part of every software application. Sellenand Nicol (1990) have identified five types of questions that should beanswered by an on-line help system: (a) goal-oriented, (b) descriptive,(c) procedural, (d) interpretive, and (e) navigational.

Goal-oriented questions are questions about the overall purpose of the application.The amount of detail required is dependent on the expected knowledge levelof the user. Usually, there are at least some novice users expected, therefore,this part of the help should be fairly detailed (Sellen & Nicol, 1990).

Descriptive help answers questions about specific user interface objects.It covers questions like "How does this work?" and "Whatis this for?" (Sellen & Nicol, 1990).

Procedural questions are questions that cannot be answered by descriptivehelp, becau se they involve manipulating a series of user interface objects.Sellen and Nicol (1990) state that this is the most popular kind of question,yet is the question least likely to be answered by on-line help.

Interpretive questions usually arise from errors. The user wants to knowwhy the operation didn't work. Help for these questions are usually providedby dialog boxes when an error occurs (Sellen & Nicol, 1990).

Navigation questions are very common in hypertext applications. Studentscan get lost in hypertext systems after following just a few hyperlinks.A typical solution to this problem is to include a map of the applicationwith some indication of where the student is with respect to the whole application.The map acts like a job aid to help the student navigate their way throughthe system (Sellen & Nicol, 1990).

Training Component
Puterbaugh (1990) claims that there are important differences between theway instruction should be built for traditional CBT lessons and the wayins truction should be built for use in PSS. Puterbaugh discusses some ofthe differences and gives a small list of guidelines for developing theinstruction for PSS.

Traditional CBT lessons.

Instructional designers frequently use task analysis (Gagne', 1991) to determinethe sections required to meet an instructional goal of the CBT lesson andthe desired sequencing of the sections in the CBT lesson. Many instructionaldesigners would then use the nine external events of instruction (Gagne',1985) as a template to sequence the presentation of information in a sectionof the CBT lesson. The external events are: (a) gain attention, (b) informthe learner of the objective of the lesson, (c) stimulate the recall ofprior learning, (d) present the stimuli, (e) guide learning, (f) elicitperformance, (g) provide informative feedback, (h) assess performance, and(i) enhance retention.

CBT systems are often set up to enforce the sequencing of section s (Puterbaugh,1990) determined by the task analysis. The enforcing of the sequencing relieson the system keeping student records. The system checks the student's recordto determine if the student has completed the required prerequisite sectionssatisfactorily before allowing the student to enter a new section of thelesson.

Instruction in PSSs.

Puterbaugh (1990) offers several suggestions for how instruction in PSSsshould be different from traditional CBT.

Enforcing the viewing of prerequisite knowledge should not be present inPSSs (Puterbaugh, 1990). In a PSS, it should be up to the students to determineif they need to review the prerequisites. Links to the prerequisites shouldbe provided, but viewing them should not be mandatory.

Student records should not be kept, if at all possible (Puterbaugh, 1990).One of the main purposes for keeping student records it to enforce the sequencingof instruction. Because the learner chooses their own sequence in PSSs,there is no need to keep student records for sequencing purposes.

Lesson segments should be as small as possible (Puterbaugh, 1990). In PSSs,the students usually enter the instructional component to learn a specifictask. The lesson segment should only cover that specific task. If the lessoncomponent covers more information, the student takes longer to completethe original task.

Lesson segments should be self contained (Puterbaugh, 1990). The instructionaldeveloper should try to limit the dependence on other lesson segments, becausethere is no guarantee that the student has viewed the other lesson segments.The instructional developer should try to minimize any undefined acronymsand backward reference.

Rely on the information in the other components of the PSS (Puterbaugh,1990). The instructional developer should not repeat information in theinformation component during the presentation of material. Instead, usea link to that information. Avoiding the duplication of in formation willgreatly reduce the cost of development and maintenance of the PSS.

Make it easy for the student to use the instructional component (Puterbaugh,1990). Don't make the student sign in to the instructional component, andallow the student to escape from the training at any point.

User Interface Component
The only PSS specific guideline for user interface design is to providea consistent user interface across all the other components of the PSS.This concept is critical to the success of PSSs. There are many sourcesof information on how to design a good user interface for any type of application.An important place to start is with the human interface guidelines for theoperating system that the PSS will run on. For Apple computers, the referenceis the Human Interface Guidelines: The Apple Desktop Interface(Apple Computer, Inc., 1987). Another consideration is if the developmentenvironment has a set of guidelines. For example, the HyperCardreg. developmentenvironment has a set of guidelines published in the HyperCard StackDesign Guidelines (Apple Computer, Inc., 1989). Marcus (1990), Mountford(1990), and Shneiderman (1987) give examples of user interface design guidancethat is independent of the operating system and development environment.

Advantages and Drawbacks of Performance Support Systems

Advantages

PSSs use a wide range of technologies in their components. It is the combinationsof these technologies that give the PSS advantages over any of the technologiesused as a standalone system. By integrating these technologies in a seamlessway, the result is that the whole is greater than the sum of its parts.The following paragraphs illustrate this concept using CBT and job aid technologies.

PSSs incorporate job aids and therefore have the same advantages as jobaids. PSSs also incorporate CBT lessons. The CBT lessons provide the mechanismfor the employees to gain an unde rstanding of the content of the job aids.The availability of the CBT lessons minimizes the limitations of the jobaids.

The CBT lessons are taken at the moment of need, at the employee's workstation. The lessons are typically very focused, using the smallest unitpossible. The timing, location, and scope of the training integrates thelearning experience with the job performance. This minimizes the lag betweenthe training event and the execution of the learned information, and therefore,minimizes learning degradation. However, there is some question as to whetherthe employee will gain a deep understanding of the content (Clark, 1992).This issue is discussed further in the next section.

The above paragraphs illustrate that CBT and job aids are a powerful combinationof technologies. PSSs provides the user a seamless interface to move betweenthe CBT lessons and the job aids. Without the user interface, an employeemight have to learn different systems to access CBT lessons and to accessthe job aids. PSSs reduce th e cognitive overhead of manipulating more thanone system.
 
 

Limitations

As happens with most new technologies, some people will promote PSSs asthe answer to all problems. Like most technologies, PSS has its limitations.Ruth Clark (1992) has been one of the few to offer criticisms of PSSs. Clarktakes issue with five assertions that Gery (1991) states. Each of Clark'sissues and an opposing view are presented below:

1. Learning should take place on the job in small increments. Clark(1992)suggests that learners need a framework on which to build their knowledge.By getting training in small chunks that are not explicitly tied to theframework, the learner will not develop the big picture. Clark also warnsthat users might not take the time to go through the training provided.Lastly, Clark warns that some jobs have response times that prohibit just-in-timetraining.

The issue of whether employees will learn the big pictur e by using a PSSis a major issue that needs to be addressed. Unfortunately, it is a questionthat is outside of the scope of this thesis. Getting employees to use thetraining is an organizational issue and therefore out of the scope of thisthesis.

2. Learners should be in control of their learning. Clark (1992) warns thatthe research has shown mixed results when evaluating learner controlledlearning. Learner control seems to work best when the learner has some experiencein the area. When the learners are novices, they seem to have more troublelearning with learner control when compared to instructional control.

3. Automated tools will automatically improve performance. Clark (1992)reminds us that there has been very little research on how automated toolseffect performance.

4. Skills will be lost. Clark (1992) points out that skills might be lostwith the heavy reliance on automated tools. How will the corporate knowledgebase increase if it is only encoded in automated tools?

A well designed PSS should include the training lessons to teach the employeesthe knowledge to understand the reason and rationale behind the automatedtools. If the PSS does not have this information, then the PSS was poorlydesigned or the responsible organization determined that this knowledgewas not required by people using the PSS.

5. Complex technology can fix problems with simpler technology. Clark (1992)argues that it is hard to justify the cost to develop traditional CBT. Howare people going to justify the cost of developing the much more complicatedPSS.

Examples of Performance Support Systems

There are few studies showing the effectiveness of PSSs. One reason forthis, is that most PSS development is taking place in industry. Industryis interested in performance improvement and not scientific study of thereasons that performance improved. Therefore, improvements in performancethat happen when PSSs are implemented cannot be scientifically attribute dto the PSSs. This section will present some of the PSSs being built by industryand the results of those PSSs.

Apple Computer has developed an PSS to inform its sales people about newproducts. The PSS is called Apple's Reference Performance and Learning Expert(ARPLE). No data on the impact of PSS is provided (Geber, 1994).

Northern Telecom developed a PSS to supplement a five day course for newmanagers. The PSS helps new managers draw up budgets. No data on the impactof PSS is provided (Geber, 1994).

Prime Computer, Inc. has developed an PSS to reduce information overloadfor its field sales organization. After the beta test, the results haveshown that the savings from distributing the information on CD-ROM insteadof paper will pay for the development of the PSS. However, no data on theperformance increase is available (Raybould, 1990b).

Steelcase has developed a PSS to support its customer service agents. ThePSS provides the customer service agents with the data they need to answerthe custo mer's questions. No data on the impact of PSS is provided (Laffey,1994).

Stone and Hutson, (1984) used a hypertext system to build a computer assistedjob aid for an assembly task procedure. While the system is not a true PSS(it lacked the instructional component) it did provide some insight intohow effective hypertext might be when used inside a PSS.

Stone and Hutson found that the performance of subjects on the assemblytask was much better using the hypertext system than using other systems.When using the hypertext system, 95% (all but one) of the subjects completedthe assemble procedure without error. An earlier study (using the same textand graphics) had three groups of students viewing a text and/or graphicalrepresentation of the assemble procedure on a slide projector. The firstgroup used only the textual representation and none of these students completedthe assembly procedure without error. The second group of students vieweda graphical represe ntation of the assembly procedure and only 9% of thestudents successfully completed the assembly procedure without error. Thethird group of students viewed both representations of the assembly procedureand only 25% of the students successfully completed the assembly procedurewithout error.

While these results seem to indicate strong trends, the study did not tryto test the effectiveness of the hypertext directly and therefore the resultscould be to external factors. However, the results indicate that furtherstudies should be conducted.

Summary

Gery (1991) theorized that a new paradigm (PSS) is required to get the performanceincrease necessary to remain competitive in international competition. PSSsare a set of integrated components that provide the user with information,guidance, and learning experiences wherever and whenever a user needs it.The key to this new paradigm is to provide a seamless user interface thatintegrates the components that compose the PSS. PSSs use a wide range oftechnolo gies in their components. It is the combination of these technologiesthat give the PSS advantages over any of the technologies used individually.In a PSS, the limitations of one technology are usually overcome by anothertechnology. The end result is that the whole is greater than the sum ofits parts.

While PSSs are being used in business, there are few studies that test theeffectiveness of PSSs. What information that is available on PSSs indicatesthat PSSs have the potential to significantly improve employees' performance.This potential performance improvement is the reason that PSSs should beevaluated and the actual improvement determined.

Performance support systems could be a significant benefit to the next generationof training and educational technology. The time is ripe for a paradigmshift. PSSs should be examined as a possible direction for that shift.
 

References and Bibliography

Apple Computer, Inc. (1987). Huma n interface guidelines: The appledesktop interface. Reading, MA: Addison-Wesley.

Apple Computer, Inc. (1989). HyperCard stack design guidelines.Reading, MA: Addison-Wesley.

Banathy, B.H. (1991). Comprehensive system design in education. EducationalTechnology, May. 54-59.

Bowsher, J. (1992). What can we learn from corporate education about systemic change? Educational Technology, 32(11), 51-54.

Carlisle, K. & Coulter, P. (1990). The performance technology of job aids. Educational-Technology, 30(5), 26-31.

Carr, C. (1991). PSS! Help when you need it. Training and Development,46</U>(6), 30-38.

Clark, R. (1989). Teaching the how without the why. CBT Directions,2(8), 8-9.

Clark, R. (1992). EPSS - Look before you leap: Some cautions about applicationsof electronic performance support systems. Performance and Instruction,31(5), 22-25.

Courseware/Anderson Consulting. (1990). An integrated performance supportsystem (IPPS): How can it help develop a competitive workforce in the `90s.San Diego, CA. (ERIC Document Reproduction Service No. ED 329 229)

Date, C. (1990). An introduction to database systems: Volume I(5th ed.). Reading, MA: Addison-Wesley.

David, J.L. (1991). Restructuring and technology: Partners in Change. PhiDelta Kappan, September. 37-40, 78-82.

Duncan, C. (1985). Job performance aids. Job aids can work: A study of themilitary application of job aid technology. Performance and Instruction,23(5), 1-4.

Gagne', R. (1985). The conditions of learning (4th ed.). NewYork: Holt, Rinehart, and Winston.

Gagne', R. (1991). Analysis of objectives. In L. Briggs, K. Gustafson, &M. Tillman (Eds.), Instructional design: Principles and applications(2nd ed.) (pp. 123 - 150). Englewood C liffs, NJ: Educational Technology.

Gagne', R. & Rojas, A. (1991). Planning and authoring computer-assistedinstruction lessons. In L. Briggs, K. Gustafson, & M. Tillman (Eds.),Instructional design: Principles and applications (2nd ed.)(pp. 211 - 226). Englewood Cliffs, NJ: Educational Technology.

Gayeski, D. (1993). Making sense of multimedia: Introduction to this volume.In D. Gayeski (Ed.), Multimedia for learning: Development, application,evaluation (pp. 3 - 14). Englewood Cliffs, NJ: Educational Technology.

Geber, B. (1994). Re-engineering the training department. Training,31(5), 27-34.

Gery, G. (1991). Electronic performance support systems. Boston,MA: Weingarten.

Heathman, D. & Kleiner, B. (1991). Training + technology: The futureis now. Training and Development, 45(9), 49-51.

Industry report 1994: An overview of employee training in America (1994).Training, 31 (10), 29-75.

Knirk, F. & Montague, W. (1994, February). What works in industrialand military training: A research summary on adult learning. Paperpresented at the 1994 National Convention of the Association for EducationalCommunications and Technology, Nashville, TN.

Kuhn, T.S. (1970). The structure of scientific revolutions. Chicago:University of Chicago Press.

Laffey, J. (1994). Dynamism in electronic performance support systems.Manuscript submitted for publication.

Lawson, P. (1986). Job Aids: Give readers what they want. British-Columbia,Canada. (ERIC Document Reproduction Service No. ED 273 746)

Marcus, A. (1990, August). Designing graphical user interfaces. UnixWorld, pp. 107-115.

McGraw, B. & McGraw K. (1993). The performance support system as a trainingsolution: Cost/benefit considerations. Journal of Instructional DeliverySystems, Fall, 7-15.

McGraw, K. (1994, March). The performance-based user interface: Amethodology for developing embedded performance support systems.Paper presented at the 2nd Annual Mid-Atlantic Human Factors and ErgonomicsSociety Conference, Reston, VA.

Milheim, W. (1992). Performance support systems: Guidelines for system designand integration. Canadian Journal of Educational Communication,21(3), 243-252.

Morgan, R.M. (1994). Educational reform: Top down or bottom-up?. In C.M.Reigeluth & R.J. Garfinkle (Eds.). Systemic change in education..Englewood Cliffs, NJ. Educational Technology Publications.

Mountford, S. (1990). Tools and techniques for creative design. In B. Laurel(Ed.), The art of human-computer interface design (pp. 17-30).Reading, MA: Addison-Wesley.

Puterbaugh, G. (1990). CBT and performance support. CBT Directions,3(6), 18-25.

Raybould, B. (1990a). Choosing the right hypertext product for per formancesupport. CBT Directions, 4(7), 13-20.

Raybould, B. (1990b). Solving human performance problems with computers.A case study: Building an electronic performance support system. Performanceand Instruction, 29(10), 4-14.

Reigeluth, C.M. & Garfinkle, R.J. (Eds.) (1994). Systemic changein education. Englewood Cliffs, NJ. Educational Technology Publications.

Sellen, A. & Nicol, A. (1990). Building user-centered on-line help.In B. Laurel (Ed.), The art of human-computer interface design(pp. 143-153). Reading, MA: Addison-Wesley.

Scales, G. & Yang, C. (1993). Perspectives on electronic performancesupport systems. Paper presented at the Annual Meeting of the EasternEducational Research Association, Clearwater Beach, FL, February. (ERICDocument Reproduction Service No. ED 354 883)

Shneiderman, B. (1987). Designing the user interface: Strategies foreffective human-computer interaction. Reading, MA: Addison-Wesley.

Stone, D. & Hutson, B. (1984). Computer-based job aiding: Problem solvingat work. (Technical Report No. 11). Ithaca, NY: Cornell University, Dept.of Education. (ERIC Document Reproduction Service No. ED 244 613)


Research Online | Features | Links | Information
Instructional Technology Research Online