Contextual Support for Remote Cooperative Troubleshooting
Contents
1. Evans M 2004 Knowledge and Work in Context A Case of Distributed Troubleshooting Across Ship and Shore Ph D Dissertation Indiana University Finholt T Sproull L and Kiesler S 1991 Communication and performance in ad hoc task groups In Galegher J Kraut R and Egido C eds Intellectual Teamwork Hillsdale NJ Lawrence Erlbaum Press 291 325 Georgakopoulos D Hornick M F and Sheth A P 1995 An overview of workflow management From process modeling to workflow automation infrastructure Distributed and Parallel Databases 3 2 119 153 Karchenasse N 1997 The hierarchical case based diagnosis In 5th German Workshop on Case Based Reasoning Kolodner J 1993 Case Based Reasoning San Mateo CA Morgan Kaufmann Leake D Birnbaum L Hammond K Marlow C and Yang H 1999 Integrating diverse information resources in a case based design environment Engineering Applica tions of Artificial Intelligence 12 6 705 716 Leake D 1996 CBR in context The present and future In Leake D ed CaseBased Reasoning Experiences Lessons amp Future Directions AAAI Press 3 30 Torasso P 2004 Case based reasoning in diagnostic problem solving Alternative or complementary to mbr In Proceedings of the 15th International Workshop on Principles of Diagnosis Valente G and Rigallo A 2004 Using case based reasoning to support operational knowledge management In Proceedings of the Fourteenth Internat2. needs for successful remote troubleshooting The second building on the first examines resulting requirements for contextual support from the perspective of human centered computing focusing on practical methods for maximizing performance of the combined human computer system by exploiting the capabilities of each The third applies these results describing our current progress in building a system to support remote collab oration by reducing uncertainty about participants actions and capturing and transmitting contextual information to aid situation assessment and knowledge reuse This work serves two goals the scientific goal of understanding of the types of contextual factors affecting remote cooperative troubleshooting and the practical goal of developing troubleshooting support tools that make relevant contextual information explicitly available to both experts and novices 2 Context in Distributed Cooperative Troubleshooting Br zillon 1999 has characterized problem solving context as what constrains a problem solving event without intervening in it explicitly Studies show that the context in which a problem is placed can have a significant effect on a human problem solver s decision process Albers 1999 making it important to provide appropriate contextual information to the decision maker However when cooperative problem solving is conducted at a distance providing con text is problematic Ahn et al 2000 obse
3. H J Hong J L Kyehyun C and Sung J P 2000 Utilizing knowledge context in virtual collaborative work Decision Support Systems 39 563 582 Albers M 1999 Information design considerations for improving situation awareness in complex problem solving In Proceedings of the 17th annual international conference on Computer documentation 154 158 Amann P and Quirchmayr G 2003 Foundation of a framework to support knowl edge management in the field of context aware and pervasive computing In Proceedings of the Australasian information security workshop conference on ACSW frontiers 2003 volume 21 119 131 Bartsch Sp rl B 1997 How to introduce CBR applications in customer support In Proceedings of the 5th German CBR Workshop Br zillon P Borges M Pino J A and Pomerol J C 2004 Context awareness in group work Three case studies In Decision Support in an Uncertain and Complex World The IFIP TC8 WG8 3 International Conference Br zillon P 1999 Context in problem solving A survey The Knowledge Engineering Review 14 1 1 34 Br zillon P 2003 Individual and team contexts in a design process In Proceedings of the 36th Hawaii Conference on System Sciences IEEE Burkhard H and Pirk P 1996 Technical diagnosis Fallexperte d In 4th German Workshop on CBR System Development and Evaluation Eisenhardt K M 1989 Building theories from case study research Academy of Man agement Review 14 4 532 550
4. downstream in a work process Leake et al 1999 the current system extends this model to apply cases for communication within a dialogue With each diagnostic step cases are transmitted back and forth receiving updates to reflect each user s actions and circumstances Likewise prior cases are used as a resource to draw on both for questions to transmit during the cooperative troubleshooting dialogue and for potential diagnoses and corrective actions The troubleshooting steps are e A fault is detected by either a functionality failure or during periodic preventative maintenance e A sailor follows standard procedures as presented by the system helping to stabi lize the sailor s context by assuring that the sailor will have actually carried out the prescribed troubleshooting steps Associated data collection and actions taken are stored in a new case e If standard procedures are exhausted without the sailor resolving the fault the sailor contacts the SME via the system which transmits to the SME the case describing the situation so far e Using the SME interface the SME examines the case so far transmitted from the sailor interface and the system automatically retrieves cases of similar past trou bleshooting scenarios to present to the SME e The SME uses similar past cases and personal experience to suggest additional mea surements and actions for the sailor to take The system suggests potentially relevant questions from p
5. knowledge capture transfer and sharing to aid communication between sailors and tech nicians We have developed a testbed system that captures and conveys information about previous diagnoses based on a case based reasoning CBR system Kolodner 1993 Leake 1996 integrated into the task processes CBR is an artificial intelligence methodology for reasoning and learning from specific experiences that has been extensively applied to diagnosis tasks as will be described in Section 7 Thus it appears to be a natural approach for capturing and conveying information about specific troubleshooting episodes CBR systems capture knowledge in the form of traces of problem solving episodes and generate new solutions by adapting solutions for similar prior problems These problem solution pairs can in turn be stored as cases in the case base to be used for future problem solving In our system an interactive variant of CBR known as conversational case based reasoning CCBR guides users in diagnosis and suggests solutions This is done in part by building up a case for the current problem in which the system defines the values of a case s attributes according to the actions of the sailor and other available data A major contribution of this work compared to normal CCBR is that cases are treated as a vehicle for cooperative knowledge sharing rather than simply as a resource for a single user We previously proposed the use of cases for information flow
6. Contextual Support for Remote Cooperative Troubleshooting Lessons From a Naturalistic Study David Leake Steven Bogaerts Michael Evans Donald F McMullen Computer Science Department Indiana University Lindley Hall 215 Bloomington IN 47405 U S A Pervasive Technologies Labs Indiana University 501 N Morton Suite 224 Bloomington IN 47404 U S A leake sbogaert cs indiana edu micaevan mcmullen indiana edu Keywords Context Collaboration Case Based Reasoning Ethnography Troubleshoot ing Abstract Distributed troubleshooting often requires experts to support non experts at a distance The resulting separation between collaborators can impoverish their understanding of each other s task contexts impeding their collaboration Consequently this paper ar gues that developing effective troubleshooting support systems requires determining 1 what contextual information the participants require to perform their tasks and 2 how support systems can be designed to help provide that contextual information Based on an ethnographic study of real world remote diagnosis of electronic devices by ad hoc teams we have identified 12 types of contextual information affecting the remote interaction of expert and non expert troubleshooters and we argue that software tools for remote cooperative troubleshooting should be explicitly designed to bridge the gap between participant contexts by capturing and transmitting these types of i
7. abling the SME to attach relevant bulletins etc to problem cases for just in time presentation to the sailor as the sailor follows instructions or procedures e The non expert s training experience When the current case is generated the system logs the sailor s actions to assure that the case contains information that the sailor may otherwise forget or need to be prompted to provide This decreases the gap when an inexperienced sailor must attempt to provide a useful description of the situation to the expert Communication from the SME to sailor is facilitated by enabling the SME to trans mit questions from prior cases in a form that is integrated into the sailor s normal troubleshooting interface and that can contain detailed guidance to reflect the sailor s level of training e The sailor s pre contact actions These actions are all logged automatically by the system and transmitted to the SME in the form of a case Thus the SME can have confidence in the accuracy of the case without spending additional time in verifica tion The system also stabilizes the sailor context by requiring the sailor to complete the standard actions in order to be allowed to proceed e Institutional regulations and responsibilities The sailor s activities are guided by the system and logged in a case so there is less chance for deviation from official procedure This case can then be passed to and trusted by the SME The following co
8. ctice In observed current practice non expert sailors first rely on standard diagnostic flowcharts and technical manuals to isolate one or more detected faults When standardized methods and support materials are exhausted the sailor contacts a remote shore based expert for assistance During their collaboration sailor s and expert technician s often have only basic media support in the form of email chat and paper based documents Thus to resolve a problem successfully 1 the technician must request a detailed account from the sailor of all actions taken prior to the request for technical assistance 2 the sailor must convey any unique aspects of the operating environment including recent upgrades and climatic conditions 3 based on data received the technician must diagnose the problem and prescribe corrective actions and 4 the sailor must correctly perform the corrective actions and confirm successful resolution of fault s In current practice no technological support is available for transmitting contextual information Instead the subject matter expert or SME must simply ask the sailor for clarification and reminders of context Maintaining a view of context is aggravated because the SME may asynchronously handle many problems concurrently with substantial delays between e mail messages from the sailor 4 Contextual Factors Shaping Remote Troubleshooting In all remote troubleshooting some basic explicitly relevant prob
9. d or standard procedures Admin istrative workflows involve automation of simple and predictable processes while leaving users to handle the more complex aspects of collaboration In production workflows more complex yet still predictable processes are handled automatically to assist users in their collaborative efforts The information captured by cases could assist both in transmission of basic information and in making accessible the contextual information needed by different participants 7 4 Opportunities for Ubiquitous Computing A future direction for this work is to examine methods for increasing system context awareness and automatic responsiveness The contextual information captured in our cases is based on capturing traces of sailors and SMEs with information systems While con siderable contextual information is available in this form significant opportunities exist for drawing on additional information sources For example as sensors are installed on devices or as device information can be automatically captured during preventative main tenance it may be possible to broaden the range of environmental and device information captured and transmitted automatically Ubiquitous computing methods promise to facili tate the sailor s troubleshooting task and facilitate information capture 8 Conclusion Human problem solving depends on numerous contextual factors Consequently when relevant contextual information is difficult
10. e Division under contracts N00164 04 C 6514 and N00164 04 C 6515 factors underlying this collaboration and illustrates strategies for supporting this task by stabilizing the task circumstances to reduce uncertainty about participants actions and by capturing and transmitting contextual information This project has been conducted by the Indiana University Knowledge Acquisition and Projection Laboratory KAPLab for the domain of troubleshooting complex shipboard electronic systems In the current U S Navy troubleshooting process shipboard sailors often with limited expertise are paired with shore based experts on an ad hoc basis to assist them in dealing with problems that they cannot diagnose Communication options are limited and often asynchronous with e mail messages a primary means of communi cation At any time the expert may be responsible for a number of open cases To perform effectively the sailor must maintain a picture of the contextual factors constraining the troubleshooting situation and process and communicate them to the expert Likewise the expert must construct and maintain an understanding of the sailor s context for each open case The paper is divided into three parts The first presents results from a naturalistic study of real world cooperative distributed troubleshooting conducted over a nine month period Evans 2004 This study examines the factors affecting the troubleshooting process and participants
11. for users to obtain directly systems that furnish them with this information can provide important benefits In remote cooperative trou bleshooting by ad hoc teams participants may begin their tasks with limited knowledge of team members contexts Based on an extensive study of the types of contextual factors shaping performance for this troubleshooting task we have illustrated the rich context that it involves Our analysis of types of contextual information used identifies a number of opportunities for context transmission to bridge gaps between non experts and the experts who attempt to aid their troubleshooting We have sketched directions for responding to some of these opportunities and their application in an implemented testbed system which uses a conversational CBR interface to guide sailor actions and cases as a vehicle to help bridge contextual gaps between experts and non experts Informal feedback on the system from potential users has been encouraging One next step is to perform controlled tests to assess the benefits and tradeoffs of the system both to contribute to further design and as a form of validation of the set of contextual factors identified as important by our naturalistic study Another future step is to further exploit the study s results by exploring how the additional types of contextual information identified by the study can be extracted from the workflow for this task and presented when needed References Ahn
12. has been extensively studied as well Valente and Rigallo apply conversational CBR to operational knowledge management to facilitate sharing of knowl edge for equipment installation and maintenance tasks Valente amp Rigallo 2004 However they focus on expert diagnosis rather than distance support involving a non expert with out the explicit contextual considerations we examine in this paper Karchenasse et al lay a foundation for hierarchical case based machine diagnosis in which the primary contextual factor is whether the machine is on line and off line Karchenasse 1997 Burkhard and Pirk 1996 argue that CBR is particularly suitable for diagnosis domains They criticize a common approach of diagnosis as classification in CBR and advocate an approach more closely mimicking expert reasoning In this approach newly defined attributes may be added to cases and missing values must be handled as the user works towards case completion searching for sufficient information to solve a problem This is in contrast to standard classification approaches in which a fixed set of attributes is used and is in the spirit of our work on monitoring actions to build up cases over time 7 3 Computer Support for Collaborative Work Georgakopoulos and Hornick 1995 describe three major workflow management strate gies In ad hoc workflows collaboration efforts are done as needed by the participants themselves with no overarching control by a system an
13. ional Conference on Knowledge Engineering and Knowledge Management Yin R K 1994 Case study research design and methods Thousand Oaks CA Sage 10
14. lem solving information is available to both participants For example in our fault diagnosis domain in which a subject matter expert SME guides a sailor in diagnosing a problem this basic information consists of e The specification of the device being diagnosed e The symptoms of the fault to diagnose e Official diagnostic resources and e Reports on the actions such as voltage tests etc carried out by the sailor under direction of the expert However our analysis shows that the troubleshooting process is also influenced by an ex tensive set of contextual factors beyond this explicit specification any of which can have important effects on the course of troubleshooting The lack of support for providing the expert with such contextual information makes the development of context based support a pressing need The following sections discuss 12 types of contextual factors identified in our tran scripts as important to effective cooperation with examples Participants are identified as the shipboard non expert sailor and the shore based subject matter expert SME who assists the sailor remotely 4 1 Characteristics of the Participants Much context comes from the characteristics of the non expert and expert participants 1 The non expert s access to knowledge updates A sailor whose knowledge has not been updated since training may need to be advised of recent developments SME there s been an advisory put out about tha
15. nd HVDU s 11 Schedule constraints Both sailors and SMEs operate under strict schedules Thus expectations for their interactions depend on the prioritization of their tasks and their availability to pursue the task at hand SME reassuring a sailor Z will keep checking my mail this weekend so I won t leave you hanging 12 Higher level priorities and goals Repairs are prioritized and the goals of trou bleshooting may depend on external factors For example under emergency cir cumstances a quick temporary fix for a crucial component may be preferable to a long term repair which would require waiting for a new part 5 Lessons for Troubleshooting Support Systems Our study of current practice illustrates the rich range of context that can be useful to remote troubleshooting all of these context types would be valuable within support systems for remote cooperative troubleshooting It also shows that baseline support is needed simply to help convey the explicit problem as illustrated in the following exchange SME I am very confused as to what the exact symptoms are I want you to go to High Voltage special functions and do the following tests After you do this I will have a better understanding your problem Then I will walk you through more instructions Our strategy in applying these lessons is first to provide this basic support and then to augment it with additional capabilities The overall aim was a combined system for
16. nformation As a means for performing this task we propose a support framework based on conver sational case based reasoning The paper illustrates the application of this approach in an implemented testbed system 1 Introduction In on site cooperative troubleshooting team members have direct access to a rich shared context Br zillon 2003 Finholt Sproull amp Kiesler 1991 They can jointly observe and adjust their behavior based on team members actions and reactions and the circumstances in which troubleshooting is done In contrast when troubleshooting teams are formed as needed from various worksites for remote collaboration as in ad hoc help desk collab orations the need to collaborate at a distance can significantly hinder the development of a shared context Consequently aiding participants at understanding each other s con texts can play an important role in effective collaboration and it is desirable for sys tems supporting remote cooperative troubleshooting to support this as well To design systems that support the sharing of contextual information it is necessary to understand the types of contextual information that may influence the task process Albers 1999 Amann amp Quirchmayr 2003 This paper presents a case study of the role of context in remote distributed collaboration for a real world diagnosis task assesses the contextual This work is supported in part by the U S Department of the Navy NSWC Cran
17. nitial inferences about steps the sailor has already performed However it may also decrease the accuracy of sailor provided information if the sailor is reluctant to reveal a deviation from policies 8 Social and cultural standards The decision of when to seek unofficial help before contacting an SME depends on social factors Sailor YJou might want to ask your Chief or the DivO4 Hey we have been at this for a while can we ask the USS Whatever for some assis tance because this sounds like a problem they had a couple of months ago And if he wants too then we ll make a request but maybe my Chief doesn t want the USS Whatever to know that our system is screwed up It is a pride issue 4 4 Capability Constraints The expert s guidance to the non expert and the overall course of troubleshooting are also shaped by external limitations in capabilities 9 Environmental constraints Adverse environmental conditions can limit the diag nostic actions an SME may request Sailor e mail message to SME The weather is pretty bad right now so I can t go out topside to get into the transmitter 10 Resource constraints The sailor and SME must adjust instructions according to the resources and facilities on hand Sailor e mail message to SME I would like to verify the power supply and HVDU but I don t know if Guam or Yokosuka Japan have any facil ities that have the bench test equipment for power supplies a
18. ntextual factors are not currently addressed in our system but could be addressed to some extent without significant additional development e The expert s skill experience When a case is resolved and stored for retrieval in future problem solving a record of the SME who solved the problem can be stored as part of that case e History of the problem setting Cases can include information on the ship and parts involved as potential retrieval cues to obtain a history of similar problems specific to a ship and or part or for data mining for problem trends e Higher level priorities and goals Descriptions of the immediate problem can be augmented with information about priorities influencing the choice of repair e g an emergency situation to use an initial filter in retrieval to obtain quick fix cases The capture of environmental factors and constraints social and cultural standards and resource constraints is a subject for future research and is likely to require integrations with other methods 7 Perspective The issues addressed in this paper relate to studies of communication of contextual knowl edge computer support for cooperative work and case based reasoning ubiquitous com puting provides interesting future directions 7 1 Communication of Contextual Knowledge Our view of context in problem solving has important commonalities with Br zillon s ap proach in Br zillon 2003 which argues that a significant por
19. rior cases which the SME can select to transmit e The sailor receives and executes these suggestions with the results added to the current case by the system as an evolving record of the sailor s steps and the sailor SME interaction If the fault is still unresolved the sailor sends the SME the updated case and the cycle continues 6 Contextual Support Using Cases As we applied cases to collect and transmit information about the explicit problem de scription we observed that cases because they capture specific information about multi ple types of information within an episode Kolodner 1993 also appeared promising for collecting and transmitting many of the types of contextual information discussed in the previous section We have implemented the capture and sharing of a number of types of contextual information in the system SMEapp to bridge particular gaps between expert and non expert contexts and reflect contextual constraints and additional extensions are under development We illustrate below how the contextual factors are reflected for our specific task domain However we note these types of contextual information will be relevant in many domains making the proposed approach promising for broader use e The non expert s access to knowledge updates Because a sailor may have missed or forgotten knowledge updates our system aims to assure that the non expert s un derstanding context will include relevant documents by en
20. rve that ad hoc virtual cooperative problem solving presents at least three key impediments to the capture of context First because the collaborations are transient involving changing sets of participants context regarding knowledge and skills involved in a problem solving session may be lost Second the dis tributed and heterogeneous nature of virtual collaborations limits the breadth and scope of context that is practical to convey Finally when virtual collaborations are intensive and non routine contextual information is easily lost in the effort to reach resolution A goal of our project is to develop technological support methods to alleviate these difficulties 3 An Analysis of Real World Troubleshooting Context The naturalistic inquiry into the virtual organization of naval technical communities in cluded nine months December 2002 August 2003 of data collection from interviews with naval and civilian personnel technical documents and on site observations at three lo cations in the continental U S Over 50 hours of semi structured interviews were conducted to gather both factual information and the respondents opinions about events following the general framework described in Yin 1994 p 84 Length constraints preclude dis cussing these further in this paper but additional information on these methods is available in Eisenhardt 1989 Yin 1994 Full details of the study are available in Evans 2004 3 1 Current Pra
21. t s just the way it is Maybe the SME can say Oh I know This ship always has a problem with this circuit card So we re going to have to look in this particular area to begin troubleshooting 5 Environmental effects Environmental conditions may make particular failures more likely making it appropriate to look outside the prescribed procedures Sailor Say you lose RF radio frequency on your port forward quad rant The worst thing that is going to happen is on an aircraft carrier the cord antenna is right next to CAT4 a catapult for launching aircraft so there is a lot of vibration So a lot of the time the cable will shake loose 6 Pre contact actions Sailors commonly start a dialogue by providing this context informing the SME of what was tried and did not work In practice it is difficult for SME s to obtain an accurate picture of this context Because they do not necessarily trust sailor reports to be accurate they may ask the sailor to repeat some tasks which should have been done pre contact to ensure that they were done fully and correctly 4 3 Institutional Social and Cultural Factors The troubleshooting process is shaped by institutional social and cultural factors as well 7 Institutional regulations and responsibilities Navy policies require the sailor to follow prescribed procedures in a set order This context limits the sailor s auton omy but enables the SME to make some i
22. t but these young third class guys would never know there was an advisory out 2 The non expert s training experience More highly skilled sailors can be given less detailed instructions and can convey their problems more effectively SME T f you don t have an experienced tech on the other end of the line you can do distance support until you re blue in the face but if he can t describe the problem to you 3 The expert s skill experience While all SMEs can be expected to have current knowledge and comparable training in practice it is recognized that the SME s ex perience may play a key role in their suitability for a particular problem SME You know someone will take the email and think Hey this guy s really up to speed on high voltage so they ll give it to me Or this guy you know it s ano load problem and let s say someone else is better suited to take the assist 4 2 The Problem Setting Characteristics of the problem setting both historical and current play an important role in doing situation assessment on the current problem suggesting possible diagnoses and selecting diagnostic actions 4 Problem history Different ships may configure the same equipment differently re sulting in different failure trends Previous problem information can provide valuable guidance for focusing diagnostic effort SME Quite often some ship s systems they like to eat a particular circuit card Tha
23. tion of problem solving in volves the construction via communication of contextual knowledge to be shared by all participants This knowledge consists of the externalization of implicit knowledge in the form of a series of proceduralized contexts one for each step in the problem solving pro cess These proceduralized contexts can then be logged so that the process can be repeated or modified for new situations Building on this work Br zillon et al link the concept of awareness the understanding of the activities of others with the development of a group context Br zillon et al 2004 They describe a knowledge processing procedure for obtaining awareness and group context in cooperative efforts describing storage and retrieval of contextual information Their approach stores this information in a centralized database while we propose that it be operationalized within a case based reasoning frame work They distinguish collaboration in individual workspaces individual contexts and a group workspace group context their individual workspaces are mapped to our individ ual applications for non expert and expert while their group workspace corresponds to our case under development edited and added to by each participant 7 2 CBR in Diagnosis CBR applications are widely used in diagnosis with help desks a classic application area Bartsch Sporl 1997 The integration of CBR with other methods e g model based di agnosis Torasso 2004
Download Pdf Manuals
Related Search