可用性测试手册.pdf

Handbook ofUsability TestingHandbook ofUsability TestingSecond EditionHow to Plan,Design,andConduct Effective TestsJeff RubinDana ChisnellWiley Publishing,Inc.Handbook of Usability Testing,Second Edition:How to Plan,Design,and ConductEffective TestsPublished byWiley Publishing,Inc.10475 Crosspoint BoulevardIndianapolis,IN 46256Copyright 2008 by Wiley Publishing,Inc.,Indianapolis,IndianaPublished simultaneously in CanadaISBN:978-0-470-18548-3Manufactured in the United States of America10 9 8 7 6 5 4 3 2 1No part of this publication may be reproduced,stored in a retrieval system or transmittedin any form or by any means,electronic,mechanical,photocopying,recording,scanning orotherwise,exceptaspermittedunderSections107or108ofthe1976UnitedStatesCopyrightAct,without either the prior written permission of the Publisher,or authorization throughpayment of the appropriate per-copy fee to the Copyright Clearance Center,222 RosewoodDrive,Danvers,MA 01923,(978)750-8400,fax(978)646-8600.Requests to the Publisherfor permission should be addressed to the Legal Department,Wiley Publishing,Inc.,10475Crosspoint Blvd.,Indianapolis,IN 46256,(317)572-3447,fax(317)572-4355,or online athttp:/ of Liability/Disclaimer of Warranty:The publisher and the author make no repre-sentations or warranties with respect to the accuracy or completeness of the contents ofthis work and specifically disclaim all warranties,including without limitation warrantiesof fitness for a particular purpose.No warranty may be created or extended by sales orpromotional materials.The advice and strategies contained herein may not be suitable forevery situation.This work is sold with the understanding that the publisher is not engagedin rendering legal,accounting,or other professional services.If professional assistance isrequired,the services of a competent professional person should be sought.Neither thepublisher nor the author shall be liable for damages arising herefrom.The fact that anorganization or Website is referred to in this work as a citation and/or a potential source offurther information does not mean that the author or the publisher endorses the informa-tion the organization or Website may provide or recommendations it may make.Further,readers should be aware that Internet Websites listed in this work may have changed ordisappeared between when this work was written and when it is read.For general information on our other products and services or to obtain technical support,please contact our Customer Care Department within the U.S.at(800)762-2974,outside theU.S.at(317)572-3993 or fax(317)572-4002.Library of Congress Cataloging-in-Publication Data is available from the publisher.Trademarks:Wiley,the Wiley logo,and related trade dress are trademarks or registeredtrademarks of John Wiley&Sons,Inc.and/or its affiliates,in the United States and othercountries,and may not be used without written permission.All other trademarks arethe property of their respective owners.Wiley Publishing,Inc.is not associated with anyproduct or vendor mentioned in this book.Wiley also publishes its books in a variety of electronic formats.Some content that appearsin print may not be available in electronic books.Dedicated to those for whom usability and user-centered designis a way of life and their work a joyful expression of theirgenuine concern for others.JeffTo my parents,Jan and Duane Chisnell,who believe mewhen I tell them that I am working for world peace through userresearch and usability testing.DanaAbout the AuthorsJeff Rubin has more than 30 years experience as a human factors/usabilityspecialist in the technology arena.While at the Bell Laboratories Human Per-formanceTechnologyCenter,hedevelopedandrefinedtestingmethodologies,and conducted research on the usability criteria of software,documentation,and training materials.During his career,Jeff has provided consulting services and workshops onthe planning,design,and evaluation of computer-based products and servicesfor hundreds of companies including Hewlett Packard,Citigroup,TexasInstruments,AT&T,the Ford Motor Company,FedEx,Arbitron,Sprint,andState Farm.He was cofounder and managing partner of The Usability Groupfrom 19992005,a leading usability consulting firm that offered user-centereddesign and technology adoption strategies.Jeff served on the Board of theUsability Professionals Association from 19992001.Jeff holds a degree in Experimental Psychology from Lehigh University.Hisextensive experience in the application of user-centered design principles tocustomer research,along with his ability to communicate complex principlesand techniques in nontechnical language,make him especially qualified towrite on the subject of usability testing.He is currently retired from usability consulting and pursuing other pas-sionate interests in the nonprofit sector.Dana Chisnell is an independent usability consultant and user researcheroperating UsabilityWorks in San Francisco,CA.She has been doing usabilityresearch,user interface design,and technical communications consulting anddevelopment since 1982.Dana took part in her first usability test in 1983,while she was working asa research assistant at the Document Design Center.It was on a mainframeoffice system developed by IBM.She was still very wet behind the ears.SinceviiviiiAbout the Authorsthen,she has worked with hundreds of study participants for dozens of clientsto learn about design issues in software,hardware,web sites,online services,games,and ballots(and probably other things that are better forgotten about).She has helped companies like Yahoo!,Intuit,AARP,Wells Fargo,E*TRADE,Sun Microsystems,and RLG(now OCLC)perform usability tests and otheruserresearchtoinformandimprovethedesignsoftheirproductsandservices.Danas colleagues consider her an expert in usability issues for older adultsandplainlanguage.(Shesaysshesstilllearning.)Lately,shehasbeenworkingon issues related to ballot design and usability and accessibility in voting.She has a bachelors degree in English from Michigan State University.Shelives in the best neighborhood in the best city in the world.CreditsExecutive EditorBob ElliottDevelopment EditorMaureen SpearsTechnical EditorJanice JamesProduction EditorEric CharbonneauCopy EditorFoxxe Editorial ServicesEditorial ManagerMary Beth WakefieldProduction ManagerTim TateVice President and ExecutiveGroup PublisherRichard SwadleyVice President and ExecutivePublisherJoseph B.WikertProject Coordinator,CoverLynsey StanfordProofreaderNancy BellIndexerJack LewisCover ImageGetty Images/Photodisc/McMillan Digital ArtixAcknowledgmentsFrom Jeff RubinFrom the first edition,I would like to acknowledge:Dean Vitello and Roberta Cross,who edited the entire first manuscript.Michele Baliestero,administrative assistant extraordinaire.John Wilkinson,who reviewed the original outline and several chaptersof the manuscript.Pamela Adams,who reviewed the original outline and most of themanuscript,and with whom I worked on several usability projects.Terri Hudson from Wiley,who initially suggested I write a book on thistopic.Ellen Mason,who brought me into Hewlett Packard to implement auser-centered design initiative and allowed me to try out new researchprotocols.For this second edition,I would like to acknowledge:Dave Rinehart,my partner in crime at The Usability Group,and co-developer of many user research strategies.The staff of The Usability Group,especially to Ann Wanschura,who wasalways loyal and kind,and who never met a screener questionnaire shecould not master.Last,thanks to all the clients down through the years who showed con-fidence and trust in me and my colleagues to do the right thing for theircustomers.xixiiAcknowledgmentsFrom Dana ChisnellThe obvious person to thank first is Jeff Rubin.Jeff wrote Handbook of UsabilityTesting,one of the seminal books about usability testing,at a time when itwas very unusual for companies to invest resources in performing a realitycheck on the usability of their products.The first edition had staying power.Itbecame such a classic that apparently people want more.For better or worse,the world still needs books about usability testing.So,a thousand thank-yousto Jeff for writing the first edition,which helped many of us get started withusability testing over the last 14 years.Thanks,too,Jeff,for inviting me towork with you on the second edition.I am truly honored.And thank you foroffering your patience,diligence,humor,and great wisdom to me and to theproject of updating the Handbook.Ginny Redish and Joe Dumas deserve great thanks as well.Their book,APractical Guide to Usability Testing,which came out at the same time as Jeffsbook,formed my approach to usability testing.Ginny has been my mentor forseveral years.In some weird twist of fate,it was Ginny who suggested me toJeff.The circle is complete.A lot of people will be thankful that this edition is done,none of them morethan I.But Janice James probably comes a close second.Her excellent technicalreview of every last word of the second edition kept Jeff and me honest onthe methodology and the modern realities of conducting usability tests.Sheinspired dozens of important updates and expansions in this edition.So did friends and colleagues who gave us feedback on the first edition toinform the new one.JoAnn Hackos,Linda Urban,and Susan Becker all gavedetailed comments about where they felt the usability world had changed,what their students had said would be more helpful,and insights about whatthey might do differently if it were their book.Arnold Arcolio,who also gave extensive,specific comments before therevising started,generously spot-checked and re-reviewed drafts as the newedition took form.Sandra Olson deserves thanks for helping me to develop a basic philosophyabout how to recruit participants for user research and usability studies.Herexcellent work as a recruiting consultant and her close review informed muchthat is new about recruiting in this book.Ken Kellogg,Neil Fitzgerald,Christy Wells,and Tim Kiernan helped meunderstand what it takes to implement programs within companies thatinclude usability testing and that attend closely to their users experiences.Other colleagues have been generous with stories,sources,answers torandom questions,and examples(which you will see sprinkled throughoutthe book),as well.Chief among them are my former workmates at Tec-Ed,especially Stephanie Rosenbaum,Laurie Kantner,and Lori Anschuetz.AcknowledgmentsxiiiJared Spool of UIE has also been encouraging and supportive throughout,starting with thorough,thoughtful feedback about the first edition and con-tinuing through liberal permissions to include techniques and examples fromhis companys research practice in the second edition.Thanks also go to those Ive learned from over the years who are part of thelarger user experience and usability community,including some I have nevermet face to face but know through online discussions,papers,articles,reports,and books.TotheclientsandcompaniesIhaveworkedwithover25years,aswellasthehundreds of study participants,I also owe thanks.Some of the examples andstories here reflect composites of my experiences with all of those importantpeople.Thanks also go to Bob Elliott at Wiley for contacting Jeff about reviv-ing the Handbook in the first place,and Maureen Spears for managing thedevelopmental edit of a time-tested resource with humor,flexibility,andunderstanding.Finally,I thank my friends and family for nodding politely and pouringme a drink when I might have gone over the top on some point of usabilityesoterica(to them)at the dinner table.My parents,Jan and Duane Chisnell,and Doris Ditner deserve special thanks for giving me time and space so Icould hole up and write.ContentsAcknowledgmentsxiForewordxxixPreface to the Second EditionxxxiiiPart OneUsability Testing:An OverviewChapter 1What Makes Something Usable?3What Do We Mean by Usable?4What Makes Something Less Usable?6Five Reasons Why Products Are Hard to Use6Reason 1:Development Focuses on the Machine or System7Reason 2:Target Audiences Expand and Adapt8Reason 3:Designing Usable Products Is Difficult9Reason 4:Team Specialists Dont Always Work inIntegrated Ways9Reason 5:Design and Implementation Dont AlwaysMatch11What Makes Products More Usable?12An Early Focus on Users and Tasks13Evaluation and Measurement of Product Usage13Iterative Design and Testing14Attributes of Organizations That Practice UCD14Phases That Include User Input14A Multidisciplinary Team Approach14Concerned,Enlightened Management15A Learn as You Go Perspective15Defined Usability Goals and Objectives16xvxviContentsWhat Are Techniques for Building in Usability?16Ethnographic Research16Participatory Design17Focus Group Research17Surveys17Walk-Throughs18Open and Closed Card Sorting18Paper Prototyping18Expert or Heuristic Evaluations19Usability Testing19Follow-Up Studies20Chapter 2What Is Usability Testing?21Why Test?Goals of Testing21Informing Design22Eliminating Design Problems and Frustration22Improving Profitability22Basics of the Methodology23Basic Elements of Usability Testing25Limitations of Testing25Chapter 3When Should You Test?27Our Types of Tests:An Overview27Exploratory or Formative Study29When29Objective29Overview of the Methodology30Example of Exploratory Study32Assessment or Summative Test34When34Objective34Overview of the Methodology35Validation or Verification Test35When35Objective35Overview of the Methodology36Comparison Test37When37Objective37Overview of the Methodology38Iterative Testing:Test Types through the Lifecycle39Test 1:Exploratory/Comparison Test39The situation39Main Research Questions40ContentsxviiBrief Summary of Outcome41Test 2:Assessment Test41The Situation41Main Test Objectives41Brief Summary of Test Outcome42Test 3:Verification Test42The Situation42Test Objectives43Brief Summary of Test Outcome43Chapter 4Skills for Test Moderators45Who Should Moderate?45Human Factors Specialist46Marketing Specialist46Technical Communicator47Rotating Team Members47External Consultant47Characteristics of a Good Test Moderator48Grounding in the Basics of User-Centered Design48Quick Learner48Instant Rapport with Participants49Excellent Memory49Good Listener49Comfortable with Ambiguity50Flexibility50Long Attention Span51Empathic People Person51Big Picture Thinker51Good Communicator52Good Organizer and Coordinator52Getting the Most out of Your Participants52Choose the Right Format53Sit-By Sessions versus Observing from Elsewhere53Think-Aloud Advantages and Disadvantages54Retrospective Review54Give Participants Time to Work through Hindrances55Offer Appropriate Encouragement55Troubleshooting Typical Moderating Problems56Leading Rather than Enabling57Too Involved with the Act of Data Collection57Acting Too Knowledgeable57Too Rigid with the Test Plan58Not Relating Well to Each Participant58xviiiContentsJumping to Conclusions58How to Improve Your Session-Moderating Skills58Learn the Basic Principles of Human Factors/Ergonomics59Learn from W