Test Report - NET Legion
Contents
1. Adobe Photoshop has a separate floating window for the layers and this approach should be emulated Layers Normal iv Opacity 100 gt Lock O Fill 100 gt The solution is not perfect merging layers is still difficult Yet it is clear that the layer display cannot fit on the same toolbar as the drawing tools and the colors Cosmetic Problem Only Severity Rating 1 Finding 1 Participants did not read the tooltips placed on buttons Evidence Tool tips showed up too slowly to be useful Some of the user problems could have been eliminated if they were able to read the tool tips Recommendation A tutorial for beginner users could take over the assistant role now occupied by the tooltips Finding 1 There is no redo Evidence One participant was looking for a redo command and was not able to find it Recommendation The current implementation makes redo very difficult to implement However it would be a nice feature to include in future releases PaintSI Usability Test Report 6 Conclusion References The usability test clearly demonstrated the shortcoming of PaintSI s interface Users expected greater functionality from a drawing application whether it was in the form of a rub erase eraser moveable objects or easier graphics importing Surprisingly however users enjoyed using the application after discovering its conventions Two of the participants wanted to play with the canv2. not identified with you personally I will collect your name at the onset along with contact information if we have any follow up questions but the identifying information will be destroyed at the end of the semester If you have any questions please contact Stan Drozdetski at sdrozdet umich edu I agree to participate in this study Signature Date Print your name Print your name PaintS Usability Test Report 8 Appendix B Task Description Task 1 Clown Draw a clown figure Feel free to be as creative as you would like The figure does not have to be involved it can simply be an assembly of geometrical shapes If any of the functions are confusing you may consult the user manual at any time Task 2 Sun Start a new file Draw a house The level of detail is once again up to you Switch to another layer Draw the sun shining over the house on that layer Task 3 Erase Sun Now erase the sun and add another building to the drawing you may keep the layer the same Task 4 Save and Erase Save the drawing to a file on the desktop naming the file with your favorite color Erase everything drawn so far Task 5 Butterfly Draw a butterfly on the third layer Task 6 Image Import Import an image file named testpic jpe into the application Move it into the bottom right corner of your screen That would be all Thank you for your participation PaintS Usability Test Report 9
3. was after about 30 seconds and reverted to visible mode drew polygon did not double click had to undo figured out how to draw a polygon after receiving a hint Image Import found the color palette by accidentally clicking on the color insert image no problems could not move the image at first figured out how later selecting did not work went back to reselect the tool even though the tool was already selected wondered if it was possible to fill a shape drawn with lines used Backspace to delete used Erase Canvas Sun thought when the color was white the shape was not filled had no idea what the preview in the color picker did thought polygon was a star used lines for rays tried to delete current shape but a previous one was still selected Erase Sun Select to Delete menu entry did not work layer switch no problems Save and Erase save erase no problems Butterfly used two ovals for wings found that right clicking did nothing went back to the toolbar to select a tool PaintS Usability Test Report 3 Task 6 Participant 3 Task 1 Task 2 Task 3 Task 4 Task 5 Task 6 Findings Image Import was looking for the import function could not locate it consulted the user manual Clown used circle arc line tools found the color palette right away tried moving an arch with the Selector tool wished that it would be possible to pas
4. Paints Usability Test Report File Edit Tools Stan Drozdetski Winter 2004 Professor Ackerman EECS 493 Introduction Methods A test evaluating the usability of PaintSI has been performed in the week of April 18 25 2004 Three trial runs have been conducted with three different subjects resulting in a set of suggestion for further improvement PaintSI is a lightweight Java based graphics program that allows simple manipulation of geometric shapes and images Its functionality is somewhat limited by the small number of tools provided However the application still attempts to implement a vector based scheme with layering undo functionality and an easy to use graphical user interface Participants Table 1 provides demographic information of the participants Participant Age Gender Occupation Major P1 24 female graduate student Chem P2 32 male graduate student HCI P3 26 male electrical engineer Table 1 Participant Information Participants were recruited through a personal network Because of the short duration of the test they were not given honorariums Tasks A set of six tasks Appendix B was assembled in order to test the suspected weak points of the interface Task 1 required the participants to draw a clown figure They were encouraged to keep the drawing simple in order to keep the experiment manageable Task 2 evaluated had participants draw t
5. as after the test was over creating some interesting drawings Most of the problems found during the test cascaded from a deign decision to use vector based graphics While some transformations were easier as compared to the bitmap format e g merging layers and selecting shapes others became very difficult e g drawing freehand Much more work would be required to make PaintSI as instantly usable as for example the bitmap based MS Paint Drozdetski S Kim Y Kong J Lee K 2004 MSN Messenger Whiteboard Usability Test Report Nielsen J 1999 Severity Ratings for Usability Problems http www useit com papers heuristic severityrating html PaintS Usability Test Report 7 Appendix A Consent Form User Interface Evaluation I am assessing the functionality and usability of PaintSI a drawing application I would like to watch you use this product in a set of set tasks to see how the interface performs I would also like to interview you about your opinions about this product The whole session will take less than half an hour I am collecting this information to write an evaluation report for my class at the University of Michigan User Interface Development EECS 493 taught by Professor Mark Ackerman You are not required to participate in this and you may skip questions or stop at any point in the experiment Your interaction with program will be recorded using a background process The data collected is
6. as confused when drawing white shapes on a white background thought he was drawing unfilled shapes To adhere to the convention the selection indicator should be a broken line that follows the outline of a shape Deleting shapes is not straight forward Clicking on a shape to delete it is not natural to the users Instead they tried rub erasing parts of the shapes finding that this was impossible One participant tried to drag a selection box around a cluster of objects to delete them all but was not able to do that The vector nature of the program prohibits rub erasing There is no good solution to this problem short of eliminating the Select to Delete tool and relying solely on using the keyboard for deletion Implementing rub erasing is unfortunately outside the scope of this project Selecting shapes worked only in certain cases Two participants encountered problems with selecting shapes they were able to select some shapes and not others PaintS Usability Test Report 5 Recommendation This problem was caused by faulty code and has already been fixed Minor Usability Problem Severity Rating 2 Finding 2 Buttons used for layer merging are confused for layer selection buttons Evidence Two participants first resorted to the layer merge buttons before figuring out that they needed to click on the layer name to select it Recommendation In order to be truly functional the layer display has to be more involved
7. he house and switch to another layer Next to test the erase tool the participant was asked to erase the sun Task 4 involved saving a file To complete the testing of drawing tools a butterfly was the next object to be drawn in Task 5 Finally the participants were asked to import an image file into the application and move it to a new position Procedure A participant was briefed on the purpose of the experiment and offered a consent form modeled after the standard IRB form Appendix A He or she was then invited to sit behind an IBM T20 laptop computer and to go through a series of tasks at their own pace Participants were encouraged to verbalize or think aloud PaintS Usability Test Report 1 Results Participant 1 Task 1 Task 2 The screen image along with audio was continuously captured by Camtasia Studio 2 02 and later saved in the AVI format Trial runs determined that Camtasia did not noticeably impact computer processing time Notes were taken for further review Timing was not performed because of the unique nature of every task Data Analysis Every finding from test notes and recordings was marked down on a separate index card These cards were then sorted to combine duplicated notes and to cluster findings by issue To make the final presentation cohesive findings were sorted by severity Severity rankings Nielsen 1999 are defined as 4 Usability catastrophe imperative to fix before produc
8. on the canvas All participants expressed the desire to move shapes Being able to move shapes would have made copy and paste more useful Unfortunately the programming effort involved in making all shapes moveable was too great every shape is stored in a unique format because of the differences in how they are drawn Ideally each shape would know the coordinates of where it is located in order to make moving easier The polygon shape is difficult for users to draw No participant expected the shape to be closeable with a double click One participant tried adding vertices in the order that the application did not support The polygon tool is inherently difficult to use Whatever the method used to specify vertices the polygon still has to be completed when the last vertex is added This however is unnatural to the users other shapes are drawn much easier by dragging The solution would be to eliminate the polygon tool entirely and replace it with more shape templates A triangle tool is definitely needed and perhaps a rhombus and an octagon tools Major Usability Problem Severity Rating 3 Finding 3 Evidence Recommendation Finding 3 Evidence Recommendation Finding 3 Evidence Some users do not recognize a solid rectangular outline as the selection indicator Participants wondered why a black outline was disappearing around the object that they last created after they drew a new one One user w
9. t can be released 3 Major usability problem important to fix should be given high priority 2 Minor usability problem fixing this should be given low priority 1 Cosmetic problem only need not be fixed unless extra time is available Problems of equal severity are ordered based on how easy the solution is with easily fixed problems being presented first method adopted from Drozdetski et al 2004 The following describes the details of participant performance on each task Clown was confused about changing color tried rub erase failed wanted to change the color of the shape found that there was no redo command used arch polygon rectangle line circle tools polygon could not figure how to end it wondered if there is a move function used Backspace to erase a shape Sun tried to figure out what the layer merge button did tried Backspace during the process of drawing wanted to draw unfilled shapes looked up how to switch to another layer in the manual PaintS Usability Test Report 2 Task 3 Task 4 Task 5 Task 6 Participant 2 Task 1 Task 2 Task 3 Task 4 Task 5 Erase Sun made everything besides the sun layer invisible erased the sun using the Select to Delete tool used rectangle tool Save and Erase save no problems used New File to erase contents Butterfly drew on Layer 3 that was set to invisible figured out what the problems
10. te and immediately move a tool Sun clicked on the merge button first had to consult the manual for layer switching wanted to change the color of the selected shape mentioned that a gradient would be nice to have Erase Sun tried to select by pulling a selection box around objects failed used the Selector tool to select objects and deleted them using Backspace Save and Erase save no problem looked around for erase found it Butterfly used oval for the body struggled with the polygon tool because it was not finished the first vertex of the new polygon was actually the last vertex of the last one asa workaround switched tools every time a polygon had to be completed Image Import import no problem tried using the arrow keys after some experimentation used the Selector tool properly tried the text tool used it properly Eight clusters of findings are presented each complete with evidence as observed during the test and proposed solution The findings are not complete in order to test the entirety of the system more tasks with a greater number of participants are sorely needed Unfortunately the design of the current user test was limited by time constraints PaintS Usability Test Report 4 Usability Catastrophes Severity Rating 4 Finding 4 Evidence Recommendation Finding 4 Evidence Recommendation An image is the only shape that can be moved
Download Pdf Manuals
Related Search