g3-decide

<< User Testing Stage = =

= = =DECIDE framework and test scripts= toc

__Annmarie__
//Determine overall goals of the evaluation// The purpose of this study is to pinpoint weaknesses of the prototype website concerning usability, information architecture, and visual effectiveness of the interface. It is imperative that the site be intuitive and well organized to facilitate an effective learning and browsing experience. As well, the visual design should be unobtrusive to the main focus of the site—the content.

//Explore the specific questions to be answered//
 * Is ‘drag and drop’ functionality an appropriate and effective way to collect images?
 * Since ‘drag and drop’ is not readily used in websites, users may be unfamiliar with the unique function and may experience some difficulty in learning the process of saving images. We want to ensure that users can adapt to the interface as well as evaluate the appropriateness of the drag and drop application for the proposed purpose//.//
 * Is the visual design of the site appropriate to subject matter and audience?
 * We want to verify that the visual design of the site allows users to focus on the content, as well as gather responses on what they expect the site should look like. It would also be worthwhile to explore the balance between the visual identity of the ccca and practicality as a subtle interface.
 * How easy is it to find specific information?
 * We want to determine if the categories on the site are inline with the user’s mental model in order for information to be located with ease. The effectiveness of hierarchy and layout in contribution with this task will also be examined.

//Choose the evaluation paradigm and techniques to answer questions// A structured method of testing would be ideal for the study since our project is fairly specific in the number of functions it can perform. As well, many of the links are not active and some of the interactive aspects are not functioning at a level that would facilitate free action.

//Identify the practical issues that must be addressed// The testing environment should be in a quiet room with a computer. Firefox is the preferred browser and javascript must be enabled to ensure the site works as intended.

//Decide how to deal with the ethical issues// Ethical issues concerning the collection of confidential information from users may arise. To address this matter, a consent form verifying the tester’s agreement of participation will be presented prior to testing. It will be made known that information gathered will only be used for the purposes of our study and that they have the right to terminate their participation at any time.

//Evaluate, interpret and present the data// We will identify and analyze any similarities and difference in the data, taking into consideration their expectations and familiarity with the subject matter. Results will be prioritized in order of importance, which will inform further changes.

//User satisfaction interview//
 * What do you think about the visual design of the interface?
 * Would you use this site? For what?
 * Did you enjoy interacting with the system?
 * What did you like about the system?
 * What didn’t you like about the system?
 * What do you suggest in order to improve the site?

__Joanna__
//Determine overall goals of the evaluation// This user test is designed to test the strengths and weaknesses of our proposed system for the CCCA in terms of ease of use and clarity. The site must be easy to use so the user does not get frustrated with the interface and lose interest in the site. Also, we would like to see if the user truly benefits from the different tools we provide them for engaging with the material. While we may feel that the tools we provided will benefit the user, we do not know if the user would find it useful for themselves.

//Explore the questions that need to be answered// • Does the interface of the site allow for clear navigation? • How useful are the features that we have given users to help with using the site? Are they helpful or is it a waste of space? • Are the features clearly presented on the site and easy to spot?

//Choose the evaluation paradigm and techniques to answer questions// The user will be given a specific task that revolves them having to navigate through the various pages of the site as well as using the contextualizer and other methods to find related artworks.

//Identify the practical issues that must be addressed// We must ensure that our site works with the testing computer that we would be using and that there will not be any major problems around it. Also, we must make sure that users understand the instructions given to them while not leading them through in any way possible to avoid skewing test results.

//Decide how to deal with the ethical issues// In order to minimize the amount of ethical issues that may arise from our user testing, we have provided a consent form to the user to sign saying that they agree to our user testing. However, they have full rights to terminate the test at any time and we must abide to that decision.

//Evaluate interpret and present the data// We will look at the different opinions expressed by our test users to find the major similarities and differences amongst them. The results will be categorized either a strength or a weakness of the system. We will also look at suggestions to the site given to us by the users and see which ones are feasible to implement on the site.

__Edeline__
//Determine overall goals of the evaluation// The primary goal of this test is to observe and evaluate the usability of the proposed design solution for the CCCA Canadian Art Database. We want to confirm which areas of the website are working successfully (or as intended) and discover areas that have errors or need improvement. Specific goals include the following:


 * **Efficiency:** We want to know whether the structure of the system is clear, accurate, appropriate, and coherent. We want to determine that the current design enables the users to get to the information they want with the least amount of time/clicks.
 * **User satisfaction:** We want to ensure that the identified needs for the website (along with the corresponding features) matches the user's actual needs. We want the website to be a useful tool that not only meets the user's intended purpose, but also one that is engaging and enhances their overall experience of interacting with the system.
 * **Learnability:** We want to check whether the website is successful in providing relevant information about artworks, artists, and their interrelationships; and whether or not those information/the presentation of those information (i.e. drag and drop) are easily accessed and understood by the user.


 * **Usability Goal for the "Curator"** || We want to determine whether the Contextualizer is the most intuitive way of finding interrelations between artworks or artists. We also want to know whether the users would find it relevant, helpful, or interesting, by either looking at it / considering clicking on it while performing the task. We want to test whether the users will prefer clicking on the Flickr-inspired "relevant artworks & artists" list, or whether or not they would look for metatags instead. Would they immediately consider the Contextualizer as metatags, or would having a brief description beside it take away from its tagging nature? ||

//Explore the questions that need to be answered//
 * Is the interface intuitive (enough)? Did we provide enough elements to hint them of certain functions available on the website? Did they use the drag-and-drop function with ease? How long did they take to figure out the drag-and-drop system (that you can drag artworks and timelines and do a comparison with more than two items)? Is it user-friendly enough, or were they intimidated by it?
 * Is this website an improvement to the old CCCA Canadian Art Database? Did it solve the (some/all) problems Bill Kirby highlighted with the previous system?
 * Is the structure of the website easily understandable? Is it clear, and does it make sense?
 * Do the users find the contextualizer interesting and/or helpful? Is it necessary?
 * What is their overall impression before and after visiting the website? Was their experience stressful, or did they find it enjoyable/pleasing? Would they visit the website again? Would they recommend the website to other people?

//Choose the evaluation paradigm and techniques to answer questions// As a way to effectively measure the progress towards our usability goals, we are to perform usability tests with 3 different people (i.e. curator/art history major, student, and a casual viewer/one who is not studying art but has some interest in it), particularly those we have specifically identified as target audience for our focus. All three participants are to be asked to perform the same sets of predefined tasks, and are to be evaluated in terms of (a) how different the behaviors/expectations of the audience from each other, (b) which features appeal to which audience, and (c) whether or not there is a pattern as to how successful/unsuccessful each of them performed specific tasks. Such test with a structured format (i.e. controlled) would give a more specific result that would be beneficial in directly identifying system flaws and in quantifying the website's capacity in meeting its intended purpose.

//Identify the practical issues that must be addressed// The testing environment should be in a quiet place that would make the participant(s) feel comfortable. There should be a computer or laptop available (preferrably Mac) and would have the browser that the prototype has been tested on prior the user-testing. One concern is the assumption that the participants are technology-savvy, or that they would be familiar in working in a Mac environment as opposed to PC's. The testing //might be// video recorded, if the participant allows it. At least two people from the group (us) must be present for the user testing to be valid: one would be talking to the participant while the other would be taking notes and observing.

//Decide how to deal with the ethical issues// Because the methodology includes testing with people, we must ensure that before the test, we ask/require the participants to sign a consent form that outlines the following keypoints:
 * their right to not participate, or terminate the participation at any time without prejudice
 * their right for anonymity and confidentiality
 * their permission and/or knowledge that they may be recorded on tape/video/email for the research
 * their right to ask questions, and to know who to contact (i.e. instructor) should they have further questions about the research or their rights
 * risks and benefits

//Evaluate, interpret and present the data// As mentioned in one of the previous points, having a critical eye for patterns is key to interpreting the data gathered. As a group, we will collectively evaluate the user testing experience: What could be improved? Did it match our expectations? Did we find anything surprising? Were our questions or tasks effective? How were the participants behaving differently, and what could have been the reason? After the evaluation, we will arrange the feedback by priority (i.e. which feature did most people have a problem with and discuss ways as to how we could improve it).

__Annmarie__

 * You are a student researching painters in Ontario. There is an artist with the initials P.A. that you would like to look up. Your task is to find his or her most recent work.
 * Your next task is to save the image and the entire timeline. Compare the two.

__Joanna__
You are a casual viewer who saw a painting (show painting) in a magazine and wish to know more about it by visiting the CCCA. You also am interested in looking for other pieces of artwork that is related. (say by the colour green)

__Edeline__
You are a curator for a small museum and are considering of putting together an exhibition that would include the piece //Theory on Separation// (1970). You are still unsure of which other artists you will include in this exhibition but are certain that the exhibition theme will relate to this piece in context. You decided to visit the CCCA Canadian Art Database to look for inspiration.

Your task is to find a potential theme for your exhibition and discover two other artists that you might look further into. You wish to have //Theory on Separation// (1970) bookmarked, as well as the two other artists' timelines so you can compare all three side-by-side.

When you feel you have completed this task, please say so.