Designer+User+Testing

Designer User Testing
Group 1: Jason, Steven, Kara

Project Brief:
We are third year students in the York/Sheridan Design Program enlisted in a course called Design Systems. Throughout this course was have been collaborating with the Contemporary Canadian Art Database (CCCA) in developing/designing a new system which they can organize their data and information. This group's focus was to represent information in more visual ways and have users explore the CCCA and find all the gems it contains. Since we are currently studying design, we felt it would be important to focus on the design section and create prototypes that can be used as a template for the other sections. The exploring aspect was an interesting approach we wanted to take, and by having many ways of exploring we felt this would make viewing, discovering, nagivating throughout the CCCA database a more enjoyable one. With your help you will be contributing valuable information which we can use to improve our system. In no way are we judging you, we are merely focused on the ways in which users will experience our design and with your input we will be sure to make the CCCA an enjoyable experience.

Final Stage
Refined Prototype

**Jason - Timeline**

**DECIDE**

 * Determine overall goals of the evaluation**
 * To see if users will understand how to navigate through the different methods of exploring/browsing
 * This is important because it will determine if this new design is a better way of organizing, finding and viewing information as compared to the current CCCA.


 * Explore the questions that need to be answered**
 * Are users enjoying this presentation of information?
 * Can they navigate through the system with ease?
 * Will Users be able to use the timeline/ understand how information is presented
 * Can users recognize the interactive elements in the exploring section > colour categorizing for time line.


 * Choose the evaluation paradigm and techniques to answer questions**
 * To properly test the site and the exploring aspect, users will be given tasks within each explore section
 * Specific tasks will give the users a goal to look for certain information and we will see if they way we present the information will help them in completing their goal.
 * Example: One Task : Find a work by Saturday Night in the 1970s


 * Identify the practical issues that must be addressed**
 * Describe the testing environment, equipment and participants ? Are there any possible concerns?
 * Testing environment should be a quiet space where I can properly evaluate the system.
 * It should make the user feel comfortable. Comfortable enough to share their thoughts about the system out loud.
 * Technology need will be a computer with a mouse and a browser program.
 * Participants > Ideally to have students/professors within the Fine Art Department because they are the ones using it. But Also open to casual users who would want to learn more about art/design.
 * Decide how to deal with the ethical issues**
 * There should not really be any ethical issues because this is an evaluation of the system.
 * Confidentiality
 * Having the option for the users to stop whenever they want
 * Evaluate, interpret and present the data**
 * Tracking the path the user takes with each given task.

Test Script
In your art history course, you professor has given you an assignment to relate designers from the 60-80's. You will be given specific piece which you will find using the CCCA database. Use the CCCA database to find the specific pieces as well as document who the designers were and what year each piece was made.

Tasks

 * Navigate throughout the CCCA
 * Using the explore options find a Saturday Night Poster made in the 1970's
 * Using the timeline find a religious work by Fred Peter
 * Using the timeline in the 80's find artwork Wilson Watson

Questions

 * Can you tell me your interpretation of the timeline?
 * Is there any confusion in the way information is presented?
 * what are some links/thing that you would click first?
 * Do you know where you are in the system?
 * Is there any other information you would like to see on this page?
 * How would you describe the way information is presented?
 * What are some things you would like to do which you could not currently do?

The Testing Procedure
The testing process was a new experience. I tried to keep it a relaxed setting and have the tester feel comfortable. I felt this was accomplished and that the testers were really open as to what they were thinking during the test as well as how they felt about the current prototype. Two of the tests were done on design students and one of the tests was done with a person interested in art who does not have a fine arts background. The test were enlightening, seeing how people would interpret the information was different than I expected. Challenges that occurred were parts of the system that were too calm or not clear enough like the main page. Often users were stuck, trying to figure out where to click. Another challenge was trying to get the users to the next point without telling them where to go. I would sometimes have to repeat the task so they would get a better idea of what they need to do. Surprises were how different users navigate through the system. Some users could find everything without any guidance, but others really searched for links and ways to find information. Example When users navigate through the decades.

Findings
Some feedback that I found interesting was that links in the current system were not obvious enough to be links. Users suggested to add some feedback or interactivity to links to better show these as clickable links. This was shown when users were stuck trying to find the explore link. After seeing it through the users eyes, it is almost hidden in the layout, even with the links colour, and position on the page. As far as the timeline pages, users clearly understood the colour coding of images in their sections. Users said the timeline was simple, but clear, they properly understood where the generations/years met. However, in the new prototype there will be more working images on the timeline. Some users wanted to click almost all the images. Also users did not understand the header, all three first clicked the header which was not functional.

Reflection
The process was enlightening. Seeing how people interpret our design, and the ways they would use a system/website. I think when we were developing the system, we designed it in a way which we would use the system (what made sense to us). But I think through the testing, things that may be obvious to us, might not be so obvious to others. I think the testing was useful is seeing that there are problems with our navigation which was one of big goals/requirements when we chose to take on the design section. I think the current system is to calm, and there needs to be more colour and ques for people to want to **EXPLORE** and find the artwork in the database.

=Steven - Designers=

//Determine overall goals of the evaluation.// In terms of the designers exploration, there are two main factors that influence the end user accessing the final data. Those are the visual design and the system structure. It is our goal for the user testing to root out both the small glitches in both these aspects, but also (and more importantly) to gain a view from a person who is outside the development of the project and a little more in touch with the end user that we are envisioning, in order that we may be able to see through their interactions, what overarching visual and structural pro's and con's there are to our approach.

//Explore the specific questions to be answered.// • Is the chronological/alphabetical approach to exploring designers helpful/useful or could there be other possibilities to exploring through them. • How has the experience of exploring through works and designers, opposed looking specifically? Is it easy to find the information? • Are the visual aspects of the site a hindrance or a help? Do they overwhelm the work? Are they appropriate for the target?

//Choose the evaluation paradigm and techniques to answer questions.// Because of the nature of our project focus being on the exploration of information, it would be ideal to have an unstructured task for the tester to be able to run through, in order that they would be able to give better feedback to us in terms of usability and where things work and don't work. But because we have limited schedules and abilities, we will most likely have a structured plan that will have the tester be running through specific tasks. But this will also be a benefit because it will allow the tester to better focus on critiquing the structure and design of the website.

//Identify the practical issues that must be addressed.// The testing environment should be a place where the user can focus on the tasks that we request of them, and a place where they feel comfortable (such as the computer, operating system, surrounding environment etc). Some concerns that we might have would be that the tester would give generic feedback that would seem common-sense and not the information that we would need to be able to further develop (and enhance) our system.

//Identify the practical issues that must be addressed.// Due to the nature of our project, I would not forsee any ethical issues surrounding the user testing. But the participant would be able to sellect their participation level based on these grounds.

• I have read and understood this form • The participation is voluntary, and hereby agree to participate in this project • Agreement to allow the student/s to use the research result in her/his/their course project (as stated above) both in print and online • I acknowledge that I have received a copy of this consent statement

1. You are a college student who has an essay based on this designer (_). He was born in the 1940's and you want to find information about him, and his work, as well as some examples of his work. Starting at the the Explore page, navigate through the link and search for this designer.
 * User Test Script**

2. You are interesting in finding more about design in the 1960's. Beginning at the Explore page, navigate through the links until you find a specific designer from that time period.

=Test Results=

=
Because of the nature of our interactive system having three different aspects and each of our group members in charge of a section, we found it fit to each find our own three candidates and conduct the testing this way. The candidate that I conducted the user test on were not the ideal targets that I had planned for in the beginning stages but I think were almost as suitable for the task. We have defined the targets for our system as a broad scope of people who would be interested in exploring and discovering more about Canadian designers. My three test users were all university students; one specializing in research, one in medicine and one in design. The tests themselves seemed like they went well in most respects. I was able to conduct the tests in environments where the user would have felt very comfortable. Most of the time that was on the users own computer, and in a relaxed environment. I think for this being the first time that I have conducted a user test using an official test script and having goals in mind, there were definitely things I can improve on for the next time, both in terms of the way the tests were conducted and conveyed and also having the system prepared for the user's tests. But overall I think they were a success. I happened to know all of the users personally which was a sort of cushion for the test routine and results, but I suppose that if I were to conduct user tests in the future, that I would not necessarily know the test subjects and the think that would change my approach a little bit.======

=
The two biggest things I have learned from these test results is that 1) people love the click things, 2) they love to do things fast. I realize that because I have been looking at the system for so long, and perhaps the fact that I designed it, means that I make many assumptions about the 'flow-ability' of a user going through the system. There are many aspects of the system that are not yet active such as buttons or images, and I have glazed over those, knowing that they are not active and not remembering that the user won't exactly know what to click on and where they are going.======

=
One user pointed out that in terms of the design, that the pages seemed sort of empty and sparse, and that there was not enough hierarchy (I sort of offered a few word suggestions). They said that the colors scheme was too flat, with not enough depth. Not necessarily more colors, just more contrast in order that they would be able to see the most important aspects first such as navigation and information. In terms of the sparseness of the layout, this user suggested compacting things a little bit more, so that their eye did not have to wander so much (which makes them tire more easily). I can see the accuracy of this critique practically worked out. I designed the system on my 24-inch desktop monitor, but the test users were using their smaller laptops and the design filled the page which in that context seemed too overwhelming. It was difficult to filter through what the user was relaying to me in terms of advice for an area that I already knew needed work and knew exactly what they meant. Such as the navigation bar not being active, and filler text in some boxes.======

=
I received much more insight into the visual design aspects of the site, versus the structural layout of the system. But there was some input in terms of the timeline that the designers slotted into. There was with all three users a confusion with the criteria with which the timeline was using. Everyone was confused by the timeline in terms of now knowing whether the dates shown were dates of birth, or dates where the designer was active or other areas.======

=
I can see that there is a lot of work that has to be done both in terms of the visual and the structural aspects of the system. I think the users test were successful in that they showed me areas that I need to improve the system. I had foreseen some of the suggestions and insights that the test users came up with, but there were some in areas that I had either looked over or not thought about, most likely because I have designed the systems and I was so used to it. In terms of what I need to do, is that most of the improvements need to be visual. The physical size of each page needs to be brought down so that it does not seem so light and airy, and in order that the user would be able to more easily see the hierarchy of each page. There needs to be some more contrast in the color scheme as well as compactness of information. The timeline criteria need to be more clearly defined and easier to navigate through. If I were to do another set of user tests, I think the improvement would need to be made on the system, by making it more ready for user tests. It would have helped to have more that one thread of working navigation, in order that the user would be able to do what the system is meant for; exploration.======

= = =Kara - Images=


 * 1. Determine overall goals of the evaluation**

For this test, there are two things I must focus on in order to properly format my thumbnail browsing page; one, being how a test subject would naturally browse throughout the thumbnail viewing portion of the site, without a defined goal (ex; without a specific designer/image in mind to arrive at in the end). Second, being to find out how the test subjects would naturally navigate their way through imagery, through only means of keywords, in order to find an image they were specifically looking for in the beginning. The thought process at this point is reliant on the imagery seen on the page.


 * 2. Explore the questions that need to be answered**

- What was their first reaction when seeing the main page for thumbnail browsing? (click on imagery, look for information, enter keyword, etc…) - Were they able to navigate quickly and easily? - Was there a need for explanation of how to use the thumbnail browsing page? - Was it confusing to navigate through?


 * 3. Choose the evaluation paradigm and techniques to answer questions**

Mostly using the idea of having a main goal to reach by the end of the test, this will help for us to understand the thought process of going from thousands of images, to focusing in on one specific image (titles, Designer names, categories, genres or dates will be provided to test subject).


 * 4. Identify the practical issues that must be addressed**

The environment in which the tester will be active must be a place where there are no distractions in which can alter or adjust the tester’s outcomes. For comfort purposes, the room should be large enough to the testers are not intimidated or distracted in any way by the observers (product developers = us). The tester must be able to focus on the task and hand, and complete it without interruptions. Main products for testing to occur are basic table and seating and fully operational computer.


 * 5. Decide how to deal with the ethical issues**

Seeing as how this is just a basic system test, there should be no encounters with ethical issues. Of course, the test subject has the right to leave the test at any time without question, and is free to let us know if they do in fact have any concerns. There is full confidentiality, and no information disclosed before, during or after the test will be discussed outside of the testing facilities.


 * 6. Evaluate, interpret and present the data**

In progress.


 * User Test Script:**

1. With some time to spare, you decide you want to learn more about design that appeals to you. Starting from scratch, use the Thumbnail browsing page to search the entire CCCA site for images in which appeal to you aesthetically. Verbalization of thought process is desired during the testing in order for us to get a better understanding of what can be done to improve the system.

2. A college of yours has asked you whether or not you knew more about a specific design piece they had found and grown a liking to. You offer to do a bit of research and gather information about it (the title, name of designer, year, etc…) Use the Thumbnail browsing page to narrow down the thumbnails to the one specific image you are looking for.


 * The Testing:**

Before continuing with the testing, I informed the test subject of the project being conducted. I let them know everything they had to in order to be relaxed and natural in the testing environment. I asked them to think out loud during their testing process; relay their thoughts at any given time to me, express any confusion or frustration, as well as point out anything they may find either hard to understand, difficult to navigate through or even what they find helpful and easy to use.

The overall environment was kept simple, with little to no distractions and all of the necessary equipment (a computer, chair, desk, etc.) in order to proceed with the testing. The test subjects felt comfortable and calm, and thus made the testing procedure smooth and easy. There were no disruptions with any of the three test subjects, and thus the results are unaffected by anything outside of the individual and the application.

The three individuals I used for the testing had no formal training in graphic design. One of which majoring in business, another in media information technology and the last was a university graduate with a background in psychology. Without the knowledge of how the CCCA website works currently, I was able to easily get solid information on what could be improved and changed in the application for easier navigation.

The actual testing showed some interesting results. Some of the users attempted to search through the site by constantly clicking text that showed some sort of hierarchy in size, colour and placement. By navigating through the site, they seemed to all agree that although they were confused in some areas of the overall application, the thumbnail exploring option seemed to be rather straightforward and easy to understand. When given the simple task of navigating through the thumbnail search section, all three of the testers automatically went to click on the thumbnails right off the bat. After explanation, they explained exactly what they would type in, and were pleased with the outcomes (both informationally and visually).

The second task was to narrow down a search from scratch to a specific image they were shown. The test subjects expressed verbally what they were looking at; keywords they were thinking as well as questioning which keywords may or may not work. One asked how the images would be connected to the words they typed in, and after a brief explanation of the ‘tagging’ system, they were excited to test it out with more images and designs.


 * Results:**

The results were basically similar, with minor differences between the individuals. The feedback was helpful and allowed me to re-think some of the design flaws in the application. One of the common suggestions was to, obviously, make the prototype more refined in order to give the tester a more realistic experience (this means to make the links actually clickable, the keyword search engine usable [currently being done], and to find a way to have the thumbnails actually generate according to keywords and phrases typed. I noticed without the actual interactive aspect of the system, they users tended to double-think and question how they were navigating through the site. I noticed, especially with one, that the way in which they would attempt to search for the given image was constantly doubted due to the prior knowledge (through exploring themselves) of what ‘links’ and features worked.

The users all agreed that the pages were easy to navigate through, and the headers and buttons for the thumbnail explore engine were very clear and easily recognized as clickable and not clickable.


 * My Thoughts:**

Overall, the whole testing process ran very smoothly, with no great difficulty on the users behalf, and a clear understanding of the user’s expectations for the testing. There was a clear line of communication, which made it easier for me to understand how other users will use the site, and allowed me to brainstorm ways in which I can improve it. I noticed there was a common agreement that the thumbnails were a very interesting and simple way to navigate, as well as visually appealing to the user. With the visuals being the main focus, the testers became interested in the site, wanting to do more and continue the searches with other images. The colours of the text made the overall process a little more ‘fun’ and kept interest without being distracting. I think if we can maintain a fun experience for the user, the site will obviously become a higher-traffic page. My main focus is to have the user think of the CCCA website as a reliable, easy-to-use and effort-free experience that they will use as a primary source of design and art information (much how wikipedia is used). By designing the site to be informative, interesting and appealing, we will be able to attract not only individuals with art and design knowledge/background/education, but as well as anybody just interested in learning something new.