G4Phase3

INDEX: ENVISION | PROTOTYPING | USER TESTING | REFINED PROTOTYPE Artists' Online Portfolio 

A USER-CENTERED INVESTIGATION EUGENE, SAMMY & TRACY

 **__PHASE THREE__** USER TESTING 



 __Project Brief __ We are students from the York University Sheridan College Joint Program in Design who are working within the area of systems design. We are currently working on the potential implementation of a new interaction system for the Center for Contemporary Canadian Art Database, which is a website and database that hosts the oeuvre of works of hundreds of major contributors to the world of Canadian art. The users of this site range from students, researchers, curators and artists.

We hope to implement a system which will enable the uploading, editing and managing one’s own web space on this database. We hope that this experience will not only be effectual but also efficient and worthwhile. And so we would like to solicit your brief involvement with our team, so as to bring to light the flaws or locations of underdevelopment that we may have failed to notice from our singular point of view. We want to see if this web-editing implementation is indeed helpful. In no way are we testing your abilities in using our webpage, but are rather looking for ways to better suit its capabilities to users such as yourself.

We would like to urge you to speak your thoughts out loud while using the system. This will provide us with a real-time experience of someone using our system. You remain the right to aboard this user-testing at any point.



Decide Framework

__**TRACY **__

Does the idea of being able to edit your own content easily and effectively seem worthwhile? Is this a good alternative to making it out of HTML? On a scale of 1-10, how will you rate the
 * Determine the overall goals of the evaluation**

a) The learn-ability of our material?

b) The suitability of this editing system to the needs of an artist?

With this user testing we would like to find out how difficult it is to grasp how different elements on the website are mapped out on the wordpress interface, and if we need to re-skin/re-map some of these elements according to the needs of the users.


 * Explore the questions that need to be answered**
 * How difficult is it to grasp that both the news section and the artwork section share the same output area on wordpress?
 * Do you find that you need to be able to update your own C.V or Biography?
 * Will you be willing to take some time to better understand how Wordpress works?

_ A big part of our system is its ability to allow the user to engage in the backbone developing process. In order to understand if the presented material can entice the user to begin a user-relationship with it, we will first have an session of unguided, unstructured roaming throughout both the wordpress interface as well as the artist’s front-end website. We will then invite the user to present any questions they have while using the interface. The second and subsequent sessions will involve several structured task-oriented testings,
 * Choose the evaluation paradigm and techniques to answer questions**

The testing will take place in LAB 3 where the user will not feel intimidated to speak out any concerns he or she will encounter during the testing. We will also run a video recorder that will record the screen so as to serve both as a documentary of the process and as a reference for us after the fact.

_ Our group will provide the subject with a media release form. In addition we have prepared a consent form outlining several key facts: their right to aboard the user testing at any point they may raise questions at any point they may be asked questions their actions on screen will be recorded their concent to be contacted via email
 * Identify the practical issues that must be addressed**

_ Because the user testing will have an audio recording component, our group will provide the subject with a media release form. In addition we have prepared a consent form outlining several key points: _ The video recording will be replayed and analyzed. During the session the tester will remain close by the subject, answering questions raised and will take brief notes.  **
 * Decide upon the ethical issues**
 * their right to aboard the user testing
 * at any point they may raise questions
 * at any point they may be asked questions
 * their actions on screen will be recorded
 * their concent to be contacted via email
 * Evaluate, interpret and present the data**

Eugene **

 **GOALS** > The main goal of this user testing is to determine the suitability of the system for the purposes of organizing an online portfolio for people with little or no IT knowledge. The secondary goal is to find out if the system's interactivity is comfortable and intuitive enough for the general public who is going to visit the web site.

> Part 1: ARTIST PORTFOLIO > Part 2: GENERAL USER EXPERIENCE
 * Questions to be answered**
 * 1) Is the logistics of the system clear and understandable? (no questions such as 'how does this work?'')
 * 2) Are the specific tools (post message, upload picture) are clearly represented, recognized by the user and work flawlessly?
 * 3) How much time does a user need to understand the system and the interface? (learning)
 * 4) Is the layout working well, and users can use the system quickly knowing the principles? (ie. Buttons are labeled to do what they do and placed in the expected place.)
 * 1) How easy is it fo the user to get to the desired information?
 * 2) Is the layout working well, and users can use the system quickly, without questioning?


 * Evaluation techniques**


 * For the portfolio part of the test the user will be given a list of simple tasks, such as: add user information, upload the work and document it, post a message, edit the information.
 * For the general user test, participants will be given a task of finding a block of information; for example artists that are connected to some network; This task is broader and more interesting then finding information of a single artwork or artist. Additionally, some time should be given for free exploration in order to allow the completely unexpected bugs to come up.


 * Concerns and considerations**


 * This test is designed for people without broad or any IT knowledge; therefore, the task instructions should be clean and explicitly explained. This is a system test, there is no need to confuse the user by providing vague and unexplained tasks. (i.e "Add a picture BLABLA.jpg located on your desktop" instead of "add a picture" - which was cause frustration as user might ask " what picture should i upload? where do i get the picture?" etc.)


 * Ethical Considerations**


 * Use the approved York University consent form.
 * Make the participants comfortable.
 * Make them understand that THEY test the system not, the other way around. If something does not work it is not their but our fault.
 * Encourage criticism.


 * Data analysis**


 * The the participant results results should be compared to the benchmark - a user test run with the designer of the system who knows perfectly well what to do. This will help to determine the problematic places where time differs much, as well as system parts that work well.
 * Any remarks and comment made by the participants should be take notes off.
 * Ass an addition a simple question " Well, what do you think?" that might bring out absolutely unexpected issues.

**Sammy **

 //**What are the goals of this evaluation?**// > The goal of this evaluation is to see whether the website would appeal to individual who may have an interest in the Canadian art scene, who normally might not. Specifically, does the website create some kind of awareness of art within Canada. Does the structure of the website navigable so as to present the artwork that desires to be seen?

//**What questions will be considered or asked for this evaluation?**//
 * Does the website appeal to you?
 * What did you learn?
 * Is the interface/design of the website good?
 * What is the overall feeling you get from observing and experiencing the CCCA website?
 * Did you detect any glitches/problems?
 * Was there anything that you found annoying?
 * Judging from this site, can one grasp what Canadian art is? And how do you define its style?

//**What techniques will be implemented during this evaluation?**//
 * User will test structure/interface/navigation of the website over a duration of 20 minutes.
 * User will then be interviewed and will be asked questions.
 * Tester will monitor the subject and how their approach to the function of the website.

//**What concerns must be considered for this test?**//
 * Is the website navigable? Is the layout pleasing?
 * Does the overall design work well with the function and operation of the website?
 * What was the main reason to include specific galleries/artworks on the website?

//**What are the ethical considerations for this test?**//
 * Subject will sign a testing consent form.
 * Subject will not be under any obligation to complete.
 * Subject will not be under any pressure.
 * Subject will be safe and not be under any harm or danger.

//**How will the data collected be used and analyzed?**// <span style="display: block; text-align: center; font-family: 'Courier New',Courier,monospace; font-size: 200%;">
 * Subject will remain seated in front of computer which displays the CCCA prototype website through the duration of the test.
 * Answers will be interpreted to better augment the CCCA website for the better.



__<span style="display: block; text-align: center; font-family: 'Courier New',Courier,monospace; font-size: 200%;">CASE SCENARIOS __

<span style="display: block; font-size: 200%; font-family: Arial,Helvetica,sans-serif; text-align: center;"> We have mapped out wordpress onto a custom skin. Our scenarios focus not so much on whether or not the skin works as a portfolio but on the way the user can interact with customizable elements of their page through an abstract engine.
 * <span style="text-align: left; display: block; font-size: 200%; font-family: Arial,Helvetica,sans-serif;">TRACY & EUGENE **

You are an artist who is currently a member of the CCCA art database. You would like to update your C.V due to the fact that you have recently exhibited in three gallery spaces in Toronto. Now, go on your website log-in and begin the procedure of editing your C.V.
 * SCENARIO 1**

You would like to post up a new painting series. You have digital pictures of the new artwork and you would like to upload all of them. You also have the title of the series as well as a written explanation of your concepts, and you would like to have all of these appear from one link on the main page.
 * SCENARIO 2**

You feel like as an artist you would like to post a new statement about the kind of work that you do and why you do it. You would like this to be listed out on the top, along side your biography and your exhibitions. Please go about finding out how to do that on your own. Please don’t hesitate to mess around with wordpress.
 * SCENARIO 3**

You are an artist currently a member of the CCCA art database. You are an installation artist who also occationally does sculptural work. You find out that the categories in the default medium categories that CCCA has provided for you does not match your areas of interest. Please go about finding out how to do so at your own pace. You are an artist that have received an invitation to participate and exhibit your work on CCCA. Your profile has been created, and the password was given to you. Your task is to edit your personal information into the profile section. For this part of the test you may enter any information you want, it does not have to be actual or make sense. Please write a couple of sentenses. This test is aimed to find how comfortable is the user interface in the profile section.
 * SCENARIO 4**
 * SCENARIO 5**

Some time has passed after you entered your information, now it is time to update your profile. The task is to log in and effectively change the previously entered information in the profile section. As the last time, the information entered does not have to be actual. This part of the test will investigate how easy it is to make changes in the CCCA profile.
 * SCENARIO 6**

As you see when you scroll down, the site also features a News section. You are an artist who has just finished reading a book and would like to post about the inspiration you have thus drawn from this new book. Please post a new entry into the Blog section at your own pace.
 * SCENARIO 7**

**<span style="display: block; font-size: 200%; font-family: 'Courier New',Courier,monospace; text-align: left;">Sammy **

You are new in Toronto, and you wish to explore the all the art and culture that this city has to offer. You decide to log onto the artists' website to engage in their daily activities and to find out more about whats going on (ie. exhibition, new artworks, etc.).
 * SCENARIO 1**

You are an art student who wishes to be inspired by the art of others. You decide to search for art from British Columbia. The wood-working art pieces really appeal to you and you spend about an hour or so looking through the gallery. You start to envision something and become inspired by the pieces to create something of your own through wooden art.
 * SCENARIO 2**

You are an artist who owns a CCCA profile page. You decide to upload photos taken from a photography session in the city. You then upload your best pieces in the photography section of the gallery, to showcase your work to others. Captions are also added for more info about that particular photograph.
 * SCENARIO 3**



<span style="display: block; text-align: center; font-family: 'Courier New',Courier,monospace; font-size: 200%;">USER TEST REPORT <span style="display: block; font-size: 200%; font-family: Arial,Helvetica,sans-serif; text-align: center;">

**<span style="text-align: left; display: block; font-size: 200%; font-family: Arial,Helvetica,sans-serif;">TRACY **


 * THE TESTING PROCEDURE**

The group members each organized a timeslot for a suitable candidate. The users were seated with a tripod and a camcorder behind them. We notified them that we have consent forms for them and announced that they will not be in the video personally, that only the screen is recorded for the purpose of reviewing the process at real-time. This made them more comfortable. The environment was 4:30 in the afternoon and there were several groups of people bustling about in Lab 3. The lights were off, it was a partly cloudly day but the blinds were up and daylight filled the room. The place had a lively and steady room tone, but it was not offensively noisy. Eugene and I had originally felt that this was not the sleek, ideal environment for user-testing, but the users felt that this was a relaxed environment in which they are not self-conscious of their own voice. At that point we began setting up the equipment while the users introduced themselves to each other and chatter briefly. We each sat with our respective candidates. My user and I were close friends, so we took on a light-hearted approach and created an environment where everything is said in hyperboles. I listed out of ethical issues, read to him the test script, and described his role as a would-be artist member of the CCCA. I encouraged this person to speak out his thoughts as he went along, which was no problem as it seems to be a constant habit of his to talk a lot at any given time. This role-playing actually helped in allowing the user to voice out his complete opinion, because it was his own personal opinion disguised in the form of the opinion of an imaginary person.

People love scrolling up and down when looking for things!
 * FINDINGS**

__On Learnability__ There were no hesitations on his part in voicing out his dislike for certain utilities and particularly for choices in aesthetics. Many of them were very interesting and had over time become banal for Eugene and I because we had been staring at it for so long. Many of the criticism was actually geared towards Wordpress. Although we felt slightly helpless, we realized a repeated pattern where although it was very difficult for them to grasp the location of some of the vital buttons, or they bemoaned the innumerable functionalities, they did not notice that after one single trial the next time they come back to it seems like second nature. As noted by Eugene, Wordpress is not without its own share of usability issues but the learning curve of is extremely steep. The entire platform is rooted upon similar repeated gestures and actions.

__On Prototyping on an Existing Platfom__ What differed our group from other groups was that we chose an existing platform on which to run our prototype. The benefit of this seems clear: we were able to create various situations where everything worked, there were no “pretend” functionalities, empty buttons or broken links. And so we had no problems when dealing with issues like continuity. The biggest challenge was, on the other hand, that there were too many capabilities and too many fields of possible data input. This was an observation of the users themselves during the testing: “there are so many fields. Do I need to fill in all of them?” We noticed immediately that the candidates felt like they had to fill in every form even though most of them were not yet “mapped” onto the skin.

__On Mapping and Interfacing__ Contrary to our initial thoughts on this matter, the users seem to understand the fact that what they were working with was essentially the back-end interface. The nature of the exercise, combined with the regular user-name-and-password routine seems to trigger an administrative mentality when dealing with the web. My user recognized immediately that the forms and checkboxes that he was working with affected the look of the page – the fact remained that they needed to find the specific button where they can preview the site; but as stated earlier any difficulties in way-finding dissolved as the user became more accustomed to the engine they were working with. The most difficult point to grasp was, as predicted, the use of Categories and Sub-categories for the separation between Artwork and Updates publishing. For one, there is a sort of inconsistency between data input and output. The subcategory of Artwork pertained to the medium of the artpiece, while the subcategory of Updates corresponded with different columns on the skin in addition to be independent categories like “Blog”, “Exhibitions” and “Current Project”. One category published little thumbnails, while the other doesn’t; one category published at the top while the other published at the bottom. The heart of the problem lies upon the decision of the developers in their way of “skinning” Wordpress. Wordpress functions merely as a data-organizing platform. It offers different methods to accomplish the same goals. One has to determine how he or she wants what type of data to be posted, where such said data will appear and how to go about mapping one thing to another. I had experience only with making a wordpress skin for my own use, so I knew exactly how to achieve the kind of look I want by working around inputting certain parameters for the data package. Working for users in mind is the direct opposite.

This process was actually quite funny and enjoyable. I had a lot of trouble putting the DECIDE Framework to use. In the end I had so much trouble in trying to fulfill in the presusmed prerequisites that I decided to disregard the whole thing and submerge myself in the experience. I thought: “Alright, I made something and I really just want to know if the thing works, so I decided to merely trust my intuition.” For me it was a spontaneous on-the-spot experience: only when I disregarded all of the restrictions that I had “predicted”, everything I had to “watch out for”, and ignoring everything I posted on the wiki previously (I created moments of tension by searching frantically on the wiki trying to find the case scenarios I posted), was I able to truly engage with both the user, his mentality and the consequences of his actions. In the end one realizes that you learn a lot even by watching someone use something you’ve created with a certain conviction for its usability. Coming up with questions to ask the user on the spot proved most efficient. Preoccupations with things like recording the actions with a camcorder, administering tasks according to what you wrote down days ago one after another only hinders the creative collaborative process and creates a stale environment where the only point of the exercise is to complete it for homework. In the end the four of us got along and had brief moments of fun. In a lot of the silliness some genuine points were taken.
 * SELF-REFLECTION**

<span style="display: block; text-align: center; font-family: 'Courier New',Courier,monospace;">
 * <span style="display: block; font-size: 200%; font-family: 'Courier New',Courier,monospace; text-align: left;">Sammy **

//**TESTING PROCEDURE**//

The test subject selected for the User Testing procedure of the website prototype was a man in his late 60s whom had good working knowledge of computers, and web browsing. The subject was seated at a desk with a laptop placed in front of him. He was then asked to browse the CCCA website prototype for about one (1) minute, and was then interviewed. He was asked a series of questions of which most were answered. The entire duration took approximately twenty (20) minutes and the audio was recorded on another laptop placed next to him. The group member sat right next to him with a notebook in hand.

//**Findings**// The subject was very fond of the website and found many aspects of it interesting and useful (text and images).

__//Aesthetics//__ Subject noted that the images seen on the Home page were of interest and were eye-catching. He noted that the artwork on the pages had a lot of deep philosophical meaning. The imagination and skill of the webdesigner(s) was captivating and lead to some interesting observations of the artwork. The artwork was perhaps more observed than the actual design of the interface. He also noted that the website was for a particular audience and whom would find the layout of the website interesting, as opposed to "ordinary" people who generally do not surf art/design websites.

//__Function__// Subject was observed to have keen interest in the navigation of the website, specifically from the menu placed at the top. The galleries were well-displayed and organized. Upon showcasing the thickbox function used for the displaying of images, he noted that they were of benefit in the viewing of the portfolio pieces. User also made some humorous remarks about the text and titles seen throughout the prototype (ie. titles given to artworks, dummy text, etc.).

//__Content__// The images which displayed the artists' artworks were of interest to the user. The subject was able to browse through the images, much like the same way in a real art gallery. He critiqued some of the artworks and made comments about what they were (ie. what they looked like) and the meanings that encompassed them. It was observed that some of the artworks were a little bit bizarre and pondered as to why they were like that. He remained on the optimistic side and observed that there weren't any problems with the content or the website in general. On the topic of Canadian art, it remains a bit blurred. Comparison must be made when looking at art from various countries to display contrast and uniqueness. Nonetheless, the user and I had some fun looking at the conceptuals and visuals displayed on the prototype.

__//Reflection//__ In the beginning, I was not sure how to go upon with the User Testing part (Phase 3) of this project. However, I felt that it was necessary to have various point-of-views when looking at this website so as to augment it for the particular target group. What struck me the most was perhaps not the artwork displayed on the prototype, but the architecture and interface of the website. It was well-crafted, sharp, and eclectic. It was not necessarily just a display of artwork, but the website itself was a mode of communication that also captivated the eyes. What must be noted was the crisp display and organization of the text that created a sense of being "in-touch" with the Canadian art scene.

In terms of the user testing, I felt that my opinions of the prototype differed greatly from that of the subject chosen for this test. His enthusiasm for interface and computer knowledge was what got the test off to a good start. I had written down my questions prior to the interview and were asked after one minute of viewing of the website. During the interview, the subject made some interesting philosophical statements on the random (dummy) text found on the prototype. While the testing may not have been as serious as it should have been, I thoroughly enjoyed the interview and working with the subject.