jpast

User testing

> The main goal of this user testing is to determine the suitability of the system for the purposes of organizing an online portfolio for people with little or no IT knowledge. The secondary goal is to find out if the system's interactivity is comfortable and intuitive enough for the general public who is going to visit the web site.
 * GOALS**

> Part 1: ARTIST PORTFOLIO > Part 2: GENERAL USER EXPERIENCE
 * Questions to be answered**
 * 1) Is the logistics of the system clear and understandable? (no questions such as 'how does this work?'')
 * 2) Are the specific tools (post message, upload picture) are clearly represented, recognized by the user and work flawlessly?
 * 3) How much time does a user need to understand the system and the interface? (learning)
 * 4) Is the layout working well, and users can use the system quickly knowing the principles? (ie. Buttons are labeled to do what they do and placed in the expected place.)
 * 1) How easy is it fo the user to get to the desired information?
 * 2) Is the layout working well, and users can use the system quickly, without questioning?

Evaluation techniques


 * For the portfolio part of the test the user will be given a list of simple tasks, such as: add user information, upload the work and document it, post a message, edit the information.
 * For the general user test, participants will be given a task of finding a block of information; for example artists that are connected to some network; This task is broader and more interesting then finding information of a single artwork or artist. Additionally, some time should be given for free exploration in order to allow the completely unexpected bugs to come up.

Concerns and considerations


 * This test is designed for people without broad or any IT knowledge; therefore, the task instructions should be clean and explicitly explained. This is a system test, there is no need to confuse the user by providing vague and unexplained tasks. (i.e "Add a picture BLABLA.jpg located on your desktop" instead of "add a picture" - which was cause frustration as user might ask " what picture should i upload? where do i get the picture?" etc.)

Ethical Considerations


 * Use the approved York University consent form.
 * Make the participants comfortable.
 * Make them understand that THEY test the system not, the other way around. If something does not work it is not their but our fault.
 * Encourage criticism.

Data analysis


 * The the participant results results should be compared to the benchmark - a user test run with the designer of the system who knows perfectly well what to do. This will help to determine the problematic places where time differs much, as well as system parts that work well.
 * Any remarks and comment made by the participants should be take notes off.
 * Ass an addition a simple question " Well, what do you think?" that might bring out absolutely unexpected issues.


 * THE SCRIPT**

In my user testing I am going to focus on basic information input and editing in artist's profile.


 * Scenario 1.** You are an artist that have received an invitation to participate and exhibit your work on CCCA. Your profile has been created, and the password was given to you. Your task is to edit your personal information into the profile section. For this part of the test you may enter any information you want, it does not have to be actual or make sense. Please write a couple of sentenses. This test is aimed to find how comfortable is the user interface in the profile section.


 * Scenario 2**. Some time has passed after you entered your information, now it is time to update your profile. The task is to log in and effectively change the previously entered information in the profile section. As the last time, the information entered does not have to be actual. This part of the test will investigate how easy it is to make changes in the CCCA profile.

Test Report

The test procedure The testers met with the developers on Thursday, May 7, at 4 30 pm. The test procedure was explained to the testers, who in their turn signed the consent forms. It was explained that the purpose of this test was to determine the usability of the system, that the videos recorded are for the analysis purposes only and that they would provide an invaluable insight for the improving of the existing system. Test participants felt relaxed and comfortable. Some time was taken in order to set up the cameras, load the film and explain the test scenarios.

At around 4 50 pm the first participant Domas run Eugene's profile test. Everything went smoothly; after he was done, the second participant Marshall run Tracy's tests, followed by Eugene's tests. After that the first participant returned completed Tracy's tests. During the test the developers stayed with the testers. While developers did not direct the testers through the system, they helped the participants to stay relaxed and comfortable, in order to avoid any frustration. This way, all participants run all tests by all group members. The testing was complete at around 5 30 pm; testers were thanked for their time and invaluable input.

Test findings During the testing it was found that the website has to undergo minor visual structure changes to make some parts more defined and understandable. The firsts participant Domas provided a good insight into the problems that the system has, as he had only basic internet knowledge and mainstream application experience. The second participant Marshall had much more experience and demonstrated usability for the more advanced user. It was determined that the system can be learned quickly and, once learned, used comfortably and fast. On the other hand it might present some difficulties and confusion for the first time user, especially if s/he does not have an advanced internet system's knowledge

Accessing the profile page proved to be inconvenient, as participants did not manage to locate the needed button quickly due to its placement. When the profile page was accessed, everything went smoothly for both participants. During Tracy's test it was found that the posting interface has to undergo some changes as well, as the participants slowed down trying to find the image upload button and the title field. Finalizing steps for both Eugene's and Tracy's steps proved to be not intuitive, as page had to be updated by pressing the 'update button' that was not sufficiently defined too suggest the participants that it had to be pressed as a final step. The more complete record of the test may be found further in this paper.

Overly, it turned out that the web site has to undergo minor changes in order for the user to go through it more smoothly. As the interface was built on Wordpress, it may be difficult to change the content creating end of the Web site, but is possible through the modifying the Wordpres php files.

Reflection The tests went smoothly and relaxed. There were no difficulties for the participants and the developments in understanding each other. Both parties showed their understanding and willing to help. The testers helped to point out a series of flaws, and gave the developers something to work with. There were some difficulties with the screen capture during the tests, as developers were inexperienced with the filming equipment provided by the school. The tests instructions proved to be good, as testers had no trouble understanding and following them. The atmosphere was friendly and relaxed, and both testers and developers sincerely enjoyed their time. For future experiences, the developers have learned that every aspect of the test should by pre-tested by the test organizers themselves in order to avoid difficulties during the test itself. Even such apparently trivial procedure as filming the test might turn to be problematic if the hardware is unfamiliar.