Assignment 3: Getting the Design Right

Overview

Now that you have chosen a design, it is time to iteratively refine that design. This group assignment, spanning multiple weeks of the course, tackles the problem of getting the design right through paper prototyping, inspection, and user testing. Design refinement culminates in a digital mockup.

This assignment is worth 14% of your overall course grade:

The content of the associated presentation is worth 5% of your overall course grade.

Your delivery of the associated presentation is worth 2% of your overall course grade.

Milestones

This is a group assignment, consisting of five milestones.

Project Description

In this assignment, you will iteratively improve your design. You will construct a paper prototype informed by your earlier work. You will then use inspection to identify and correct issues in the paper prototype. You will next conduct a series of three usability tests, further identifying and correcting issues in your paper prototype. You will create a digital mockup that captures the result of your design process. Finally, you will present your design process in a report and presentation.

  1. Develop a paper prototype for your design. It should convey all critical aspects of your design, including support for testing your two primary tasks.

    This prototype has been informed by your prior sketching and storyboarding. But it also contains more details than your prior exploratory designs. If necessary or appropriate, revise your tasks or scenarios based on what your learn in prototyping.

    (Assignment 3a)

  2. Conduct a heuristic evaluation of your paper prototype. You will perform an evaluation in class, and can supplement this with your own evaluation.

    Revise your paper prototype based on what you learn in heuristic evaluation. Your tasks and scenarios are likely stable, but revise them if your evaluation reveals a need.

    (Assignment 3b)

  3. Conduct usability tests with three participants, each completing your two primary tasks.

    Make a log of critical incidents during the test, including both positive and negative events. For example, a person might make a mistake or might comment on something they like. Write it down along with a description of the context of the incident. Later you will discuss and prioritize these events, assigning a severity rating to problems (ranging from 0 for no problem to 4 for usability catastrophe).

    You should not use friends, members of the class, or people you know to have engineering backgrounds. You also should not use people who have been previously exposed to your project. Target an appropriate set of participants based on your contextual inquiry and resulting design.

    Be sure to divide your team into roles when conducting the usability test. There is simply too much for a single person to conduct a test. As discussed in class, these roles include the facilitator, the computer, and one or more observers.

    As part of your introduction, have your facilitator show the participant how to interact with your design. Do not show participants how to perform your tasks you will later test. Instead show them how paper prototyping works, how your system generally works, and give an example of something specific that is sufficiently different from your tasks.

    Consider recording your tests to allow you to see exactly what happened. If you do this, ensure your recording preserves the anonymity of the participant. Frame your shot such that you capture interaction with the prototype without unnecessarily capturing the participant.

    Each participant should perform both tasks. Keep the data separate for each participant and each task. Keep participant names confidential, and preserve anonymity in your reporting. Remember participation must be voluntary, participants should be free to stop at any time, and you should emphasize that your focus is on identifying usability issues in the design.

    Revise your paper prototype based on what you learn in usability testing. This can and should include fixing major issues as soon as they are detected, before additional tests. Your tasks and scenarios are likely stable, but revise them if your evaluation reveals a need.

    (Assignment 3d), (Assignment 3e)

  4. After refining your paper prototype based on your usability testing, construct a digital mockup of your design.

    Do not begin working in digital prototyping until you finish revising your paper prototype. It is much easier to iterate on your design in the paper prototyping phase. As you transition into digital prototyping, you will consider other aspects of the design. These details you may not have considered (e.g., visual design, layout, color, typography), but can also include constraints or mismatches introduced by the technology itself.

    We have no strong preference to what digital tools you utilize to create your mockup. But it must be digital (e.g., you may not use pictures of a hand-drawn sketch).

    Some recommendations from the course staff include:

    You will use this digital mockup to effectively communicate your design in your final report, website, and poster. Some tools support creation of interactive mockups (e.g., allow clicking active regions to move between screens). This would add additional capabilities to your website, and you are welcome to use such a tool. But interactive mockups are neither expected nor required.

    Implement your digital mockup to include key frames that illustrate your design. It should convey all critical aspects of your design, including walkthroughs for your two primary tasks.

    (Assignment 3e)

  5. Summarize your design process in a report and presentation.

    (Assignment 3f), (Presentation)

Deliverables

3a: Paper Prototype

Due: Uploaded the night before class Tuesday, November 10, 2015.

Develop a paper prototype for your design. It should convey all critical aspects of your design, including support for testing your two primary tasks.

Submit detailed images of your paper prototype. Include both:

  • An overview image that shows your entire paper prototype.
  • Detailed images showing each component as you walk through your two primary tasks.

Examples from Prior Offerings

Examples from prior offerings include:

Submission

No more than one page of text in PDF format.

Images do not count against your page limit, and are therefore effectively free. You should embed images throughout your PDF, keeping them near the text that references them. The limit applies to the approximate amount of text you would have if all images were removed.

Submit via Canvas here:

https://canvas.uw.edu/courses/987987/assignments/3035453

Bring your prototype to class. You will need it to conduct Heuristic Evaluation.

Grading

This milestone will be graded on a scale of 9 points:

  1. Overall Completeness and Appropriateness of Paper Prototype: (3 points)
  2. Completeness, Appropriateness, and Documentation of Each Task: (3 points per task)

3b: Heuristic Evaluations

Due: Conducted in class Tuesday, November 10, 2015.

Conduct a heuristic evaluation of your paper prototype. You will not be able to complete this if you do not bring your prototype to class on Tuesday, November 10, 2015.

Each team will split in half to conduct heuristic evaluations, pairing with another half-team from a different project. For 15 minutes, one half-team will conduct a heuristic evaluation of the other’s paper prototype. After 15 minutes, groups will reverse roles. Each half-team will therefore conduct and receive one heuristic evaluation. Each team will therefore receive two evaluations of their prototype.

We will provide a worksheet with Nielsen’s heuristics:

3b-nielsen-heuristics.pdf

Evaluators will use index cards to record issues identified in evaluation. Include what heuristic is violated, and a severity rating (ranging from 0 for no problem to 4 for usability catastrophe).

Revise your paper prototype based on what you learn in heuristic evaluation. Your tasks and scenarios are likely stable, but revise them if your evaluation reveals a need.

Submission

The evaluated project will keep the created index cards, so they can use them to improve their design. The evaluated project should be careful not to lose these, as their contents will be submitted in the next assignment.

After class, submit images of your notes in PDF format. Submit via Canvas here:

https://canvas.uw.edu/courses/987987/assignments/3035454

Grading

This milestone will be graded on a scale of 4 points:

  1. Participation of Each Half-Team: (2 points per half-team)

3c: Usability Testing Check-In

Due: Uploaded the night before section Friday, November 13, 2015.

You received two heuristic evaluations of your paper prototype. If desired, you may also conduct a heuristic evaluation or cognitive walkthrough for your own paper prototype. Also complete one usability test with your paper prototype.

Present your iterative revisions and your plan going forward:

  • Present a table or list of results from your inspection-based methods. For each identified issue, include:
    • An image of the relevant portion of the prototype.
    • The identified issue, including the heuristic that is violated or the believability issue identified in walkthrough.
    • The severity of the issue assigned by the team.
    • An image and explanation of any revision implemented as a result.

  • Discuss your first usability test. Provide a description of the test, including the participant, the environment, why you chose this participant and environment, the test protocol, and the roles of each team member who participated in the test. Discuss anything you learned about the testing process itself, or any revisions you decided you need to make to your testing process.

  • Present a table or list of results from your first usability test. For each critical incident, include:
    • An image of the relevant portion of the prototype.
    • Description of the incident, remembering that these can be either positive or negative.
    • For negative incidents, the severity of the issue assigned by the team.
    • For negative incidents, an image and explanation of any revision implemented as a result.
  • Submit detailed images of your current paper prototype, as revised in this process. Include both:
    • An overview image that shows your entire paper prototype.
    • Detailed images showing each component as you walk through your two primary tasks.
  • Present your plan for the remainder of your usability tests. Describe who you will target for participation, what goals you have for the additional tests, planned roles for each team member, and any new approaches you will employ.

Examples from Prior Offerings

Examples from prior offerings include:

Submission

No more than three pages of text in PDF format.

Images do not count against your page limit, and are therefore effectively free. You should embed images throughout your PDF, keeping them near the text that references them. The limit applies to the approximate amount of text you would have if all images were removed.

Submit via Canvas here:

https://canvas.uw.edu/courses/987987/assignments/3040077

In section, be prepared to discuss your iterative revision of your paper prototype. Also bring your prototype to section. You will need it to demonstrate for critique.

Grading

This milestone will be graded on a scale of 12 points:

  1. Issues Identified from Inspection, Revisions Made in Response: (3 points)
  2. Discussion of Usability Test and Plan: (3 points)
  3. Issues Identified in Usability Test, Revisions Made in Response: (3 points)
  4. Updated Images of Paper Prototype: (3 points)

3d: Usability Testing Review

Due: Uploaded the night before Tuesday, November 17, 2015.

Complete your three usability tests and finalize your paper prototype according to what you learn.

Present the results of your usability testing and your final paper prototype:

  • Discuss your three usability tests. Provide a description of each test, including the participant, the environment, why you chose this participant and environment, the test protocol, and the roles of each team member who participated in the test. Briefly discuss revisions you made in your process and prototype over the course of your testing.

  • Present a table or list of results from your three usability tests. For each critical incident, include:
    • An image of the relevant portion of the prototype.
    • Description of the incident, remembering that these can be either positive or negative.
    • For negative incidents, the severity of the issue assigned by the team.
    • For negative incidents, an image and explanation of any revision implemented as a result.
  • Submit detailed images of your final paper prototype, as revised in this process. Include both:
    • An overview image that shows your entire paper prototype.
    • Detailed images showing each component as you walk through your two primary tasks.
  • Discuss two to four revisions that you consider to be the most salient and important modifications to your paper prototype based on issues identified in inspection, usability testing, or critique. These issues may already be included in the above list, but discuss them in more detail and argue why detecting and fixing them is important to your design.

Examples from Prior Offerings

Examples from prior offerings include:

Submission

No more than three pages of text in PDF format.

Images do not count against your page limit, and are therefore effectively free. You should embed images throughout your PDF, keeping them near the text that references them. The limit applies to the approximate amount of text you would have if all images were removed.

Submit via Canvas here:

https://canvas.uw.edu/courses/987987/assignments/3040177

In-Class Peer Critique

In lecture, be prepared to discuss your usability testing with other teams and the course staff.

Be sure to take notes during critique. We will provide a worksheet you can use:

3d-critique-worksheet-testingreview.pdf

After class, submit images of your notes in PDF format. Submit via Canvas here:

https://canvas.uw.edu/courses/987987/assignments/3046463

Grading

This milestone will be graded on a scale of 12 points:

  1. Discussion of Usability Tests and Plan: (3 points)
  2. Issues Identified in Usability Tests, Revisions Made in Response: (3 points)
  3. Updated Images of Paper Prototype: (3 points)
  4. Discussion of Key Revisions in Design Process: (3 points)

3e: Digital Mockup

Due: Uploaded the night before section Friday, November 20, 2015.

Create your digital mockup based on your final paper prototype. The mockup should effectively communicate your design in a manner appropriate for your final report, website, and poster. Implement your digital mockup to include key frames that illustrate your design. It should convey all critical aspects of your design, including walkthroughs for your two primary tasks.

Note decisions or changes you need to make as you transition from paper to a digital representation.

Present your digital mockup together with discussion of the decisions and changes you made:

  • Submit detailed images of your digital mockup. Include both:
    • Overview images that show your digital mockup.
    • Detailed images showing each component as you walk through your two primary tasks.
  • Discuss decisions and changes you made in implementing the digital mockup.

Examples from Prior Offerings

Examples from prior offerings include:

Submission

No more than one page of text in PDF format.

Images do not count against your page limit, and are therefore effectively free. You should embed images throughout your PDF, keeping them near the text that references them. The limit applies to the approximate amount of text you would have if all images were removed.

Submit via Canvas here:

https://canvas.uw.edu/courses/987987/assignments/3040179

Grading

This milestone will be graded on a scale of 12 points:

  1. Overall Completeness and Appropriateness of Digital Mockup: (3 points)
  2. Completeness, Appropriateness, and Documentation of Each Task: (3 points)
  3. Discussion of Decisions and Changes in Implementation: (3 points)

3f: Final Report

Due: Uploaded the night before class Tuesday, December 1, 2015.

Prepare a report documenting your process of getting the design right. Your report should follow the outline below, and will be graded using the guidelines that follow. The provided page allocations are estimates, intended to help convey how to divide up the space.

If you completed all of your milestones above, you will have much of the content for this report. But it is critical that you revise and update that content. You have received extensive feedback throughout your design process, and evaluation of your report will include how you have addressed and incorporated that feedback to improve relative to your prior milestones.

  1. Title

    A short, creative, and marketable title capturing the key idea.

  2. Each Team Member’s Name and Role

  3. Problem and Solution Overview: (1 paragraph)

    A concise statement of the problem you are tackling and a brief synopsis of your proposed solution. Consider updating this from your previous report to reflect the scope of your final design.

  4. Initial Paper Prototype: (1 page)

    Present your original paper prototype, as a baseline against which this report will present your iteration. Convey the critical aspects of your design, including your two primary tasks.

  5. Testing Process: (1 page)

    Present your testing process, including your method and your participants. This process probably evolved some in the course of your project. For your report, present a retrospective on how you refined your design.

  6. Testing Results: (2 pages)

    Summarize the results of your paper prototype testing and refinement. Present what you learned about each version of your prototype and what changes you made as a result. This will include changes from your heuristic evaluation, usability testing, and design critique.

  7. Final Paper Prototype: (1 page)

    Present your final version of your paper prototype, as you did for your initial paper prototype above. Convey the critical aspects of your design, including your two primary tasks.

  8. Digital Mockup: (1 page)

    Present your digital mockup. Convey the critical aspects of your design, including your two primary tasks. Briefly discuss any changes you needed to make as you switched to your digital tools instead of paper. Briefly discuss any changes you made in response to critique. Include descriptions of how your design supports each of your primary tasks (e.g., one paragraph per task).

  9. Discussion: (1 page)

    Reflect upon and discuss your project and your results. For example:

    • What did you learn from the process of iterative design?
    • How did the process shape your final design?
    • How have your tasks changed as a result of your usability tests?
    • Do you think you could have used more, or fewer, iterations upon your design?

  10. Appendix:

    Attach copies of all materials involved in your testing. Includes any instructions or task descriptions you handed out or read aloud to your participants. Include identified critical incidents from your usability testing. The appendix materials and screenshots do not count in your page limit.

Examples from Prior Offerings

Examples from prior offerings include:

Submission

No more than eight pages of text in PDF format, following the above outline.

Images and your appendix do not count against your page limit, and are therefore effectively free. You should embed images throughout your PDF, keeping them near the text that references them. The limit applies to the approximate amount of text you would have if all images were removed.

Submit via Canvas here:

https://canvas.uw.edu/courses/987987/assignments/3052941

Grading

This report will be graded on a scale of 25 points:

  1. Title: (1 point)
  2. Team Member Names and Roles: (1 point)
  3. Problem and Solution Overview: (2 points)
  4. Initial Paper Prototype: (3 points)
  5. Testing Process: (2 points)
  6. Testing Results: (3 points)
  7. Final Paper Prototype: (3 points)
  8. Digital Mockup: (3 points)
  9. Discussion: (3 points)
  10. Report Clarity and Presentation: (4 points)

Presentation

Due: Uploaded the night before class Thursday, December 3, 2015.

Prepare a presentation of your process in getting the design right. It should encompass all of your work in Assignment 3.

  • Two members of your team should deliver the presentation, each speaking to relatively equals portions.
  • An eight minute time limit will be strictly enforced, with additional time for questions.
  • Your presenters should be different from your presenters in Assignment 2, such that everybody has presented at least once this quarter. If there is some reason this cannot work for your team, be sure you have communicated with us.

A suggested organization of this presentation is:

  1. Title:

    A short, creative, and marketable title capturing the key idea. Include team member names and roles.

  2. Overall Problem:

    Tell this as a story, instead of simply reading the slide. Motivate your audience to be interested in your problem.

  3. Initial Paper Prototype:

    Present your initial paper prototype. Convey the critical aspects of your design, including your two primary tasks.

  4. Testing Process and Results:

    Describe your testing process, what you learned from it, and how you refined your prototype as a result.

  5. Final Paper Prototype:

    Present your final paper prototype. Convey the critical aspects of your design, including your two primary tasks. This might be shorter than your presentation of the initial paper prototype, because you already presented changes and are about to present the digital mockup. Keep your presentation appropriately paced instead of getting bogged down here.

  6. Digital Mockup:

    Present your digital mockup. Convey the critical aspects of your design, including your two primary tasks.

  7. Summary:

    Summarize the lessons learned in your design process.

We strongly recommend rehearsing your presentation beforehand. For example, arrange to practice together with another group or two, giving each other feedback on your presentations.

Note that we have not provided recommendations for the number of slides in each section of this presentation. You can deliver your presentation using as many or as few slides as you want, as long as you successfully address the above points and the presentation falls within the eight-minute time restriction.

Examples from Prior Offerings

Examples from prior offerings include:

The projects we have been posting this quarter:

Additional projects from the same prior offerings:

Additional examples of CSE 441 presentations. These cover a different scope of content, but were polished based on an initial round of feedback. So their structure will be different than your talks, but you may benefit from seeing their presentation.

Submission

Your presentation may be in PPT, PPTX, or PDF format.

To minimize switching time, we will have all presentations on a single laptop running Microsoft Windows. You should optimize your presentation for portability (e.g., ensure any necessary fonts are embedded). If we detect any obvious formatting issues on the presentation machine, we may fix them or contact you to fix them. But you are ultimately responsible for your presentation.

Submit via Canvas here:

https://canvas.uw.edu/courses/987987/assignments/3052942

In-Class Feedback Forms

The course staff will have a feedback form they keep during your presentation:

3g-presentation-instructor-form.pdf

Grading

The content of this presentation will be graded on a scale of 10 points:

  • Presentation shows appropriate preparation, with visual aids that are effective, properly prepared, and properly employed.
  • Slides are legible, such that people in the back of the room can still see them.
  • Presentation should not have an outline slide. It is short enough to be told as a story of your process.
  • Problem is presented in a manner that is compelling and achievable.
  • First paper prototype effectively demonstrates completion of a scenario.
  • Testing process is appropriate for the design.
  • Testing process is carried out in an appropriate manner.
  • Testing results are illuminating in terms of the design.
  • Final paper prototype has a strong connection to the results of the testing process.
  • Digital mockup effectively implements the final paper prototype.
  • Digital mockup is at the proper fidelity.
  • Presentation covered the required scope within the 8 minute time period.

The delivery of this presentation will be graded on a scale of 4 points:

  • Presenter makes eye contact with the audience.
  • Presenter projects their voice well and is audible throughout the room.
  • Presenter feels casual and engaged with the content, not just reading it.
  • Presentation covered the required scope within the 8 minute time period.