1. CONCEPTUAL DESIGN (MODEL)
    1. Interface Metaphor (physical entity)
      1. Conceptualizing what users are doing
      2. A conceptual model instantiated at the interface
      3. Visualizing an operation
      4. Choosing
        1. Understand functionality
        2. Identify potential problems areas
        3. Generate metaphors
        4. Evaluate suitability of metaphors
      5. Advantages
        1. makes it easy to learn new systems
        2. helps to better understand the conceptual model
        3. supports a diverse set of users
        4. strengthen innovation
      6. Disadvantages
        1. constrains the conceptual design to metaphors
        2. limit the user to understand the system in terms of metaphor
        3. poor designs can be used as the metaphor
        4. rely on metaphors might hinder coming up with new conceptual models
        5. some times may break conventional and cultural rules
    2. Interaction Types
      1. Instructing
        1. tell the system what to do
          1. e.g. typing in commands
          2. e.g.selecting menus
      2. Conversing
        1. have a dialog with the system
          1. menu-based dialogues
          2. text-based dialogues
          3. virtual agents
      3. Manipulating
        1. manipulating objects and users experiences
        2. based on users experience with real objects (affordance)
      4. Exploring
        1. moving through virtual or physical environments
        2. e.g. VR
      5. Responding
        1. initiative to alert, describe, or show the user sth. of interesting
        2. relevance to time or context
    3. Interface Types
      1. Types of input and output methods
      2. Interface used by the users to support the interaction
      3. Choose the most appropriate or a combination
  2. ENVISIONMENT
    1. make ideas visible and externalize thoughts
    2. represent design work
    3. occurs throughout development
    4. Different representations
      1. Sketches
        1. Ideas and thoughts can be quickly visualized
        2. Quick, timely, inexpensive, disposable and plentiful
          1. o Allow quick test of new ideas during brainstorming
          2. o Reduce attachment to design
        3. Basic elements, People, Objects - depends on the Purpose
        4. Context, User view, Snapshot
        5. advantages
      2. Storyboards
        1. Sequence of actions or events
        2. User Journey
          1. 3-7 steps
          2. each picture labelled with 1 short description
          3. context of interaction is visible
          4. correct level of details
      3. Wireframes (e.g. Wireflow)
        1. Single screen or interaction page
        2. Plan the layout and interaction patterns
        3. Different level of details
      4. Prototypes
        1. Low-fidelity
          1. medium unlike the final medium
          2. capture early design thinking
          3. quick and easy to produce
        2. High-fidelity
          1. similar in look and feel with anticipated final product
          2. detailed evaluation of the main design elements
        3. Paper prototypes
          1. produce quickly
          2. enables non-technical people to interact easily with the design team
          3. flexibility - 'redesigned'
          4. Advantages
        4. Faking interaction
          1. Wizard of Oz (lo-fi)
          2. human is responding to output rather than the system
          3. Video prototype
          4. how the prototype is 'used' in real-life
          5. Early stage - fake interaction
          6. Later stage - communicate what product looks like and can do
          7. Focus on information to be conveyed
          8. limited by imgination, time and materials
        5. Compromises
          1. Horizontal - wide range but little details
          2. Vertical - lots of details but a few functions
  3. Computer Prototyping tools
    1. Fidelity of Prototype
      1. Level of details and functionality built into a prototype
      2. Low-fidelity
        1. Limited functionality and interactivity
        2. Examples: Paper prototype
      3. High-fidelity
        1. Close resemblance to the final design
        2. High functionality and interactivity
        3. Examples: Digital prototypes
    2. What are prototyping tools?
      1. Tools develop for the sole purpose of prototyping
        1. Code-based
        2. Code-free
    3. Software Prototyping Tools
      1. Certain degree of coding required
        1. Web UIs
          1. HTML5 with a lot of libraries
          2. Three.js
        2. User interface builders
          1. Visual Studio, XCode, Visual Basic
      2. Finished design can be used for final implementation
        1. Processing.org
          1. A programming IDE for prototyping
        2. Supports many libraries
          1. Video, Audio, Network, Animation, Vision, ML
        3. Based on Java
      3. Comparison
    4. Designing Tools
      1. Tools allow you to design within or import from other softwares
      2. Different tools, different range of fidelity
      3. Software suitable to create
        1. Balsamiq
        2. AdobeXD
    5. Linking to create Clickthroughs
      1. Prototype that links multiple screens together via hotspots
      2. Hotspots area that is interact-able by the user.
      3. Moving from paper to digital prototype
        1. Upload existing images
        2. Add hotspots
    6. Sharing the prototype
      1. Purpose
        1. Collaboration
          1. Add team members to project (cloud-based tools)
          2. Edit and comment on design
        2. Presentation and testing
          1. User participants and stakeholders
          2. View and use prototype
      2. Different types of shareable/export formats
        1. Web Link
        2. PDF file with hyperlinks
        3. View in iOS/Android phones
        4. HTML files
    7. Choosing a Prototype Tool
      1. Fidelity
        1. Layout and navigation design
        2. Visual design and micro-interactios
      2. Ease of collaboration (Teamwork support)
        1. Ease to pick up tool
        2. Number of people on the same project
        3. Platform - Mac/Window/Cloud
      3. Integration with workflow
        1. Import ad export previous work
        2. Assert libraries
      4. Costs
        1. Free/Trial
        2. Subscription
  4. Physical Prototypes
    1. What are the physical prototypes?
      1. Mostly focus on electrical products
        1. Wearable technology
        2. Tangible UI
      2. Same principle - test out ideas quickly
        1. Test out idea quickly
      3. Resources to support development
        1. Physical computing kits
          1. Build and code prototypes and devices using electronics
          2. Arduino
          3. Open-source electronics platform based on easy-to-use hardware and software
          4. Toolkit comprises of two parts
          5. Arduino board
          6. Arduino IDE - program sketch to board
          7. Sketch - Unit of code
          8. BBC micro: bit
          9. Similar to Arduino
          10. Add to external components at the edge connector
          11. Teach programming in schools - Scratch, Python
          12. MaKey MaKey
        2. Rapid fabrication
          1. Computer aided production tools
          2. 3D printers (additive manufacturing)
          3. Laser cutters (Subtractive)
          4. Helps to quickly fabricate high quality physical prototypes
          5. Easy to modify and change
    2. Subtopic 2
  5. Introducing Evaluation
    1. Ethics
      1. Inform participants about their rights during the study
      2. Protect participants during study
        1. Physical or emotional endangerment
        2. Privacy of participants
      3. Ethics approval must be obtained before study is conducted.
      4. University Human Ethics Policy
        1. Ethics approval helps to
          1. protect the welfare, rights, dignity and safety of research participants
          2. protect researchers' rights to conduct legitimate investigation
          3. protect the University's reputation for research conducted and sponsored by it.
          4. Minimize the potential for claims of negligence made against individual researchers and the University
          5. Human research
          6. Research conducted with or about people, their biological materials or information.
          7. It covers activities including:
          8. taking part in surveys, interviews, or focus groups
          9. undergoing psychological, physiological or medical testing or treatment
          10. being observed by researchers
          11. assessing personal documents or other materials
          12. collection and use of biological materials
          13. assess to personal information as part of an existing published or unpublished source.
          14. Before the Session
          15. Don't waste the user's time
          16. make sure experiment is designed well
          17. be prepared
          18. Make users feel comfortable
          19. communicate only the system will be tested, not the user
          20. indicate that the software may have problems
          21. inform that they can stop at any time
          22. Maintain privacy
          23. tell the user that results will be anonymized (if applicable)
          24. Inform the user
          25. explain what is being recorded (video, audio, data logging, etc.)
          26. answer user's questions (but avoid bias)
          27. Do not coerce users
          28. obtain informed consent
          29. During the Session
          30. Don't waste the user's time
          31. do not ask to perform unnecessary tasks
          32. Make users feel confortable
          33. give early success experience (pre-trials)
          34. keep a relaxed atmosphere
          35. sufficient breaks (e.g. coffee breaks)
          36. hand out test tasks one at a time
          37. do not show displeasure
          38. avoid disruptions
          39. stop the test if participant show discomfort
          40. Maintain privacy
          41. external people should not be present
          42. After the Session
          43. Make users feel comfortable
          44. thank the user and inform they have helped
          45. Provide additional information if necessary
          46. answer any other remaining questions user had
          47. e.g. something that could have lead to a bias
          48. Maintain privacy
          49. report the data without compromising privacy
          50. only share audio visual data with expression permission
          51. store all the data in a secure location
          52. university has a dedicated research data storage
          53. Research Computing Optimized Storage (RCOS)
    2. Main steps in Evaluation
      1. 1. Establish aims of evaluation
      2. 2. Select evaluation methods. Good to have combination of participant (with users) and non-participant methods (without users).
      3. 3. Carry out non-participant methods first.
      4. 4. Use results from non-participant methods to plan participant testing
      5. 5. Plan session, recruit participants and setup equipment
      6. 6. Carry out evaluation
      7. 7. Analyze results, document and report
    3. Selecting and Combining Methods
      1. Use a combination of methods to obtain richer understanding of users and product
      2. Controlled - test hypothesis about specific features
      3. Uncontrolled - insight to people's experience of interacting with technology in the context of daily life
      4. Examples
        1. Combination of usability testing in labs combined with field studies
        2. Cognitive walkthrough to test run the prototype before actual usability testing in the lab
    4. When to Evaluate?
      1. During Iterative Design - Check if
        1. Design matches the requirements
        2. Problems with the design
      2. Before deployment - For Acceptance Testing
        1. Does the system meet expected performance
      3. Continuous Evaluation after Deploying
        1. "Performance beta"
        2. Continuous evaluation
          1. In the wild, bug reports, field studies
    5. Where to Evaluate?
      1. Usability lab
        1. Testing room constructed for usability testing
          1. Instrumented
          2. camera, microphones, data recording, etc.
          3. Separate observation room
          4. connected by one-way mirror
        2. Benefits
          1. Controlled situation
          2. Ideal to study one precise aspect
          3. Many equipment available
          4. Only option if real location is dangerous or remote
        3. Problems
          1. Does not represent a natural situation
          2. Hard to generalize results
      2. Research lab
      3. Naturalistic setting
        1. Observation occurs in realistic setting
          1. Real life
          2. Workplace / Home
          3. In-situ
        2. Benefits
          1. More realistic (e.g. external effects)
          2. Situation and behavior more natural
          3. Better suited for long-term studies
          4. Well-suited for user experience studies
        3. Problems
          1. Hard to arrange and run
          2. Time consuming
          3. Task is difficult to control
          4. Environment is difficult to control (e.g. distractions)
      4. Remote study
    6. What to Evaluate?
      1. Conceptual model
        1. Focus is on (standard) usability issues
        2. product is close to final / feature rich
        3. comparative results
      2. Early and subsequent prototypes of a new system
        1. get early feedback on a design
        2. Low-fidelity prototype
        3. can fix design issues in advance
      3. Final product
        1. How the product works for new markets / user groups
        2. Existing product, already evaluated for one market
    7. Why Evaluate?
      1. To judge system features / functionality
        1. Does it facilitate users' tasks and match their requirements?
        2. Does it offer the right features?
      2. To judge effects on users
        1. How easy is the system to learn and use?
        2. How do users feel about the system?
      3. To discover unforeseen problems
        1. What unexpected / confusing situations come up?
      4. To compare your solution against competitors
        1. Important for marketing / sales department
  6. EVALUATIONS WITHOUT uSERS
    1. Inspections
      1. Heuristic Evaluation
        1. a review guided by a set of heuristics
        2. Small set of evaluators examine the interface and judge its compliance with recognized usability principles.
        3. Original heuristics - Nielsen TEN Usability Heuristics derived empirically from an analysis of 249 usability problems.
          1. Number of Evaluators
          2. On average Five evaluators identify 75-80 percent of usability problems
          3. Choice of Heuristics
          4. Should depend on goals of the evaluations
          5. Suggest to use category-specific heuristics that apply to a specific class of product as a supplement to the general heuristics.
          6. Can tailor original heuristics with other design guidelines, market research and requirements documents for this purpose.
          7. How to Heuristics Evaluation
          8. Briefing session to tell experts what to do
          9. Evaluation period of 1-2 hours in which
          10. Each expert works separately
          11. Take one pass to get a feel for the product
          12. Take a second pass to focus on specific features
          13. Debriefing session in which experts work together to prioritize problems.
          14. Subtopic 1
        4. Benefits
          1. Few ethical and practical issues to consider because users not involved
          2. Best experts have knowledge of application domain and users
        5. Problems
          1. Can be difficult and expensive to find experts
          2. Many trivial problems are often identified, such as false alarms
          3. Experts have biases
      2. Cognitive Walkthrough
        1. Involve stepping through a pre-planned scenario noting potential problems
        2. Focus on ease of learning
        3. Designer presents an aspect of the design and usage scenarios
        4. Expert is told the assumptions about user population, context of use, task details.
        5. One or more experts walk through the design prototype with the secenario
        6. How to Cognitive Walkthrough
          1. UX researchers walk through the action sequences for each task.
          2. As they do this, answer the following questions:
          3. Will the correct action be sufficiently evident to the user?
          4. Will the user notice that the correct action is available?
          5. Will the user associate and interpret the response from the action correctly?
          6. Record problems.
        7. Benefits
          1. Can be done without users
          2. Considers users' task
          3. Quick and inexpensive to apply
        8. Problems
          1. Limited by skills of the evaluator
          2. Labor intensive - answering and discussing questions may take a long time.
    2. Analytics
      1. Web Analytics
        1. A form of interaction logging that analyzes users' activities on website
          1. Total number of people
          2. Length of stay
          3. Content site visits
        2. Outcomes can be used to improve their design
        3. When designs don't meet users' needs, users will not return to the site (one-time users)
        4. Example - SparkPlus
      2. Learning Analytics
        1. Web analytics applied to field of education
        2. Learner's activity in massive open online courses ( MOOCs) and Open Education Resources (OERs).
      3. A/B Testing
        1. A large-scale experiment
        2. Offers another way to evaluate a website, application of app running on a mobile device
        3. Often used for evaluating changes in design on social media applications
        4. Compares how two groups of users perform on two versions of a design
    3. Model
      1. Predictive models evaluate a system without users being present.
      2. Fitts' Law
        1. Time taken to hit a screen target is independent on distance of cursor and size of target.
  7. eVALUATION WITH USERS (1) - Usability test
    1. Usability Testing
      1. Involves recording performance of typical users doing typical tasks
      2. Users are observed and timed
      3. Data is recorded on video, and key presses are logged
      4. User satisfaction is evaluated using questionnaires and interviews
      5. Team roles during testing
        1. All members are encouraged to participate in the evaluations
        2. Facilitator
          1. Person in the lab together with participant
          2. Responsibilities
          3. Plan and execute session
          4. Set up lab for session
          5. Responsible for putting participant at ease during session
          6. Must have people skills
        3. Prototype executor
          1. Person to 'execute' the prototype and move it through its paces as users interact
          2. Only if you are using a low-fidelity prototype
          3. Must have thorough technical knowledge of how design works
          4. Poker face and should not speak a single word during the session.
        4. Quantitative data Collector
          1. Works
          2. Time to complete task
          3. Number and type of errors per task
          4. Number of errors per unit of time
          5. Number of navigations to online help or manuals
          6. Record into spreadsheet directly
          7. Tools - stopwatch and counter.
        5. Qualitative data Collector
          1. Observation notes
          2. Critical incidents
          3. Think aloud comments
        6. Supporting Actors
          1. Optional
          2. If part of the setting or task requires participant to interact with someone.
          3. Manage the props needed in the evaluation (other than the prototype execution).
          4. Example: Call client on the telephone.
      6. Tasks during session
        1. Representative, frequent and critical tasks that apply to the key work role and user class represented by each participant.
        2. Prepare corresponding task description and UX target metrics to guide data collection and compare observed results.
        3. Test conditions same for every participant.
        4. Task description
          1. what to do, no hints about how to do
      7. Recuiting Participants
        1. Find representative users (usually outside you team and outside project organization)
        2. Recruitment methods and screening
          1. People around you - spouses, children, friends.
          2. Post ads in public spaces
          3. Announcements at meetings of user groups and professional organizations if the group matches your user class needs
          4. Temporary employment agencies.
        3. Number of participants
          1. Schedule for testinng
          2. Availability of participants
          3. Costs of running tests
          4. Famous rule of thumb
          5. 3 - 5 participants
          6. typically 5-10 participants
          7. OR: Test until no new insights are gained
      8. Planning the Session
        1. If it is in the lab, configure the lab to your needs.
          1. Computer / Device
          2. Placement of participants, facilitator and executor
        2. Set up hardware, e.g. eye-trackers, timers, counters etc.
        3. Determine length of session for one participant
          1. Typical length: 30 mins to 120 mins
          2. Strategies to manage long sessions
          3. Warn participants in advance
          4. Schedule breaks between task - exercise, toilet break, refreshments.
          5. Prepare food and water in advance to keep participant at ease
        4. Prepare necessary paperwork
          1. Informed consent (important)
          2. Formal and signed permission given to UX professional by participants to use data gathered within stipulated limits.
          3. Preparation for informed consent begins with institutional review board (IRB) / Ethics Approval Committee
          4. Evaluators / Project manager to prepare application.
          5. USYD Ethics Application
          6. Participant Information Statement (PIS)
          7. Participant Consent Forms (PCF)
          8. advertisements, letters and emails seeking participants.
          9. interview or focus group questions / themes
          10. letters of support or permission from organizations assisting in the research in any way
          11. external research declarations (for researchers not affiliated with the University)
          12. Participants must read both PIS and PCF before session
          13. Allow participants / guardians to ask questions
          14. Participants / guardian must sign PCF before session
          15. Prepare two copies for the session
          16. One for participant to keep
          17. One for submission
          18. Other data collection forms
          19. Non-disclosure Agreements (NDAs)
          20. If required by developer or customer organizations to protect intellectual property (IP) contained in design.
          21. Must be included during signing of PCF.
          22. Questionnaires
          23. If your evaluation plan includes administration of one.
          24. SUS - System Usability Scale
          25. Usefulness, Satisfaction and Ease of Use (USE) questionnaire
      9. On the Big Day
        1. Before Session
          1. invite participant into the lab
          2. offer refreshments and paperwork
          3. explain details of study and check for questions
          4. participant to complete requested forms
          5. [Optional] interview participant to check responses on questionnaire
        2. During Session
          1. hand out task (one at a time)
          2. Encourage participant to think - aloud
          3. Describe his actions and why
          4. Stop test if participant is in distress
          5. To help or not to help participant during the task
          6. Depends on purpose of test
          7. Guide users if questions are asked
        3. After Session
          1. post-session probs if have any
          2. debrief participant - answer remaining questions that the participant has and what you will do next
          3. thank the participant and give token of appreciation for their time
          4. prepare for the next participant.
  8. EVALUATION WITH USERS (2) - Experiments
    1. Usability Testing vs. Experiments
      1. Usability testing is applied experimentation
      2. Developers check that the system is usable by the intended user population by collecting data about participants' performance on prescribed tasks
      3. Experiments test hypotheses to discover new knowledge by investigating the relationship between two or more variables.
    2. Experiments
      1. Basics
        1. test hypothesis
        2. predict the relationship between two or more variables
        3. independent variable is manipulated by the researcher
        4. dependent variable influenced by the independent variable
        5. typical experimental designs have one or two independent variables
        6. validated statistically and replicable
      2. Designs
        1. When dealing with human subjects, need to determine which participants to involve for which conditions in an experiments.
        2. Experience in participating in one condition will affect performance of those participants if asked to participate in another condition.
        3. Using multimedia materials in class will improve students' learning.
        4. Different - participants design (between - subjects design)
          1. Single group of participants is allocated randomly to each of the experimental conditions so that different participants perform in different conditions.
          2. Need many participants to minimize individual differences- differences in experience and expertise. Perform pre-testing to identify participants that differ strongly.
        5. Same - participants design (within - subjects design)
          1. All participants perform in all conditions
          2. Must perform counter-balancing to reduce effects of ordering.
          3. Effects of ordering - learning from pervious task affects performance of subsequent task.
        6. Matched - participant design (pair-wise design)
          1. Participants are matched in pairs based on certain user characteristics such as expertise and gender.
          2. Each pair is then randomly allocated to each experimental condition.
          3. Does not consider important variables that may influence the results.
      3. How to Experiments
        1. 1. Determine goals, explore the questions then formulate hypothesis
        2. 2. Design experiment, define experimental variables
        3. 3. Choose subjects
        4. 4. Run pilot experiment
        5. 5. Iteratively improve experiment design
        6. 6. Run experiment
        7. 7. Interpret results to accept or reject hypothesis
      4. Crowdsourcing
        1. Internet - source to recruit participants and run large-scale experiments
          1. Amazon Mechanical Turk (Mturk)
          2. Turkers - volunteers to perform human intelligence tasks (HIT)
  9. EVALUATION WITH USERS (3) - Field Study
    1. Field Studies
      1. Done in natural settings
      2. "In the wild" is a term for prototypes being used freely in natural settings
      3. Seek to understand what users do naturally and how technology impacts them
      4. Field studies are used in product design to
        1. identify opportunities for new technology
        2. determine design requirements
        3. design how best to introduce new technology
        4. evaluate technology in use
      5. Range from a few minutes to longitudinal studies (few years)
      6. Data collection that is obtrusive but informative
        1. Self-reports of problems encountered when occur
        2. Interval logging triggered by smartphone notifications
        3. Logging software - monitor frequency / patterns of daily activities
      7. Conundrums with Field Studies
        1. Informing participants that they are studied
          1. Knowledge of study would make people conscious about how they behave.
          2. Without informing people of their participation, how do you get their consent for participation?
        2. Privacy of participants
          1. Studies done in people's homes will always be intrusive
          2. Agreement between participant and research the activities that can or cannot be recorded.
        3. What to do with the prototype?
          1. Event of breakdown
          2. Security arrangements if deployed in public spaces
      8. E.G. Pain Monitoring
  10. special permission and leads to privacy issues
  11. User Journey
  12. Wireflow
  13. Adv and Disadv
  14. Experts use their knowledge of users and technology to review software usability / Expert critiques can be formal or informal
  15. A variety of users' actions can be recorded by software automatically - key presses, time spent searching web page, looking at help systems. / Unobtrusive provided system's performance not affected. / Large volumes logged automatically, explored and analyzed. / Ethical issues - observing without knowledge.