Difference between revisions of "Analysis & Evaluation"

From MPDLMediaWiki
Jump to navigation Jump to search
m (#REDIRECT Protocol)
 
m (Redirecting to User Interface Evaluation)
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{Template:TemplateUIE_Activities}}
#REDIRECT [[User Interface Evaluation]]
<div style="float:left; width:70%; margin-bottom:3em;">
 
{| align="center"
  | __TOC__
  |}
 
==Evaluation==
 
Interfaces where a usability life cycle is applied are subject to evaluation. Evaluation, done by the User Interface Engineering team can be carried out with different Methods:
 
#Usability Interviews and tests with 'Thinking Aloud'
#GUI Workshops 'e.g. Card Sorting'
#Software based evaluation
 
=== Usability Interviews (Thinking Aloud/Think Aloud Protocol<ref>http://en.wikipedia.org/wiki/Think_aloud_protocol Wikipedia on 'Think Aloud Protocol'</ref>) ===
 
From our point of view the method of choice because it balances result and effort.
 
{|{{Prettytable}}
|-
!bgcolor = #eee |Participants
|
1
|-
!bgcolor = #eee |Supervisor
|
1 plus 1 visitor (usually developers)
|-
!bgcolor = #eee |Equipment
|
<ol>
<li>PC</li>
<li>Internet connection</li>
<li>Tasklist (provided by UIE)</li>
<li>Test Form (provided by UIE)</li>
<li>Recording Device</li>
</ol>
|-
!bgcolor = #eee |Duration
|
1h
|-
!bgcolor = #eee |Results
|
Document with notes on user actions for each action/step.
 
The results are analysed after 6 - 8 interviews towards a statistic, providing GUI flaws within use cases.
|-
!bgcolor = #eee |Steps
|
<ol>
<li>Short Introduction</li>
<li>Participant solves task independently</li>
<li>Open questions/diskussion</li>
</ol>
|}
 
 
Usability interviews are conducted to discover, document and classify usability issues for
 
* a specific release (e.g. PubMan 2.0.0.1)
* a functional prototype
* a prototype draft
 
Up to 8 interviews are usually sufficient to discover the main flaws of a GUI. The interviewer just provides tasks for his participant. The participant does all tasks on his own and should be encouraged to comment on his actions. It is not recommended for the interviewer to interfere in any form. If the participant gets stuck and there are important steps to follow, he gets a hint how to continue.
 
Interviews can be recorded optionally if participants agree. If a participant is not able to solve a task or step it will be noted as a fatal usability issue. For later interviews with a more standardized set of tasks the issues are to be classified in the following way:
 
#Fatal (Task could not be finished successfully or in a proper way)
#Serious (Task could not be finished on the first attempt or user performance is bad)
#Minor = (The user hesitates, is not sure or does not feel comfortable with the flow)
 
For each interview the UIE team provides an interview outcome. The document contains all tasks, observation notes and comments of participants. Additionally the interviews collect demographic characteristics as well to prepare data needed for personas. After a valid number of interviews exists they are analysed and summarized.
 
===Interviews for PubMan Release 2===
'''Interview series I'''
*Munich: 2 Interviews
'''Interview series II'''
*Nijmegen: 9 Interviews
'''[[User Interface Evaluation/Workshops/Interview Analysis|Interview Analysis]]'''
 
===Interviews for PubMan Release 3===
 
'''Interview series I'''
{|style= color:black; background-color:#fff; cellpadding=5; cellspacing=0; border=0
|<b>Place</b>
|
*Max Planck Institute of Molecular Plant Physiology, Berlin: 2 Interviews
*Max Planck Institute for Gravitational Physics (Albert Einstein Institute), Golm 2 Interviews
*Max Planck Institute for Human Development, Berlin 2 Interviews
|-
|<b>Date/Time</b>
|August 6/7
|-
|<b>Status</b>
|Closed
|-
|}
 
'''Interview Series II'''
 
{|style= color:black; background-color:#fff; cellpadding=5; cellspacing=0; border=0
|<b>Place</b>
|
*Harnack-Haus, Berlin: 3 Interviews
|-
|<b>Date/Time</b>
|Mittwoch, 05. November 2008/09:00-12:00
|-
|<b>Status</b>
|Closed
|-
|}
 
'''[[User Interface Evaluation/Workshops/R3 Interview Analysis|R3 Interview Analysis]]'''
 
===Interviews for PubMan Release 4===
 
'''Interview series I'''
{|style= color:black; background-color:#fff; cellpadding=5; cellspacing=0; border=0
|<b>Place</b>
|
*Max Planck Digital Library, Munich: 4 Interviews
|-
|<b>Date/Time</b>
|April - June
|-
|<b>Status</b>
|Open
|-
|}
 
'''Interview Series II'''
 
{|style= color:black; background-color:#fff; cellpadding=5; cellspacing=0; border=0
|<b>Place</b>
|
*Max Planck Institute for Foreign and International Social Law, Munich: 2 Interviews
|-
|<b>Date/Time</b>
|April - June
|-
|<b>Status</b>
|Open
|-
|}
 
'''[[User Interface Evaluation/Workshops/R4 Interview Analysis|R4 Interview Analysis]]'''
 
===Interviews for FACES Release 3 (Prototype test)===
{|style= color:black; background-color:#fff; cellpadding=5; cellspacing=0; border=0
|<b>Place</b>
|
*Max Planck Institute for Human Development
|-
|<b>Date/Time</b>
|[http://www.doodle.com/9zy24f297pkbzsaz To be scheduled]
|-
|<b>Please note</b>
|We need an information about your status (expert/non-expert) Please take a note behind your name.
|-
|<b>Status</b>
|Planned
|-
|}
 
==Expert Interviews==
 
{|{{Prettytable}}
|-
!bgcolor = #eee |Participants
|
1
|-
!bgcolor = #eee |Supervisor
|
1 - 2
|-
!bgcolor = #eee |Equipment
|
<ol>
<li>PC</li>
<li>Internet connection</li>
<li>Interview agenda (provided by UIE)</li>
<li>Audio recording device (provided by UIE)</li>
</ol>
|-
!bgcolor = #eee |Duration
|
2h - 4h
|-
!bgcolor = #eee |Results
|
Document with protocol of interview. Prototype draft.
|-
!bgcolor = #eee |Steps
|
<ol>
<li>Introduction</li>
<li>Q & A (Questions and Answers)</li>
<li>Summary</li>
</ol>
|}
 
Expert interviews are conducted to get a better understanding of one specific working process and how it can be organized/reorganized within the user interface. A very domain specific knowledge is needed to understand interface requirements better. Functional aspects come additionally and can not be separated from interface needs.
 
===Expert Interviews 2007===
 
*'''[[User Interface Evaluation/Expert Interviews/2007 Garching]]'''
*'''[[User Interface Evaluation/Expert Interviews/2007 Munich]]'''
*'''[[User Interface Evaluation/Expert Interviews/Easy Submission Summary]]'''
 
===Expert Interviews 2009===
 
*'''Max Planck Institute for Human Development (Planned)'''
 
==Workshops==
 
{|{{Prettytable}}
|-
!bgcolor = #eee |Participants
|
5 - 8
|-
!bgcolor = #eee |Supervisor
|
1 - 2
|-
!bgcolor = #eee |Equipment
|
Equipment provided by UIE Team
|-
!bgcolor = #eee |Duration
|
4h - 6h
 
|-
!bgcolor = #eee |Result
|
A rough draft of a graphical user interface part.
|-
!bgcolor = #eee |Steps
|
<ol>
<li>Introduction to a GUI task</li>
<li>Participants work on paper prototypes</li>
<li>Question and Answers</li>
<li>Comparison of Results</li>
<li>Clustering</li>
<li>Rework if necessary</li>
<li>Preparation plus dissamination of results</li>
 
</ol>
|}
 
 
For each workshop the UIE team prepares a topic. The following topics are already covered:
 
===Workshops 2008===
*'''Scheduled: 09.12.2008 - 10.12.2008, Bibliotheca Hertziana - Max Planck Institute for Art History'''
<b>Topics</b>: A brainstorming can be found on [[Faces_GUI#User_Interface_Evaluation|Faces GUI]].
 
*'''[[User Interface Evaluation/Adding Lists of Authors]], MPDL Berlin'''
*'''[[User Interface Evaluation/Easy Submission]], Nijmegen'''
 
===Workshops 2007===
*'''[[User Interface Evaluation/Short Item View]], MPDL Munich'''
 
==Software based Evaluation==
 
Currently the pubman prototype is monitored by an open source application called [http://www.labsmedia.com/clickheat/index.html ClickHeat]. This solution generates pictures called heat maps. Heat maps show where users click (dots) and how often (colour).
 
Example:
 
[[Image:clickheat_depositor.png|framed|center|Heat Map of Depositor Workspace]]
 
Software based evaluation is not conducted for R3. Currently no open source application is available for analysing dynamic pages.
 
</div>
 
[[Category:Analysis & Evaluation]]
[[Category:User Interface Engineering]]

Latest revision as of 13:45, 19 June 2009

Redirect to:

Retrieved from "https://colab.mpdl.mpg.de/mw010/index.php?title=Analysis_%26_Evaluation&oldid=37234"