Web evaluation

Heuristic evaluation vs. user testing

Wei siong Tan, Dahai Liu, Ramaratnam R Bishu

Research output: Contribution to journalArticle

51 Citations (Scopus)

Abstract

It is very important that designers recognize the benefits and limitations of different usability inspection methods. This is because the quality of the usability evaluation is dependent on the method used. Two of the most popular usability evaluation techniques are user testing and heuristic analysis. The main objective of this study was to compare the efficiency and effectiveness between user testing and heuristic analysis in evaluating four different commercial web sites. The results showed that both user testing and heuristic analysis addressed different usability problems. Analysis by severity of problems found and diminishing return analysis model on the relationship between the number of new problems discovered with users and evaluators used showed that both methods are equally efficient and effective in addressing different categories of usability problems. These significant differences found between these two methods suggested that the two methods are complimentary and should not be competing. In order for better evaluation results, both user testing and heuristic analysis are still needed. Relevance to industry: The research findings from this study will be of particular value to the web development industry and communities. Knowledge regarding the differences between user testing and heuristic evaluation will enable appropriate business decisions to be made on when and how to apply these methods to improve the overall efficiency of the design process.

Original languageEnglish (US)
Pages (from-to)621-627
Number of pages7
JournalInternational Journal of Industrial Ergonomics
Volume39
Issue number4
DOIs
StatePublished - Jul 1 2009

Fingerprint

heuristics
Testing
evaluation
Industry
Efficiency
efficiency
industry
model analysis
Websites
Inspection
Heuristics
Research
community

Keywords

  • Heuristic evaluation
  • Usability testing

ASJC Scopus subject areas

  • Human Factors and Ergonomics
  • Public Health, Environmental and Occupational Health

Cite this

Web evaluation : Heuristic evaluation vs. user testing. / Tan, Wei siong; Liu, Dahai; Bishu, Ramaratnam R.

In: International Journal of Industrial Ergonomics, Vol. 39, No. 4, 01.07.2009, p. 621-627.

Research output: Contribution to journalArticle

@article{c6f46a20c5164603b2c8f2a8c08e70a9,
title = "Web evaluation: Heuristic evaluation vs. user testing",
abstract = "It is very important that designers recognize the benefits and limitations of different usability inspection methods. This is because the quality of the usability evaluation is dependent on the method used. Two of the most popular usability evaluation techniques are user testing and heuristic analysis. The main objective of this study was to compare the efficiency and effectiveness between user testing and heuristic analysis in evaluating four different commercial web sites. The results showed that both user testing and heuristic analysis addressed different usability problems. Analysis by severity of problems found and diminishing return analysis model on the relationship between the number of new problems discovered with users and evaluators used showed that both methods are equally efficient and effective in addressing different categories of usability problems. These significant differences found between these two methods suggested that the two methods are complimentary and should not be competing. In order for better evaluation results, both user testing and heuristic analysis are still needed. Relevance to industry: The research findings from this study will be of particular value to the web development industry and communities. Knowledge regarding the differences between user testing and heuristic evaluation will enable appropriate business decisions to be made on when and how to apply these methods to improve the overall efficiency of the design process.",
keywords = "Heuristic evaluation, Usability testing",
author = "Tan, {Wei siong} and Dahai Liu and Bishu, {Ramaratnam R}",
year = "2009",
month = "7",
day = "1",
doi = "10.1016/j.ergon.2008.02.012",
language = "English (US)",
volume = "39",
pages = "621--627",
journal = "International Journal of Industrial Ergonomics",
issn = "0169-8141",
publisher = "Elsevier",
number = "4",

}

TY - JOUR

T1 - Web evaluation

T2 - Heuristic evaluation vs. user testing

AU - Tan, Wei siong

AU - Liu, Dahai

AU - Bishu, Ramaratnam R

PY - 2009/7/1

Y1 - 2009/7/1

N2 - It is very important that designers recognize the benefits and limitations of different usability inspection methods. This is because the quality of the usability evaluation is dependent on the method used. Two of the most popular usability evaluation techniques are user testing and heuristic analysis. The main objective of this study was to compare the efficiency and effectiveness between user testing and heuristic analysis in evaluating four different commercial web sites. The results showed that both user testing and heuristic analysis addressed different usability problems. Analysis by severity of problems found and diminishing return analysis model on the relationship between the number of new problems discovered with users and evaluators used showed that both methods are equally efficient and effective in addressing different categories of usability problems. These significant differences found between these two methods suggested that the two methods are complimentary and should not be competing. In order for better evaluation results, both user testing and heuristic analysis are still needed. Relevance to industry: The research findings from this study will be of particular value to the web development industry and communities. Knowledge regarding the differences between user testing and heuristic evaluation will enable appropriate business decisions to be made on when and how to apply these methods to improve the overall efficiency of the design process.

AB - It is very important that designers recognize the benefits and limitations of different usability inspection methods. This is because the quality of the usability evaluation is dependent on the method used. Two of the most popular usability evaluation techniques are user testing and heuristic analysis. The main objective of this study was to compare the efficiency and effectiveness between user testing and heuristic analysis in evaluating four different commercial web sites. The results showed that both user testing and heuristic analysis addressed different usability problems. Analysis by severity of problems found and diminishing return analysis model on the relationship between the number of new problems discovered with users and evaluators used showed that both methods are equally efficient and effective in addressing different categories of usability problems. These significant differences found between these two methods suggested that the two methods are complimentary and should not be competing. In order for better evaluation results, both user testing and heuristic analysis are still needed. Relevance to industry: The research findings from this study will be of particular value to the web development industry and communities. Knowledge regarding the differences between user testing and heuristic evaluation will enable appropriate business decisions to be made on when and how to apply these methods to improve the overall efficiency of the design process.

KW - Heuristic evaluation

KW - Usability testing

UR - http://www.scopus.com/inward/record.url?scp=67349225428&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=67349225428&partnerID=8YFLogxK

U2 - 10.1016/j.ergon.2008.02.012

DO - 10.1016/j.ergon.2008.02.012

M3 - Article

VL - 39

SP - 621

EP - 627

JO - International Journal of Industrial Ergonomics

JF - International Journal of Industrial Ergonomics

SN - 0169-8141

IS - 4

ER -