Melody Y. Ivory
University of California, Berkeley
Berkeley, CA 94720-1776 USA
In the Proceeding of ACM CHI 00, Conference on Human Factors in Computing Systems, Student Posters, April, 2000.
Web site usability is even more critical as the number of sites grows
exponentially and the number of users increases dramatically. We
describe a new automated methodology and tool, Web TANGO, being
developed to allow designers to explore alternative designs of
information-centric web sites prior to implementation.
Web Design, Automated Usability Evaluation, Information Retrieval
Despite the abundance of design recommendations and guidelines for building a usable web site [4, 5, 10, 11], web site usability continues to be a pressing human-computer interaction issue. Most of the difficulties that users experience in web sites can be attributed to poor information architecture [2, 8, 9, 10]- the grouping of information into categories and the addition of navigational elements over this information structure . Existing approaches and tools provide very limited support for evaluating information architectures and other aspects of web sites, especially during the design phase.
The goal of this research is to develop a new automated methodology and tool, Web TANGO (Tool for Assessing Navigations and Organization), to allow designers of information-centric web sites (i.e., sites designed primarily to provide information) to evaluate the usability of their designs early. Our technique will employ Monte Carlo simulation and information retrieval algorithms to approximate a user's information seeking behavior and report quantitative usability measures (e.g. number of errors and navigation time). This will enable the comparison of different design options before undergoing the costly process of implementation.
We conducted an extensive survey of automated usability evaluation techniques for Web interfaces . Our survey revealed that the majority of existing methods can only be employed late in the design phase (i.e., with existing web sites). Furthermore, none of these methods attempt to evaluate the information architecture of sites.
Our survey revealed one simulation method, WebCriteria's Site Profile , that attempts to mimic a user's information-seeking behavior within a model of an implemented site. Site Profile uses a idealist web user model that follows an explicit navigation path through the site, estimates page load time and navigation time for the shortest path corresponding to the supplied path, and measures content freshness and page composition (e.g. amount of text and graphics). Currently, it does not employ other user models, attempt to predict navigation paths or consider the impact of other page features, such as the number of colors or fonts, in estimating navigation time.
Web TANGO will employ Monte Carlo simulation and information retrieval algorithms to approximate a user's information seeking behavior and report quantitative usability measures (e.g. number of errors and navigation time) as well as simulated navigation paths. Figure 1 depicts Web TANGO's architecture and underlying model of information-seeking behavior, which is based on a study by Bryne et al. in .
Figure 1: Web TANGO architecture.
A typical design scenario entails the designer initially creating several designs (i.e., site models) in TANGO either by specifying information about each page, including page title, metadata, page complexity and link structure, or importing this information from an existing site. The designer would also specify details about the server's latency and load (i.e., server model) and users' information tasks (e.g. destination pages and associated topics of interest). Finally, the designer would specify models of anticipated users with key parameters, such as the reading speed, connection speed, probability that a user will complete a task, read a page, make an error, etc. The designer will also be able to specify constraints in the user model, such as an upper bound on navigation time or a small screen size, using production rules. We will create several reusable models based on observed user behavior in future studies.
After specifying these models, the designer would then run the simulator for each design. The designer specifies starting pages or uses randomly-chosen one. During the simulation run, the simulator mimics a user navigating the site as depicted in Figure 2. The simulator uses information from the server and site model to compute a load time for each page. The simulator then estimates time for scanning the page (i.e., read and think time) based on the page complexity and correlated usability measures. The simulator also updates the user's conceptual model to mimic a user employing scent (i.e., information cues) in locating information .
Figure 2: Simulator behavior during a run.
The simulator must also mimic a user surveying navigation options by computing think time, ranking the options, and selecting an option. We are planning to employ information foraging theory [6, 7] for the ranking algorithm. This approach has been used to present users with relevant web pages in a site , and could be adapted to rank navigation options in TANGO.
After conducting simulations of each design, the designer would use simulation results to select the best design and to inform design improvements.
Web TANGO is a work in progress. Future work entails conducting an online study to correlate page composition (e.g. number of words, links, graphics, fonts, color changes, reading complexity, etc.) with perceived page complexity. The results of this study will enable us to make more accurate predictions of scanning time when page elements are known. We also plan to conduct a user study to develop and validate a navigation prediction algorithm based on prior work on information foraging theory. We will also extract several user models from this study. We will then implement the simulation tool and use a healthcare intranet as an evaluation testbed. We will validate simulator results for comparable designs with observed usage.
Web TANGO: Towards Automated Comparison of Information-centric Web Site Designs
This document was generated using the LaTeX2HTML translator Version 96.1 (Feb 5, 1996) Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
The command line arguments were:
latex2html -split 0 -t Web TANGO: Towards Automated Comparison of Information-centric Web Site Designs poster.
The translation was initiated by Melody Ivory on Thu Apr 13 08:15:45 PDT 2000