Andrina Granić, Ivica Mitrović, Nikola Marangunić



The paper reports on the experience regarding usability evaluation of broad-reach web portals. The designed methodology advocates a number of usability test methods along with specialists' inspection and provides a comprehensive quantitative and qualitative assessment feedback. Our experience accords with the claim that we should not rely on isolated evaluations, but instead on combine assessment methods. In this paper we present and discuss mainly the results obtained through user-based usability evaluation. The results indicate that chosen research instruments, measures and methods for usability testing were consistent. Conversely, the results of the designed guideline inspection did not conform to the ones obtained through the end user testing. Although proved promising, some of the guideline-based evaluation aspects need improvement.


  1. Bevan, N., Macleod, M., 1994. Usability measurement in context. Behaviour & Information Technology, 13, pp. 132-145.
  2. Boye, J., 2006. Improving portal usability. CMS Watch. http://www.cmswatch.com/Feature/151-PortalUsability.
  3. Brantley, S., Armstrong, A., Lewis, K.M., 2006. Usability Testing of a Customizable Library Web Portal. College & Research Libraries 67, 2.
  4. Brooke, J., 1996. SUS: a "quick and dirty" usability scale. In Jordan, P.W., Thomas, B., Weerdmeester, B.A, McClelland, A.L. (Eds.): Usability Evaluation in Industry. Taylor and Francis, London.
  5. Carstens, D.S., Patterson, P., 2005. Usability Study of travel websites. Journal of Usability Studies, 1, 1.
  6. Fang, X., Rau, P.L.P., 2003. Culture differences in design of portal sites. Ergonomics 46, 1-3.GemiusAudience, 2007. http://www.cati.hr/ GemiusAudience-ozujak.pdf.
  7. GFK Croatia, 2006. http://www.gfk.hr/press1/ internet.htm.
  8. Granic, A., Glavinic, V., Stankov, S., 2004. Usability Evaluation Methodology for Web-based Educational Systems. In Stary, C., Stephanidis, C. (Eds.): Proceedings of the 8th ERCIM Workshop "User Interfaces for All". ERCIM, Heraklion (Crete), Greece, 28.1-28.15.
  9. Hassenzahl, M., 2004. The interplay of beauty, goodness, and usability in interactive products. HumanComputer interaction 19, 4.
  10. Holzinger, A., 2005. Usability Engineering methods for Software developers. Communications of the ACM 48, 1, 71-74.
  11. Hornbaek, K., 2006. Current practice in measuring usability: Challenges to usability studies and research. International Journal of Man-Machine Studies 64, 2.
  12. Hornbaek, K., Law, E.L-C., 2007. Meta-Analysis of Correlations Among Usability Measures. CHI 2007 Proceedings, April 28-May 3, San Jose, California, USA.
  13. Hvannberg, E., Law, E.L-C., Larusdottir, M., 2007. Heuristic evaluation: Comparing ways of finding and reporting usability problems. Interacting with Computers 19, 225-240.
  14. ISO/IEC 25062:2006, 2006. Software engineering - Software product Quality Requirements and Evaluation (SQuaRE) - Common Industry Format (CIF) for usability test reports.
  15. Klausegger, C., 2006. Evaluating Internet Portals - An Empirical Study of Acceptance Measurement Based on the Austrian National Tourist Office's Service Portal. Journal of Quality Assurance in Hospitality & Tourism 6, 3-4.
  16. Lewis, J. R., 2005. Introduction to Usability Testing. Tutorial given at HCI International 2005. July 22-27. Las Vegas, Nevada.MIT Usability Guidelines, 2004. http://web.mit.edu/is/usability/selected.guidelines.pdf.
  17. Nielsen, J., 1993. Usability Engineering. Academic Press Limited. Boston.
  18. Nielsen, J., 1994. Heuristic Evaluation. In Nielsen, J., Mack, R.L. (Eds.): Usability Inspection Methods. John Wiley and Sons Inc, New York.
  19. Nielsen, J., Tahir, M., 2002. Homepage Usability: 50 Websites Deconstructed. New Riders Publishing, Indianapolis.
  20. Preece, J., Rogers, Y., Sharp, H., 2002. Interaction design: beyond human-computer interaction. John Wiley & Sons, New York.
  21. Shackel, B., 1991. Usability - context, framework, design and evaluation. In Shackel, B., Richardson, S. (eds.): Human Factors for Informatics Usability, Cambridge University Press, Cambridge, 21-38.
  22. Shelat, B., Stewart, T., 2004. Transport Direct Portal. Usability Testing Report. http:// www.transportdirect.gov.uk/research/pdf/mr61r1.pdf Tatnall, A. (Ed.), 2005. Web portals: the new gateways to Internet information and services. Idea Group Publishing, Hershey.
  23. Theng, Y.L., Soh, E.S., 2005. An Asian Study of Healthcare Web Portals: Implications for Healthcare Digital Libraries. Proceedings of the 8th international conference on Asian digital libraries, ICADL. Bangkok.
  24. Tomic-Koludrovic, I., Petric, M., 2004. Identities on the Net: Gender and National Stereotypes on Croatian Broad-Reach Portals. Društvena istraživanja 13, 4-5.
  25. Tractinsky, N., Katz, A., Ikar, D., 2000. What is beautiful is usable. Interacting with Computers 13, 2.
  26. Tsui, W.C., Paynter, J., 2004. Cultural Usability in the Globalisation of News Portal. In Masoodian, M., Jones, S., and Rogers, B. (Eds.): Computer Human Interaction, Proceedings of the 6th Asia Pacific Conference, APCHI. Lecture Notes in Computer Science 3101. Springer.
  27. Tullis, T.S., Stetson, J.N., 2004. A Comparison of Questionnaires for Assessing Website Usability. Proceedings of UPA Conference. Minneapolis. http://home.comcast.net/tomtullis/publications/UPA2 004TullisStetson.pdf.
  28. Uldall-Espersen, T., Frøkjaer, E., Hornbaek, K., 2007. Tracing Impact in a Usability Improvement. Interacting with Computers. (in press)
  29. Waloszek, G., 2001. Portal Usability - Is There Such A Thing? SAP Design Guild, Edition 3. http:// www.sapdesignguild.org/editions/edition3/ overview_edition3.asp.
  30. Wood, J., 2004. Usability Heuristics Explained, iQ Content.http://www.iqcontent.com/publications/featur es/article_32.
  31. Wright, P., Monk, A., 1991. A cost-effective evaluation method for use by designers. International Journal of Man-Machine Studies 35, 6, 891-912.

Paper Citation

in Harvard Style

Granić A., Mitrović I. and Marangunić N. (2008). EXPERIENCE WITH USABILITY TESTING OF WEB PORTALS . In Proceedings of the Fourth International Conference on Web Information Systems and Technologies - Volume 2: WEBIST, ISBN 978-989-8111-27-2, pages 161-168. DOI: 10.5220/0001520801610168

in Bibtex Style

author={Andrina Granić and Ivica Mitrović and Nikola Marangunić},
booktitle={Proceedings of the Fourth International Conference on Web Information Systems and Technologies - Volume 2: WEBIST,},

in EndNote Style

JO - Proceedings of the Fourth International Conference on Web Information Systems and Technologies - Volume 2: WEBIST,
SN - 978-989-8111-27-2
AU - Granić A.
AU - Mitrović I.
AU - Marangunić N.
PY - 2008
SP - 161
EP - 168
DO - 10.5220/0001520801610168