Usability Evaluation of Mobile Weather Hazard Alert Applications

Main Article Content

Abdulrahman Khamaj
Ziho Kang


Cell phones enable us to receive and respond to critical incidents, such as: severe storms, tornadoes, and flash floods. However, due to the small display size of cell phones, and regardless of simplified symbols or alert messages, it is possible to overlook users’ ability to interact with the available features and understand the messages in a timely manner. Untrained and trained users of the Weather Radio application participated in an experiment to perform three search tasks; (task 1: location search, task 2: alert settings, and task 3: map settings). In task 4, they evaluated two types of weather alert messages: original National Weather Service (NWS) messages vs. filtered (proposed) messages. By recording users’ completion time on the search tasks, the results showed that the time of the typing in text bar method for task 1 was significantly less than the pin on map method, while much more time was required to complete tasks 2 and 3 by the untrained users compared to the trained users. It was also revealed that the proposed messages were more effective than the original messages by both user groups. This research of user-centered designs provides a foundation to support the designs of time-critical mobile alert systems.

Article Details

How to Cite
Khamaj, A., & Kang, Z. (2018). Usability Evaluation of Mobile Weather Hazard Alert Applications. Industrial and Systems Engineering Review, 6(1), 21-40.


Ahmadi, H., & Kong, J. (2008). Efficient web browsing on small screens. In Proceedings of the working conference on Advanced visual interfaces (pp. 23-30). ACM.

Alluri, A. (2013). Usability testing of android applications (Master’s thesis). Retrieved from

Baharuddin, R., Singh, D., & Razali, R. (2013). Usability dimensions for mobile applications- A review. Research Journal of Applied Sciences, Engineering and Technology, 5, 2225-2231.

Drogalis, T. J., Keyes, E., & Dhyani, L. (2015). The Weather Channel application usability test. Retrieved from

Drost, R., Trobec, J., Steffke, C., & Libarkin, J. (2015). Eye tracking: evaluating the impact of gesturing during televised weather forecasts. Bulletin of the American Meteorological Society, 96(3), 387-392.

Frekjmr, E., Hertzum, M. & Hornbaek, K. (2000). Measuring Usability: Are Effectiveness, Efficiency, and Satisfaction Really Correlated? Retrieved from

Geven, A., Sefelin, R., & Tscheligi, M. (2006). Depth and breadth away from the desktop: the optimal information hierarchy for mobile use. In Proceedings of the 8th conference on Human-computer interaction with mobile devices and services (pp. 157-164). ACM.

Han, S. H., Kim, K. J., Yun, M. H., Hong, S. W., & Kim, J. (2004). Identifying mobile phone design features critical to user satisfaction. Human Factors and Ergonomics in Manufacturing, 14(1), 15–29

International Standards Organization, (1997). ISO 9241: Ergonomic requirements for office work with visual display terminals (VDTs)-Part 11: Guidance on usability. Geneva, Switzerland: International Standards Organization.

John, B. E., & Kieras, D. E. (1996). Using GOMS for user interface design and evaluation: Which technique? ACM Transactions on Computer-Human Interaction (TOCHI), 3(4), 287-319.

Kang, Z. and Landry, S.J. (2015). An eye movement analysis algorithm for a multi-element target tracking task: Maximum transition-based agglomerative hierarchical clustering. IEEE Transactions on Human-Machine Systems, 45(1), 13-24.

Khanum, M. A., & Trivedi, M. C. (2012). Take care: a study on usability evaluation methods for children, International Journal of Advanced Research in Computer Science, 3(2), 101-105.

Krueger, R. A., & Casey, M. A. (2002). Designing and conducting focus group interviews. Social Analysis, Selected Tools and Techniques, 4-23.

Nayebi, F., Desharnais, J. M., & Abran, A. (2012). The state of the art of mobile application usability evaluation. In Proceeding of the 25th Annual IEEE Canadian Conference on Electrical and Computer Engineering (pp. 1-4), IEEE Canada.

Nielsen, J. (1994). Usability engineering. Mountain View, CA: Elsevier.

Nielsen, J. (2012). How Many Test Users in a Usability Study? Retrieved from

Panach, J. I., Condori-Fernández, N., Valverde, F., Aquino, N., & Pastor, Ó. (2008). Understandability measurement in an early usability evaluation for model-driven development: an empirical study. In Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement (pp. 354-356). ACM.

Poole, A., & Ball, L. J. (2006). Eye tracking in HCI and usability research. Encyclopedia of human computer interaction, 1, 211-219.

Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., & Carey, T. (1994). Human-computer interaction. Essex, UK: Addison-Wesley Longman Limited.

Saleh, A. M., & Ismail, R. B. (2015). Usability evaluation frameworks of mobile application: a mini-systematic literature review. In Proceeding of the 3rd Global Summit on Education GSE (pp. 231–239), Global Summit on Education GSE.

Schmiedl, G., Seidl, M., & Temper, K. (2009). Mobile phone web browsing: a study on usage and usability of the mobile web. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services (p. 70). ACM.

Shackel, B. (1991). Usability-context, framework, definition, design and evaluation. Human factors for informatics usability, 21-37.

Shrestha, S. (2007). Mobile web browsing: usability study. In Proceedings of the 4th international conference on mobile technology, applications, and systems and the 1st international symposium on Computer human interaction in mobile technology (pp. 187-194). ACM.

Singhal, B. (2011). Usability testing of iphone (Master’s thesis). Retrieved from

Statistica (2017). Number of smartphone users in the United States from 2010 to 2020. Retrieved from

van Elzakker, C. P., Delikostidis, I., & van Oosterom, P. J. (2008). Field-based usability evaluation methodology for mobile geo-applications. The Cartographic Journal, 45(2), 139-149.

Weather Decision Technology (2016). Retrieved from

Weather Radio (2016). Retrieved from

Whitenton, W (2016). Website Forms Usability: Top 10 Recommendations. Retrieved from

Xu, Q., Erman, J., Gerber, A., Mao, Z., Pang, J., & Venkataraman, S. (2011). Identifying diverse usage behaviors of smartphone apps. In Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference (pp. 329-344). New York, NY: Association for Computing Machinery.