REACT AI
Timely and detailed information is critical during a disaster. With the mainstreaming of social-media-based civic voluntarism during disasters, disaster managers often find themselves in a situation where they get overwhelmed by the information streaming in through various channels. Real-time Emergency Assessment, Coordination, & Triage (REACT) AI is a research project that aims to automate the triage of this incoming image and text information to help disaster managers focus on areas that need the most attention. By working directly with Emergency Operations Centers in Japanese cities, this project closely integrates non-government and community-led organizations in disaster response, strengthening long-term resilience planning.
To understand the existing response protocols, challenges, and opportunities from a community perspective, the research team (Mercy Corps, MIT Urban Risk Lab, and Tsukuba University) designed and conducted a range of semi-structured in-depth interviews. Furthermore, working with the city, we have generated a labeled dataset of over 600 images, including field-level reporting of damage due to flooding and comparative “blue sky” images of everyday situations.
AI for Triage
Interviews with local emergency managers revealed that, when they receive a report, assessing its importance and initiating a response takes an average of 10 minutes. The goal of the AI module is to reduce the time required for assessment by utilizing convolutional neural network (CNN) models that yield efficient and accurate predictions from image data in crowdsourced crisis reports, providing quick categorization tags that the EOC can use to construct an aggregate summary of the unfolding disaster event. We have designed the system to predict four classification tasks: damage severity, humanitarian categories, informativeness, and flood presence. Based on the local EOC protocols, this categorization instantaneously provides a suite of informative labels. In the subsequent phase, an authenticated dashboard will visualize these labels as geolocated clusters of areas of interest. This dashboard will also display a visual interpretability layer and a UI to correct labels that can be used for retraining. We leverage state-of-the-art CNNs to provide these labels in seconds.
Next Steps
While this project successfully demonstrates that AI-based triage can improve real-time assessment, it needs to be refined to consider the subjectivity in multi-label abstract tasks such as categorizing damage severity. We aim to continue working with cities in Japan to assimilate the latent knowledge disaster managers use and create a standardized matrix for labeling disaster-time image data. Furthermore, with Google's support, we aim to scale this approach to integrate different types of flood hazards — mainly riverine floods in Bangladesh and Vietnam and flash floods in Nepal and East Timor.
This work relies in part on research funded by Google for Social Good and Google.org. However, any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors alone and do not necessarily reflect the views of the funding organization.