Characterization of the Construction Period of Buildings in Beirut from Street-View Photos Using Deep Learning

Abstract

Multiple efforts have contributed to collecting data on the built environment in Beirut. However, these efforts usually rely on field visits that are time-consuming. Until now no complete dataset that provides the construction periods and the number of stories of all the buildings of Beirut exists. Such datasets are crucial to perform seismic risk and vulnerability assessments.

This work presents a framework that uses artificial intelligence practices for automated building characterization, with a focus on the construction period of buildings in Beirut. A total of five construction periods are identified and used in this work which are: pre1935, 19351-955, 1956-1971, 1972-1990, and post1990. The framework is implemented into a tool that can predict the period of construction of a building from clear street-view photographs of its facade. The tool relies on deep learning methods to analyze the features of the facade image and output the estimated building’s construction period. Two approaches are used. The first approach only requires as input a street-view image of the building’s façade, and relies on a deep learning model to autonomously learn the features in the image and classify it into a construction period. The second approach requires tabular data as input in addition to a street-view image of the façade, and relies on a multi-modal machine learning model that uses late fusion to join the features extracted from two different models: the first is a deep learning model that uses the street-view image, and the second is a fully connected neural network model that learns from the tabular input, in particular the number of floors and socioeconomic background.

To develop all these models, a unified dataset on the building stock of Beirut is assembled that includes a construction period label and at least one street-view photo for each building. For this purpose, data is compiled from multiple sources that provide building street-view photos and/or construction periods. In addition, information on the number of floors and location of the building is stored in the database when available. After collecting and pre-processing the data, and validating, correcting or adding construction period and number of floors label when possible, we end up with a total of 7,198 data points that all contain street-view images and are labeled with a construction period. Among them, 6,480 also have a number of floors label and contain information on their location (either exact or approximate, e.g., the name of the neighborhood).

In approach number one, three state of the art models are trained and compared: DenseNet121, ResNet50, and Swin Transformer v2. The latter is found to be the best performing model, reaching an accuracy of 75% on the test set. Utilizing additional tabular data as input in the second approach is found to be beneficial to the overall performance of the model, with a slight increase in accuracy to 78%. Nonetheless, both approaches have a weakness when trying to predict buildings in the “1972-1990” period, with an accuracy less than 50% for this period which can be correlated to the lack of image examples in the mentioned period. The developed model can be used to rapidly characterize cultural heritage buildings, and help in urban planning efforts through the characterization of the built environment. Furthermore, the usefulness of the proposed tool for seismic risk assessment is illustrated by developing seismic damage and loss maps for a target neighborhood in Beirut.