Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video

Soroush Shahi; Rawan Alharbi; Yang Gao; Sougata Sen; Aggelos K. Katsaggelos; Josiah Hester; Nabil Alshurafa

doi:10.1109/PerComWorkshops53856.2022.9767447

Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video

Soroush Shahi
, Rawan Alharbi
, Yang Gao
, Sougata Sen
, Aggelos K. Katsaggelos
, Josiah Hester
, Nabil Alshurafa

Northwestern University
Birla Institute of Technology and Science Pilani

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Scopus citations

Abstract

Automated detection and validation of fine-grained human activities from egocentric vision has gained increased attention in recent years due to the rich information afforded by RGB images. However, it is not easy to discern how much rich information is necessary to detect the activity of interest reliably. Localization of hands and objects in the image has proven helpful to distinguishing between hand-related fine-grained activities. This paper describes the design of a hand-object-based mask obfuscation method (HOBM) and assesses its effect on automated recognition of fine-grained human activities. HOBM masks all pixels other than the hand and object in-hand, improving the protection of personal user information (PUI). We test a deep learning model trained with and without obfuscation using a public egocentric activity dataset with 86 class labels and achieve almost similar classification accuracies (2% decrease with obfuscation). Our findings show that it is possible to protect PUI at smaller image utility costs (loss of accuracy).

Original language	English
Title of host publication	Proceedings - 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	341-346
Number of pages	6
ISBN (Electronic)	9781665416474
DOIs	https://doi.org/10.1109/PerComWorkshops53856.2022.9767447
State	Published - 2022
Externally published	Yes
Event	20th IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022 - Pisa, Italy Duration: 21 Mar 2022 → 25 Mar 2022

Publication series

Name	2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022

Conference

Conference	20th IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022
Country/Territory	Italy
City	Pisa
Period	21/03/22 → 25/03/22

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

Deep Learning
Human Activity Recognition
Image Obfuscation
Wearable Camera

Access to Document

10.1109/PerComWorkshops53856.2022.9767447

Cite this

Shahi, S., Alharbi, R., Gao, Y., Sen, S., Katsaggelos, A. K., Hester, J., & Alshurafa, N. (2022). Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video. In Proceedings - 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022 (pp. 341-346). (2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/PerComWorkshops53856.2022.9767447

Shahi, Soroush ; Alharbi, Rawan ; Gao, Yang et al. / Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video. Proceedings - 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 341-346 (2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022).

@inproceedings{da1f76daef334382adb28ec0569dfdd7,

title = "Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video",

abstract = "Automated detection and validation of fine-grained human activities from egocentric vision has gained increased attention in recent years due to the rich information afforded by RGB images. However, it is not easy to discern how much rich information is necessary to detect the activity of interest reliably. Localization of hands and objects in the image has proven helpful to distinguishing between hand-related fine-grained activities. This paper describes the design of a hand-object-based mask obfuscation method (HOBM) and assesses its effect on automated recognition of fine-grained human activities. HOBM masks all pixels other than the hand and object in-hand, improving the protection of personal user information (PUI). We test a deep learning model trained with and without obfuscation using a public egocentric activity dataset with 86 class labels and achieve almost similar classification accuracies (2\% decrease with obfuscation). Our findings show that it is possible to protect PUI at smaller image utility costs (loss of accuracy).",

keywords = "Deep Learning, Human Activity Recognition, Image Obfuscation, Wearable Camera",

author = "Soroush Shahi and Rawan Alharbi and Yang Gao and Sougata Sen and Katsaggelos, \{Aggelos K.\} and Josiah Hester and Nabil Alshurafa",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 20th IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022 ; Conference date: 21-03-2022 Through 25-03-2022",

year = "2022",

doi = "10.1109/PerComWorkshops53856.2022.9767447",

language = "英语",

series = "2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "341--346",

booktitle = "Proceedings - 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022",

address = "美国",

}

Shahi, S, Alharbi, R, Gao, Y, Sen, S, Katsaggelos, AK, Hester, J & Alshurafa, N 2022, Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video. in Proceedings - 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022. 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022, Institute of Electrical and Electronics Engineers Inc., pp. 341-346, 20th IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022, Pisa, Italy, 21/03/22. https://doi.org/10.1109/PerComWorkshops53856.2022.9767447

Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video. / Shahi, Soroush; Alharbi, Rawan; Gao, Yang et al.
Proceedings - 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 341-346 (2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video

AU - Shahi, Soroush

AU - Alharbi, Rawan

AU - Gao, Yang

AU - Sen, Sougata

AU - Katsaggelos, Aggelos K.

AU - Hester, Josiah

AU - Alshurafa, Nabil

PY - 2022

Y1 - 2022

N2 - Automated detection and validation of fine-grained human activities from egocentric vision has gained increased attention in recent years due to the rich information afforded by RGB images. However, it is not easy to discern how much rich information is necessary to detect the activity of interest reliably. Localization of hands and objects in the image has proven helpful to distinguishing between hand-related fine-grained activities. This paper describes the design of a hand-object-based mask obfuscation method (HOBM) and assesses its effect on automated recognition of fine-grained human activities. HOBM masks all pixels other than the hand and object in-hand, improving the protection of personal user information (PUI). We test a deep learning model trained with and without obfuscation using a public egocentric activity dataset with 86 class labels and achieve almost similar classification accuracies (2% decrease with obfuscation). Our findings show that it is possible to protect PUI at smaller image utility costs (loss of accuracy).

AB - Automated detection and validation of fine-grained human activities from egocentric vision has gained increased attention in recent years due to the rich information afforded by RGB images. However, it is not easy to discern how much rich information is necessary to detect the activity of interest reliably. Localization of hands and objects in the image has proven helpful to distinguishing between hand-related fine-grained activities. This paper describes the design of a hand-object-based mask obfuscation method (HOBM) and assesses its effect on automated recognition of fine-grained human activities. HOBM masks all pixels other than the hand and object in-hand, improving the protection of personal user information (PUI). We test a deep learning model trained with and without obfuscation using a public egocentric activity dataset with 86 class labels and achieve almost similar classification accuracies (2% decrease with obfuscation). Our findings show that it is possible to protect PUI at smaller image utility costs (loss of accuracy).

KW - Deep Learning

KW - Human Activity Recognition

KW - Image Obfuscation

KW - Wearable Camera

UR - https://www.scopus.com/pages/publications/85130619122

U2 - 10.1109/PerComWorkshops53856.2022.9767447

DO - 10.1109/PerComWorkshops53856.2022.9767447

M3 - 会议稿件

AN - SCOPUS:85130619122

T3 - 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022

SP - 341

EP - 346

BT - Proceedings - 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 20th IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022

Y2 - 21 March 2022 through 25 March 2022

ER -

Shahi S, Alharbi R, Gao Y, Sen S, Katsaggelos AK, Hester J et al. Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video. In Proceedings - 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 341-346. (2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events, PerCom Workshops 2022). doi: 10.1109/PerComWorkshops53856.2022.9767447

Impacts of Image Obfuscation on Fine-grained Activity Recognition in Egocentric Video

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this