Roberto Amoroso

ELLIS PhD Student, AI & Computer Vision
International Doctorate in ICT

LMU @ Ludwig-Maximilians-Universität of Munich

AImageLab @ University of Modena and Reggio Emilia

About me

My name is Roberto Amoroso. I am an ELLIS PhD student enrolled in the International Doctorate in ICT program at the AImageLab research group of the University of Modena and Reggio Emilia 🇮🇹, under the supervision of Prof. Rita Cucchiara and Prof. Lorenzo Baraldi. I am involved in the study and development of novel Computer Vision and Deep Learning techniques.

I am currently a PhD Intern at LMU - Ludwig-Maximilians-Universität of Munich, in Germany 🇩🇪, focusing on Multimodal Video Understanding with Large Language Models (LLM) and Open-vocabulary Image Segmentation, under the co-supervision of Prof. Volker Tresp.

Prior to joining AImageLab, I was Research Scholar at the Networking Research Group in Saint Louis, USA 🇺🇸, working on Super-resolution techniques applied to Internet traffic matrices.

My primary research focus is on Image Segmentation, which includes multimodal machine learning approaches for learning new semantic concepts from textual information in a weakly/un-supervised way, aka Open-vocabulary Segmentation. In addition, I have also conducted research on the optimization of Transformer-based architecture for image classification.

My primary areas of research are Open-vocabulary Segmentation and Multimodal Video Understanding. In addition, I have also conducted research on the pre-training and optimization of Transformer-based architecture for image classification, self-supervised learning, deepfake detection of synthetic images, and the development of image watermarking systems for artwork protection.

Feel free to reach me out if you have any questions or curiosities! :)

Interests

Computer Vision
Deep Learning
Machine Learning
Image Segmentation
Multimodal Machine Learning

Education

ELLIS PhD in AI and Computer Vision, 2024
University of Modena and Reggio Emilia
MS in Artificial Intelligence, 2020
University of Modena and Reggio Emilia
BS in Computer Engineering, 2018
University of Modena and Reggio Emilia

Recent News

All news »

[Oct. 2023] 2 papers accepted @ WACV 2024

[Sep. 2023] I attended the ELLIS Summer School on Large-Scale AI in Modena, Italy 🇮🇹

[Sep. 2023] I attended the ELLIS Doctoral Symposium 2023 (EDS2023) in Helsinki, Finland 🇫🇮

[Aug. 2023] Our paper “Superpixel Positional Encoding to Improve ViT-based Semantic Segmentation Models” has been accepted @ BMVC 2023

[Jul. 2023] Our paper “Enhancing Open-Vocabulary Semantic Segmentation with Prototype Retrieval” has been accepted @ ICIAP 2023

[Jun. 2023] Started my PhD Internship @ LMU in Munich, Germany 🇩🇪, working on Multimodal Video Perception under the co-supervision of Prof. Volker Tresp

[Jun. 2023] Read our new preprint “MaPeT: Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training”

[Apr. 2023] Read our new preprint “Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images”

[Oct. 2022] Our paper “A Federated Learning Approach to Traffic Matrix Estimation using Super-resolution Techniques” has been accepted @ CCNC 2023

[Sep. 2022] I gave coding classes about “Image Processing and Advanced deep architectures for visual cognitive systems” at the School in AI: Deep Learning, Vision and Language for Industry - second edition

[Jul. 2022] I attended the International Computer Vision Summer School (ICVSS) 2022

[May 2022] Our paper “Investigating Bidimensional Downsampling in Vision Transformer Models” received the Best Paper Award sponsored by NVIDIA @ ICIAP 2021

[Mar. 2022] I gave a short course about “AI & HPC for Industries” on AI, computer vision, and HPC organized by CINECA and Leonardo AI Labs as part of the EuroCC Italy project

[Feb. 2022] I gave coding classes about “Image Processing and Computer Vision” at the School in AI: Deep Learning, Vision and Language for Industry - first edition

[Nov. 2021] I started my ELLIS PhD and International Doctorate in ICT at the AImageLab research group at UNIMORE, Italy

[Jun. 2021] I presented our paper “Assessing the Role of Boundary-Level Objectives in Indoor Semantic Segmentation” @ CAIP 2021

[Jun. 2021] I presented our paper “Improving Indoor Semantic Segmentation with Boundary-Level Objectives” @ IWANN 2021

[Feb. 2021] I joined the AImageLab group at UNIMORE in Italy 🇮🇹 as a Research Fellow

[Dec. 2020] Our paper “Estimation of Traffic Matrices via Super-resolution and Federated Learning” received the Best Poster Award @ CoNEXT 2020

[Mar. 2020] I joined the NRG group at Saint Louis University in the USA 🇺🇸 as a Research Scholar

All news »

Experience

PhD Intern

LMU @ Ludwig-Maximilians-Universität of Munich

Jun 2023 – Present Munich, Germany 🇩🇪

Research activity focused on the development of novel Multimodal Video Understanding and Open-vocabulary Image Segmentation techniques, under the co-supervision of Prof. Volker Tresp.

ELLIS PhD Student | International Doctorate in ICT

AImageLab @ University of Modena and Reggio Emilia

Nov 2021 – Present Modena, Italy 🇮🇹

The European Laboratory for Learning and Intelligent Systems (ELLIS) supports cutting-edge machine learning research in Europe. ELLIS PhD students (<5% 2021 acceptance rate) are selected on the basis of outstanding academic achievement. My research activity is focused on multimodal self-attentive Transformer-based models for visual semantic understanding, with applications to generative problems and visual semantic tasks.
1st in the ranking of student candidates for the International Doctorate in ICT.

Research Fellow

AImageLab @ University of Modena and Reggio Emilia

Feb 2021 – Nov 2021 Modena, Italy 🇮🇹

Research activity under the supervision of Prof. Rita Cucchiara and Prof. Lorenzo Baraldi, aimed at the study, analysis, and development of novel Computer Vision and Deep Learning techniques.

Research Engineer

CINI - Consorzio Interuniversitario Nazionale per l’Informatica

Nov 2020 – Jan 2021 Modena, Italy 🇮🇹

Development of a web platform for the management of data concerning the activities of European research centers, as part of the HumanE-AI-NET project, funded by the EU Framework Programme for Research and Innovation Horizon 2020.

Research Scholar

Saint Louis University

Mar 2020 – Sep 2020 St. Louis, USA 🇺🇸

Carried out research aimed at writing my MS thesis
Created an architecture that allows estimating Internet traffic matrices, combining a Computer Vision technique called Super-resolution with a distributed machine learning technique called Federated Learning

Erasmus+

AGH Akademia Górniczo-Hutnicza

Sep 2019 – Feb 2020 Krakow, Poland 🇵🇱

Attended the following courses:

Advanced Python Programming
Computer Vision
Cybersecurity and Cryptography
Programming in Javascript
Mobile App Development

Honors and Awards

[Sep. 2023] Best Project Award @ ELLIS Summer School on Large-Scale AI for the project “Radioactive Watermarks”, in which we investigate the “radioactivity” of watermarked texts, i.e. , whether they contaminate other models when used as fine-tuning data. The project was evaluated by Tal Hassner (Meta AI) and Marc’Aurelio Ranzato (DeepMind)

[Jul. 2022] ICVSS 2022 Reading Group Competition Award sponsored by Amazon Web Services (AWS) at ICVSS 2022 for participation in the reading group led by Prof. Dr. Stefano Soatto (leading science for AI Applications at AWS and Professor at the University of California Los Angeles)

[Mar. 2022] Best Paper Award sponsored by NVIDIA at ICIAP 2021 for our paper “Investigating Bidimensional Downsampling in Vision Transformer Models”

[Dec. 2020] Best Poster Award at CoNEXT 2020 for our paper “Estimation of Traffic Matrices via Super-resolution and Federated Learning”

Featured Publications

Roberto Amoroso, Matteo Tomei, Lorenzo Baraldi, Rita Cucchiara

August, 2023 In British Machine Vision Conference (BMVC) 2023 Vision Transformer, Segmentation, Superpixel, Positional Encoding

Superpixel Positional Encoding to Improve ViT-based Semantic Segmentation Models

[ BMVC 2023 ] We present a novel superpixel-based positional encoding technique that combines Vision Transformer (ViT) features with superpixels priors to improve the performance of semantic segmentation architectures.

Lorenzo Baraldi, Roberto Amoroso, Marcella Cornia, Lorenzo_Baraldi, Andrea Pilzer, Rita Cucchiara

June, 2023 Self-supervised learning, Vision-and-Language, CLIP

MaPeT: Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training

We propose a novel self-supervised pre-training technique for Vision Transformer called MaPeT and a novel image tokenizer called k-CLIP which directly employs discretized CLIP features.

Roberto Amoroso, Davide Morelli, Marcella Cornia, Lorenzo Baraldi, Alberto Del Bimbo, Rita Cucchiara

April, 2023 Multimodal Deepfakes, Vision-and-Language, Generative Models

Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images

We propose a novel deepfake detection method for images generated through Diffusion Models and created a new dataset COCO-Fake consisting of 650K generated fake images.

Paolo Bruno, Roberto Amoroso, Marcella Cornia, Silvia Cascianelli, Lorenzo Baraldi, Rita Cucchiara

March, 2022 In International Conference on Image Analysis and Processing (ICIAP) 2021 Vision Transformer, Classification, 2D Pooling

Investigating Bidimensional Downsampling in Vision Transformer Models

[ ICIAP 2021 | Best Paper Award sponsored by NVIDIA ] We explore the application of a 2D max-pooling operator to improve the efficiency of Transformer-based architecture for classification.

Roberto Amoroso, Flavio Esposito, Maria Luisa Merani

December, 2020 In International Conference on Emerging Networking EXperiments and Technologies (CoNEXT) 2020 Deep Learning, Super-resolution, Network measurement, Network management

Estimation of Traffic Matrices via Super-resolution and Federated Learning

[ CoNEXT 2020 | Best Poster Award ] Inspired by the field of image processing, we propose a super-resolution technique for Internet traffic matrix inference.