The Priberam Machine Learning Lunch Seminars are a series of informal meetings which occur every two weeks at Instituto Superior Técnico, in Lisbon. It works as a discussion forum involving different research groups, from IST and elsewhere. Its participants are interested in areas such as (but not limited to): statistical machine learning, signal processing, pattern recognition, computer vision, natural language processing, computational biology, neural networks, control systems, reinforcement learning, or anything related (even if vaguely) with machine learning.

The seminars last for about one hour (including time for discussion and questions) and revolve around the general topic of Machine Learning. The speaker is a volunteer who decides the topic of his/her presentation. Past seminars have included presentations about state-of-the-art research, surveys and tutorials, practicing a conference talk, presenting a challenging problem and asking for help, and illustrating an interesting application of Machine Learning such as a prototype or finished product.

Presenters can have any background: undergrads, graduate students, academic researchers, company staff, etc. Anyone is welcome both to attend the seminar as well as to present it. Ocasionally we will have invited speakers. See below for a list of all seminars, including the speakers, titles and abstracts.

Note: The seminars are held at lunch-time, and include delicious free food.

Feel free to join our mailing list, where seminar topics are announced beforehand. You may also visit the group webpage. Anyone can attend the seminars. If you would like to present something, please send us an email.

The seminars are usually held every other Thursday (previously on Tuesday), from 1 PM to 2 PM, at the IST campus in Alameda. This sometimes changes due to availability of the speakers, so check regularly!

Thursday, February 21st 2019, 13h00 - 14h00

João Cartucho, (ISR)

Robust Object Recognition Through Symbiotic Deep Learning In Mobile Robots


Instituto Superior Técnico - Alameda


Despite the recent success of state-of-the-art deep learning algorithms in object recognition, when these are deployed as-is on a mobile service robot, we observed that they failed to recognize many objects in real human environments. In this paper, we introduce a learning algorithm in which robots address this flaw by asking humans for help, also known as a symbiotic autonomy approach. In particular, we bootstrap YOLOv2, a state-of-the-art deep neural network and train a new neural network, that we call HHELP, using only data collected from human help. Using an RGB camera and an on-board tablet, the robot proactively seeks human input to assist it in labeling surrounding objects. Pepper, located at CMU, and Monarch Mbot, located at ISR-Lisbon, were the service robots that we used to validate the proposed approach. We conducted a study in a realistic domestic environment over the course of 20 days with 6 research participants. To improve object detection, we used the two neural networks, YOLOv2 + HHELP, in parallel. Following this methodology, the robot was able to detect twice the number of objects compared to the initial YOLOv2 neural network, and achieved a higher mAP (mean Average Precision) score. Using the learning algorithm the robot also collected data about where an object was located and to whom it belonged to by asking humans. This enabled us to explore a future use case where robots can search for a specific person's object. We view the contribution of this work to be relevant for service robots in general, in addition to Pepper, and Mbot.


Bio: João Cartucho, a IST alumni at ISR, former MSc student co-supervised by Manuela Veloso and Rodrigo Ventura, was finalist to the Best Paper Award at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018).

Eventbrite - Robust Object Recognition Through Symbiotic Deep Learning In Mobile Robots