Neural Networks for Data Science Applications

Master's Degree in Data Science (2020-2021)

Important Info

A Google Group is active to receive all info on the course: https://groups.google.com/a/uniroma1.it/forum/#!forum/neural-networks-for-data-science-applications-20202021

The exam is composed of a homework and an oral part. For the homework, refer to the instructions for the final homework below.

Timetable (updated): Wednesday 9-11 AM, Thursday 8-11 AM. Lectures will begin on October 5 (official notice).

In-person attendance: Via Ariosto 25, Room A4 (Wednesday), Room B2 (Thursday).

Remote attendance (Zoom): 863 3832 0837 (Wednesday), 829 0383 9086 (Thursday). Passcodes will be provided on the Google Group only.

General overview

The course will introduce neural networks in the context of data science applications. After an overview on supervised learning and numerical optimization, we will describe recent techniques and algorithms (going under the broad name of “deep learning” or differentiable programming), that allows to successfully apply neural networks to a wide range of problems, e.g., in computer vision and natural language processing.

Students will be introduced to convolutional networks (e.g., for image analysis), to recurrent neural networks (for sequential problems), and to recent attention-based models. We will also introduce problems of robustness, fairness, and interpretability. Optional topics include graph-based model and generative architectures.

Theory will be supplemented by practical laboratories where all concepts will be developed on realistic use cases through the use of the TensorFlow 2.x library.

Slides and notebooks

  Date Content Material
1 07/10/2020 About the course Slides
Video
2 08/10/2020 Introduction (key concepts, history, …) Slides
Video
Lab 1 14-15/10/2020 Lab: preliminaries (linear algebra, probability, gradients) Chapter 2 from the book
Video (Part 1)
Video (Part 2, until 1h30m)
3 15/10/2020 Linear regression and classification Slides
Video (Part 1, from 1h45m)
Video (Part 2)
Video (Part 3)
Lab 2 22/10/2020 Lab: linear regression from scratch Notebook
Video
4 28-29/10/2020 Feedforward neural networks Slides
Video (Part 1)
Video (Part 2)
Lab 3 04-05/11/2020 Lab: feedforward neural networks & tf.keras Notebook
Video (Part 1)
Video (Part 2)
5 12/11/2020 Convolutional neural networks Slides
Video (Part 1)
Video (Part 2)
Lab 4 12-18-19/11/2020 Lab: steering a car with convolutional networks Notebook
Video (Part 1)
Video (Part 2)
Video (Part 3)
Homework 1 NA Implementing a custom activation function. Deadline: 26/11/2020 (03/12/2020 - postponed) Template
Solution
Evaluation
6 19-25-26/11/2020 Building deeper convolutional networks Slides
Video (Part 1)
Video (Part 2)
Video (Part 3)
Lab 5 26/11/2020 Implementing a deep CNN from scratch Notebook
Video (Part 1)
Video (Part 2)
7 03-10/12/2020 Going beyond image classification Slides
Video (Part 1)
Video (Part 2)
Video (Part 3)
Lab 6 03/12/2020 Lab: audio classification and hyperparameter tuning Notebook
Video (Part 1)
Video (Part 2)
Extra 10/12/2020 Notebook on handling word embeddings with TensorFlow Notebook
8 16/12/2020 Fairness, robustness, and interpretability Slides
Video (Part 1)
Video (Part 2)
Homework 2 NA Putting it all together. Deadline: two days prior to the exam Template
Video
9 21/12/2020 Recurrent neural networks and seq2seq models Slides
Video

Environment setup

Students are invited to bring their own laptop for the lab sessions. In order to have a working Python installation with all prerequisites, you can install the Anaconda distribution.

We will use TensorFlow 2.x in the course, that you can install following the instructions from the website.

Alternatively, you can run all notebooks freely using the Google Colaboratory service (which you can access with a standard Gmail account or the uniroma1.it account).

Reading material

The main reference book for the course is Dive into Deep Learning. Each set of slides will mention the corresponding sections in the book.