Neural Networks for Data Science Applications
Master's Degree in Data Science (2020-2021)
Important Info
A Google Group is active to receive all info on the course: https://groups.google.com/a/uniroma1.it/forum/#!forum/neural-networks-for-data-science-applications-20202021
The exam is composed of a homework and an oral part. For the homework, refer to the instructions for the final homework below.
Timetable (updated): Wednesday 9-11 AM, Thursday 8-11 AM. Lectures will begin on October 5 (official notice).
In-person attendance: Via Ariosto 25, Room A4 (Wednesday), Room B2 (Thursday).
Remote attendance (Zoom): 863 3832 0837 (Wednesday), 829 0383 9086 (Thursday). Passcodes will be provided on the Google Group only.
General overview
The course will introduce neural networks in the context of data science applications. After an overview on supervised learning and numerical optimization, we will describe recent techniques and algorithms (going under the broad name of “deep learning” or differentiable programming), that allows to successfully apply neural networks to a wide range of problems, e.g., in computer vision and natural language processing.
Students will be introduced to convolutional networks (e.g., for image analysis), to recurrent neural networks (for sequential problems), and to recent attention-based models. We will also introduce problems of robustness, fairness, and interpretability. Optional topics include graph-based model and generative architectures.
Theory will be supplemented by practical laboratories where all concepts will be developed on realistic use cases through the use of the TensorFlow 2.x library.
Slides and notebooks
| Â | Date | Content | Material |
|---|---|---|---|
| 1 | 07/10/2020 | About the course | Slides Video |
| 2 | 08/10/2020 | Introduction (key concepts, history, …) | Slides Video |
| Lab 1 | 14-15/10/2020 | Lab: preliminaries (linear algebra, probability, gradients) | Chapter 2 from the book Video (Part 1) Video (Part 2, until 1h30m) |
| 3 | 15/10/2020 | Linear regression and classification | Slides Video (Part 1, from 1h45m) Video (Part 2) Video (Part 3) |
| Lab 2 | 22/10/2020 | Lab: linear regression from scratch | Notebook Video |
| 4 | 28-29/10/2020 | Feedforward neural networks | Slides Video (Part 1) Video (Part 2) |
| Lab 3 | 04-05/11/2020 | Lab: feedforward neural networks & tf.keras | Notebook Video (Part 1) Video (Part 2) |
| 5 | 12/11/2020 | Convolutional neural networks | Slides Video (Part 1) Video (Part 2) |
| Lab 4 | 12-18-19/11/2020 | Lab: steering a car with convolutional networks | Notebook Video (Part 1) Video (Part 2) Video (Part 3) |
| Homework 1 | NA | Implementing a custom activation function. Deadline: |
Template Solution Evaluation |
| 6 | 19-25-26/11/2020 | Building deeper convolutional networks | Slides Video (Part 1) Video (Part 2) Video (Part 3) |
| Lab 5 | 26/11/2020 | Implementing a deep CNN from scratch | Notebook Video (Part 1) Video (Part 2) |
| 7 | 03-10/12/2020 | Going beyond image classification | Slides Video (Part 1) Video (Part 2) Video (Part 3) |
| Lab 6 | 03/12/2020 | Lab: audio classification and hyperparameter tuning | Notebook Video (Part 1) Video (Part 2) |
| Extra | 10/12/2020 | Notebook on handling word embeddings with TensorFlow | Notebook |
| 8 | 16/12/2020 | Fairness, robustness, and interpretability | Slides Video (Part 1) Video (Part 2) |
| Homework 2 | NA | Putting it all together. Deadline: two days prior to the exam | Template Video |
| 9 | 21/12/2020 | Recurrent neural networks and seq2seq models | Slides Video |
Environment setup
Students are invited to bring their own laptop for the lab sessions. In order to have a working Python installation with all prerequisites, you can install the Anaconda distribution.
We will use TensorFlow 2.x in the course, that you can install following the instructions from the website.
Alternatively, you can run all notebooks freely using the Google Colaboratory service (which you can access with a standard Gmail account or the uniroma1.it account).
Reading material
The main reference book for the course is Dive into Deep Learning. Each set of slides will mention the corresponding sections in the book.