Introduction to Deep Learning
(Tentative)
Overview
This course explores fundamentals of deep neural networks and their applications in various machine learning tasks. We will cover the fundamentals of deep learning (the learning problem, backpropagation, optimization, loss functions, regularization, generalization, etc.), various neural network architectures (CNN, RNN, transformers), generative models, self-supervised training, and techniques for efficiency and interpretability. Students will get familiar with deep learning, understand various design choices, gain hands-on experience in implementing and applying deep learning through assignments and a research- or engineering-focused course project.
Official description in the catalog: 4 Units, Lecture, 3 hours; research, 3 hours. Prerequisite(s): CS 224 or EE 231 or EE 236 or EE 244 or CS 171 or EE 142; graduate standing; or consent of instructor. Explores fundamentals of deep neural networks and their applications in various machine learning tasks. Includes the fundamentals of perception, approximation, neural network architectures, loss functions, and generalization. Addresses optimization methods including backpropagation, automatic differentiation, and regularization. Covers non-standard problems including auto-encoders and probabilistic models. Presents applications in machine learning/computer vision. May be taken Satisfactory (S) or No Credit (NC) with consent of instructor and graduate advisor. Cross-listed with EE 228.
Course Details
Time: MWF 3-3:50 PM
Location: Gordon Watkins Hall, Room 2240
Instructor: Zhouxing Shi
- Email: zhouxing.shi@ucr.edu
- Office hours: Tuesday 4-5 PM, at WCH 356
TA:
Credits and type: 4 Units. Lecture: 3 hours. Research (outside): 3 hours.
Prerequisites:
- Understanding of linear algebra, calculus, and probability.
- Programming with Python.
Tentatitive Schedule
| Week | Monday | Wednesday | Friday |
|---|---|---|---|
| 1 | Introduction | Learning problem | Multi-Layer Perceptrons |
| 2 | Backpropagation | Optimization for NNs | Optimization for NNs |
| 3 | Training techniques | Convolutional NNs | Convolutional NNs |
| 4 | Recurrent NNs | Recurrent NNs | [Async at test center] Midterm I |
| 5 | Attention | Transformers | Transformers |
| 6 | Transformers | Self-supervised learning | [Zoom slots] Project discussion |
| 7 | Autoencoders | Variational Auto Encoders | Generative Adversarial Networks |
| 8 | Diffusion | Diffusion | [Async at test center] Midterm II |
| 9 | [No Class] Memorial Day | Efficiency | Efficiency |
| 10 | Interpretability | Project presentations | Project presentations |
Tentative Grading
Overview:
- Assignments: 30%
- Midterm exams: 30%
- Project: 40%
- Participation bonus: up to 5%
Assignments
There will be about three coding assignments for you to get started with deep learning implementations by PyTorch, understand model architectures and design choices, and gain hands-on experience with training.
The assignments should be individually completed. Discussion with peers is allowed but should be acknowledged. Using AI to generate solutions is not allowed. Late penalty is 3^k where k is the number of days late (rounded up).
Exams
There will be two async midterm exams on Week 4 and Week 8, respectively, conducted at the CSE Test Center. Each exam will run from Thursday through Friday, and you will reserve a 50-min window to take the exam at your convenience. On those two weeks, there will be no lecture on Friday. There will be no final exam.
Project
You will work on a team project throughout the quarter to strengthen hands-on experience in deep learning. Each team should typically have 3 students.
Sign up (due on 4/12, 5%): Please form a team and sign up.
Grading criteria: If the sign up is done on time.
Proposal (due on 4/19, 10%): You will choose a task for your project and submit a 1-page writeup. The writeup should cover: 1) An introduction and the background of the task; 2) An overview of existing methods, including deep learning approaches and/or others (if any); 3) Datasets and metrics. You do not have to propose any new method at this stage.
Grading criteria: If the proposal is reasonably completed.
Milestone report (due on 5/6, 15%): You will submit a new writeup, updated and extended from your proposal. In addition to the task and background, it should also cover: 1) Initial progress: reproduction of one or two baseline methods (reuse existing code if possible), or an implementation of a straightforward or naive method; 2) Proposal on next steps that propose and implement a new method, conduct a new analysis, or develop a new tool or demo.
Grading criteria: If the milestone report is reasonably completed.
Midterm project discussion (Friday, 5/8, optional): We will dedicate the class time on 5/8 to project discussion between each group and the instructor. We will have 15-min slots for meetings on Zoom. The instructor may provide extended availability after the regular class time or for adhoc appointments. Participation is encouraged but optional.
Final presentation (in-class on Week 10, 20%): You will deliver a 10-min (TBD) final presentation. The presentation should cover: 1) A recap of the problem and background (with any new changes); 2) Proposed method/study and its merits; 3) Experiments and analysis; 4) Anything else you feel valuable.
Final report and code submission (due on 6/12, 50%): You will submit a final report for the project. Page limit is 4 pages, not including any references.
Conduct and Academic Integrity
See https://conduct.ucr.edu for details. Here at UCR we are committed to upholding and promoting the values of the Tartan Soul: Integrity, Accountability, Excellence, and Respect. As a student in this class, it is your responsibility to act in accordance with these values by completing all assignments in the manner described, and by informing the instructor of suspected acts of academic misconduct by your peers. By so doing, you will not only affirm your own integrity, but also the integrity of the intellectual work of this University, and the degree which it represents. Should you choose to commit academic misconduct in this class, you will be held accountable according to the policies set forth by the University, and will incur appropriate consequences both in this class and from Student Conduct and Academic Integrity Programs. For more information regarding University policy and its enforcement, please visit: conduct.ucr.edu.
Test Center. This course makes use of the BCOE Computer Science & Engineering Test Center for conducting the course exams. The course policies align with the Test Center policies, and any violations of academic integrity pertaining to the Test Center are considered violations within this course as well. If you require any special accommodations for exams, please inform your course instructor as soon as possible. The Test Center will try to include any special accommodations as communicated by the course instructor. If you encounter any problems during an exam, please promptly notify the proctor. Collaborate with the proctor to address the issue before ending your session. If you fail to inform a proctor about an issue during the test, your issue may not be addressed later. Please review all the instructions and policies about the Test Center here prior to your first exam for a smooth testing experience.