Date | Topic | Presenters | Notes |
---|
Week 1 Thu, Aug 27 | Lecture Introduction: Towards General-Purpose Robot Autonomy
| | [slides] |
Part I: Robot Perception |
Week 2 Tue, Sept 1 | Lecture Overview of Robot Perception
- The Limits and Potentials of Deep Learning for Robotics. Niko Sünderhauf, Oliver Brock, Walter Scheirer, Raia Hadsell, Dieter Fox, Jürgen Leitner, Ben Upcroft, Pieter Abbeel, Wolfram Burgard, Michael Milford, Peter Corke (2018)
- A Sensorimotor Account of Vision and Visual Consciousness. Kevin O'Regan and Alva Noë (2001)
| | [slides] |
Week 2 Thu, Sept 3 | Object Detection
| Tianwei Yue | |
Week 3 Tue, Sept 8 | 3D Data Processing
| Bo Yan | |
Week 3 Wed, Sept 9 | Tutorial Tutorial on physical simulations for robot learning | | [slides] |
Week 3 Thu, Sept 10 | Representation Learning I: Priors
| Tongrui Jang Hyun | |
Week 4 Tue, Sept 15 | Representation Learning II: Motions
- Learning Features by Watching Objects Move. Deepak Pathak, Ross Girshick, Piotr Dollar, Trevor Darrell, Bharath Hariharan (2016)
- Time-Contrastive Networks: Self-Supervised Learning from Video. Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine (2017)
| Mofei Jeffrey | |
Week 4 Thu, Sept 17 | Multimodal Perception
- Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks. Michelle A. Lee, Yuke Zhu, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, Jeannette Bohg (2018)
- Audio-Visual Embodied Navigation. Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, Kristen Grauman (2020)
- More Than a Feeling: Learning to Grasp and Regrasp Using Vision and Touch. Roberto Calandra, Andrew Owens, Dinesh Jayaraman, Justin Lin, Wenzhen Yuan, Jitendra Malik, Edward H. Adelson, Sergey Levine (2018)
- Deep Multimodal Embedding: Manipulating Novel Objects with Point-clouds, Language and Trajectories. Jaeyong Sung, Ian Lenz, Ashutosh Saxena (2015)
| Farzan Changan | Project Proposal Due |
Week 5 Tue, Sept 22 | Recursive State Estimation
| Jiaru Haresh | |
Week 5 Thu, Sept 24 | Pose Estimation
- Implicit 3D Orientation Learning for 6D Object Detection from RGB Images. Martin Sundermeyer, Zoltan-Csaba Marton, Maximilian Durner, Manuel Brucker, Rudolph Triebel (2019)
- DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion. Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, Silvio Savarese (2019)
| Shailesh Zihang | |
Week 6 Mon, Sept 28 | Tutorial Tutorial on the PyTorch deep learning framework | | [slides] |
Week 6 Tue, Sept 29 | Visual Tracking
| Sung Yeon Shivam | |
Week 6 Thu, Oct 1 | Active Perception
- Interactive Perception: Leveraging Action in Perception and Perception in Action. Jeannette Bohg, Karol Hausman, Bharath Sankaran, Oliver Brock, Danica Kragic, Stefan Schaal, Gaurav Sukhatme (2016)
- The Curious Robot: Learning Visual Representations via Physical Interactions. Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, Abhinav Gupta (2016)
| Sagnik Jerry | |
Part II: Robot Decision Making |
Week 7 Tue, Oct 6 | Lecture Overview of Robot Decision Making
| | [slides] |
Week 7 Thu, Oct 8 | Model-free Reinforcement Learning
- Trust Region Policy Optimization. John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, Pieter Abbeel (2015)
- Soft Actor-Critic Algorithms and Applications. Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, Sergey Levine (2018)
- Continuous Control with Deep Reinforcement Learning. Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra (2015)
- Proximal Policy Optimization Algorithms. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov (2017)
| Dian Jiaxun | |
Week 8 Tue, Oct 13 | Model-based Reinforcement Learning
- Benchmarking Model-Based Reinforcement Learning. Tingwu Wang, Xuchan Bao, Ignasi Clavera, Jerrick Hoang, Yeming Wen, Eric Langlois, Shunshi Zhang, Guodong Zhang, Pieter Abbeel, Jimmy Ba (2017)
- Deep Visual Foresight for Planning Robot Motion. Chelsea Finn, Sergey Levine (2016)
| Adam Priyanka | |
Week 8 Thu, Oct 15 | Imitation as Supervised Learning
- Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation. Tianhao Zhang, Zoe McCarthy, Owen Jow, Dennis Lee, Xi Chen, Ken Goldberg, Pieter Abbeel (2017)
- Learning Latent Plans from Play. Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, Pierre Sermanet (2019)
| Srinath Satya | Project Milestone Due |
Week 9 Tue, Oct 20 | Inverse Reinforcement Learning
| Ruohan Yuchen | |
Week 9 Thu, Oct 22 | Adversarial Imitation Learning
- InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations. Yunzhu Li, Jiaming Song, Stefano Ermon (2017)
- Reinforcement and Imitation Learning for Diverse Visuomotor Skills. Yuke Zhu, Ziyu Wang, Josh Merel, Andrei Rusu, Tom Erez, Serkan Cabi, Saran Tunyasuvunakool, János Kramár, Raia Hadsell, Nando de Freitas, Nicolas Heess (2018)
| Zhendong Yuguang | |
Part III: Robots and General Intelligence |
Week 10 Tue, Oct 27 | Learning to Learn I: Meta-Learning
| Caroline Lucas | |
Week 10 Thu, Oct 29 | Learning to Learn II: Lifelong Learning
| Maohua Garrett | |
Week 11 Tue, Nov 3 | Compositionality I: Hierarchy
| Prakhar Soroush | |
Week 11 Thu, Nov 5 | Compositionality II: Task and Motion
| Oleg Yifeng | |
Week 12 Tue, Nov 10 | Causal Reasoning
| Jordan Hussein | |
Part IV: Robots in the Real World |
Week 12 Thu, Nov 12 | Simulation-Reality Gap
- Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks. Stephen James, Paul Wohlhart, Mrinal Kalakrishnan, Dmitry Kalashnikov, Alex Irpan, Julian Ibarz, Sergey Levine, Raia Hadsell, Konstantinos Bousmalis (2019)
- Meta-Sim: Learning to Generate Synthetic Datasets. Amlan Kar, Aayush Prakash, Ming-Yu Liu, Eric Cameracci, Justin Yuan, Matt Rusiniak, David Acuna, Antonio Torralba, Sanja Fidler (2019)
| Sai Kiran Ziyang | |
Week 13 Tue, Nov 17 | Data-driven Grasping
- Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics. Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, Ken Goldberg (2017)
- Learning a Visuomotor Controller for Real World Robotic Grasping using Simulated Depth Images. Ulrich Viereck, Andreas ten Pas, Kate Saenko, Robert Platt (2017)
| Zhenyu Satyam | |
Week 13 Thu, Nov 19 | Building Robotic Systems
- QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation. Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, Sergey Levine (2018)
- Learning Agile and Dynamic Motor Skills for Legged Robots. Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, Marco Hutter (2019)
- TossingBot: Learning to Throw Arbitrary Objects with Residual Physics. Andy Zeng, Shuran Song, Johnny Lee, Alberto Rodriguez, Thomas Funkhouser (2019)
- Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems. Clemens Eppner, Sebastian Höfer, Rico Jonschkowski, Roberto Martíın-Martín, Arne Sieverling, Vincent Wall, Oliver Brock (2016)
| Mofei Mingyo | |
Week 14 Tue, Nov 24 | Lecture Conclusion: Open Questions in Robot Learning
| | [slides] |
Week 14 Thu, Nov 26 | No Class - Thanksgiving Holidays | | |
Week 15 Tue, Dec 1 | Spotlight Final Project Spotlights I | | Video Due Nov 29 |
Week 15 Thu, Dec 3 | Spotlight Final Project Spotlights II | | |
Week 16 Fri, Dec 11 | No Class | | Final Report Due |