This course is a graduate seminar whose intent is to get you as quickly up to speed on the state-of-the-art so that you are able to read papers or participate in multi-agent learning research. As such, it is by necessity not an in-depth course on any of the particular topics and the emphasis is on reading papers and course projects over homework. This means that as you go through the material you will find yourself not fully understanding it. That is an intended outcome; this course is not about basics but about giving you a roadmap of multi-agent learning research as well as just enough comfort to confidently dive in deeper.
At the conclusion of the course, I expect to cover:
As two and a half-hours is a very long time to listen to a lecture, the second half of many of the classes will be a reading group. For more details on how the reading groups will work, see Sec. Reading Group. Please read this entire document, I've put a lot of work into it and understanding the logic of it should help you engage with the course more deeply.
This is a graduate course intended to help you to potentially incorporate ideas from multi-agent learning into your research. As such, the primary element of the course is a project that you will develop over the duration of the class. There are three elements of the project:
The final project is intended to be a paper written in the style of an RLC submission. See the associated style-file on that page. The paper should roughly be eight pages, though it can be longer, and should conform to the style of a paper e.g. it should either demonstrate a new result, describe the construction of an engineering project in detail, or be an in-depth investigation of a topic. As an alternative option to new work, I will also allow for the construction of a clear write-up of a paper that would allow a beginner to understand it. See the ICLR blog post track for examples of what I mean. Note that longer papers will not receive additional credit for being so: eight pages is the expectation and the longer limit is merely to give you extra space if you need it. Similarly, if a substantive result can be described in fewer than eight pages that is also fine.
The project proposal should be a one-page, LaTeX document outlining a concrete research question or engineering task. I'm being insistent about LaTeX here because if you don't know LaTeX yet you probably need to learn it at this point. We are doing it quite early in the course, possibly before you have all of the relevant background, so that I can help you refine your project proposal.
At this point I expect you to have a 3 page writeup that outlines your progress so far, any open questions that you have not been able to resolve, and a list of additional work that remains to be done before the completion of the final project.
Reading groups can be fun and engaging or very dull. To try to ensure that everyone has a reason to be engaged, we're going to use the following format from Colin Raffel: Role-playing reading group seminars. This is put as reading for the second class because we will start using it before the third class. Note that the hacker role will not be available every week.
Additionally, each week of reading group I will expect to receive a short summary of the paper before the reading group. I will provide a format for the summary. The reasoning for this is that I want you to get used to taking a large paper and condensing it into a few key ideas that you could easily explain to someone else. In addition, having you write up a summary allows me to provide feedback on how well you understand the paper. I do not expect this to take more than a few minutes if you have read the paper. Note, you will not be graded on if your summary is correct, only that you have done it.
Because of highly variable backgrounds and the logic of the course organization, some of you will will not have the necessary background for some of the papers. This also means that some of the papers for the reading groups will be challenging very early on! My suggestion is to view this as an exciting challenge that is very analogous to your first year attending a seminar in a new topic. When you start such a seminar, you can only catch fragments of what is being discussed; the pace is simply too fast and too much of the material as new. Keep in mind that learning is a process, you don't transition from not knowing to knowing in a single leap.
As such, your goal in each reading group is to take away 3 new things. These can be a new ideas, a new technique, a question. This way, even if you don't understand everything, you've come away more knowledgeable than you came in.
This is a graduate course and grading is intended to be fairly lenient; the expectation is that you are excited to learn things and do not need to be cajoled to do so by fear of a bad grade. As such, most of the grade is through participation. If you do the work, you should expect to receive a very good grade. There will be 2 straightforward homework assignments through the course and a small number of quizzes that are intended to mimic spaced repetition and help you assess whether you have appropriately understood the materials. The grading will be as follows:
Cheating is obviously not allowed. Copying answers or code from another student or the internet constitutes cheating. However, collaborating with another student is allowed as long as you indicate the student that you collaborated with and your answers and writing are in your own words.
I love ChatGPT and use it all the time. However, one goal of learning is to develop fluency, the process in which you can come up with ideas and use tools and knowledge without reference to an external data store. This is very similar to development of language fluency; imagine if instead of learning a foreign language you tried to look every word up in a dictionary! The same thing happens in research; there are ideas you want to be able to pull out without having to look them up every time. I will try to make clear in the course what those foundational concepts are. A similar thing applies to writing; you want to be able to write quickly and thoughtfully and the only way to get there is practice and repetition.
Using ChatGPT to skip steps delays the development of fluency. In this way, overuse of ChatGPT will cause you to be a worse researcher in the long term and harms your educational experience. As such, my rules for ChatGPT are the following:
Note, I acknowledge that there's basically no way for me to check whether you have followed these rules. My hope is that you use it in the ways outlined above because the alternative will harm your development as a researcher, not out of fear of consequence.
I will host a 1 hour office hour twice a week. The time will be updated here once it is scheduled. Please use this time to come talk to me about homeworks, research, whatever. It's your time to use and I am excited to talk to you!
The NYU Tandon School of Engineering values an inclusive and equitable environment for all our students. I hope to foster a sense of community in this class and consider it a place where individuals of all backgrounds, beliefs, ethnicities, national origins, gender identities, sexual orientations, religious and political affiliations, and abilities will be treated with respect. It is my intent that all students’ learning needs be addressed, and that the diversity that students bring to this class be viewed as a resource, strength and benefit. If this standard is not being upheld, please feel free to speak with me.
If you are student with a disability who is requesting accommodations, please contact New York University’s Moses Center for Students with Disabilities at 212-998-4980 or mosescsd@nyu.edu. You must be registered with CSD to receive accommodations. Information about the Moses Center can be found at www.nyu.edu/csd. The Moses Center is located at 726 Broadway on the 2nd floor.
A quick overview of the logic of the course schedule. Based on preliminary discussions, not everyone entering the course has the background in RL that is necessary to get to the multi-agent learning pieces that form their project. However, we need to get some of the multi-agent learning pieces in so that you actually have some tools and ideas with which to propose your project! As such, we're going to start with a quick overview of some multi-agent topics. We're then going to detour for a few weeks to an overview of RL before returning back to the multi-agent component.
| Date | Topics Covered | Expected Learning Outcome | Weekly Requirements | Course materials |
|---|---|---|---|---|
| 1/24 |
|
|
||
| 1/31 |
|
|
||
| 2/7 |
|
|
||
| 2/14 |
|
|
|
|
| 2/21 |
|
|
|
|
| 2/28 |
|
|
||
| 3/6 |
|
|
|
|
| 3/13 |
|
|
|
|
| 3/20 | Break! | |||
| 3/27 |
|
|
4/3 |
|
|
|
| 4/10 |
|
|
|
|
| 4/17 |
|
|
||
| 4/24 |
|
|
|
|
| 5/1 | Course project presentations! |