Course Project

An integral part of the course is the class project (35% of the grade), which gives students a chance to apply the algorithms discussed in class to a research oriented project. This semester the theme is Art and Machine Learning.

Some example project titles are as follows:

• Identifying Hip Hop Musicians . Predicting the hip hop artists from song lyrics.
• Art Forgery Detection. Identifying fake artworks from real ones
• Controllable Poem Generation. Generating poems using a given set of keywords and themes.
• Classification of Turkish Makams. Classifying Makams of Turkish classical and folk songs from raw audio.

Sample Research Papers on this years project theme

1. Ahmed Elgammal, Yan Kang, Milko Den Leeuw. Picasso, Matisse, or a Fake? Automated Analysis of Drawings at the Stroke Level for Attribution and Authentication. AAAI, 2018
2. Amy Zhao, Guha Balakrishnan, Kathleen M. Lewis, Frédo Durand, John V. Guttag, Adrian V. Dalca. Painting Many Pasts: Synthesizing Time Lapse Videos of Paintings. CVPR, 2020
3. Rajat Agarwal, Katharina Kann. Acrostic Poem Generation. EMNLP, 2020
4. Prithviraj Ammanabrolu, Wesley Cheung, William Broniec, Mark O. Riedl. Automated Storytelling via Causal, Commonsense Plot Ordering. AAAI, 2021

Resources

• The Magenta project
• Google's Artists + Machine Intelligence program
• Art in the age of machine intelligence: A TED Talk by Refik Anadol
• NeurIPS 2020 Workshop on Machine Learning for Creativity and Design
• The 4th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature
• IEEE Workshop on Artificial Intelligence for Art Creation

In your projects, you may use a dataset available on the web or collect your own data. However, if you choose the latter option, you must you must keep in mind that data collection can be fun and exciting, but it is also time-consuming.

Please also keep in mind that complexity of your project topic will be taken into consideration while grading.

Software and Libraries

You are encouraged to learn and use the following machine learning and deep learning frameworks in your projects. Links to some useful NLP tools are also provided.

• PyTorch: An Open Source Machine Learning Library
• Tensorflow: An Open Source Software Library for Machine Intelligence
• Caffe: A deep learning framework made with expression, speed, and modularity in mind
• Theano: A Python framework for fast computation of mathematical expressions.
• Keras: Keras: Deep Learning library for Theano and TensorFlow
• MatConvNet: CNNs for MATLAB
• mxnet: Flexible and Efficient Library for Deep Learning
• Torch: A scientific computing framework for LuaJIT
• LIBSVM: A Library for Support Vector Machines
• scikit-learn: Machine learning in Python
• Stanford CoreNLP: A suite of core NLP tools
• NLTK: Natural Language Toolkit

Deliverables

• Proposals: March 31, 2021.
• Project progress reports: May 5, 2021
• Final project presentations: May 26, 2021
• Final reports: May 30, 2021
  
  

In preparing your progress and final project reports, you should use the provided LaTeX template and submit them electronically in PDF format. Late submissions will be penalized.

Collaboration Policy

Each project should be done in groups of 3 students. Of course, there may be some exceptions, depending on the enrollment. Note that students without a team will be randomly assigned to one project group.

Grading

• Proposal (2%)
• Blog posts (4%)
• GitHub commits and meetings with TAs(5%)
• Progress report (6%)
• Presentation (8%)
• Final report and code (10%)

Project Proposal

Each project group should submit a half page project proposal on their specific project idea by March 31, 2021. The proposal should provide

• The research topic to be investigated,
• What data you will use,
• A list of related papers.

Blog posts/GitHub commits/Meetings with TAs

Each project group should maintain a blog sharing their steady progress, ideas, and experiments, and they must write at least one blog post per week (excluding exam weeks). Moreover, they will regularly meet with TAs to discuss their progress and get feedback. Each group should maintain a GitHub repository for their project (must be viewable to the TAs and instructor). The frequency of your commits to GitHub will also be graded.

Progress Report

Due: May 5, 2021 (11:59pm)

Each project group should submit a project progress report by May 5, 2021. The report should be 3-4 pages and should describe the following points as clearly as possible:

• Problem to be addressed. Give a short description of the problem that you will explore. Explain why you find it interesting.
• Related work. Briefly review the major works related to your research topic.
• Methodology to be employed. Describe the machine learning method that is expected to form the basis of the project. State whether you will extend an existing method or you are going to devise your own approach.
• Experimental evaluation. Briefly explain how you will evaluate your results. State which dataset(s) you will employ in your evaluation. Provide your preliminary results (if any).

Project Presentations

Due: May 26, 2021 (in class)

Each project group will have ~8 mins to present their work in class. The suggested outline for the presentations are as follows:

• High-level overview of the paper (main contributions)
• Problem statement and motivation (clear definition of the problem, why it is interesting and important)
• Key technical ideas (overview of the approach)
• Experimental set-up (datasets, evaluation metrics, applications)
• Strengths and weaknesses (discussion of the results obtained)

In addition to classroom presentations, each group should also prepare an engaging video presentation of their work using online tools such as PowToon, moovly or GoAnimate. The deadline is May 28, 2021.

Final Report

Due: May 30, 2021 (11:59pm)

As the last deliverable of the course project, each group is expected to submit a project report prepared using the style files provided in the course web page. The report should be 6-8 pages and should be structured as a research paper. It will be graded based on clarity of presentation and technical content. A typical organization of a report might follow:

• Title, Author(s).
• Abstract.
• Introduction. This section introduces the problem that you investigated by providing a general motivation and briefly discusses the approach(es) that you explored to solve this problem.
• Related Work. This section discusses relevant literature for your project topic.
• The Approach. This section gives the technical details about your project work. You should describe the representation(s) and the algorithm(s) that you employed or proposed as detailed and specific as possible.
• Experimental Results. This section presents some experiments in which you analyze the performance of the approach(es) you proposed or explored. You should provide a qualitative and/or quantitative analysis, and comment on your findings. You may also demonstrate the limitations of the approach(es).
• Conclusions. This section summarizes all your project work, focusing on the key results you obtained. You may also suggest possible directions for future work.
• References. This section gives a list of all related work you reviewed or used.