CSCI-B 659 Topics in Artificial Intelligence

LING-L 665 Applying Machine Learning Techniques in Computational Linguistics - Neural Networks, Deep Learning for CL/NLP

Course Material Spring 2020

Focusing on:

The new course (Spring 2020) extends the previous discussions and focuses on newest approaches to Natural Language Processing (NLP), Knowledge Graphs and Computational Semantics, using data-driven Deep Learning and knowledge-driven hybrid machine learning approaches.

Introduction

This is a graduate course that focuses on machine learning techniques that are used in Computational Linguistics, Natural Language Processing, Cognitive Modeling.

Machine learning problems in language processing are rather non-typical for machine learning because natural language includes a significant level of exceptions, parallel and cross-linguistic-level dependencies, or complex connections to other cognitive systems and forms of knowledge. The course will provide an overview of the most important machine learning algorithms, but it will mostly focus on how to apply current popular machine learning to language problems such as parsing, co-reference resolution, morphological analysis, semantic disambiguation and coreference analysis, word sense disambiguation, entity and relation extraction, and linking to general knowledge representations. We will put a strong focus on data-driven deep learning and knowledge-driven graph-based methods for NLP and computational semantics (reasoning, knowledge linking, multi-modal aspects of language processing).

Among others, we will cover word vector representations (embedding models), window-based neural networks, recurrent neural networks, long-short-term-memory models and various forms of recursive, convolutional, or graph neural networks as applied in NLP and Computational Semantics.

We will experiment with and discuss the limitations and problems of popular Deep Learning approaches as applied in the domain of NLP, Computational Semantics, Reasoning, and other aspects of Knowledge processing, and consider new approaches or models addressing Natural Language Understanding.

The course is a series of lectures and hands-on programming exercises.

The course is using material provided by:

These courses are accompanied by videos, slides, research papers, links to supplemental material and tutorials, and other very valuable information. Please use these resources during our course.

Prerequisites and Requirements

I expect that you are able or acquire the skills to follow example code in Python (3.x), and provide your own solutions using either Python, Java, C(++), Julia, or Go. If you have no programming experience, follow the different links here and on the mentioned course sites and learn Python and Numpy.

Recommended:

Work through all the relevant Jupyter notebooks at: Python tutorials for NLP, ML, AI

Literature

I do not require any textbook, I recommend the following:

If you are new to natural language, computational linguistics, NLP, take a look at this book:

We will read the following papers:

Recommended tutorials:

Policies

Academic Integrity

(from the Dean for Academic Standards and Opportunities) As a student at IU, you are expected to adhere to the standards and policies detailed in the Code of Student Rights, Responsibilities, and Conduct (http://www.iu.edu/~code/). When you submit an assignment with your name on it, you are signifying that the work contained therein is all yours, unless otherwise cited or referenced. Any ideas or materials taken from another source for either written or oral use must be fully acknowledged. If you are unsure about the expectations for completing an assignment or taking a test or exam, be sure to seek clarification beforehand. All suspected violations of the Code will be handled according to University policies. Sanctions for academic misconduct may include a failing grade on the assignment, reduction in your final course grade, a failing grade in the course, among other possibilities, and must include a report to the Dean of Students who may impose additional disciplinary sanctions.

Students with Disabilities

Students who need an accommodation based on the impact of a disability should contact me to arrange an appointment as soon as possible to discuss the course format, to anticipate needs, and to explore potential accommodations.

I rely on Disability Services for Students for assistance in verifying the need for accommodations and developing accommodation strategies. Students who have not previously contacted Disability Services are encouraged to do so (812-855-7578; http://www.indiana.edu/~iubdss/).

CAPS

One benefit of a school like IU is that there are many, many resources available to you. School and life can be intense at times, and if your academic responsibilities or other personal concerns are distracting or weighing on you this semester, I encourage you to contact Counseling and Psychological Services (CAPS) (812) 855-5711, http://healthcenter.indiana.edu/counseling/). The people there can be a resource and a source of support, not just in times of crisis, but also when you need an extra ear or a little extra support. I’m happy to be a listening ear, as well, but I have no counseling training and the folks at CAPS do. Note, too, that I am required to report certain things (e.g., reports of sexual assault, suicidal thoughts).

Note Selling

Several commercial services have approached students regarding selling class notes/study guides to their classmates. Selling the instructor’s notes/study guides in this course is not permitted. Violations of this policy will be reported to the Dean of Students as academic misconduct (violation of course rules). Sanctions for academic misconduct may include a failing grade on the assignment for which the notes/study guides are being sold, a reduction in your final course grade, a failing grade in the course, among other possibilities. Additionally, you should know that selling a faculty member’s notes/study guides individually or on behalf of one of these services using IU email, or via Canvas may also constitute a violation of IU information technology and IU intellectual property policies and additional consequences may result.

Sexual Misconduct Policies at IU

As your instructor, one of my responsibilities is to create a positive learning environment for all students. Title IX and IU’s Sexual Misconduct Policy prohibit sexual misconduct in any form, including sexual harassment, sexual assault, stalking, and dating and domestic violence. If you have experienced sexual misconduct, or know someone who has, the University can help. If you are seeking help and would like to speak to someone confidentially, you can make an appointment with:

It is also important that you know that Title IX and University policy require me to share any information brought to my attention about potential sexual misconduct, with the campus Deputy Title IX Coordinator or IU’s Title IX Coordinator. In that event, those individuals will work to ensure that appropriate measures are taken and resources are made available. Protecting student privacy is of utmost concern, and information will only be shared with those that need to know to ensure the University can respond and assist.

I encourage you to visit stopsexualviolence.iu.edu to learn more.

Disclaimer

This syllabus is subject to change and likely will change. All important changes will be made in writing, with ample time for adjustment.

Old Material

An old syllabus from Spring 2018 is available online.

Old material (including topic presentations, project presentations and reports, code examples) is available on the 2018 course page.