CSCI-B 659 Topics in Artificial Intelligence - Advanced Natural Language Processing

LING-L 645 Advanced Natural Language Processing

This is the course page for Topics in Artificial Intelligence / Advanced Natural Language Processing (NLP) by Damir Cavar.

Damir Cavar

– August 2025 –

Course Arrangements

Meeting time: Tuesday and Thursday, 11:10 AM - 12:25 PM

Course website: Assignments, slides, and other material will be posted on Canvas.

Credits: 3

Instructor: Associate Professor, Dr. Damir Cavar

Office: Ballantine Hall (BH) 511

Office hours: Thursday 1-2 PM and by arrangement

Course Description

Symbolic, statistical, and neural methods are at the core of Computational Linguistics and Natural Language Processing (NLP) in research and applications. This course introduces advanced techniques for NLP based on statistical modeling and machine learning algorithms, including neural network Deep Learning approaches, and transformers, bringing them together with symbolic and knowledge-based systems. We aim to bridge research and insights from language and linguistic disciplines and the application of NLP and linguistic technologies from the computer and information science perspective.

This course will cover fundamental notions in probability and information theory, focusing on the concepts needed for common NLP tasks. We will discuss N-gram models, exemplified by an approach to document classification or Part-of-Speech (PoS) tagging. In the next step, we will extend to probabilistic methods and to sentiment analysis. We will study advanced neural network approaches (Deep Learning) for NLP, used for various speech and language processing tasks.

Additionally, we will cover concrete topics such as information extraction and graph-based knowledge representations used for text classification, natural language understanding, dialog systems (AIs), or information retrieval, and how to use various NLP methods in the context of such systems. There is space to focus in part on topics of interest related to the choice of concrete applications of NLP methods.

We are discussing advanced hybrid NLP methods, covering symbolic, statistical, and neural network methods in the context of particular tasks. All the methods we use apply to a range of tasks in NLP. The mission is to teach students techniques, algorithms, and existing environments to enable them to develop their own strategies to analyze linguistic phenomena using language data, to apply NLP in the domain of information extraction from unstructured data, or to research in the field of AI, psycholinguistic or cognitive language faculty, verbal behavior, and general speech and language technologies.

Learning Outcomes

Crucial aspects of course outcomes are:

Understand basic concepts of machine learning as applied in Natural Language Processing
Understand differences in supervised and unsupervised learning
Understand advanced NLP algorithms and methods
Understand the linguistic annotations, analyses, and outputs that these methods generate
Acquire the skills and ability to develop own models and to tune such methods to apply NLP to entirely new problems and research areas
Apply machine learning concepts to sample problems in NLP
Understand how Large Language Models and Generative AI work
Reinforce concepts of programming in Python
Learn to apply well-documented scientific libraries in Python

This course provides an essential platform for further work in NLP.

Assessment

Grades are based on the following schema:

45% Written assignments, assigned 7.
10% Class participation
40% Final project – written report
5% Final project – oral report

Coding and Computational Experiments

Students are encouraged to bring their laptops or other computational devices to class.

The readings and exercises will be accompanied by practical examples using:

Python 3 (with Numpy, Scipy) (and Cython) with NLTK, spaCy, Stanza, Stanford CoreNLP, Spark NLP, PyTorch
Code using Rust, Java, C, C++, or other modern programming languages will be accepted and can be used in projects

Literature

Main textbook

We will be using the most recent 3rd edition of the textbook and additional material shared on Canvas.

Jurafsky, Dan and James H. Martin (2008) Speech and Language Processing. 2nd ed. The 3rd edition is available online (https://web.stanford.edu/~jurafsky/slp3/).

Additional books and articles

Glass, Lelia, Markus Dickinson, Chris Brew, Detmar Meurers (2024) Language and computers. (Textbooks in Language Sciences 14). Berlin: Language Science Press.
Manning, Christopher and Schütze, Hinrich (1999) Foundations of Statistical Natural Language Processing. Cambridge, MA: MIT Press.
Bird, Steven, Ewan Klein, Edward Loper (2009) Natural Language Processing with Python. O’Reilly Media.
Hogg, Robert V., Elliot A. Tanis (2001) Probability and statistical inference. 6th ed. Upper Saddle River, NJ: Prentice Hall.
Goldberg, Yoav (2015) A Primer on Neural Network Models for Natural Language Processing.

General Information and Notes

Participation in the NLP-Lab Projects

Students are welcome to participate in NLP-Lab meetings and projects after consultation with the instructor. See for more details: https://nlp-lab.org/

Disclaimer

This syllabus is subject to change and likely will change. All critical changes will be made in writing, with ample time for adjustment.