DA5005 · BS elective · 4 credits

Introduction to Natural Language Processing

The NLP pipeline from tokenisation to transformers, with the linguistics that motivates each step.

Instructor Prof. Rahul Mishra & Prof. Parameswari Krishnamurthy Term As offered Prereq CS3004

§ 1Calendar

  • Quiz 1—
  • Quiz 2—
  • End-term examination—

§ 2Previous Year Questions

No papers uploaded yet.

§ 3Resources

DA5005 · BS elective · 4 credits

Syllabus

The official course syllabus, transcribed from the IITM BS Data Science & Applications course booklet.

§ 1Course Description

Natural language is the primary mode of human communication, and the rapid growth of text data has led to the need for automated processing through NLP . This course covers the challenges of language processing, linguistic and statistical methods, and modern deep learning approaches, including transformers and large language models. It also develops the ability to understand research, apply NLP techniques to real-world problems, and build NLP-based applications. This course is also offered as an elective for PG-diploma students.

Lectures-Tutorials-Practicals-Credits: 4-0-0-4

§ 2Faculty

  • Prof. Rahul Mishra, Language Technologies Research Center, IIIT Hyderabad
  • Prof. Parameswari Krishnamurthy, Language Technologies Research Centre, IIIT Hyderabad

§ 3Week-by-week

Week 1: Introduction to Natural Language (NL) Why is it hard to process a natural language? Levels of Language Processing, Linguistic Fundamentals for NLP

Week 2: Text preprocessing techniques; Tokenization and normalization; Stemming, lemmatization, and morphological analysis.

Week 3: Part-of-speech tagging; Named entity recognition; Sequence models (HMM, MEMM, CRF); Classical classification methods.

Week 4: Syntax and parsing (CFGs, constituency, dependency); Semantic analysis and representation; Semantic role labeling.

Week 5: Distributional semantics (n-grams, Word2Vec, GloVe); Coreference resolution and discourse; Machine translation.

Week 6: Recurrent neural networks (RNNs); LSTM architectures; NLP preprocessing pipelines; PyTorch basics.

Week 7: Transformers and attention mechanisms; Large language models (LLMs); Transformer implementation.

Week 8: Parameter-efficient fine-tuning (LoRA, QLoRA); Instruction tuning; Human alignment (RLHF, DPO); Hugging Face workflows.

Week 9: Natural language generation (NLG); Decoding strategies (beam search, sampling); Fine-tuning LLMs.

Week 10: Retrieval-augmented generation (RAG); Applications (question answering, summarization, code tasks).

Week 11: Model explainability; Attention and attribution methods; Interpretability techniques (integrated gradients, attention maps).

Week 12: Ethics in NLP; Bias in datasets and models; Evaluation metrics; Bias mitigation and responsible AI practices.