Welcome to Transformers

This is the documentation site of the Transformers course for modern-day NLP - a WIP.

One day (hopefully sooner rather than later), you will be able to find explanations through articles and visuals that will guide the beginner practitioner through the wonderful world of transformers models for language application.

If you’re interested in keeping up-to date with any updates, you can find me on:

Medium

Twitter

YouTube

For now, I will leave this rough outline for how I’m expecting this to pan out:

├── Before Transformers
│   ├── RNNs
│   ├── LSTMs and GRUs
│   ├── Information Bottleneck
│   ├── Attention
│   └── Attention Is All You Need
│
├── Attention
│   ├── An Overview (incl. where is has led to)
│   ├── Dot-Product Attention (in Numpy - same for next 3)
│   ├── Causal Attention
│   ├── Bidirectional Attention
│   └── Multi-head Attention
│
├── Preprocessing
│   ├── Why Text Needs Cleaning
│   ├── Stemming
│   ├── Lemmazation
│   ├── Normalization
│   └── Tokenization
│
├── Encoding
│   ├── One Hot
│   ├── Word2Vec
│   ├── SentencePiece
│   └── Byte Piece Encoding (BPE)
│
├── Decoding
│   ├── Greedy Decoding
│   ├── Random Sampling
│   ├── Beam Search
│   └── MBR
│
├── Machine Translation
│   ├── From RNNs to Transformers (short overview/timeline)
│   ├── Attention in Translation
│   ├── Pretrained Translation Transformer
│   ├── Prediction
│   ├── Performance With ROUGE
│   └── Building a Custom Translator
│
├── Language Classification
│   ├── Classification in ML
│   ├── Sentiment Analysis Overview
│   ├── Sentiment With Flair
│   ├── Pretrained Classifier with Transformers
│   ├── Fine-tuning
│   ├── Making Predictions
│   └── Performance Metrics
│
├── Automatic Summarization
│   ├── ???
│   └── Performance With ROUGE
│
├── Language Generation
│   ├── Overview and GPT-3
│   ├── ???
│   ├── Autocompletion
│   └── Performance With ROUGE
│
├── Question-Answering
│   ├── QA Overview
│   ├── Open Book
│   ├── Closed Book
│   ├── SQuAD
│   ├── Pretrained QA with Transformers
│   ├── Fine-tuning
│   ├── Making Predictions
│   └── Performance Metrics
│
├── FAISS and DPR
│   ├── What is FAISS?
│   ├── DPR
│   ├── Similarity Search Architecture With FAISS + DPR
│   ├── Formatting Data for Haystack
│   ├── Indexing Using FAISS
│   └── Similarity Search
│
├── Question-Answering For Tables
│   ├── ???
│   └── ???
│
└── Chatbots
    ├── ???
    └── ???