Trust, but Verify

Using and trusting LLMs: evaluation, RAG, agents, and safety.

01

Evaluating LLMs

Benchmarks, contamination, judges, and pass@k.

02

Retrieval-augmented generation

Look the answer up, then write it.

03

Agents & tool use

A model that calls functions and comes back.

04

LLM security

Jailbreaks, prompt injection, and poisoning — the ideas.

05

Memorization & factuality

What the model stored, and when that is a problem.

Inspired by the work of Mohit Iyyer, Umar Jamil.