Ai-Engineering
Quantization and Distillation: Compression for Inference
Quantization and distillation methods for reducing inference memory, latency, and serving cost.
LoRA and Parameter-Efficient Fine-Tuning
How LoRA and QLoRA reduce trainable parameters, GPU memory, and adapter deployment cost.
DPO and Modern Alignment
Direct Preference Optimization, its common variants, production workflow, and failure modes.
From Pre-Training to RLHF
LLM pretraining, supervised fine-tuning, preference optimization, and verifiable rewards.
Agent Budgets and Runaway Prevention
How agent harnesses enforce step, time, token, cost, and tool-call limits.
PII Detection and Data Privacy for LLM Systems
How to detect, transform, retain, and delete personal data in LLM request and storage paths.
Guardrails: Input and Output Safety Layers for LLM Systems
How input, output, and tool-call guardrails enforce application safety policies.
Fine-Tuning vs RAG: When to Choose Which
How to choose fine-tuning, retrieval, or both based on behavior, knowledge freshness, cost, and evaluation.
Cost Optimization and Model Routing
How predictive routers and cascades trade model cost, quality, and tail latency per request.
Speculative Decoding and Draft Models
Speculative decoding with draft models, Medusa, EAGLE, and prompt lookup for lower decode latency.
Inference Latency: Prefill, Decode, and Batching
Inference latency across prefill, decode, batching, chunked prefill, and disaggregated serving.
Sleep-Time Compute for AI Agents
Background consolidation and speculative work for stateful agents with explicit consistency controls.
Human-in-the-Loop Feedback Loops for LLM Systems
How to capture, sample, label, and route production feedback into evaluation and product changes.
Drift Detection and Regression Testing for LLM Systems
How to detect distribution shifts and test model upgrades with paired evaluation, shadow runs, and canaries.
Production Tracing and Observability for LLM Systems
How to trace model calls, tools, retrieval, cost, cache usage, and evaluation in LLM applications.
LLM-as-Judge: Pointwise and Pairwise
How to design and calibrate pointwise and pairwise LLM judges against human labels.
Eval-Driven Development for LLM Systems
Evaluation-driven development with error analysis, golden sets, layered checks, and CI gates.
Production Memory Frameworks: MemGPT/Letta, mem0, Zep, Graphiti
How Letta, mem0, Zep, and Graphiti differ in storage, retrieval, and operational cost.
Memory Evaluation: Benchmarks and Custom Evals
How public benchmarks and custom evals measure memory recall, updates, temporal reasoning, and cost.
Conversation Compaction: Keeping Long Sessions Alive
How long agent sessions compact context using preemptive triggers, cache-aware deletion, rollback, and journals.
Anatomy of an Agent Harness
Agent harness responsibilities including context assembly, tool dispatch, streaming, caching, recovery, cost accounting, and telemetry.
Long-Horizon Task Reliability
How checkpointing, abort policies, and recovery limit drift in long-running agents.
Computer Use and Browser Agents
Computer-use agents based on screenshots or DOM access, with grounding, sandboxing, injection defense, and OSWorld evaluation.
Memory Privacy, Isolation, and Multi-Tenancy
How to isolate tenant memory, resist stored prompt injection, and verify deletion.
Multi-Agent Shared Memory
Shared memory for multiple agents with explicit scope, consistency, conflict handling, and audit.
Cross-Session Identity and Personalization
How agents load durable user preferences across sessions without weakening privacy boundaries.
Procedural Memory and Skill Caching
Procedural memory for agents using success-gated skills, task-shape keys, and retrieval-time validation.
Memory Conflict, Forgetting, and Embedding Drift
Agent-memory conflict resolution, active forgetting, and embedding-model migration.
Temporal Reasoning and Memory Provenance
Temporal memory retrieval with as-of queries, staleness scoring, and evidence provenance.
Memory Retrieval Policies: Recency, Relevance, Importance
How recency, importance, and similarity combine to rank memories for the current query.
Summarization and Context Compression
How recursive summarization, structured notes, and API compaction reduce agent context safely.
Reflection: From Experiences to Beliefs
Memory reflection that derives evidence-linked beliefs from episodic records.
Episode Segmentation and Salience Scoring
How episode boundaries and salience scores shape long-term memory retrieval.
Memory Write Policies: What's Worth Remembering
Agent-memory admission control with extraction, distillation, deduplication, and linking.
Hierarchical Memory: Working / Episodic / Semantic Tiers
Hierarchical agent memory with core, recall, and archival tiers plus promotion and demotion policies.
Knowledge Graphs as Structured Memory
How knowledge graphs represent entities, relations, temporal validity, and hybrid graph-vector retrieval.
Long-Term Memory: Vector-Backed Episodic Storage
Long-term episodic memory with vector storage, episode boundaries, selective writes, and recency-weighted retrieval.
Working Memory: Scratchpads, Blackboards, and Agent Notebooks
Agent working memory using scratchpads, typed state, notebooks, and multi-agent blackboards.
Short-Term Memory: Managing the Conversation Buffer
How token budgets, protected messages, and head-tail eviction manage conversation context.
The Cognitive Taxonomy: Semantic, Episodic, Procedural
How working, episodic, semantic, and procedural memory differ in agent systems.
The Memory Stack: A Map of AI Memory
How agent memory separates working state, durable storage, cognitive types, and maintenance paths.
Tool Selection at Scale: MCP and Dynamic Routing
Tool selection at scale using MCP discovery, deferred loading, semantic retrieval, and namespaces.
Multi-Agent Orchestration
How supervisor, swarm, and hierarchical agent patterns trade specialization for coordination cost.
Planning Agents vs Reactive Agents
How ReAct, plan-and-execute, ReWOO, and search-based planning trade cost for control.
The Agent Loop: ReAct and Its Descendants
How ReAct agents execute tools, process observations, enforce stop conditions, and compare with plan-and-execute.
Constrained Decoding: Grammars, Regex, and FSMs
How vocabulary masks, finite-state machines, grammars, and schema compilation constrain model output.
Prompt Caching: Reusing the KV Cache Across Calls
Prompt caching across Anthropic, OpenAI, and Gemini, including prefix layout and cost calculations.
Streaming and Backpressure
How SSE streaming handles partial output, cancellation, tool-call JSON, and backpressure.
Function Calling and Tool Use
LLM tool use as typed RPC with schema validation, dispatch, retries, and result handling.
Structured Output: JSON Mode and Schema Coercion
Schema-constrained LLM output using JSON mode, strict schemas, tool calls, and validation retries.
Context Engineering: JIT vs AOT Context Loading
Ahead-of-time context assembly and just-in-time retrieval compared by attention, latency, and token cost.
RAG Evaluation: Recall, Faithfulness, and Answer Quality
RAG evaluation with recall, ranking, faithfulness, answer relevance, and context metrics.
Query Transformations: Rewriting, HyDE, and Multi-Query
How rewriting, HyDE, multi-query, decomposition, and step-back prompting affect retrieval.
Reranking: Cross-Encoders and Cascades
Cross-encoder and hosted reranking for improving retrieval precision within a fixed latency budget.
Hybrid Search: BM25 Meets Dense Vectors
How BM25 and dense retrieval combine through reciprocal-rank or weighted fusion.
Chunking Strategies for Retrieval
How fixed-size, recursive, semantic, structural, and parent-document chunking affect retrieval.
Vector Databases & ANN Indexes
How HNSW, IVF, and ScaNN trade recall, latency, memory, and update cost.
Text Embeddings: Turning Meaning into Geometry
How text becomes dense vectors for semantic similarity and retrieval.
LLM Inference: Tokens, Context, and Sampling
LLM inference fundamentals covering tokenization, context windows, KV caching, and sampling.