Skip to content
Jatin Bansal
Theme

AI Engineering

Last entry May 26, 2026. RSS feed

  1. ai-engineering Quantization and Distillation: Compression for Inference Quantization and distillation methods for reducing inference memory, latency, and serving cost. Reading time: 11 minutes
  2. ai-engineering LoRA and Parameter-Efficient Fine-Tuning How LoRA and QLoRA reduce trainable parameters, GPU memory, and adapter deployment cost. Reading time: 11 minutes
  3. ai-engineering DPO and Modern Alignment Direct Preference Optimization, its common variants, production workflow, and failure modes. Reading time: 12 minutes
  4. ai-engineering From Pre-Training to RLHF LLM pretraining, supervised fine-tuning, preference optimization, and verifiable rewards. Reading time: 8 minutes
  5. ai-engineering Agent Budgets and Runaway Prevention How agent harnesses enforce step, time, token, cost, and tool-call limits. Reading time: 12 minutes
  6. ai-engineering PII Detection and Data Privacy for LLM Systems How to detect, transform, retain, and delete personal data in LLM request and storage paths. Reading time: 12 minutes
  7. ai-engineering Guardrails: Input and Output Safety Layers for LLM Systems How input, output, and tool-call guardrails enforce application safety policies. Reading time: 10 minutes
  8. ai-engineering Fine-Tuning vs RAG: When to Choose Which How to choose fine-tuning, retrieval, or both based on behavior, knowledge freshness, cost, and evaluation. Reading time: 12 minutes
  9. ai-engineering Cost Optimization and Model Routing How predictive routers and cascades trade model cost, quality, and tail latency per request. Reading time: 11 minutes
  10. ai-engineering Speculative Decoding and Draft Models Speculative decoding with draft models, Medusa, EAGLE, and prompt lookup for lower decode latency. Reading time: 11 minutes
  11. ai-engineering Inference Latency: Prefill, Decode, and Batching Inference latency across prefill, decode, batching, chunked prefill, and disaggregated serving. Reading time: 13 minutes
  12. ai-engineering Sleep-Time Compute for AI Agents Background consolidation and speculative work for stateful agents with explicit consistency controls. Reading time: 8 minutes
  13. ai-engineering Human-in-the-Loop Feedback Loops for LLM Systems How to capture, sample, label, and route production feedback into evaluation and product changes. Reading time: 13 minutes
  14. ai-engineering Drift Detection and Regression Testing for LLM Systems How to detect distribution shifts and test model upgrades with paired evaluation, shadow runs, and canaries. Reading time: 13 minutes
  15. ai-engineering Production Tracing and Observability for LLM Systems How to trace model calls, tools, retrieval, cost, cache usage, and evaluation in LLM applications. Reading time: 11 minutes
  16. ai-engineering LLM-as-Judge: Pointwise and Pairwise How to design and calibrate pointwise and pairwise LLM judges against human labels. Reading time: 12 minutes
  17. ai-engineering Eval-Driven Development for LLM Systems Evaluation-driven development with error analysis, golden sets, layered checks, and CI gates. Reading time: 9 minutes
  18. ai-engineering Production Memory Frameworks: MemGPT/Letta, mem0, Zep, Graphiti How Letta, mem0, Zep, and Graphiti differ in storage, retrieval, and operational cost. Reading time: 9 minutes
  19. ai-engineering Memory Evaluation: Benchmarks and Custom Evals How public benchmarks and custom evals measure memory recall, updates, temporal reasoning, and cost. Reading time: 9 minutes
  20. ai-engineering Conversation Compaction: Keeping Long Sessions Alive How long agent sessions compact context using preemptive triggers, cache-aware deletion, rollback, and journals. Reading time: 13 minutes
  21. ai-engineering Anatomy of an Agent Harness Agent harness responsibilities including context assembly, tool dispatch, streaming, caching, recovery, cost accounting, and telemetry. Reading time: 13 minutes
  22. ai-engineering Long-Horizon Task Reliability How checkpointing, abort policies, and recovery limit drift in long-running agents. Reading time: 10 minutes
  23. ai-engineering Computer Use and Browser Agents Computer-use agents based on screenshots or DOM access, with grounding, sandboxing, injection defense, and OSWorld evaluation. Reading time: 12 minutes
  24. ai-engineering Memory Privacy, Isolation, and Multi-Tenancy How to isolate tenant memory, resist stored prompt injection, and verify deletion. Reading time: 15 minutes
  25. ai-engineering Multi-Agent Shared Memory Shared memory for multiple agents with explicit scope, consistency, conflict handling, and audit. Reading time: 15 minutes
  26. ai-engineering Cross-Session Identity and Personalization How agents load durable user preferences across sessions without weakening privacy boundaries. Reading time: 8 minutes
  27. ai-engineering Procedural Memory and Skill Caching Procedural memory for agents using success-gated skills, task-shape keys, and retrieval-time validation. Reading time: 12 minutes
  28. ai-engineering Memory Conflict, Forgetting, and Embedding Drift Agent-memory conflict resolution, active forgetting, and embedding-model migration. Reading time: 12 minutes
  29. ai-engineering Temporal Reasoning and Memory Provenance Temporal memory retrieval with as-of queries, staleness scoring, and evidence provenance. Reading time: 13 minutes
  30. ai-engineering Memory Retrieval Policies: Recency, Relevance, Importance How recency, importance, and similarity combine to rank memories for the current query. Reading time: 13 minutes
  31. ai-engineering Summarization and Context Compression How recursive summarization, structured notes, and API compaction reduce agent context safely. Reading time: 15 minutes
  32. ai-engineering Reflection: From Experiences to Beliefs Memory reflection that derives evidence-linked beliefs from episodic records. Reading time: 14 minutes
  33. ai-engineering Episode Segmentation and Salience Scoring How episode boundaries and salience scores shape long-term memory retrieval. Reading time: 12 minutes
  34. ai-engineering Memory Write Policies: What's Worth Remembering Agent-memory admission control with extraction, distillation, deduplication, and linking. Reading time: 14 minutes
  35. ai-engineering Hierarchical Memory: Working / Episodic / Semantic Tiers Hierarchical agent memory with core, recall, and archival tiers plus promotion and demotion policies. Reading time: 14 minutes
  36. ai-engineering Knowledge Graphs as Structured Memory How knowledge graphs represent entities, relations, temporal validity, and hybrid graph-vector retrieval. Reading time: 12 minutes
  37. ai-engineering Long-Term Memory: Vector-Backed Episodic Storage Long-term episodic memory with vector storage, episode boundaries, selective writes, and recency-weighted retrieval. Reading time: 12 minutes
  38. ai-engineering Working Memory: Scratchpads, Blackboards, and Agent Notebooks Agent working memory using scratchpads, typed state, notebooks, and multi-agent blackboards. Reading time: 12 minutes
  39. ai-engineering Short-Term Memory: Managing the Conversation Buffer How token budgets, protected messages, and head-tail eviction manage conversation context. Reading time: 12 minutes
  40. ai-engineering The Cognitive Taxonomy: Semantic, Episodic, Procedural How working, episodic, semantic, and procedural memory differ in agent systems. Reading time: 15 minutes
  41. ai-engineering The Memory Stack: A Map of AI Memory How agent memory separates working state, durable storage, cognitive types, and maintenance paths. Reading time: 10 minutes
  42. ai-engineering Tool Selection at Scale: MCP and Dynamic Routing Tool selection at scale using MCP discovery, deferred loading, semantic retrieval, and namespaces. Reading time: 16 minutes
  43. ai-engineering Multi-Agent Orchestration How supervisor, swarm, and hierarchical agent patterns trade specialization for coordination cost. Reading time: 12 minutes
  44. ai-engineering Planning Agents vs Reactive Agents How ReAct, plan-and-execute, ReWOO, and search-based planning trade cost for control. Reading time: 10 minutes
  45. ai-engineering The Agent Loop: ReAct and Its Descendants How ReAct agents execute tools, process observations, enforce stop conditions, and compare with plan-and-execute. Reading time: 11 minutes
  46. ai-engineering Constrained Decoding: Grammars, Regex, and FSMs How vocabulary masks, finite-state machines, grammars, and schema compilation constrain model output. Reading time: 9 minutes
  47. ai-engineering Prompt Caching: Reusing the KV Cache Across Calls Prompt caching across Anthropic, OpenAI, and Gemini, including prefix layout and cost calculations. Reading time: 8 minutes
  48. ai-engineering Streaming and Backpressure How SSE streaming handles partial output, cancellation, tool-call JSON, and backpressure. Reading time: 10 minutes
  49. ai-engineering Function Calling and Tool Use LLM tool use as typed RPC with schema validation, dispatch, retries, and result handling. Reading time: 9 minutes
  50. ai-engineering Structured Output: JSON Mode and Schema Coercion Schema-constrained LLM output using JSON mode, strict schemas, tool calls, and validation retries. Reading time: 11 minutes
  51. ai-engineering Context Engineering: JIT vs AOT Context Loading Ahead-of-time context assembly and just-in-time retrieval compared by attention, latency, and token cost. Reading time: 9 minutes
  52. ai-engineering RAG Evaluation: Recall, Faithfulness, and Answer Quality RAG evaluation with recall, ranking, faithfulness, answer relevance, and context metrics. Reading time: 11 minutes
  53. ai-engineering Query Transformations: Rewriting, HyDE, and Multi-Query How rewriting, HyDE, multi-query, decomposition, and step-back prompting affect retrieval. Reading time: 11 minutes
  54. ai-engineering Reranking: Cross-Encoders and Cascades Cross-encoder and hosted reranking for improving retrieval precision within a fixed latency budget. Reading time: 9 minutes
  55. ai-engineering Hybrid Search: BM25 Meets Dense Vectors How BM25 and dense retrieval combine through reciprocal-rank or weighted fusion. Reading time: 11 minutes
  56. ai-engineering Chunking Strategies for Retrieval How fixed-size, recursive, semantic, structural, and parent-document chunking affect retrieval. Reading time: 9 minutes
  57. ai-engineering Vector Databases & ANN Indexes How HNSW, IVF, and ScaNN trade recall, latency, memory, and update cost. Reading time: 9 minutes
  58. ai-engineering Text Embeddings: Turning Meaning into Geometry How text becomes dense vectors for semantic similarity and retrieval. Reading time: 9 minutes
  59. ai-engineering LLM Inference: Tokens, Context, and Sampling LLM inference fundamentals covering tokenization, context windows, KV caching, and sampling. Reading time: 8 minutes