Skip to main content

Explore Problems

Certifications Pricing Discuss

Production AI Engineering (LLMOps)

Course Contents

Inference Performance Metrics (TTFT, TPOT, Throughput)

Batching & Continuous Batching

Speculative Decoding & Other Tricks

Serving with vLLM / TGI

Streaming for Perceived Latency

Token Economics: Where the Money Goes

Semantic Caching for Agents

Model Routing & Cascades (Heterogeneous Architectures, SLMs)

Batch APIs & Distillation for Cost

The Reference Architecture (Full Diagram)

The Model Gateway & Router

Adding Caches (Prompt + Semantic)

Orchestration & Pipelines

Putting Guardrails in the Loop

The Monitoring & Observability Layer

Treating Latency as a Product Feature

Streaming UX (TTFT, Token-by-Token, Markdown Buffering)

Showing Your Work: Citations, Sources & Confidence

Defensive UX: Designing for Uncertainty & Failure

Feedback UI & Human-in-the-Loop (The Data Flywheel)

Multi-Turn Conversation & Session Management

Agent UX: Showing Reasoning, Progress & Approvals

Input Guardrails (Validation, PII, Moderation)

Output Guardrails (Format, Safety, Grounding Checks)

Hallucination Mitigation Strategies

Fallbacks, Retries & Circuit Breakers

Content Moderation & Responsible AI

AI Governance & Compliance (EU AI Act, NIST AI RMF)

Model Cards, Audit Logs & Incident Response

Hands-On: Build a Guardrail Layer

The OWASP LLM Top 10

Prompt Injection in Depth (Direct & Indirect)

Data Exfiltration & Tool Abuse

Securing Agents & MCP

Red-Teaming Your AI Application

Designing Your AI API (Sync, Async, Streaming)

Rate Limiting & Quotas

CI/CD for Prompts & Evals

Versioning Prompts, Models & Datasets

Scaling & Reliability Patterns

Wiring Up Observability End-to-End

Vision: Image Understanding & VQA

Document AI: OCR → Agentic Extraction

Audio: Speech-to-Text & Text-to-Speech

Building Voice Agents

Image & Video Generation (Diffusion Basics)

Multimodal RAG & Pipelines

Reasoning Models & Test-Time Compute

Computer-Use & Browser Agents

Coding Agents (How Tools Like Claude Code Work)

The Open Research Problems

Staying Current: How to Keep Learning

The Model Gateway & Router

The Model Gateway & Router on CodePeet.

Loading...

Loading content...