AI Lab (RunPod)
AI Lab — Distributed Inference R&D
A distributed RAG lab split in two: a local 'brain' orchestrating the pipeline and a RunPod GPU 'muscle' serving 4-bit quantized inference.
Proof
Why this matters
MLOps discipline: total separation of orchestration and inference, VRAM-safe model loading, and embedding parity between the local FAISS knowledge base and the remote /embed endpoint.
'Brain & Muscle' split: local orchestrator + Dockerized RunPod inference server
Read the code
Source on GitHub ↗ModelManager 'VRAM dance' loads 4-bit quantized models without OOM crashes
Built under real friction
Talk to Juan →FAISS knowledge base with embedding parity between local build and remote /embed
Read the code
Source on GitHub ↗Build → verify locally → deploy: Docker images tested before touching the GPU
Built under real friction
Talk to Juan →Stack
What this proves
Capabilities backed by this system
Evidence
Visual showcase
Founder note
From the builder
“The RunPod AI Lab shows Juan's MLOps approach to cost-efficient distributed inference.”
Want the full story?
Talk to Juan →Get it
Try it where it lives
Related systems
Explore next
Explore more of Amorosi Labs