Ananda Poudel
Eight years building backend and platform systems in production. Most recently built an enterprise agentic AI platform from scratch: model routing, sandboxed execution, agent memory, and the observability and guardrails that keep autonomous workflows safe at 500+ requests per second. Mostly open source infrastructure, treated as a distributed systems problem, not an API wrapper.
EXPERIENCE
Senior Software Engineer, Cargill — 2018 to present, remote
Agentic AI platform
- Built the platform end to end for 500+ concurrent requests per second: LiteLLM as the orchestration gateway, routing each task to the model that fits on latency and cost, with standardized tool calling
- Designed the execution layer on Kubernetes with E2B: ephemeral sandboxes for autonomous workflows, agent code isolated from the host network by default
- Built two tier agent memory: Redis for short term session state, Milvus for long term vector storage, so long running workflows keep context across sessions
- Instrumented the reasoning loop with OpenTelemetry into Langfuse, making every decision step inspectable
- Layered Guardrails AI over tool execution to enforce programmatic policy: unverified tool calls dropped over 80 percent
LLM tooling and developer experience
- Own the LLM based compilation tooling that turns internal modeling definitions into executable pipeline code: prompt construction, context assembly, structured output parsing, and a validation layer gating production
- Built its evaluation harness: rubric scoring plus human review against a held out regression set on every prompt or model change
- Introduced LLM assisted code review and ran the prompt engineering training: review turnaround down about 25 percent with no drop in defect catch rate
Backend and infrastructure
- Build and operate the REST and GraphQL services other teams build against: six core services across Node, Go, and C# / .NET carrying about 800k requests per day for four product teams, with contract versioning
- Built the hybrid on prem to cloud data layer: ETL moving multi TB datasets in near real time at 99.9 percent accuracy
- Cut P99 API latency 300ms and raised peak throughput 50 percent via SQL tuning and Redis caching; SLO monitoring and distributed tracing hold 99 percent availability
- Own CI/CD and Bicep IaC: zero downtime deployments, about 40 percent less manual infra work, managed identities and Key Vault removing hardcoded secrets entirely
- Run the RFC and design review process across teams; mentor junior engineers; product owner tour alongside engineering
SELECTED PROJECTS
- playhal.app — live AI party game with a hidden agenda mechanic
- citecheck.me — AI visibility audit and tracker for site owners
- dejabit.app — writing tool for comedians: drafting, punch up, and originality checking
- Notes From Prod — engineering blog, 37 posts on distributed systems, security, product, and AI risk
SKILLS
agentic AI / LLM infra: LiteLLM, E2B, Milvus, Langfuse, Guardrails AI, OpenTelemetry, model routing and fallback, agent memory, RAG, embeddings, prompt engineering, structured output validation, eval harnesses, token and cost monitoring, Anthropic and OpenAI APIs
languages / backend: Go, Python, TypeScript, Node/Express, C#, .NET Core, SQL, GraphQL, gRPC, REST, microservices, event driven systems
infra / cloud: Kubernetes, Docker, Redis, Kafka, PostgreSQL, Linux, Azure (AKS, App Insights, Key Vault), Bicep, GitHub Actions, CI/CD, zero downtime deploys, observability and tracing
data / frontend: ETL pipelines, data lake architecture, stream processing, React, Next.js, React Native
EDUCATION
B.S. Software Engineering, Saint Cloud State University, 2016 to 2019. GPA 3.9.
Publication: "Interior Design with AR," MICS proceedings.