Applied AI Engineer
CodeRound is hiring for this role.
- Applied AI Engineer
- Applied Machine Learning
- NLP
- Large Language Models
- LLMs
- Retrieval-Augmented Generation
- RAG
- Information Retrieval
- Semantic Search
- Vector Databases
- Embeddings
- Reranking
- Prompt Engineering
- LLM Evaluation
- Evaluation Frameworks
- Eval Harnesses
- Agentic AI
- AI Agents
- Multi-Agent Systems
- LangChain
- LlamaIndex
- OpenAI
- Anthropic Claude
- Python
- FastAPI
- Pinecone
- Elasticsearch
- pgvector
- AWS
- Azure
- Context Engineering
- Model Optimization
- LLM Observability
- Latency Optimization
- Cost Optimization
- Enterprise AI
We're looking for an Applied AI Engineer to build and optimize production-grade AI systems that combine retrieval, reasoning, and generation to solve complex enterprise problems. You'll own end-to-end RAG pipelines, develop evaluation frameworks, and build intelligent agentic workflows that deliver accurate, reliable, and context-aware outputs. This role is ideal for engineers who enjoy solving challenging NLP problems, improving LLM performance through rigorous evaluation, and taking AI solutions from experimentation to production.
What you'll do
- Design, build, and own end-to-end RAG pipelines, covering document ingestion, preprocessing, chunking, embeddings, retrieval, reranking, and response generation.
- Develop and optimize retrieval architectures to improve relevance, factual accuracy, contextual understanding, and the quality of generated responses.
- Build robust evaluation frameworks by defining quality metrics, creating evaluation datasets, developing automated eval harnesses, and tracking system performance over time.
- Design and implement agentic workflows involving multi-step reasoning, tool use, orchestration, model selection, and context management.
- Continuously evaluate and improve LLM outputs through experimentation, error analysis, prompt optimization, retrieval tuning, and systematic benchmarking.
- Optimize AI pipelines for production requirements, balancing response quality, latency, scalability, reliability, and computational costs.
- Collaborate with backend engineers to integrate and serve NLP outputs through reliable, scalable APIs that meet production performance requirements.
- Work with enterprise data from multiple sources, transforming unstructured and heterogeneous information into accurate, contextual intelligence for downstream workflows.
- Stay current with advancements in LLMs, RAG, evaluation techniques, and agentic systems, identifying promising approaches and translating them into measurable product improvements.
Must have
3–6 years of experience in production NLP, Machine Learning, or Applied AI, with a track record of shipping models or pipelines used by real users. Hands-on experience designing, building, and optimizing production-grade Retrieval-Augmented Generation (RAG) pipelines, including retrieval architecture, chunking, reranking, and prompt design. Strong understanding of LLMs, NLP, semantic search, embeddings, context management, and retrieval-based systems. Experience defining quality metrics, building evaluation frameworks, developing evaluation harnesses, and systematically measuring LLM output quality. Strong Python engineering skills, including writing clean, maintainable, tested, production-grade code beyond experimentation in notebooks. Practical experience integrating LLMs such as OpenAI GPT models or Anthropic Claude into production applications. Familiarity with agentic frameworks such as LangChain, LlamaIndex, or equivalent tools for building LLM-powered workflows. Experience working with vector databases and search technologies such as Pinecone, Elasticsearch, or pgvector. Experience deploying or supporting AI/ML applications on cloud platforms such as AWS or Azure, with an understanding of production reliability and performance.
Good to have
Experience designing multi-agent systems, multi-step reasoning workflows, tool calling, and agent orchestration patterns. Familiarity with enterprise data sources, including heterogeneous document formats, noisy inputs, inconsistent schemas, and unstructured business data. Experience optimizing LLM-powered applications for latency, throughput, token usage, and inference costs. Background in B2B SaaS, sales technology, CRM systems, revenue intelligence, or enterprise workflow automation. Experience with advanced retrieval techniques, including hybrid search, query expansion, metadata filtering, and cross-encoder reranking. Familiarity with FastAPI or similar frameworks for building scalable APIs and serving AI/NLP pipelines. Experience developing automated evaluation pipelines, regression testing systems, and continuous monitoring for LLM applications. Understanding of model selection, prompt optimization, context-window management, and trade-offs between quality, latency, and cost. Ability to track emerging research in LLMs, retrieval, and agentic AI and translate relevant advances into production-ready solutions.