Associate AI Data Engineer

Key Responsibilities

• Design and develop LLM-based solutions for business use cases (e.g., chatbots, summarisation, document intelligence).
• Build and optimise RAG (Retrieval Augmented Generation) pipelines including data ingestion, embeddings, and retrieval.
• Implement prompt engineering techniques (prompt design, chaining, optimisation).
• Develop backend services/APIs for AI applications using Python frameworks (FastAPI / Flask / Streamlit).
• Integrate LLM solutions with enterprise systems and structured/unstructured data sources.
• Apply basic guardrails and evaluation techniques to improve response quality and reduce hallucinations.
• Collaborate with cross-functional teams to ensure data quality, model performance, and deployment readiness.
• Document solutions and contribute to reusable components and best practices.

Must-Have Skills

Experience

• 2–4 years total experience, with exposure to AI/ML, NLP, or Data Engineering projects
• Hands-on experience or strong learning exposure to LLM / GenAI use cases (projects, POCs, academic work, or professional)

LLM / GenAI & Agentic Engineering

• Strong hands-on experience with:
  • LLMs (Claude, OpenAI, etc.)
  • RAG pipelines and retrieval optimisation
  • GPT + Agentic AI implementation experience
• Experience with:
  • LangChain, LangGraph, or similar frameworks
  • Agent orchestration and tool-calling architectures
• Deep understanding of:
  • LLM limitations, evaluation, and optimisation strategies

Core Engineering

• Strong Python/Pyspark engineering expertise (production-grade development) with proven API integration experience
• Deep data analysis experience and handling large volume of data
• Fabric/Azure Databricks/Snowflake data engineering integration skills
• Good exposure to:
  • Cloud platforms (Azure/AWS/GCP)
  • SQL
  • Containers, CI/CD, monitoring

Data / AI Foundations (Mandatory)

Prior experience in one or more:

• Data Engineering (ETL/ELT, pipelines, orchestration)
• Data Science / ML lifecycle (especially NLP)
• Analytics engineering / data products

Good-to-Have

• Exposure to agentic workflows or tool calling concepts
• Basic knowledge of fine-tuning / prompt tuning (LoRA, PEFT – optional exposure)
• Experience with Azure OpenAI / Azure AI Search or similar stacks
• Awareness of enterprise AI considerations (data security, privacy, governance)