Your career opportunity

We are a dedicated team of six engineers working on flagship AI and ML projects that are transforming the Credit and Lending landscape. Youll join a fast-moving, innovative and stable environment, partnering closely with business stakeholders to deliver high-impact solutions. This role focuses on building robust evaluation approaches for LLM and AI applications so quality, reliability and safety are proven before changes reach production.

What youll do

• Define and implement evaluation strategies for LLM and AI applications used in Credit and Lending with clear quality reliability and safety criteria
• Build and maintain representative evaluation datasets and repeatable test harnesses for prompts models Retrieval-Augmented Generation and agentic workflows
• Measure response correctness relevance groundedness completeness latency and cost using appropriate automated and human evaluation methods
• Evaluate retrieval quality source attribution and answer grounding across complex financial data and vector databases
• Test AI systems for robustness hallucinations bias fairness prompt injection and other safety risks and use findings to guide improvements
• Automate evaluation and regression tests in CI/CD so changes to prompts models or data are assessed before release
• Analyse evaluation results investigate quality regressions and communicate trade-offs and recommendations to engineering and business stakeholders
• Collaborate with engineers product managers and business experts to align evaluation coverage with user needs and strategic goals

What you need to have to succeed in this role

• Demonstrate deep expertise in Python and its ML and AI ecosystem
• Apply experience with FastAPI LangChain and data containerisation
• Use a solid understanding of microservices observability API design and concurrency models
• Build and deploy on Azure or GCP using containers Kubernetes and CI/CD tools such as Jenkins Azure DevOps or GCP Cloud Build
• Apply practical experience with Test-Driven Development Behaviour-Driven Development and automated testing
• Use a strong grasp of LLM architectures including experience fine-tuning models on domain-specific data
• Apply hands-on experience with Retrieval-Augmented Generation advanced prompt engineering and agentic architectures
• Communicate clearly in English including explaining technical solutions evaluation results and trade-offs to business partner