We are partnered with a leading quantitative trading fund that is applying frontier LLMs and foundation models to solve some of the most challenging problems within financial markets. We are looking for an ML Researcher to contribute to cutting-edge research and development in large-scale language model pre-training. This role combines deep scientific expertise with hands-on engineering to design, train and optimise SOTA transformer models at scale. You will work across the full research lifecycle, from novel algorithm development and large-scale distributed training through to evaluation, optimisation and deployment, collaborating with world-class researchers and engineers to build production-grade AI systems.
Key Responsibilities
- Conduct research into large-scale transformer architectures, self-supervised learning and foundation model pre-training.
- Design and execute pre-training strategies for large language models.
- Develop novel approaches to improve model quality, efficiency, scalability and training performance.
- Build and optimise distributed training pipelines across large GPU clusters using modern parallelisation techniques.
- Improve training throughput through innovations in data loading, communication, memory optimisation and scheduling.
- Research and implement techniques such as Mixture of Experts (MoE), retrieval augmented training, parameter-efficient learning, curriculum learning and continual pre-training.
- Develop robust evaluation frameworks to assess model capability, reasoning, coding performance, multilingual ability and factuality.
- Work closely with infrastructure engineers to maximise compute utilisation and improve training efficiency.
- Contribute to research that advances large-scale language model training and informs future model development.
- Document experimental findings and communicate results clearly with researchers and engineers.
Required Qualifications
- Bachelor’s degree in Computer Science, Machine Learning, Artificial Intelligence, Mathematics, Statistics or another highly quantitative discipline.
- Alternatively, a Master’s degree with industry research experience developing state-of-the-art machine learning systems.
- Strong experience building deep learning models using PyTorch.
- Deep understanding of transformer architectures, attention mechanisms, positional encoding, normalisation techniques, scaling laws and optimisation algorithms.
- Experience training large language models from scratch or conducting large-scale continual pre-training.
- Strong knowledge of distributed deep learning, including:
- Fully Sharded Data Parallel (FSDP)
- DeepSpeed
- Megatron-LM
- Tensor Parallelism
- Pipeline Parallelism
- Data Parallelism
- ZeRO Optimisation
- Experience working with large-scale GPU clusters.
- Strong software engineering skills in Python and modern machine learning frameworks.
- Experience designing reproducible experiments and analysing large-scale model performance.