Our client is a well-funded AI Stealth Startup based in San Francisco building frontier foundation models that push the boundaries of large language model capabilities. Founded by leading researchers and engineers, the team is tackling fundamental challenges in model architecture, scaling laws and large-scale pre-training across multiple modalities. This is an opportunity to join at an early stage, where your research will directly influence the direction of next-generation foundation models and contribute to breakthroughs in generative AI.
The Role
We are seeking a Machine Learning Researcher (MTS) with a strong background in deep learning and large-scale model training. You will work across every stage of the pre-training pipeline, researching novel model architectures, scaling laws and training methodologies while building highly efficient distributed training systems.
This is a hands-on research role suited to someone who enjoys developing new ideas, running large-scale experiments and translating research into production-ready foundation models. You will collaborate closely with researchers and infrastructure engineers to advance the state of the art in large language model pre-training, with your research directly shaping the next generation of AI products developed by the company.
Key Responsibilities
- Research and develop novel approaches to large language model pre-training.
- Investigate new model architectures, scaling laws and training methodologies for frontier foundation models.
- Design, implement and optimise large-scale pre-training pipelines using web-scale datasets.
- Improve model quality, efficiency and scalability through advances in optimisation algorithms, data curation and compute-efficient training.
- Build and optimise distributed training systems using state-of-the-art parallelisation techniques.
- Conduct research into compute scaling, data scaling and model scaling to improve training efficiency and downstream performance.
- Develop evaluation frameworks to assess reasoning, coding, instruction following, factuality and general model capability.
Required Qualifications
- Bachelor’s or Master’s degree in Computer Science, Machine Learning, Artificial Intelligence, Mathematics, Statistics or another highly AI relevant discipline.
- Strong experience developing deep learning models using PyTorch.
- Deep understanding of large language model pre-training, modern deep learning optimisation techniques and foundation model research.
- Experience researching or training large language models at scale.
- Strong understanding of transformer architectures, model scaling laws, neural network optimisation and representation learning.
- Experience with distributed deep learning frameworks such as FSDP, DeepSpeed or Megatron-LM.
- Strong knowledge of tensor parallelism, pipeline parallelism, data parallelism and ZeRO optimisation.
- Experience working with large-scale GPU clusters and optimising distributed training workloads.
- Strong Python programming skills and experience building high-performance machine learning systems.
- Experience designing rigorous experiments and analysing model performance at scale.