AI Researcher - Pre-training

Responsibilities: — Design and run foundation-model experiments — Improve training recipes — Support large-scale training runs The role involves working with machine learning technologies including TorchTitan, Megatron-LM, DeepSpeed, PyTorch, and Python, focusing on areas such as attention mechanisms, LLMs, experiment design, tensor parallelism, optimization, kernel optimization, pipeline parallelism, transformer models, scaling laws, reproducibility, probability, distributed training, expert parallelism, and data parallelism.