Senior GenAI Research Engineer - Optimization and Kernels
Senior GenAI Research Engineer - Optimization and Kernels
Senior GenAI Research Engineer - Optimization and Kernels
Databricks
Databricks is seeking a Senior GenAI Research Engineer to join their Mosaic AI organization, focusing on developing advanced AI models and systems. The role involves creating new techniques in deep learning, optimizing GPU kernels, and designing distributed training frameworks for large language models.
Qualification
- BS/MS/PhD in Computer Science or related field
- Hands-on experience writing and tuning CUDA kernels for ML training applications
- Experience in distributed training frameworks such as PyTorch DDP, DeepSpeed, Megatron-LM, FSDP
- Strong understanding of deep learning concepts and techniques
- Ability to work collaboratively in a diverse team environment
Responsibility
- Drive performance improvements through advanced optimization techniques including kernel fusion, mixed precision, memory layout optimization, tiling strategies, and tensorization for training-specific patterns
- Design, implement, and optimize high-performance GPU kernels for training workloads targeting NVIDIA architectures
- Design and implement distributed training frameworks for large language models, including parallelism strategies and optimized communication patterns
- Profile, debug, and optimize end-to-end training workflows to identify and resolve performance bottlenecks
- Collaborate with a team of researchers and engineers to advance the scientific frontier in AI
