HPC Storage Engineer
wehrtyou
Hudson River Trading (HRT) is seeking an experienced HPC Storage Engineer to join their R&D team. The role involves managing and designing storage solutions for a large-scale research environment that supports AI/ML-based trading. The position offers opportunities for research and experimentation with new hardware and software to enhance performance and usability.
Qualification
- Knowledge of distributed file systems used for large-scale cluster computing (Lustre, CephFS, GPFS, etc.)
- Ability to design a home grown NAS storage solution with redundancy and high availability
- 5+ years of experience in HPC-like environments with large scale storage deployments greater than 1PB
- 3+ years of experience with scale-out enterprise storage solutions such as Isilon, VAST, NetApp, and Pure
- Extensive experience with Linux troubleshooting and NFS
- Strong knowledge of Linux software RAID, ZFS, and other Linux filesystems
- Proficient in Python and shell scripting
- Strong cross-functional communication skills
- Ability to analyze and solve problems quickly in a fast-paced environment
- Exceptional time management skills and the ability to balance independent and collaborative workflows
- Eager to learn and apply new skills
Responsibility
- Design and manage a variety of storage solutions
- Research and experiment with new storage solutions
- Troubleshoot complex storage, OS, and networking issues
- Build tools to improve the performance and monitoring of storage clusters
