Software Engineer, Data Infrastructure
Software Engineer, Data Infrastructure
Software Engineer, Data Infrastructure
DatologyAI
DatologyAI is seeking an experienced Data Platform Engineer to join their core team in Redwood City, CA. The company focuses on improving data curation for training large models efficiently. The role involves leading the development of the data platform and collaborating with founders and researchers to enhance product capabilities.
Qualification
- Experience in leading and building production data systems
- Proficiency in scalable data processing solutions (e.g., Spark, Flink)
- Experience with data lakes or warehouses (e.g., Snowflake, Hive)
- Strong SQL query authoring skills
- Familiarity with distributed storage systems (e.g., HDFS, S3)
- Experience with workflow management tools (e.g., Airflow, Dagster)
Responsibility
- Design, build and maintain highly scalable data processing solutions
- Architect, build, and deploy back-end systems and services for the data curation platform
- Collaborate with researchers and engineers to implement new features
- Ensure system reliability, security, and customer trust
- Drive business-critical technical decisions and product direction
