Senior Software Engineer - Distributed Data Systems
Senior Software Engineer - Distributed Data Systems
Senior Software Engineer - Distributed Data Systems
Databricks
Databricks is seeking a Senior Software Engineer for its Runtime team to develop next-generation distributed data storage and processing systems. The role involves working on projects like Apache Spark, Delta Lake, and performance engineering to enhance data analysis capabilities and support diverse workloads.
Qualification
- BS or higher in Computer Science or a related technical field, or equivalent practical experience.
- Experience with distributed systems and data processing frameworks.
- Strong programming skills in languages such as Java, Scala, or Python.
- Familiarity with cloud storage solutions and data engineering concepts.
- Ability to work collaboratively in a fast-paced, technical environment.
Responsibility
- Build next-generation distributed data storage and processing systems.
- Develop and enhance Apache Spark as the open-source standard for big data.
- Create reliable and high-performance services for cloud storage backends like AWS S3 and Azure Blob Store.
- Design and implement Delta Lake for efficient data management with ACID transactions and time travel.
- Work on Delta Pipelines to simplify the orchestration of data engineering pipelines.
