Data Engineer
Obviant
Obviant is seeking a Data Engineer to build and maintain data pipelines that transform complex information into usable data for the defense market. The role involves collaborating with various teams to ensure data accuracy and accessibility, contributing to the development of AI tools for defense acquisition.
Qualification
- 2+ years of hands-on experience building ETL/ELT pipelines and data extraction systems
- Proven ability to extract and parse unstructured data from documents, PDFs, APIs, and web sources
- Experience designing scalable data schemas that support analytical workloads and business intelligence
- Track record of automating manual data workflows and reducing operational overhead
- Comfort working with ambiguous requirements and translating them into technical solutions
Responsibility
- Ingest and extract data from diverse sources (APIs, databases, web scraping, documents, PDFs) and transform raw, unstructured information into clean, structured formats
- Collaborate with product, engineering, and customer success teams to understand data requirements and translate them into technical solutions
- Design and build robust ETL/ELT pipelines that automate data extraction, validation, and loading processes
- Develop data parsers and extraction logic for complex, unstructured sources (PDFs, contracts, procurement documents, budgetary reports)
- Assist in the build of scalable data pipelines using modern orchestration tools
- Create and optimize data models and schemas in data warehouses (Postgres, Clickhouse) that enable downstream analytics
- Implement data quality checks and validation frameworks to ensure accuracy and consistency across all pipelines
- Monitor pipeline performance, troubleshoot data issues, and implement alerting for pipeline failures
- Document data flows, pipeline architecture, and transformation logic for team knowledge sharing
