Summary:

Seeking a Data Engineer skilled in designing, building, and maintaining data-intensive systems. This role involves collaborating with stakeholders to understand business needs and selecting appropriate technologies to meet those requirements.

Responsibilities:

• Develop and implement data architecture, including technology selection, coding, and storage solutions.
• Construct and maintain data pipelines to connect various systems, databases, and data warehouses for seamless data flow.
• Establish and enforce rules for data cleaning, standardization, and transformation to ensure data quality.
• Apply security controls and manage access policies to safeguard data assets.
• Monitor and optimize system performance using appropriate tools, and troubleshoot technical issues as needed.
• Collaborate with data scientists, analysts, and business teams to deliver effective data solutions based on organizational needs.
• Document technical designs, workflows, and best practices to facilitate knowledge sharing and maintain system integrity.
• Perform additional tasks as assigned to support evolving business and technical requirements.

Qualifications:

• Experience with SQL (Postgres preferred) and NoSQL databases such as MongoDB, Cassandra, InfluxDB, Clickhouse, and Elasticsearch.
• Proficiency with data pipeline orchestration tools (e.g., Kestra, Airflow).
• Demonstrated experience in designing data pipelines.
• Hands-on experience with Kafka, Kubernetes, Python, and Java.
• 10+ years of software development experience, with at least 7+ years as a Data Engineer or database administrator.
• Familiarity with IoT ingestion patterns.
• Experience implementing data pipelines using Kubernetes-ready workloads.
• Knowledge of real-time analytics frameworks such as Flink and Spark.
• Experience with Clickhouse and Kubeflow.
• Bachelor’s degree in Computer Science, Computer Engineering, or a related technical discipline, or 15+ years of relevant experience preferred.

Published Category: Technology & Data & AI Solutions