Egen

Company

Senior Data Engineer – Python & AWS

Job Description

Job Overview:

We are looking for an experienced Senior Data Engineer to design, build, and optimize scalable, high-performance data platforms using AWS cloud services and Python. The ideal candidate will play a key role in architecting end-to-end data pipelines, driving automation, ensuring data quality, and enabling analytics and AI workloads across the organization.

This role requires deep technical expertise in AWS data services, modern data architecture, and a passion for delivering reliable, high-quality data solutions at scale.

Key Responsibilities

Architect and implement scalable, fault-tolerant data pipelines using AWS Glue, Lambda, EMR, Step Functions, and Redshift

Build and optimize data lakes and data warehouses on Amazon S3, Redshift, and Athena

Develop Python-based ETL/ELT frameworks and reusable data transformation modules

Integrate multiple data sources (RDBMS, APIs, Kafka/Kinesis, SaaS systems) into unified data models

Lead efforts in data modeling, schema design, and partitioning strategies for performance and cost optimization

Drive data quality, observability, and lineage using AWS Data Catalog, Glue Data Quality, or third-party tools

Define and enforce data governance, security, and compliance best practices (IAM policies, encryption, access control)

Collaborate with cross-functional teams (Data Science, Analytics, Product, DevOps) to support analytical and ML workloads

Implement CI/CD pipelines for data workflows using AWS CodePipeline, GitHub Actions, or Cloud Build

Provide technical leadership, code reviews, and mentoring to junior engineers

Monitor data infrastructure performance, troubleshoot issues, and lead capacity planning

Required Skills & Qualifications

Bachelor’s or Master’s degree in Computer Science, Information Systems, or related field

5–10 years of hands-on experience in data engineering or data platform development

Expert-level proficiency in Python (pandas, PySpark, boto3, SQLAlchemy)

Advanced experience with AWS Data Services, including:

AWS Glue, Lambda, EMR, Step Functions, DynamoDB, EDW Redshift, Athena, S3, Kinesis, Amazon Quicksight.

IAM, CloudWatch, CloudFormation / Terraform (for infrastructure automation)

Strong experience in SQL, data modeling, and performance tuning

Proven ability to design and deploy data lakes, data warehouses, and streaming solutions

Solid understanding of ETL best practices, partitioning, error handling, and data validation

Hands-on experience in version control (Git) and CI/CD for data pipelines

Knowledge of containerization (Docker/Kubernetes) and DevOps concepts

Excellent analytical, debugging, and communication skills

Preferred Skills

Experience with Apache Spark or PySpark on AWS EMR or Glue

Familiarity with Airflow, dbt, or Dagster for workflow orchestration

Exposure to real-time data streaming (Kafka, Kinesis Data Streams, or Firehose)

Knowledge of Lake Formation, Glue Studio, or DataBrew

Experience integrating with machine learning and analytics platforms (SageMaker, QuickSight)

Certification: AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect

Soft Skills

Strong ownership mindset with focus on reliability and automation

Ability to mentor and guide data engineering teams

Effective communication with both technical and non-technical stakeholders

Please mention that you found this job on MoAIJobs, this helps us grow. Thank you!

Egen

4 jobs posted

View all Egen jobs

About the job

Posted on

Nov 18, 2025

Apply before

Dec 18, 2025

Job typeFull-time

CategoryData Engineer

Share this job opportunity

Job applications getting ignored?

Professional headshots increase response rates by 40%

Ready in 3 minutes

Save $200+ vs photographer

Perfect for LinkedIn & resumes

Get My Headshots

Similar Jobs

Discover more opportunities that match your interests

View all Data Engineer jobs

Looking for something different?

Browse all AI jobs