Position Overview
We are looking for an experienced Lead Data Engineer to join our dynamic team. If you are passionate about building scalable software solutions, and work collaboratively with cross-functional teams to define requirements and deliver solutions we would love to hear from you.
ShyftLabs is a growing data product company that was founded in early 2020 and works primarily with Fortune 500 companies. We deliver digital solutions built to help accelerate the growth of businesses in various industries, by focusing on creating value through innovation.
Job Responsibilities:
Develop and maintain data pipelines and ETL/ELT processes using PythonDesign and implement scalable, high-performance applicationsWork collaboratively with cross-functional teams to define requirements and deliver solutionsDevelop and manage near real-time data streaming solutions using Pub, Sub or Beam.Contribute to code reviews, architecture discussions, and continuous improvement initiativesMonitor and troubleshoot production systems to ensure reliability and performance,
Basic Qualifications:
5+ years of professional software development experience with PythonStrong understanding of software engineering best practices (testing, version control, CI/CD)Experience building and optimizing ETL/ELT processes and data pipelinesProficiency with SQL and database conceptsExperience with data processing frameworks (e.g., Pandas)Understanding of software design patterns and architectural principlesAbility to write clean, well-documented, and maintainable codeExperience with unit testing and test automationExperience working with any cloud provider (GCP is preferred)Experience with CI/CD pipelines and Infrastructure as codeExperience with Containerization technologies like Docker or KubernetesBachelor's degree in Computer Science, Engineering, or related field (or equivalent experience)Proven track record of delivering complex software projectsExcellent problem-solving and analytical thinking skillsStrong communication skills and ability to work in a collaborative environment,
Preferred Qualifications:
Experience with GCP services, particularly Cloud Run and DataflowExperience with stream processing technologies (Pub/Sub)Familiarity with big data technologies (Airflow)Experience with data visualization tools and librariesKnowledge of CI/CD pipelines with Gitlab and infrastructure as code with TerraformFamiliarity with platforms like Snowflake, Bigquery or Databricks,.GCP Data engineer certification We are proud to offer a competitive salary alongside a strong insurance package. We pride ourselves on the growth of our employees, offering extensive learning and development resources.
