Lead Cloud Data Engineer

This position will follow our hybrid work model, we expect the selected candidate to be in office 2-3 days a week(10-12 days a month ) at the following office location: St. Petersburg, FL/ Memphis, TN/ Denver, CO/Southfield, MI

Responsibilities:

• Design and develop scalable ETL pipelines using AWS Glue and PySpark.

• Integrate data from various sources into Amazon Redshift, ensuring data quality and consistency.

• Optimize the performance of data processing jobs and Redshift queries.

• Deploy infrastructure as code (IaC) with Terraform

• Monitor and troubleshoot data pipelines to ensure reliability and efficiency.

• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements.

• Ensure compliance with security policies and regulations.

• Document data workflows and best practices.

• Mentor and guide junior engineers on the team.

• Continuously explore new AWS services and technologies to improve existing solutions.

Skills:

• Expert level experience with AWS Cloud Services is preferred, knowledge of other Cloud Providers is a plus

• Experience and understanding of AWS Cloud Services components like Redshift, S3, Kafka, Glue, AirFlow, SageMaker, EMR, Athena, Lake Formation, DynamoDB, Aurora, VPC, IAM and many other Cloud Service modules.

• Working experience in Python is preferred, Java and other languages are a plus

• Nice to have Terraform or other Infrastructure as Code experience

• Knowledge and experience with Analytics, Data Warehouse and Big Data concepts across both batch and streaming patterns

• Solid understanding of networking, infrastructure, cost management, SRE concepts

• Creating patterns for adoption from scratch and being able to document and articulate coding and architecture standards for others to follow.

• Performance tuning and optimization of code and infrastructure

• Creating a culture of continuous improvement, performs code reviews and makes recommendations for improvement or streamlining patterns

• Relevant experience in various database platforms, ETL solutions/products, Infrastructure, ETL architecture and On-Prem or Cloud Services.

• Financial Services Industry knowledge is a plus.

Ability to:

• Identify and understand issues, problems and opportunities for automation.

• Good communication and organization skills that clearly convey information and ideas through a variety of media to individuals or groups in a manner that engages the audience and helps them understand and retain the message.

• Use effective approaches for choosing a course of action or developing appropriate solutions; recommend or take action that is consistent with available facts, constraints and probable consequences.

• Demonstrate a satisfactory level of technical and professional skill or knowledge in position-related areas; remains current with developments and trends in areas of expertise.

• Develop and use collaborative relationships to facilitate the accomplishment of work goals.

• Make internal and external clients and their needs a primary focus of actions; develop and sustain productive client relationships.

Certifications

Preferred , not required

• AWS Certified Solutions Architect

• AWS Certified Big Data

• AWS Certified Data Analytics

• AWS Certified Developer

• AWS Certified Machine Learning