Data Engineer (Pyspark/Snowflake/ AWS)

Role Overview:

We are seeking an experienced Data Engineer with expertise in PySpark, Snowflake and AWS  to join our team. The ideal candidate will design, develop, and optimize data pipelines and workflows in a cloud-based environment, leveraging AWS services, PySpark, and Snowflake for data processing and analytics.

Key Responsibilities:

Design and implement scalable ETL pipelines using PySpark on AWS.
Develop and optimize data workflows for Snowflake integration.
Manage and configure AWS services such as S3, Lambda, Glue, EMR, and Redshift.
Collaborate with data analysts and business teams to understand requirements and deliver solutions.
Ensure data security and compliance with best practices in AWS and Snowflake environments.
Monitor and troubleshoot data pipelines and workflows for performance and reliability.
Write efficient, reusable, and maintainable code for data processing and transformation.

Required Skills:
Strong experience with AWS services (S3, Lambda, Glue, MSK, etc.).
Proficiency in PySpark for large-scale data processing.
Hands-on experience with Snowflake for data warehousing and analytics.
Solid understanding of SQL and database optimization techniques.
Knowledge of data lake and data warehouse architectures.
Familiarity with CI/CD pipelines and version control systems (e.g., Git).
Strong problem-solving and debugging skills.

Experience with Terraform or CloudFormation for infrastructure as code.
Knowledge of Python for scripting and automation.
Familiarity with Apache Airflow for workflow orchestration.
Understanding of data governance and security best practices.
Certification in AWS or Snowflake is a plus.

Education and Experience:

Bachelor’s degree in Computer Science, Engineering, or related field with 6 to 10 years of experience.
5+ years of experience in AWS cloud engineering.
2+ years of experience with PySpark and Snowflake.