Senior Data Engineer

We are seeking a skilled and motivated Python and Database Developer to join our dynamic technology team. The ideal candidate will possess strong programming skills in Python, a solid understanding of database systems, and the ability to design, develop, and maintain efficient and scalable applications. This role involves working on a variety of projects, including data processing pipelines, web applications, and database integrations.

Responsibilities and Duties

• Data Engineering Leadership: Lead the design and development of data pipelines for ingestion, transformation, and loading of data from various sources (databases, APIs, streaming platforms) into our data warehouse/lake, ensuring seamless data flow and accessibility.

• Develop data models that support business requirements and analytical needs.

• Optimize data models for query performance and data accessibility.

• Ensure data models are well-documented and aligned with industry best practices.

• Database Optimization: Write optimized and maintainable SQL queries and leverage SQLAlchemy for efficient database interaction, ensuring high performance and data accuracy.

• Data Quality Assurance: Implement robust data quality checks and monitoring systems to ensure data integrity and accuracy, proactively identifying and resolving data issues.

• Data Governance Contribution: Contribute to the design and implementation of data governance policies and procedures, ensuring compliance with regulatory requirements and internal standards.

• Cross-Functional Collaboration: Collaborate closely with Data Scientists, Data Analysts, and business stakeholders to translate complex business requirements into efficient and impactful data solutions.

• Technology Innovation: Continuously research and implement new technologies and best practices to improve the efficiency, scalability, and resilience of our data platform.

• Cloud Deployment & Monitoring: Take ownership of the deployment and monitoring of data pipelines and related infrastructure on cloud platforms such as OpenShift, ECS, or Kubernetes, ensuring optimal performance and reliability.

• Operational Excellence: Ability to occasionally work a non-standard shift, including nights and/or weekends, and/or have on-call responsibilities to support critical data operations.

Required Skills and Qualifications:

• Education: Bachelor's degree in Computer Science, Software Engineering, or a related field.

• Experience: 6+ years of hands-on experience in a Data Engineering role

• Proven experience with database design, development, and administration.

• Data Visualization: Experience with data visualization tools and techniques for presenting data insights effectively.

• Agile Methodologies: Familiarity with agile development methodologies and experience working in agile teams.

• Strong proficiency in Python (version 3.6+), with experience in Python packaging and shared libraries like Pandas and NumPy.

• API Development: Experience implementing REST APIs in Python using microframeworks like Flask.

• Extensive experience working with relational databases, writing complex SQL, and optimizing queries for performance.

• Proven expertise with SQLAlchemy or similar ORM libraries for efficient database interaction.

• Experience with workflow management tools like Airflow (experience with PySpark or PyFlink is a major plus).

• Data Warehousing Knowledge: Solid understanding of data warehousing concepts and experience working with large datasets, including data modeling and ETL processes.

• Leadership & Mentorship: Ability to guide and mentor junior developers, fostering a collaborative team environment and promoting professional growth.

• Communication Skills: Strong communication skills, both written and verbal, with the ability to explain complex technical concepts to both technical and non-technical audiences.

• DevOps Practices: Experience working in a Continuous Integration and Continuous Delivery environment and familiarity with tools like Jenkins, TeamCity, SonarQube, OpenShift, ECS, or Kubernetes.

• Software Engineering Principles: Proficient in industry-standard best practices such as Design Patterns, Coding Standards, Coding modularity, and Prototyping.

• Design, develop, and maintain database schemas and models.

• Write and optimize SQL queries for data retrieval, manipulation, and reporting.

• Communicate technical concepts and solutions effectively to both technical and non-technical audiences.

• Provide technical support and troubleshooting for production systems.

• Stay up-to-date with the latest trends and technologies in Python development, database systems, and data engineering.

• Evaluate and recommend new tools and technologies to improve development efficiency and product quality.

• Contribute to the continuous improvement of development processes and practices.

------------------------------------------------------

Job Family Group:

------------------------------------------------------

Job Family:

------------------------------------------------------

Time Type:

------------------------------------------------------

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi’s EEO Policy Statement and the Know Your Rights poster.