DescriptionWe have an exciting and rewarding opportunity for you to take your software engineering career to the next level.
As a Software Engineer III at JPMorgan Chase within the Corporate Technology, Legal Reporting team you serve as a seasoned member of an agile team to design and deliver trusted market-leading technology products in a secure, stable, and scalable way. We are looking for an experienced Data Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and optimizing data pipelines using AWS services like Glue, Redshift, and Lambda. The ideal candidate should have hands-on experience with ETL processes, performance tuning, and a strong understanding of cloud-based data platforms.
.
Job responsibilities
- Develop and Maintain ETL Pipelines: Design, develop, and implement scalable ETL workflows using PySpark, Python, and AWS Glue.
- Data Transformation and Integration: Extract, transform, and load data from various sources to AWS S3 and Redshift.
- Performance Optimization: Identify and resolve performance bottlenecks in ETL processes, ensuring optimal performance across large datasets.
- Automation and Monitoring: Implement automation scripts using AWS Lambda to schedule and monitor data pipelines.
- Data Quality: Ensure data integrity and quality across all stages of the ETL pipeline.
- Collaboration: Work closely with data architects, analysts, and stakeholders to understand requirements and provide clear communication throughout the project lifecycle.
- Documentation: Create and maintain technical documentation, including data mapping, workflow designs, and ETL processes.
- Proactively identifies hidden problems and patterns in data and uses these insights to drive improvements to coding hygiene and system architecture
- Contributes to software engineering communities of practice and events that explore new and emerging technologies
- Adds to team culture of diversity, equity, inclusion, and respect
Required qualifications, capabilities, and skills
-
Formal training or certification on software engineering concepts and 3+ years of applied experience
- Hands-on experience in ETL development using PySpark, Python, and AWS services (Glue, Lambda, S3, and Redshift).
- Experience in optimizing data pipelines and troubleshooting performance issues.
- Strong understanding of SQL and relational databases.
- Familiarity with data warehousing concepts and design patterns.
- Excellent problem-solving skills and attention to detail.
- Strong communication skills, with the ability to explain technical concepts to non-technical stakeholders.
Preferred qualifications, capabilities, and skills
- Experience with other AWS services like Athena, Step Functions, and CloudWatch.
- Knowledge of CI/CD pipelines and best practices in deployment automation.
- Experience working with large-scale distributed systems and big data environments.