Bangalore (INSUBRAM) Subramanya Arcade, Grd-3rd Flrs No 12 Subramanya Arcade, Grd-3rd Flrs No 12
India
3-6 months
Time and material
$ 18-20/Hr
Description
RTH Y. FSS No of positions: 1 Locations: Bengaluru/Hyderabad /Kolkata/Pune Total Years of Experience: 8+ Years. Relevant years of Experience: 5 Years. Mandatory Skills: AWS Glue, Glue Crawler, PySpark, EMR, Lambda, EC2, Lake Formation, S3, and S3 Tables. Detailed Job Descriptio: We are seeking a highly experienced AWS Data Engineer to join our offshore team in India. The ideal candidate will have 7–8 years of deep expertise in AWS cloud data engineering, with a proven track record of architecting, developing, and maintaining scalable, secure, and efficient data solutions. This role requires advanced hands-on experience with AWS Glue, Glue Crawler, PySpark, EMR, data orchestration, containers, Lambda, EC2, Lake Formation, S3 and S3 Tables, Python, SQL, APIs, and CI/CD on AWS. Key Responsibilities: Architect, develop, and optimize complex ETL pipelines using AWS Glue, PySpark, and EMR. Automate metadata management and schema discovery using AWS Glue Crawlers. Orchestrate and monitor data workflows with AWS Step Functions, MWAA (Apache Airflow), or AWS Glue Workflows. Integrate and manage APIs for seamless data ingestion and transformation. Manage large-scale data processing using AWS EMR (Spark, Hadoop, etc.). Implement and govern data lakes using AWS Lake Formation. Design and maintain scalable, secure S3 buckets and S3 tables for structured and unstructured data. Build and deploy containerized solutions using Docker, ECS, or EKS. Develop serverless data solutions with AWS Lambda. Provision, automate, and maintain compute resources on EC2. Write robust, efficient Python code and advanced SQL for data manipulation and analytics. Establish and manage CI/CD pipelines leveraging AWS CodePipeline, CodeBuild, CodeDeploy, and CodeCommit. Ensure data security, quality, and compliance aligned with AWS best practices. Collaborate with global stakeholders and cross-functional teams to deliver robust data solutions. - IBMFG2JP00015368