RTH - Y Total Yrs of Exp : 5-6 Yrs. Detailed JD: 1. Databricks Platform Administration • Deploy, configure, and maintain Azure Databricks workspaces • Manage clusters, pools, and jobs for optimal performance. • Handle Databricks Runtime upgrades and compatibility testing. • Configure Databricks notebooks, Repos, and Delta Lake for efficient workflows. 2. Security & Access Control: • Implement Identity and Access Management (IAM) using Azure AD / AWS IAM / SCIM. • Set up and enforce Role-Based Access Control (RBAC) and network security. • Ensure compliance with data governance policies (Unity Catalog, Purview, etc.). • Manage personal access tokens (PATs) and OAuth-based authentication. 3. Performance Monitoring & Optimization: • Monitor cluster performance, auto-scaling, and cost optimization. • Analyze and fine-tune Spark jobs to improve execution efficiency. • Use Databricks SQL for querying and performance diagnostics. 4. Integration & Automation: • Integrate Databricks with AzureData Factory, Sql server and other data sources. • Automate cluster management using Terraform. • Implement CI/CD pipelines for Databricks notebooks using GitHub, Azure DevOps. 5. Troubleshooting & Support: • Investigate and resolve job failures, cluster crashes, and query performance issues. • Provide L2/L3 support for Databricks infrastructure issues. • Collaborate with Databricks support for bug fixes and escalations. Shift timings : 11:00 am to 8:00 pm. Note - Resource needs to ready for F2FIntv at IBM location based on account request and Day 1 reporting from DOJ. - U2XC6V