Azure DataLake Support

Company: Apolis
Location: Schaumburg
Posted on: January 25, 2023

Job Description:

Title: Azure DataLake Support
Location: Schaumburg, Illinois/Hybrid - Can be initially remote

JD:
--- Should have hand-on experience on Azure Data Factory, Azure Data Lake, Azure Automation Account, Storage Accounts, HDInsights, Azure Data bricks, Batch Accounts and Azure SQL
--- Hands On Experience in writing Spark and Scala Programs.
--- Experience in creating Azure Data factory pipelines using ARM template and PowerShell.
--- Experience in optimizing and tuning Hive and Spark queries.
--- Experience in data analysis and also finding root cause for Production issues.
--- To maintain the number of nodes in HDInsight, Batch Account and Data bricks clusters according to the requirements and request platform team to increase/decrease them accordingly.
--- To maintain DTU percentage of Azure SQL according to the requirements and request platform team to increase/decrease them accordingly.
--- To maintain codes and release branches in cloud (VSTS) and ensure version is maintained according to the continuous integration.
--- To implement CR/new change/new source through VSTS using CI/CD pipelines
--- To prepare and maintain deployment documents.
--- To create VSTS tasks for every code change and deployments.
--- To monitor the daily/weekly/monthly/yearly and on-demand provisioning and curation/consumption loads running on Azure Platform.
--- Ensuring that the jobs are starting on time and finishing before SLA hours.
--- Validate whether all the monthly jobs are completed as per the correct dependency.
--- To check and validate data integrity in Azure Data Lake with respect to different on-premise sources.
--- Creating incidents(Service Now tickets) for the failures encountered.
--- Migrating the ETL related changes based on the requests for different environments.
--- Analyzing the incident tickets and sometimes the resolution would require the code change or data fix.
--- Responsible for providing the resolution / fixing the issue within the SLA time

Keywords: Apolis, Schaumburg , Azure DataLake Support, Other , Schaumburg, Illinois