Azure DataLake Engineer - Schaumburg, IL

Company: Diverse Lynx
Location: Schaumburg
Posted on: November 24, 2022

Job Description:

Title: Azure DataLake Engineer
Location: Schaumburg, Illinois - onsite


JD:
" Should have hand-on experience on Azure Data Factory, Azure Data Lake, Azure Automation Account, Storage Accounts, HDInsights, Azure Data bricks, Batch Accounts and Azure SQL
" Hands On Experience in writing Spark and Scala Programs.
" Experience in creating Azure Data factory pipelines using ARM template and PowerShell.
" Experience in optimizing and tuning Hive and Spark queries.
" Experience in data analysis and also finding root cause for Production issues.
" To maintain the number of nodes in HDInsight, Batch Account and Data bricks clusters according to the requirements and request platform team to increase/decrease them accordingly.
" To maintain DTU percentage of Azure SQL according to the requirements and request platform team to increase/decrease them accordingly.
" To maintain codes and release branches in cloud (VSTS) and ensure version is maintained according to the continuous integration.
" To implement CR/new change/new source through VSTS using CI/CD pipelines
" To prepare and maintain deployment documents.
" To create VSTS tasks for every code change and deployments.
" To monitor the daily/weekly/monthly/yearly and on-demand provisioning and curation/consumption loads running on Azure Platform.
" Ensuring that the jobs are starting on time and finishing before SLA hours.
" Validate whether all the monthly jobs are completed as per the correct dependency.
" To check and validate data integrity in Azure Data Lake with respect to different on-premise sources.
" Creating incidents(Service Now tickets) for the failures encountered.
" Migrating the ETL related changes based on the requests for different environments.
" Analyzing the incident tickets and sometimes the resolution would require the code change or data fix.
" Responsible for providing the resolution / fixing the issue within the SLA time






Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

Keywords: Diverse Lynx, Schaumburg , Azure DataLake Engineer - Schaumburg, IL, Engineering , Schaumburg, Illinois