Sr Big Data Engineer

Company: DataDelivers, LLC.
Location: Schaumburg
Posted on: November 12, 2021

Job Description:

The Sr. Big Data Engineer's primary responsibilities are to create and develop solution designs to integrate and ingest data from disparate sources, supporting DataDelivers's big data ecosystem. They will work in close collaboration with key stakeholders to design and implement optimum solutions using best practices.This resource creates ETL, batch, and automated processes to best suite DataDelivers and its client's needs. This role is part of the core Technology Solutions team and heavily contributes to the team's coding and programming standards and ensure compliance to the guidelines and standards.Responsibilities:

• Define big data architectural blueprints, prototypes, solution architectures and jumpstart support to accelerate adoption of new solutions based on standards and best practices.
• Oversees the implementation of data solutions by working with global teams (onshore and offshore) to create ETL, batch, real-time, and automated processes.
• Define and own standards and ensure other team member compliance.
• Select and integrate Big Data tools and frameworks required to provide requested capabilities.
• Manage AWS landscape of EC2 clusters, Glue Jobs, Athena Tables, S3 data lakes and related services.
• Maintain data quality, integrity and availability requirements.
• Ensure the data ecosystem is highly scalable, responsive, and available.
• Acts as an SME for big data ecosystem. Ensures DataDelivers's data solutions are using the latest versions and code base.
• Monitor batch and reporting performance and implement necessary optimizations.
• Provide technical support to members of TS and other teams, as well as project support across client engagements.
• Manage Sprint Planning sessions and Team Ceremonies: Iteration Planning, Daily Scrum, and Iteration Review.
• Collaborate with DevOps team to ensure that the deliverable quality meets the approved business objectives and functionalities.
• Track, research and introduce technology innovations to drive modernization and efficiencies and improve functionality for authored applications.Required Experience:
  Qualifications:
  • (7 years of relevant experience) or (5 years of relevant experience and an advanced degree in Computer Science/IT or related field)
  • Deep technology expertise in Architecture, Development, Deployment and Evolution of large-scale data platform(s).
  • Demonstrated ability to have successfully completed multiple, complex technical projects using Onshore/Offshore model.
  • Solid expertise in Data Modeling, Star Schemas, NoSQL, ETL/ELT, Data Marts, Analytic Playgrounds, Reporting techniques.
  • Proven experience with NoSQL databases, such as DynamoDB, Redshift, Databricks, etc.
  • Deep experience with designing solutions using various ingestion / ETL techniques - Flume, Glue Jobs, Step Functions, AWS Lambda, REST API, SQS
  • Proven experience with Big Data querying tools such as Athena/Presto, Pig, Hive, etc.
  • Solid competency and hands-on experience with AWS Big Data Optimization, performance monitoring including CloudWatch, S3 Events.
  • Extensive experience with Python scripts & libraries and AI and Data Science toolkits - Sagemaker, Mahout, SparkML, H2O, etc.
  • Broad knowledge about BI and Data Visualization technologies such as Qlik, Quicksight, PowerBI.
  • Knowledge in AWS Security architecture is highly desired
  • Experience with UI / UX frameworks such as Javascript, Angular, React JS and technologies including HTML, CSS are highly desired.
  • Experience with client data management and client analytics platforms is strong plus.
  • Experience in enterprise CRM and Campaign Management platforms such as Salesforce, SendGrid desired
  • Prior experience driving Agile, Scrum methodologies required. Experience using Jira, Bitbucket are nice to have.
  • Excellent relationship building, organization, collaboration, leadership, negotiation, adaptability, and time management skills.
  • Engaging with people from many different disciplines and varying degrees of technical experience.Benefits:
    • We constantly strive to achieve a strong work/life balance; we are an employee-centric culture, with room for flexibility in work location and hours where possible
    • We provide excellent benefits, including 401K, medical, dental, vision, disability, and life insurance, and generous compensated time off policy
    • We offer a competitive compensation package with opportunities for performance rewards based on company success

Keywords: DataDelivers, LLC., Schaumburg , Sr Big Data Engineer, Engineering , Schaumburg, Illinois