Sr Big Data Engineer
Company: DataDelivers, LLC.
Posted on: November 12, 2021
The Sr. Big Data Engineer's primary responsibilities are to
create and develop solution designs to integrate and ingest data
from disparate sources, supporting DataDelivers's big data
ecosystem. They will work in close collaboration with key
stakeholders to design and implement optimum solutions using best
practices.This resource creates ETL, batch, and automated processes
to best suite DataDelivers and its client's needs. This role is
part of the core Technology Solutions team and heavily contributes
to the team's coding and programming standards and ensure
compliance to the guidelines and standards.Responsibilities:
- Define big data architectural blueprints, prototypes, solution
architectures and jumpstart support to accelerate adoption of new
solutions based on standards and best practices.
- Oversees the implementation of data solutions by working with
global teams (onshore and offshore) to create ETL, batch,
real-time, and automated processes.
- Define and own standards and ensure other team member
- Select and integrate Big Data tools and frameworks required to
provide requested capabilities.
- Manage AWS landscape of EC2 clusters, Glue Jobs, Athena Tables,
S3 data lakes and related services.
- Maintain data quality, integrity and availability
- Ensure the data ecosystem is highly scalable, responsive, and
- Acts as an SME for big data ecosystem. Ensures DataDelivers's
data solutions are using the latest versions and code base.
- Monitor batch and reporting performance and implement necessary
- Provide technical support to members of TS and other teams, as
well as project support across client engagements.
- Manage Sprint Planning sessions and Team Ceremonies: Iteration
Planning, Daily Scrum, and Iteration Review.
- Collaborate with DevOps team to ensure that the deliverable
quality meets the approved business objectives and
- Track, research and introduce technology innovations to drive
modernization and efficiencies and improve functionality for
authored applications.Required Experience:
- (7 years of relevant experience) or (5 years of relevant
experience and an advanced degree in Computer Science/IT or related
- Deep technology expertise in Architecture, Development,
Deployment and Evolution of large-scale data platform(s).
- Demonstrated ability to have successfully completed multiple,
complex technical projects using Onshore/Offshore model.
- Solid expertise in Data Modeling, Star Schemas, NoSQL, ETL/ELT,
Data Marts, Analytic Playgrounds, Reporting techniques.
- Proven experience with NoSQL databases, such as DynamoDB,
Redshift, Databricks, etc.
- Deep experience with designing solutions using various
ingestion / ETL techniques - Flume, Glue Jobs, Step Functions, AWS
Lambda, REST API, SQS
- Proven experience with Big Data querying tools such as
Athena/Presto, Pig, Hive, etc.
- Solid competency and hands-on experience with AWS Big Data
Optimization, performance monitoring including CloudWatch, S3
- Extensive experience with Python scripts & libraries and AI and
Data Science toolkits - Sagemaker, Mahout, SparkML, H2O, etc.
- Broad knowledge about BI and Data Visualization technologies
such as Qlik, Quicksight, PowerBI.
- Knowledge in AWS Security architecture is highly desired
React JS and technologies including HTML, CSS are highly
- Experience with client data management and client analytics
platforms is strong plus.
- Experience in enterprise CRM and Campaign Management platforms
such as Salesforce, SendGrid desired
- Prior experience driving Agile, Scrum methodologies required.
Experience using Jira, Bitbucket are nice to have.
- Excellent relationship building, organization, collaboration,
leadership, negotiation, adaptability, and time management
- Engaging with people from many different disciplines and
varying degrees of technical experience.Benefits:
- We constantly strive to achieve a strong work/life balance; we
are an employee-centric culture, with room for flexibility in work
location and hours where possible
- We provide excellent benefits, including 401K, medical, dental,
vision, disability, and life insurance, and generous compensated
time off policy
- We offer a competitive compensation package with opportunities
for performance rewards based on company success
Keywords: DataDelivers, LLC., Schaumburg , Sr Big Data Engineer, Engineering , Schaumburg, Illinois
Didn't find what you're looking for? Search again!