Data Engineer

AI & Data Analytics

|

Full-Time

The Data Engineer provides the technical expertise to scale our data infrastructure using the right tools and frameworks, enabling access to high quality data for Setel and our stakeholders. The responsibilities include performance tuning and monitoring of our data warehouse, developing best-in-class extract, transform and load (ETL) job architecture, and building batch or streaming data pipelines for analytics and ML use cases. This role will require someone with hands on exposure developing and implementing data engineering solutions in a cloud environment, experienced with data warehouse technologies and data processing frameworks as well as proficient in Spark/Python/Java and SQL.

In This Role You Will:

Collaborate with data team to define key data quality and timeliness SLAs for all datasets
Integrate all key data sources in the Redshift data warehouse for a single source of truth and own the performance tuning and monitoring of data warehouse
Develop and implement best-in-class extract, transform and load (ETL) architecture and processes to enhance productivity of data consumers
Implement good engineering practices of version control (Git) and quality testing into our data development processes in collaboration with data team members
Implement and maintain data orchestration tools e.g. Airflow, DBT which help efficiently automate, schedule and manage dependencies in our daily data jobs
Design, build and maintain batch and/or streaming data pipelines in the cloud for analytics, ML or external data transfer use cases
Collaboratively work with data analysts and scientists to develop tooling and platforms which helps them work more effectively
Work closely where necessary with the broader engineering team on company-wide projects like data lake / central data platform to centralize our customer data
Mentor and guide fellow data team members on good data engineering and coding best practices
Implement reasonable data governance e.g. prevent exposure of personal identifiable information data to users without proper access rights

You’re a great fit if you have:

Minimum B.S. Degree in Computer Science or a related quantitative/technical field
Experience working in a consumer product company with large amounts of data
3+ years of data engineering experience

Technical & Functional Skills:

Strong working experience with one or more general purpose programming languages such as Java, Scala, Python
Ability to write optimised and efficient SQL queries
Familiar with data warehouses and data lakes such as Redshift, Athena, and BigQuery
Familiar with distributed data processing frameworks such as Hadoop, Spark and Flink
Familiar with data warehouse modelling concepts e.g. dimensional data modelling and schema design
Familiar with data orchestration frameworks e.g. Airflow to schedule and manage ETL jobs
Experience with cloud computing technologies and environments e.g. AWS, GCP, Azure
Experience of developing and managing large scale TB-level data processing pipelines is preferred
Experience with managing application and infrastructure dependencies using containers e.g. Docker is a plus
Experience with event sourcing and stream processing frameworks e.g. Kafka is a plus
Experience developing backend APIs or deploying Machine Learning/Data Mining applications is a plus

Soft Skills:

Ability to work independently and collaboratively with others in the data team and different functional teams e.g. product and engineering
Positive team player with strong collaboration skills
Able to take ownership and drive initiatives
High energy, self-motivated and able to thrive in a fast-paced environment
Humble and customer-centric

Bonus Qualifications

–

AI & Data Analytics

|

Full-Time

Data Engineer

#WeSetelForYou

Locations

About

Customers