Lead II - Data Engineering - T

Role Description

Job Description: Senior Data Engineer

SummaryWe are seeking a skilled Azure Data Engineer to join a Data & Analytics team. This role focuses on designing, developing, implementing, and maintaining scalable and reliable data solutions on the Microsoft Azure platform. The Azure Data Engineer will build and optimize data pipelines, ensure data quality, and collaborate with analysts, data scientists, and other stakeholders to support data-driven initiatives.

Key Responsibilities Design, develop, test, and deploy ETL/ELT data pipelines using Azure services such as Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.

Implement data transformations, data cleaning, aggregation, and enrichment logic according to business requirements.

Build and maintain data structures and models within Azure Data Lake Storage and Azure Synapse Analytics (or other data warehousing solutions).

Monitor data pipeline execution, troubleshoot failures, and optimize performance and cost-effectiveness.

Implement data quality checks and processes to ensure data accuracy and reliability.

Collaborate with data analysts, data scientists, and business users to understand data requirements and translate them into technical solutions.

Work closely with senior engineers and architects, contributing to solution design and adhering to established best practices.

Create and maintain clear technical documentation for data pipelines and processes.

Experience1. Minimum 6+ years of experience in data engineering, business intelligence, or related roles involving data integration and transformation.

• Minimum 4+ years of hands-on experience developing and implementing data solutions specifically on the Microsoft Azure platform.
  
  

Education1. Bachelor's degree in Computer Science, Engineering, Information Technology, or a related quantitative field (or equivalent practical experience).

Technical Skills Core Azure Data Services: Solid experience with Azure Data Factory (ADF), Azure Synapse Analytics (SQL/Spark Pools), Azure Databricks, and Azure Data Lake Storage (ADLS Gen2).

Data Processing & Programming: Strong proficiency with SQL is essential. Good proficiency with Spark (using PySpark or Scala) and Python (preferred) or Scala for data manipulation.

Data Concepts: Understanding of data warehousing concepts, dimensional modeling, ETL/ELT patterns, and data lake principles.

Databases: Experience with relational databases (e.g., Azure SQL Database). Familiarity with NoSQL databases is a plus.

Version Control: Experience using Git for code management.

Soft Skills Strong Analytical Thinking And Problem-solving Abilities.

Good communication and interpersonal skills.

Ability to work effectively both independently and collaboratively within a team environment.

A proactive approach and eagerness to learn new cloud technologies and data engineering techniques.

Skills

data engineering,azure data factory,dimensional modeling,data warehousing,azure databricks,azure synapse analytics,azure data lake storage,azure sql database,data lake,pyspark,spark,python,etl,sql