Lead I - Software Testing (Bigdata QA)

Role Description

Job Description We are looking for a skilled Big Data QA Engineer to

join our team and ensure the quality and reliability of our data processing

pipelines. This role involves designing and executing both automated and manual

tests for batch workflows, validating data integrity across transformation

stages, and maintaining a robust Python-based test automation framework. The

ideal candidate will have hands-on experience with AWS services, big data

formats, and a strong understanding of testing methodologies in a data-centric

environment. Preferred Skills and Experience: Design, develop, and execute

automated and manual batch test cases for data processing pipelines. Validate

data integrity and transformation logic across multiple stages of batch

workflows. Monitor and troubleshoot batch job failures, performance issues, and

data mismatches. Maintain and enhance Python-based test automation suite and

framework. Evaluates system performance, reproduces issues, and works with

development teams to resolve identified problems. Collaborates effectively with

peers and management to improve application quality and testing processes.

Minimum 4 years of experience with a focus on batch testing and data

validation. Strong proficiency in Python for test automation and scripting.

Hands-on experience with AWS services such as S3, Lambda, EMR, and Athena.

Experience with data formats like JSON, Parquet, and CSV, and tools like Pandas

and PySpark. Familiarity with version control systems (e.g., Git) and CI/CD

tools (e.g., Jenkins, GitHub Actions). Solid understanding of software testing

methodologies, including regression, integration, and system testing. Ability

to analyze logs and metrics to identify root causes of failures. Experience

working with agile methodology. Excellent verbal and written communication

skills, and ability to collaborate with cross-functional teams.

Minimum 4 years of experience with a focus on batch testing and data

validation. Strong proficiency in Python for test automation and scripting.

Hands-on experience with AWS services such as S3, Lambda, EMR, and Athena.

Experience with data formats like JSON, Parquet, and CSV, and tools like Pandas

and PySpark. Familiarity with version control systems (e.g., Git) and CI/CD

tools (e.g., Jenkins, GitHub Actions). Solid understanding of software testing

methodologies, including regression, integration, and system testing. Ability

to analyze logs and metrics to identify root causes of failures. Experience

working with agile methodology. Excellent verbal and written communication

skills, and ability to collaborate with cross-functional teams. Bachelor s

degree in Computer Science or equivalent preferred.

Skills

big data testing,big data,qa,automated testing,