Big Data Engineer
Id: AT0092
2023-05-25
● Partner with internal stakeholders to understand business requirements, work with cross-functional data and product teams and build efficient and scalable data solutions.
● Work in various phases of the software development life cycle right from Requirements gathering, Analysis, Design, Development, and Testing to Production.
● Develop sustainable distributed data management solutions and workflows required for Data migration to cloud to enable analytics and derive business insights.
● Build and maintain infrastructure pipelines for data extraction, transformation and loading of data from a variety of sources. Analyze data both in real-time and batch processing utilizing Spark, Kafka, Hadoop, and AWS technologies such as Kinesis, S3, ElastiCache, and Lambda.
● Ensure overall data and code quality via Unit and Integration Tests. Troubleshoot and resolve issues in test and production environments. Employ CI/CD practices using Jenkins, Bogie and version control system GIT.
● Identify and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
Qualification:
This position requires a minimum of a Bachelor’s Degree or its equivalent in Computer Science.