Engineering
NTT DATA, Inc. Hiring for Lead Data Engineer at United States, Pennsylvania, Pittsburgh Full Time
Job Description
NTT DATA’s Client is seeking a Lead Data Engineer with 10 to 12 years working experience in data integration and pipeline development with data warehousing .
* Experience with AWS Cloud on data integration with Apache Spark, EMR, Glue, Kafka, Kinesis, and Lambda in S3, Redshift, RDS, MongoDB/DynamoDB ecosystems
* Strong real-life experience in python development especially in pySpark in AWS Cloud environment.
* Design, develop test, deploy, maintain and improve data integration pipeline.
* Experience in Python and common python libraries.
* Strong experience with Perl and Unix Scripts
Power BI experience preferred
* Strong analytical experience with database in writing complex queries, query optimization, debugging, user defined functions, views, indexes etc.
* Strong experience with source control systems such as Git, Bitbucket, and Jenkins build and continuous integration tools.
* Experience with continuous deployment(CI/CD)
* Databricks, Airflow and Apache Spark Experience is a plus.
* Experience with databases (Oracle, SQL Server, PostgreSQL, Redshift, MySQL, or similar)
* Strong experience with performance tuning, analytical understanding with business and program.
* Exposure to ETL tools including Informatica and any other .
The Company is an equal opportunity employer and makes employment decisions on the basis of merit and business needs. The Company will consider all qualified applicants for employment without regard to race, color, religious creed, citizenship, national origin, ancestry, age, sex, sexual orientation, genetic information, physical or mental disability, veteran or marital status, or any other class protected by law. To comply with applicable laws ensuring equal employment opportunities to qualified individuals with a disability, the Company will make reasonable accommodations for the known physical or mental limitations of an otherwise qualified individual with a disability who is an applicant or an employee unless undue hardship to the Company would result.