ITI Data Hiring for Hadoop Platform Administrator Job at Tampa, FL Full Time

Location: Multiple Locations in the U.S. and India

Type: Full-time

Experience: 3 – 5 years on Hadoop administration, Overall experience of 4-8 years.

Functions: Consulting, Finance, Information Technology, Big Data Engineering

Industries: Capital Markets, Investment Banking, Alternative Investments, Financial Services, Management Consulting, Information Technology and Services, Healthcare

Job Description

We are looking for a Hadoop Administrator to administer and manage Hadoop clusters and other Hadoop ecosystem resources for our Fortune 500 client. This fast-paced, high visibility initiative will integrate data across internal and external sources, provide analytical insights, and integrate with the customer’s critical systems. The Hadoop administrator may also perform DBA-like tasks with HBase, Hive and other NoSQL databases, as well as security administration and cluster administration.

Key Responsibilities

• Deploy and maintain a Hadoop cluster (preferably Hortonworks or CDP distributions), adding and removing nodes using cluster monitoring tools like Ambari and/or Cloudera Manager.
• Configure the NameNodes with high availability and monitor overall health of all running Hadoop jobs.
• Perform online upgrade of Hadoop clusters from CDH or HDP to CDP (Cloudera Data Platform) distributions.
• Manage and maintain the Linux (root-level) configurations, packages, and disk devices (filesystem) on individual nodes of the cluster.
• Perform Active Directory integration on Hadoop Cluster with secure SSO or LDAP services for authentication.
• Perform LDAP integration with Sentry or Ranger for authorization, and configure table, column, and row level ACLs on Hive/Impala/HBase databases.
• Create and administer YARN resource pools with different scheduling policies, resource limits, pre-emption, and pool/queue ACLs, to enable application multi-tenancy on the cluster.
• Setup and Perform inter-cluster data migrations using DISTCP utility, along with scalability and optimization options.
• Deploy and administer Kafka Cluster, Topics, Kafka Connect, ksqlDB, and integrations to schema registry.
• Perform service containerization on Docker and/or Kubernetes.
• Integrate external services/application into Hadoop Cluster, for masking, tokenization & encryption.
• Install and configure Apache Atlas and/or Cloudera Navigator for scanning Hadoop databases, Spark/Sqoop/Storm codebases, to render technical data lineage.
• Configure Spark executor resources, job recoverability and enable Hive-on-Spark.
• Create, manage SSL certificates, and enable HTTPs access to different web applications and REST API endpoints within the components/services of Hadoop ecosystem.
• Build Kerberos server from scratch, principals, and key tabs. Configure Hadoop Cluster services to use Kerberos tickets for user/job interaction authentication.
• Perform basic activities on AWS Cloud – setting up EC2 nodes, EMR clusters, Lambda functions, S3 buckets, with predefined ACLs on IAM.
• Build custom monitoring and alert automation services on top of Hadoop clusters, via Ambari or Cloudera Manager REST APIs.
• Responsible for capacity planning and estimating the requirements for lowering or increasing the capacity of the Hadoop cluster.
• Work closely with the security team, network team, and application teams to make sure that all the big data applications are highly available and performing as expected.
• Perform Backup and recovery tasks as required by the customer.

Requirements

• BE/B.Tech/B.Sc. in Computer Science/ Statistics, Econometrics from an accredited college or university.
• Minimum 3 years of experience as Hadoop Administration, in a recent role.
• In-depth knowledge of Hadoop ecosystem, preferably Hortonworks or CDP (HDFS, Yarn, Sqoop, Flume, Kafka, Spark, Hive, Zookeeper, Oozie, Hbase/Impala etc).
• In-depth working knowledge of security administration, Kerberisation, ACLs within various Hadoop services.
• Excellent knowledge of UNIX/LINUX OS administration.
• Additional knowledge (preferable but not required) of high degree configuration management and automation tools like Ansible, Puppet or Chef for DevOps and platform automations.
• Working knowledge of cluster management tools like Apache Ambari, Cloudera Manager.
• Basic knowledge of AWS Cloud components and services.
• Knowledge of core Java is preferable but not required.
• Understanding of OS concepts, process management and resource scheduling.
• Basic knowledge of networking, CPU, memory, and storage.
• Ability to build Shell & Python scripts.
• Ability to quickly adapt and learn.
• Ability to jump into an ambiguous situation and take the lead.
• Ability to communicate and coordinate across various teams.
• Comfortable challenging your peers and leadership team.
• Ability to prove oneself quickly and decisively.
• Excellent communication skills.
• Good Customer Centricity.
• Solution and deadline oriented.

Job Type: Full-time

Benefits:

• Dental insurance
• Health insurance
• Paid time off
• Vision insurance

Schedule:

• Monday to Friday

Education:

• Bachelor’s (Preferred)

Experience:

• Hadoop: 1 year (Preferred)
• Java: 1 year (Preferred)

Work Location:

• Multiple locations

Work Remotely:

• Temporarily due to COVID-19