Wawanesa Insurance

Receive alerts when this company posts new jobs.

Similar Jobs

Job Details

Hadoop Administrator

at Wawanesa Insurance

Posted: 7/3/2019
Job Status: Full Time
Job Reference #: 87369411262
Keywords:

Job Description

Job Summary:   The Hadoop Administrator is responsible for supporting, configuring, upgrading, and maintaining multiple Hadoop clusters, BI platform administration (Microstrategy, Tableau) and data integration tool administration (Streamsets/Informatica). Contribute to building out additional clusters, adding nodes/capacity to existing clusters, and implementing open source projects.

Principal Duties:

  1. Responsible for installation, configuration and administration of Cloudera Hadoop platform. Help maintain the Hadoop Infrastructure and Operations. Work with Infrastructure teams to setup new Hadoop clusters.
  2. Support Big Data system environments such as Cloudera CDH. 
  3. Set up Kerberos principals and perform testing. Responsible for administering Key trustee server.
  4. Install, configure, and upgrade of Tableau, Microstrategy intelligent and mobile server.  Integrate support with data sources for Microstrategy.  Ensure backups and restoration of Tableau repositories.
  5. Responsible for enabling the BI tool dashboards through Mobile devices.
  6. Manage integration with Active Directory and provide support for user and group management.
  7. Work with data engineer to setup data access security groups.  Work with delivery teams for Provisioning of users into Hadoop.
  8. Automate monitoring and server job processes.
  9. Manage the environment that supports Tableau Server and perform maintenance, upgrades, patches, and requests for new virtual machines to expand the environment.
  10. Respond to support requests that occur same-day. Resolution will depend upon the level of complexity within a maximum of 72 hours.
  11. Enable data encryption at rest and at motion with TLS/SSL to meet the security standards.
  12. Responsible for system backups, and coordinate with infrastructure team for storage and rotation of backups is accomplished.
  13. Enable Sentry for RBAC (role-based access control) to have a privilege level access to the data in HDFS as per the security policies.
  14. Perform cluster maintenance as well as creation and removal of nodes using tools like Cloudera Manager Enterprise.  Monitor Hadoop cluster connectivity and security.  Conduct performance tuning of Hadoop clusters and Hadoop MapReduce routines.
  15. Screen Hadoop cluster job performances and capacity planning.  Manage, troubleshoot and review Hadoop log files.
  16. Perform file system management and monitoring as well as HDFS and other Cloudera Hadoop components support and maintenance.
  17. Design and implement Backup and Disaster Recovery strategy based out of Cloudera BDR utility.
  18. Mentor others as necessary.
  19. Work with hosted partner IBM to configure server changes for storage, performance.

Essential Functions

  1. Work individually, with minimal supervision, as well as in team environments
  2. Detail oriented, well organized work skills
  3. Manage time and stress appropriately
  4. Prioritize work, considering the bigger picture, ensuring alignment with department and corporate objectives
  5. Analyze and solve development, maintenance, and support challenges
  6. Handle multiple projects simultaneously and independently
  7. Proven self-starter with demonstrated ability to make decisions
  8. On-call 24/7 required (rotating)
  9. Ability to work at a fast pace to deliver optimal solutions that will help improve business results
  10. Ability to exercise initiative on problem solving
  11. Strong ability to identify, prioritize and solve system related problems




 

Skill / Requirements

 Basic Qualifications:

  1. Bachelor’s degree in Computer Science, Engineering or related field preferred with 10 Years of IT experience
  2. 5+ years recent Cloudera Hadoop Administration experience and 2 years of BI tool administration with Tableau, Microstrategy administration or any BI tool administration
  3. Strong knowledge of Hadoop Architecture(HDFS), Hadoop Cluster installation, configuration, monitoring, cluster security, cluster resources management, maintenance and performance tuning
  4. Expert level knowledge of Cloudera Hadoop components such as HDFS, Sentry, Kafka, Impala, Hue, Spark, Hive, YARN, ZooKeeper and Postgres
  5. Strong knowledge of key scripting and programming languages such as Python, Scala, and Bash..
  6. Strong understanding and experience with Hadoop Stack including HDFS, Hive, HBase, Yarn, Flume, Scoop, Oozie, Impala, Spark.  
  7. Knowledge of relational databases, industry practices, techniques, standards and sensitive, PII data handling.
  8. Proficient with system upgrades, patches, maintaining compliance, and support audits.
  9. Experience with public Cloud environments including AWS and Azure. Experience in build and deployment automation.
  10. Intimate knowledge of fully integrated AD/Kerberos authentication