Log In
Sign Up
Australia
Optus Hiring for DPM Big Data DevOps Engineer Job at Sydney, Australia Full Time
br{display:none;}.css-58vpdc ul > li{margin-left:0;}.css-58vpdc li{padding:0;}]]>
Position Summary
This data management position enables Data Platform Management of Group IT to achieve the following objectives:
- Deliver Big Data technology roadmap leveraging industry standard patterns with the aim of deriving analytics from massive volumes of data
- Design and build comprehensive Data Ingestion and analytics solutions for the Big Data Platform
- Design and build reusable frameworks and pilots for enriching batch and real-time ingestion & transformation patterns.
- Enhance inhouse knowledge base, frameworks and tools management practices that fit into the big data eco-system for business data consumption
- Work with business in identifying data access patterns and building solutions
Key Responsibilities
Responsible for delivering solutions in alignment Optus Big Data roadmap. Leverage industry standard patterns and Optus technology stacks.
Responsible for designing and building data load, transformation and summarisation solution that services analytics needs, downstream feeds, and reporting requirements
Responsible for Design and build of reusable frameworks and pilots for enriching batch and real-time ingestion & transformation patterns.
Work closely with EIM Solution Architects for ensuring alignment with high level solution designs, domain patterns and architecture standards
Expected to demonstrate knowledge of framework and tools that fit into the big data eco-system for business data consumption
Expected to demonstrate knowledge of batch and real time Ingestion tool such as Informatica BDM (Big Data Management), Informatica DIS
Responsible for working with business in identifying data access patterns and building solutions
Provide a mechanism and a platform for data SMEs to discover data patterns and build hypothesis
Experience and Qualifications
- Overall 6+ Years of experience in Data Warehousing, Data Management and Big Data
- 3+ years of designing and implementation experience in big data related solutions involving data ingestion (real time and batch code), stitching, data collection and processing including curation
- 1-3 years of designing and implementation experience with building analytical solutions based on big data platforms.
- 2+ years of hands-on experience in Big Data tools (not limited to)
- Spark
- Python
- Informatica BDM
- Scala
- StreamSets / Apache Nifi
- Cloudera and /or Hortonworks toolset
- Kafka
- Associated supporting technologies such as Unix Scripting
- Experience with Visualization tools and building of analytical models
- hands-on experience in Big Data tools (not limited to)
- Spark
- Python
- Informatica BDM
- Scala
- StreamSets / Apache Nifi
- Cloudera and /or Hortonworks toolset
- Kafka
- associated supporting technologies such as Unix Scripting
- Preferred if experienced in Cloudera Administration
Show more
