Senior/Lead Data Engineer/Senior Data Manager-Canberra, Brisbane, Melbourne, Adelaide,Sydney.Hobart
Canberra, Australian Capital Territory, AustraliaFrom $12/yr
One of our leading federal govt clients is seeking to engage Senior /Lead Data Engineer for Canberra, Brisbane, Melbourne, Adelaide, Sydney locations Please note Candidates must have a NV1 security clearance Location of work- Canberra, Brisbane, Melbourne, Adelaide, Sydney.Hobart 1-Senior Data Manager - APS6 Level Preferred Technical Skills / Platforms * Collibra * Informatica * IBM Knowledge Catalogue * SQL * Python * Similar data governance tools or platforms Required Skills and Knowledge * Lead data analysis activities to support the delivery of the Data Foundations for AI (D4AI) project. * Demonstrated understanding of data governance and data management concepts, including metadata management, data quality, data lineage and data stewardship. * Understanding of enterprise data management platforms and their role in supporting data discovery, governance and reuse. * Experience analysing business needs, gathering requirements and translating them into practical data governance or platform requirements. * Ability to assess data assets, business processes and operational systems to identify governance, quality or metadata improvement opportunities. * Strong research, analysis and problem-solving skills, including the ability to develop evidence-based recommendations. * Experience supporting project delivery activities, including planning, documentation, testing, implementation and continuous improvement. * Strong stakeholder engagement skills, including the ability to work with business and technical teams to clarify needs, manage expectations and support adoption. * Strong written and verbal communication skills, including the ability to prepare clear reports, briefings and documentation for senior stakeholders. * Ability to manage competing priorities, work independently and contribute effectively in a complex project environment. * Proficiency in Microsoft Excel to support analysis, reporting, tracking and documentation. 2-Senior Data Engineer * Negative Vetting Level 1 * Canberra, Brisbane, Melbourne, Adelaide * 12 Months + 24 months Job details Technical Skills include: * Data modelling techniques (relational, dimensional, messaging and big data), data analysis and profiling using modern analytical tools and programming languages, and visualisation methods and tools. * Experience with one or more of the following - Teradata, Cloudera Data Platform. * Experience in implementing solutions on Red Hat Open shift platforms is highly desirable. * Best-practice data cleansing, validation, quality assurance and feature preparation techniques for complex datasets. * Programming \& Software Engineering – one or more of following: * Modern engineering practices supporting data products (API integration, Data Replication, Data Acquisition, Data Reconciliation, ETL/ELT using Batch and near Real Time) * Programming in one or more of the following - Python, SQL, Apache Spark, Java, Apache NiFi. * Experience in utilising Source control (Git) repositories, utilising and building CI/CD pipelines * Automated testing, Secure coding practices, Code review processes Essential criteria * 1.Demonstrated experience engaging stakeholders to define business requirements and design, develop, implement, maintain and support data engineering, enterprise data warehouse, data lake, streaming and software solutions that meet complex agency needs. * 2.Demonstrated ability to perform complex data extraction, manipulation, analysis, querying, programming, testing and troubleshooting, while applying data quality, integrity, security, risk controls and endorsed engineering standards to deliver reliable business outcomes. * 3.Demonstrated technical capability across data modelling, analysis, profiling, cleansing, validation, API integration, data replication, data acquisition, data reconciliation, ETL/ELT, batch and near real-time pipeline delivery. Desirable criteria * 1.Experience with one or more of the following platforms: Teradata, Cloudera Data Platform, Red Hat OpenShift * 2.Experience in one or more of the following: Python, SQL, Apache Spark, Java, Apache NiFi, Hadoop * 3.Experience building and supporting data products using automated testing, secure coding practices, Git repositories, DevOps pipelines, code review processes and contemporary software engineering practices. * 4.Relevant qualification or equivalent practical experience in computer science, software engineering, data engineering, enterprise data warehousing or software development. 3-Lead Data Engineer * Initial contract duration- 12 Months * Extension term-24 Months * Canberra, Brisbane, Melbourne, Adelaide.Sydney Requirements: Overview: Data Engineers prepare raw data for analytical or operational uses. They make data accessible so that it can be used for optimisation, delivery of services and enhanced analytics solutions. The Data Engineer operates under limited direction to support and maintain enterprise data assets across environments. The position will work closely with business and technical teams to deliver continuous improvement of data pipelines and processes. Key duties and responsibilities: • Develop, maintain and optimise enterprise data pipelines, Extract, Transform and Load (ETL) processes and analytical code. • Define and maintain data modelling, peer review, development and release standards. • Lead technical implementation activities from requirements analysis through to deployment and warranty support. • Develop and maintain Structured Query Language (SQL) analytical and ETL code supporting enterprise reporting and analytics. • Optimise and tune SQL code to improve performance and efficiency. • Produce and maintain technical documentation, data dictionaries and implementation artefacts. • Conduct peer reviews of code, data models, and technical designs. • Analyse and translate business requirements into technical solutions and data assets. • Collaborate with business stakeholders, developers, testers and support teams to deliver solutions. • Facilitate knowledge transfer and contribute to team capability uplift. • Support backlog prioritisation, work queue and management, and delivery planning activities. Knowledge/Qualifications: • Cyber security knowledge and experience. • Protective Security Policy Framework (PSPF). • Information Security Manual (ISM). • Essential 8 Strategies. • National Institute of Standards and Technology (NIST). • Business intelligence BINT - Level 4. • Data engineering DENG - Level 6. • Data management DATM - Level 6. • Data modelling and design DTAN - Level 5. • Data visualisation VISL - Level 5. • Database design DBDS - Level 5. • Information management IRMG - Level 6. • Measurement MEAS - Level 5. • Quality assurance QUAS - Level 5. • Storage management STMG - Level 5. Technical skills: • Structured Query Language (SQL) • Cloud Platforms: 1. Amazon Web Services 2. Google Cloud Platform 3. Microsoft Azure If you would like to apply for the above role, then please send y our updated resume with Cover Let ter to rahul.sharma@italliance.co m.au