Data Engineers Jobs

Data engineers

From $47/yr

About the role We are seeking an experienced Data Engineer with strong Google Cloud Platform (GCP) expertise to design, develop and maintain scalable, secure and reliable data solutions within a complex enterprise environment. Please note: A current Negative Vetting Level 1 (NV1) security clearance is mandatory for this position. Location of work : QLD, WA, ACT, VIC, NSW, NT, SA and TAS. Key responsibilities * Design, develop and maintain scalable batch, micro-batch and real-time data ingestion, transformation and curation pipelines. * Translate solution designs, technical specifications and data contracts into robust, production-ready data solutions. * Apply data warehousing, dimensional modelling, data quality and governance principles, including metadata, lineage, classification, retention and security. * Develop and optimise data pipelines using GCP services, including BigQuery, Dataflow, Apache Beam, Cloud Storage, Cloud Composer and Pub/Sub. * Implement DataOps practices, including CI/CD, infrastructure automation, automated testing, monitoring, alerting and release management. * Identify and address gaps between solution designs and technical implementation, ensuring requirements are fully realised. * Optimise data solutions for performance, scalability, reliability, cost efficiency and operational supportability. * Apply security best practices, including least-privilege access, encryption, secrets management and secure handling of sensitive information. * Build and maintain automated unit, integration, reconciliation and data quality tests for data pipelines and products. * Collaborate with technical teams and business stakeholders throughout the solution delivery lifecycle. * Identify technical risks, dependencies and design deficiencies, recommending sustainable improvements. * Contribute to technical analysis, estimation, solution planning, standardisation and continuous improvement initiatives. * Develop reusable data products and trusted datasets to support advanced analytics, machine learning and generative AI. Highly desirable criteria * Minimum three years of data engineering experience, including at least two years designing, building and operating production data solutions on Google Cloud Platform (GCP). * Demonstrated experience delivering and supporting production-grade data pipelines in complex enterprise environments. * Strong hands-on experience with BigQuery, including data modelling, partitioning, clustering, query optimisation, cost management and workload monitoring. * Experience developing scalable batch and streaming data pipelines using Dataflow and Apache Beam. * Demonstrated experience applying data warehousing, dimensional modelling and data quality practices. * Strong proficiency in SQL and Python for developing, testing and optimising data pipelines. * Experience implementing DataOps practices, including CI/CD, infrastructure as code, automated testing and monitoring. * Hands-on experience with Terraform, Git-based source control and CI/CD tooling such as Cloud Build or GitHub Actions. * Demonstrated understanding of cloud security, data governance, metadata management and data lifecycle principles. * Ability to translate business and technical requirements into scalable, secure and maintainable data solutions. Contact details For a confidential chat, please contact Shruthi on 0474 545 379 or via email at shruthi@exclaim-it.com.au . We would be more than happy to provide you with additional information and assist in the application process. Please note that a National Police Check might be required as part of pre-employment screening.

Data Scientist/Engineers x5 - DV Clearance Required

Role Overview Kaze Consulting is growing, driven by rising demand from clients who want to move beyond the promise of AI and apply it to solve complex, real-world problems that matter. We are looking for 5x technically strong Data Scientists/Engineers to join our Data \& AI team, working on complex data-driven problems within secure government environments. This role combines data science, software engineering and data engineering. You will work across the full data science lifecycle: understanding operational and business problems, defining data requirements, exploring and preparing large datasets, developing analytical and machine-learning solutions, evaluating performance, and taking successful proofs of concept through to production. The role suits someone with a strong technical foundation in Computer Science, Physics or a related quantitative discipline, who is comfortable working directly with large-scale data platforms. Key Responsibilities • Work with stakeholders to translate complex business and operational problems into clear analytical and data science requirements. • Contribute across the full data science lifecycle, from discovery, data requirements and collection through understanding, preparation, modelling, evaluation, deployment and feedback. • Develop proofs of concept and progress successful approaches into robust, production-ready solutions. • Work with large and complex datasets hosted within Hadoop and/or Stackable-based data platforms. • Build scalable processing and analytical workflows using Python, SQL, Spark and PySpark; query distributed datasets using Trino and Hive. • Develop and maintain Jupyter and Python data science environments, alongside scheduled analytical and processing workloads, including orchestration of Trino queries. • Perform data exploration, feature engineering, statistical analysis and machine-learning model development where appropriate. • Apply strong engineering practices, including modular development, testing, version control, reproducibility and maintainability. • Work with engineers, architects, analysts and operational stakeholders to integrate analytical capabilities into wider platforms. • Evaluate solutions against technical and operational requirements; use feedback to improve them iteratively. • Contribute to technical documentation, architecture decisions and knowledge sharing. • Work within Agile/Scrum teams, contributing to planning, refinement, reviews and retrospectives. • Deliver in accordance with the security, governance and assurance requirements of UK Government and Defence environments Must Haves: * Be based in the UK and willing to travel onsite 2-3 days per week * Hold a British passport * Currently hold a valid DV clearance If you’re motivated by challenging work, inspired by the potential of AI, and want to be part of a team shaping how it is applied in the real world, we’d like to hear from you at hello@kaze-consulting.com

Data Engineers (VP) | Global Investment Firm | Singapore

Data Engineers | Databricks, AWS \& Python | VP | Singapore Are you a hands-on data engineering leader who wants to shape enterprise-scale platforms with direct investment impact? I’m partnering with one of the world’s largest global investment organisations to hire multiple VP-level Data Engineers in Singapore. You’ll lead the design and delivery of modern data platforms supporting portfolio management, trading, risk and investment operations. These roles combine hands-on coding, technical leadership and senior stakeholder engagement. No Finance Experience Required Financial-markets experience is not essential. The team welcomes experienced data engineering leaders from technology companies, global banks, fintech firms, consultancies and other data-intensive organisations. What You’ll Be Doing * Lead the design and delivery of enterprise data and analytics platforms. * Build scalable solutions using Databricks, Python, PySpark, Spark SQL, SQL and dbt. * Design AWS-based lakehouse architecture covering modelling, governance, lineage and data quality. * Remain hands-on across coding, system design, deployment and production. * Set technical direction and raise engineering standards. * Work with architects, product teams and senior business stakeholders. * Mentor engineers and, for one position, manage permanent staff and external vendors. What We’re Looking For * Strong hands-on Databricks experience is essential. * Commercial AWS experience is mandatory. Azure-only candidates cannot be considered, even with Databricks experience. * Strong hands-on Python, PySpark, Spark SQL and SQL expertise. * Experience designing large-scale data pipelines, lakehouse platforms or analytics systems. * Strong knowledge of data architecture, modelling, governance, lineage and quality. * Exposure to dbt, Airflow, Pandas, Docker, Kubernetes, Terraform or CI/CD. * Proven technical leadership and the ability to influence senior stakeholders. * A hands-on engineering mindset—you must remain comfortable writing and reviewing code. * Team-management experience is required for one of the VP openings. * Candidates should currently be based in Singapore. Why Join? * Lead major Databricks and AWS initiatives at genuine enterprise scale. * Build platforms that directly influence global investment decisions. * Join a long-term data and technology transformation with senior-level ownership. * Gain exposure across asset classes, investment strategies and business functions. * Choose between hands-on technical-leadership and people-management paths. * Join a stable, flexible environment offering complex work and long-term career development. Please apply with your latest resume or get in touch for a confidential conversation. BAH Partners specialises in placing world-class technology and quantitative talent across Asia-Pacific’s leading financial institutions and technology-driven organisations.