Job Description Senior Data Engineer Overview As a Senior Data Engineer, you will play a critical role in designing, developing, and maintaining data products and interactive dashboards that address the complex needs of our clients.
Using your expertise in R and R Shiny, you will create tools and visualizations that are accessible, user-friendly, and fully compliant with government and industry standards.
Key Responsibilities - Lead the development of.
*RShiny web applications * to deliver innovative data solutions.
- Design and implement.
*interactive dashboards* and integrate dynamic visualizations into R Markdown documents.
- Translate complex datasets into actionable insights for non-technical stakeholders.
- Collaborate with cross-functional teams to gather requirements, prioritize features, and deliver high-impact products.
- Ensure all products adhere to compliance standards, including.*GDPR*,.
*WCAG 2.1 AA accessibility*, and.
*GDS Service Standards*.
- Maintain alignment with.
*Technical Standards* and the.
*Technology Code of Practice*.
- Contribute to an Agile development environment, participating in sprints, stand-ups, and retrospectives.
Qualifications and Skills Essential: - A minimum of.
*5 years of experience as a Data Engineer*.
- At least.
*3 years of hands-on experience coding in R*, with a strong focus on.
*R Shiny*.
- Demonstrated ability to design and develop.
*interactive dashboards* and embed.
*charts and visualizations* in R Markdown.
- Proven experience sharing data insights with.
*non-technical stakeholders*.
- Strong expertise in.
*data collection and visualization*.
- Familiarity with Agile development principles and practices.
Desirable:* - Previous experience in the education or public sector.
- Knowledge of government technical frameworks, including the.
*Government Design System*.
- Experience working with large-scale data platforms.
Requirements What we are looking for: The interview will consist of both behavioural questions which follow the STAR methodology based on your past experience (see interview tips below) as well as detailed technical interview across our main technology stack described below.
Must have: - Proven hands on track record experience managing large cloud based infrastructure deployments via IaC on Azure via terraform.
- Proven hands on track record experience managing VM deployments on Azure and via Ansible.
- Proven hands on track record experience implementing CI / CD pipelines to build, package, deploy, test and promote to higher environments infrastructure as well as applications, preferably via ADO.
- Detailed understanding of infrastructure main building blocks and design, how they all fit together and how to manage them via code.
That includes Virtual Networks, Routing, DNS, NSGs, Identity Management, Firewall Setup, Security Policies, Secrets and Certificates management.
- Experience with configuring SecOps tools, preferably SonarQube, Checkmarx, PMD and using pre commit hooks across git based repositories to support shift left approach on the organisation deliverables.
- Ability to resolve complex problems and work collaboratively with multiple technical and non technical teams with different skill sets and backgrounds.
- Last but certainly not least: Excellent verbal and written communication skills.
Nice to have: - Experience with Linux and Windows System administration including OS patching, permissions management, security settings, ssh / RDP enablement, using debugging and tracing tools to triage common problems.
- Experience with at least one market leading monitoring tool, understand how it operates, configured and managed.