Job Reference: BBBH34416
Job Duration:
Start date: ASAP
Description:
My client are a prestigious government organisation. They are seeking Data Engineer on a contract basis,initially until the end of March. The organisation have three UK locations that candidates can work from - London, Hampshire and South Wales. However offices are currently closed due to covid, so the role will start remotely.
A live SC Clearance is needed for this position.
This is an opportunity to work in the Data Architecture (DA) Division, and more specifically within the Products Branch.
The role will involve:
1. Development of ETL methods for a range of internal ONS and external data sources converting these from unstructured formats to dynamic tables and views in Hive and Impala;
2. Supporting the development andtransformation of RDMF v0.2 and the associated indexes. 4. Assisting in the development of analytical layers of data from raw HDFS files for use in producing a range of RDMF outputs.
Skills and Experience
Essential
* Extensive proven experienceof data engineering and architectural techniques, including data wrangling, data profiling, data preparation, metadata development, and data upload/download;
* Proven experience of 'big data' environments, including the Hadoop Stack (Cloudera), includingdata ingestion, processing and storage using HDFS, Spark, Hive and Impala;
* Extensive hands-on experience of developing ETLfunctionality in a cloud or onpremise environment;
* Experience of using tools such as python and SQL (Spark) to profile, query and structure large-volume data;
* Proven experience of using Cloud Services particularly in the context of Hadoop;
* Experience of developing/utilising programming and query languages e.g. SQL (Hive Impala specifically), Python (through Spark), Scala.
* Understanding of data bases and applying data models in relational database formats.
Desirable
* Experience of UKGovernment Administrative Data;
Industry: IT
Salary: £300 - £450 per day + Flexible working
Salary Benefits: Flexible working
Vacancy Type: Permanent
Job Skills: -
Contact Name: Ben Tristram
Website: -
Direct Application URL: -