Data Engineer

The data engineer considers themselves an expert in data management with cloud technologies. The data engineer leads the definition and implementation of data pipelines, data storage, and data access/security for a complex Azure hosted application environment. They are responsible for curating a sustainable and scalable data environment, servicing a mix of business. A number of operational data sources will require integration, data structuring/cleaning, ETL and data quality reviews on an ongoing basis. The data engineer works closely with the digital business infrastructure operations (DBIO) lead and analytics lead, and to ensure data quality, performance, and structure requirements scale with the customer needs.

The data engineer will be working within a team setting and will support a range of data requirements, supporting cloud operations/monitoring, mobile application analytics, and user personalization use cases for a global 100 technology leading client. The environment is fast-paced and dynamic and will provide a range of requirements expected to evolve.

Requirements:

· High-level proficiency in data engineering

· Proficiency in data engineering best practices

· Deep experience with Azure data management related services and related data architectures

· Knowledge of security standardsfor Azure data environments

· Bachelor of Science in Computer Science or other relevant experience

· Experience with Java, Python, SQL, and noSQL

Responsibilities:

· Configure data sources for data loading, batch, and continuous data streaming

· Configure and implement ETL operations

· Configuring data storage tools / platforms

· Support enterprise architect for data architecture strategy and design

· Develop data models in support of data architecture

· Lead data lake design and implementation, aligned with data architecture

· Data set curation, cleaning, loading

· Integrate analytics and monitoring tools with consolidated data environment

· Establishing and maintaining consolidated data lake environment

· Managing data storage resources, monitoring, data operations

· Identify areas for continuous improvement in data related technology and processes

· Create and communicate high quality technical documentation

· Interact regularly with managers, SMEs, and other technical leadership

Desired Skills and Experience:

· Minimum 5-10 years consulting / professional services experience Azure Architect certified (Associate or Professional)

· Azure experience (data pipeline, database, network, security, CI/CD)

· Strong written/verbal communication skills

· Ability to handle ambiguity and deliver outcomes

· Experience with Snowflake or Databricks (preferred, but not required)