Senior Big Data Engineer

Are you a Pioneering Spirit? Lowe's is looking for Visionaries, Risktakers , Adventurers. Lowe’s is looking for someone that is driven by potential—A trailblazer of innovation. We are a multibillion dollar company with 5 million MyLowes users and millions of daily site visits. We’re looking for someone that dares to take risks and is excited about the opportunity to work on problems that have never been attempted. So, if you’re an innovator, the kind of person who thinks outside the box and is always excited about your next challenge, you might just be the Pioneering Spirit we’re looking for.

PURPOSE OF ROLE:

The primary purpose of this role is to deliver data pipelines and analytics solutions on Hadoop and other platforms. The Big Data Engineer is responsible for building, maintaining, testing and evaluation complex solutions in close partnership with solution architects and other analytic partners. This role translates requirements into new big data solutions, maintenance and execution of existing processes, as well continuous improvement.

RESPONSIBILITY STATEMENTS

• Builds complex technical solutions that turn business requirements into operational processes and inject analytical insights into the business

• Aligns complex technical solutions to corporate governance requirements to ensure data security and maintain data quality standards

• Develop custom batch-oriented and real-time streaming data pipelines working within the MapReduce ecosystem

• Operationalize open source data-ingestion, data-processing, and data-analytics tools for enterprise use

• Utilize multiple development languages/tools such as Java, Python, Scala, SQL, Spark, Hive, and Pig to build prototypes and evaluate results for effectiveness and feasibility

• Ensure proper data governance policies followed by implementing or validating data lineage, quality checks and adhering to published metadata capture and management policies

• Effectively resolve defects/bugs noticed in various testing cycles and in operational environments

• Have a quality mindset, work hard to prevent bugs in the first place through unit testing, test-driven development, version control, continuous integration and deployment.

• Contribute to the design and architecture of projects

• Conduct code reviews

• Mentor junior engineers and new team members, and apply technical expertise to challenging programming and design problems

• Developer level on-call support in rotations

REQUIREMENTS

• Bachelor’s degree in Computer Science, Information Systems, Engineering or other related field.

• 3-5 years of experience in Data processing in Hadoop/Spark and related data activities

• Solid background in Java /Scala in Hadoop for data ingestion and data transformation.

• Deep understanding and hands on experience of Big Data / NoSQL frameworks.

• Programming in at least 2 of these programming languages: Java, Python, Scala, R, C++ in Linux/Unix environment

DESIRED COMPETENCIES

• Advanced degree in Computer Science, Information Systems, Engineering or other related field.

• Working knowledge of Front End technologies

• Working Experience with continuous integration and continuous delivery tools

• Test Driven Development in a lean startup like environment

About Lowe's

Lowe’s Companies, Inc. (NYSE: LOW) is a FORTUNE ® 50 home improvement company serving more than 17 million customers a week in the United States , Canada and Mexico. With fiscal year 2016 sales of $65.0 billion, Lowe’s and its related businesses operate or service more than 2,375 home improvement and hardware stores and employ over 290,000 employees. Founded in 1946 and based in Mooresville, N.C. , Lowe’s supports the communities it serves through programs that focus on K-12 public education and community improvement projects. For more information, visit Lowes.com