Data Engineer @ DICK'S Sporting Goods - Coraopolis, PA
Post a job for free in Coraopolis, PA
A Data Engineer is a key member of our team that will be responsible for developing and maintaining flexible, scalable and accurate data pipelines that facilitate the movement of data which enables Marketing, Data Science, and Enterprise Reporting teams to make effective, data-driven decisions. They will help lead the implementation and maintenance of a data catalog and drive data quality across the organization by automating data validation and alerting protocols.
- Collaborate with Engineers, Data Scientists and Analysts to build and maintain data pipelines to support Marketing, Data Science and Enterprise Reporting efforts
- Work with stakeholders including the product, data and architecture teams to assist with data-related technical issues and support their data infrastructure needs.
- Identify, design and implement internal process improvements (automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc..)
- Refine and improve our continuous integration/continuous delivery (CI/CD) pipelines to streamline deployment and product release cycles.
- Create data tools for analytics and Data Scientist and Analyst team members that assist them in building and optimizing our data processes.
- Bachelor's Degree in Computer Science, Software Engineering, Information Systems or Information Technology or related field required, or equivalent experience
- One to three years of experience in Cloud Platforms, Software Design, Database experience, CI/CD
- Experience building data pipelines, preferably utilizing Google Cloud Platform or other cloud platforms.
- Experience with Google Cloud Platform networking and Cloud Identity and Access Management
- Proficient in object-oriented and scripting languages (Python, Java, etc..)
- Experience with continuous integration/continuous delivery (CI/CD) pipelines (Concourse, Azure DevOps)
- Experience with distributed data processing technologies (Spark, Dask, Apache Beam, etc..)
- Experience with Infrastructure as Code and deployment management tools (Terraform, Ansible, GCP Deployment Manager, etc..)
- Experience with relational databases (Oracle, SQL Server, etc..) as well as NoSQL database technologies (MongoDB, BigTable, Cassandra, etc..)
- Experience with Agile Development and Agile Deployment tools and versioning using Git or similar tools
- Proficient in developing, maintaining and interacting with APIs
- Proficient in Linux/Unix environments
- Experience with message queuing, stream processing, and highly scalable ‘big data’ data stores (Kafka, Pub/Sub)