Big Data Engineer

San Francisco, CA

Apply For This Job

Avenue Code is the leading software consultancy focused on delivering end-to-end development solutions for digital transformation across every vertical. We’re privately held, profitable, and have been on a solid growth trajectory since day one. We care deeply about our clients, our partners, and our people. We prefer the word ‘partner’ over ‘vendor’, and our investment in professional relationships is a reflection of that philosophy. We pride ourselves on our technical acumen, our collaborative problem-solving ability, and the warm professionalism of our teams.

About the Opportunity:

We are seeking an energetic and talented Data Engineer to deliver high value, high-quality business capabilities to our data technology platform.  With a background in both software development and machine learning, you will collaborate with software engineers, data scientists and domain experts in Supply Chain and Inventory management to develop data and machine learning products to support our company. You will be an integral member engineering team delivering across multiple business functional areas. You will build data analysis infrastructure for effective prototyping and visualization of various data-driven approaches

Key Responsibilities:

  • Partner in building the infrastructure required for optimal extraction, transformation, visualization, and loading of data from a wide variety of data sources using SQL and big data technologies.
  • Improve our Machine Learning systems by contributing to all phases of algorithm development including ideation, prototyping, design and production
  • Scale our data processing pipelines to handle and maintain complex processes in an efficient and reliable way that are available, scalable, and fault tolerant.
  • Build scalable production systems for data collection, data transformation, feature extraction, model training, and scoring, using distributed software tools
  • Work with data science to define data ingestion standards and assist with data-related technical issues
  • Partner with product development team to understand various opportunities and use cases.
  • Maintain specifications and metadata; create and maintain documentation
  • Build Communities-of-Practice in key data technologies

Technical Skills/Qualifications:

  • 5+ years’ experience building and maintaining large-scale analytics systems
  • 5+ years’ experience in software development and delivery
  • Expertise in one or more programming languages such as Python, Scala, Java, Javascript, etc. and related frameworks (Flask, Django, Spring, AngularJS, React).
  • Good knowledge of architecting, developing, and maintaining of cloud technologies (AWS, Azure, GCP)
  • Experience with Big Data stack (Kafka, Hadoop, Storm, Spark, Hive, Airflow, ElasticSearch, TensorFlow).
  • Experience with relational (Postgres, MySQL) and/or non-relational (Cassandra, MongoDB) databases.
  • Strong machine learning or stats knowledge a plus
  • Experience in Supply Chain / Retail / eCommerce
  • Excellent communication skill and a strong team player.
  • BS/MS in Computer Science, Computer Engineering or related technical discipline
  • Strong analytical and quantitative problem-solving ability

Does this sound like you? Apply now to become an Avenue Coder!

Apply For This Job

First Name

Last Name



LinkedIn Profile

How did you hear about this job?

Do you have the right to work in the U.S. indefinitely without any sponsorship or other legal support from your employer? If no, please explain the support you would need from your employer.

What is your availability to start a new position?

What are your salary requirements?

Are you available for opportunities requiring relocation?

Attach Resume *

Choose file...

Attach Cover Letter

Choose file...