About the company:
Avenue Code is the leading software consultancy focused on delivering end-to-end development solutions for digital transformation across every vertical. We’re privately held, profitable, and have been on a solid growth trajectory since day one. We care deeply about our clients, our partners, and our people. We prefer the word ‘partner’ over ‘vendor’, and our investment in professional relationships is a reflection of that philosophy. We pride ourselves on our technical acumen, our collaborative problem-solving ability, and the warm professionalism of our teams.
Avenue Code has been believing in and promoting plurality actions for over 10 years, understanding that recognizing differences and fostering a safe environment, employment opportunities, representation, and support are the best ways to promote an increasingly equitable culture.
About the opportunity:
We are looking for a Data Engineer who can handle a wide array of data indexing automation and prompt engineering. Experience with large datasets and performance/scalability knowledge will be important.
Responsibilities:
- Design, build, and maintain scalable and efficient data pipelines to support the ingestion, transformation, and integration of data from various sources.
- Organize and optimize data in MySQL to ensure data integrity, organization, and performance.
- Set up, manage, and optimize ElasticSearch clusters, creating and tuning indices to improve query and analysis efficiency.
- Implement pipeline and ETL processes to consolidate data in data warehouse and data lake environments, using tools like Apache Airflow and Databricks.
- Collaborate with Data Science teams to provide data ready for analysis, supporting the development of predictive models and advanced analytics.
- Monitor and troubleshoot performance issues related to data ingestion processes and database queries.
- Create clear and detailed technical documentation of the developed data processes and pipelines.
Required Qualifications:
We know that each person has gone through different career paths that have provided further knowledge. If you don’t check all the points: go ahead! Apply anyways! Your experience is much more than a technology checklist.
- Fluent in English for direct communication with foreign clients.
- Solid experience in developing and optimizing queries in MySQL.
- Deep knowledge of ElasticSearch, including configuration, index creation, and performance optimization.
- Experience with ETL tools and data orchestration, such as Apache Airflow.
- Proficiency in Python for process automation and script development.
- Experience with cloud environments, especially in data-oriented services.
- Familiarity with data lake and data warehouse architecture.
Nice to Have:
However, if you have the following requirements, it will be a plus for your application:
- Kubernetes
- AWS
- Redis
- Prompt Engineering
Avenue Code reinforces its commitment to privacy and to all the principles guaranteed by the most accurate global data protection laws, such as GDPR, LGPD, CCPA and CPRA. The Candidate data shared with Avenue Code will be kept confidential and will not be transmitted to disinterested third parties, nor will it be used for purposes other than the application for open positions. As a Consultancy company, Avenue Code may share your information with its clients and other Companies from the CompassUol Group to which Avenue Code’s consultants are allocated to perform its services.
#Li-Remote