Data Engineer
Hyderabad
Job description
A Data Engineer designs, builds, and maintains systems for collecting, storing, and processing large datasets. They develop data pipelines, ensure data integrity, and optimize data workflows for analytics or machine learning. Proficient in programming languages like Python or Java and tools such as SQL, Spark, and cloud platforms, they collaborate with teams to enable efficient data access and support decision-making processes. Strong problem-solving and communication skills are essential.
Roles and Responsibilities
- Develop and enhance data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and GitLab automation.
- Collaborate with product and technology teams to design and validate the capabilities of the data platform
- Identify, design, and implement process improvements: automating manual processes, optimizing for usability, re-designing for greater scalability
- Provide technical support and usage guidance to the users of our platform’s services.
- Drive the creation and refinement of metrics, monitoring, and alerting mechanisms to give us the visibility we need into our production services.
Key Skills Required
- Programming: Proficiency in Python, Java, Scala, or similar languages.
- Database Management: Strong knowledge of SQL, NoSQL databases, and data modeling.
- ETL Development: Expertise in designing and implementing data pipelines and ETL workflows.
- Cloud Platforms: Familiarity with AWS, Azure, Google Cloud, or similar services.
- Data Warehousing: Experience with tools like Snowflake, Redshift, or BigQuery.
- Data Governance: Understanding of data quality, security, and compliance.
- Version Control: Knowledge of tools like Git for code management.
- Analytical Thinking: Strong problem-solving and debugging skills.
- Team Collaboration: Effective communication and teamwork with stakeholders and developers.
Qualifications
- Experience building and optimizing data pipelines in a distributed environment
- Experience supporting and working with cross-functional teams
- Proficiency working in Linux environment
- 4+ years of advanced working knowledge of SQL, Python, and PySpark
- 2+ years of experience with using a broad range of AWS technologies
- Experience using tools such as: Git/Bitbucket, Jenkins/CodeBuild, CodePipeline
- Experience with platform monitoring and alerts tools
Instructions for job application
Lorem ipsum dolor sit amet consectetur. Condimentum morbi eu commodo justo magna nisl ultrices aliquam. Pulvinar turpis ut cum pellentesque. Lectus scelerisque non non donec. Massa etiam elit condimentum nec et egestas sed in felis. Penatibus netus lacus interdum nunc tincidunt non sit et. Mi gravida pulvinar mattis sit tellus a eu. Vel.
Other Tangible Benefits
- Employee insurance benefits covering dependants.
- Flexible work timings.
Ready to start your journey? Email us at [email protected].
VentureSoft is an Equal Opportunity Employer. We embrace diversity and offer an inclusive workplace for everyone.
