Data Engineer
Glitch Devs
- Work mode
- Remote
- Job type
- Full time
- Team
- AI & Data
- Salary
- USD 900 to 1,500 / month
Posted 4 months ago
About the role
You will build the pipelines that move data from where it is created to where people can use it. Clients often have data spread across databases, spreadsheets and third-party tools, and your job is to bring it together in a form that is reliable and documented.
We value pipelines that fail loudly and are easy to rerun over clever ones that nobody understands.
Day to day:
- Build and maintain data pipelines
- Design warehouse tables and transformations
- Pull data from APIs, databases and files
- Add checks so bad data is caught early
- Document what each pipeline does and who depends on it
Examples of what you might work on:
- A pipeline that collects data from a client's database and three outside tools into one warehouse
- Data checks that stop a bad file from reaching a report
- A set of documented tables that analysts can query without asking for help
Tools you will use:
SQL, Python, Airflow or Dagster, BigQuery, Snowflake or PostgreSQL, dbt, Git, Docker.
Your first months:
- First weeks: Read the existing pipelines, run them, and learn where the data comes from.
- By month two: Fix a failing pipeline and add checks and documentation to it.
- By month three: Build a new pipeline from source to warehouse and hand it over with clear notes.
Requirements
- 3 to 5 years working with data pipelines
- Strong SQL and Python
- Experience with a data warehouse such as BigQuery, Snowflake, Redshift or PostgreSQL
- Experience with an orchestration tool such as Airflow or Dagster
- Understanding of data modelling
Nice to have:
- dbt
- Streaming tools such as Kafka
- Experience supporting dashboards or machine learning work
Practice for this role
Try a free 20-question test built for this role. It takes about 20 minutes and you get an explanation for every answer. It is not part of the application.
Take the practice testKnow someone who’d be a good fit?
Similar roles
See all roles →Data Engineer
Apply now