Agentic Coding for Data and Analytics Engineering Teams

Diese Seite auf Deutsch

Short answer

Coding agents can speed up data and analytics engineering, but data work fails silently: a wrong join still returns plausible numbers. Give agents schemas and metadata instead of real rows, read-only access to development data and dbt tests that run after every change. Then review generated SQL for join fan-out, NULL handling and time zones, and reconcile results against known totals.

Why data work needs extra care

When an agent writes broken application code, a test fails or the app crashes. When it writes broken SQL, you usually get a result anyway – revenue counted twice, customers missing, a week shifted by a time zone. Those numbers end up in dashboards and decisions. Agentic coding in data teams therefore needs two things application teams can sometimes skip: strict control over which data reaches the model, and a review discipline aimed at semantic errors rather than syntax.

Give the agent context, not data

An agent needs to understand your models, not see your customers. Useful context includes table definitions, dbt sources and model YAML with column descriptions, the grain of each table and a handful of synthetic example rows.

  • Never paste exports with personal or confidential data into a prompt.
  • Give the agent read-only credentials for a development dataset or sample schema, never for production.
  • Remember that query results the agent runs become part of its context and are sent to the model provider. A SELECT * on a customer table is a data transfer.
  • Set cost limits where the warehouse supports them, such as a maximum of bytes billed per query in BigQuery.

Whether a given setup is allowed under GDPR depends on your contracts and data; this is not legal advice. Our guide on anonymizing data before sending it to ChatGPT or Claude covers practical techniques.

dbt as the agent's safety net

dbt projects suit agents well, because the project structure is explicit and tests are cheap to run. Put these commands in the instruction file:

  • dbt build --select my_model+ builds and tests the model and everything downstream of it.
  • dbt compile --select my_model lets the agent and the reviewer read the final SQL.
  • Generic tests such as unique, not_null, relationships and accepted_values belong on every new model's keys.
  • In CI, state-based selection with --select state:modified+ and --defer keeps runs fast.

Also document naming layers (for example staging, intermediate, marts), materializations and which sources are off limits.

Python and notebooks

Notebooks are JSON files with embedded outputs, which makes diffs noisy and can leak data into Git. Move reusable logic into Python modules with tests and keep notebooks thin. Strip outputs before committing, for example with nbstripout, or pair notebooks with plain scripts via Jupytext so agents and reviewers work on readable code. Pin library versions in the instruction file: agents mix APIs across pandas versions, such as DataFrame.append, which pandas 2 removed, or blend pandas and Polars idioms.

Reviewing generated queries

  1. Join fan-out: check the grain of both sides and compare row counts before and after the join.
  2. Accidental inner joins: a filter on the right table in the WHERE clause turns a LEFT JOIN into an inner join.
  3. NULL semantics: NOT IN against a subquery that contains NULL returns no rows; COUNT(column) skips NULLs.
  4. Time: time zones, date truncation and half-open intervals for period boundaries.
  5. Window functions and deduplication: correct partition and order, and handling of late-arriving records.
  6. Reconciliation: compare results against known totals, such as booked revenue, before anything is published.

How jocapps helps

neooutput runs hands-on workshops, team trainings and 1:1 consulting on agentic coding – build-as-you-learn, in English and German, on-site or remote. Data teams work on instruction files for their dbt project, safe access patterns and a review routine for generated SQL, and every session ends with working results. Formats are on our agentic coding workshops page; a free 30-minute intro call is available via the meeting planner.

Frequently asked questions

Can a coding agent connect directly to our data warehouse?

It can, but give it read-only access to development or sample data only. Every query result it reads is sent to the model, so treat that access like a data export.

Are coding agents good at writing SQL?

They write syntactically correct SQL quickly. Semantic errors such as join fan-out or wrong NULL handling are the real risk, which is why tests and reconciliation matter.

How should we use agents with Jupyter notebooks?

Keep logic in tested Python modules, strip outputs before committing and use paired scripts so the agent works on readable code instead of notebook JSON.

Guides on this topic

Talk to us about your project

Honest advice on your idea, prototype or app.

  • 30 minutes
  • Free
  • No obligation

Book a free call [email protected]