> For the complete documentation index, see [llms.txt](https://data-engineering-zoomcamp-2025-t.gitbook.io/tinker0425/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://data-engineering-zoomcamp-2025-t.gitbook.io/tinker0425/module-2/introduction-to-module-2.md).

# Introduction to Module 2

{% hint style="info" %}
You will have 1 week to complete Module 2
{% endhint %}

:exclamation:Homework is due 2/03/25 (Alaska local time) -  <https://courses.datatalks.club/de-zoomcamp-2025/>

:warning: NOTE that because this module was new, there were a lot of changes/modifications to this week throughout the week. Some of my notes may not have caught all the changes. Two ISSUES were the docker-compose.yml and JSON key. In these notes, the docker-compose.yml by Bruno is great, up until the dbt module, when I used Will's, but better practice is Bruno's. The JSON private key comes into play for GCP connection. BE SURE to NOT git your key info. The video has up copy and paste it into a flow, so DO NOT git that flow. I put mine in a folder called 'private' and did not add it to my public github repo. You can find more info in slack here -&#x20;

{% embed url="<https://datatalks-club.slack.com/archives/C01FABYF2RG/p1738520204522619>" %}

:books: Module 2 Course Material from #zoomcamp:

{% @github-files/github-code-block url="<https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/02-workflow-orchestration>" %}

<figure><img src="/files/K1OipDa0vqIOKWLNsUVS" alt=""><figcaption><p><a href="https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/images/architecture">https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/images/architecture</a></p></figcaption></figure>

#### :writing\_hand: Summary

Module 2 will introduce us to Kestra, which is "an open-source, event-driven orchestration platform that simplifies building both scheduled and event-driven workflows. By adopting Infrastructure as Code practices for data and process orchestration, Kestra enables you to build reliable workflows with just a few lines of YAML."  - #dezoomcamp

{% @github-files/github-code-block url="<https://github.com/kestra-io/kestra>" %}

### :bookmark: FAQs videos

{% tabs %}
{% tab title="PostgresDB Setup and Installing pgAdmin" %}
{% embed url="<https://youtu.be/ywAPYNYFaB4?si=5X9AD0nFAT2WLWgS>" %}
{% endtab %}

{% tab title="Docker Setup" %}
{% embed url="<https://youtu.be/l2M2mW76RIU?si=oqyZ7KUaI27vi90V>" %}
{% endtab %}
{% endtabs %}

<table data-view="cards"><thead><tr><th></th><th></th><th></th></tr></thead><tbody><tr><td></td><td><a href="/pages/Luh69ZJ4PNxzGj5hILrk">2.1 - Introduction to Orchestration and Kestra</a></td><td></td></tr><tr><td></td><td><a href="/pages/33fOyrmfBc2uL3Wtaedh">2.2 - ETL Pipelines in Kestra: Detailed Walkthrough</a></td><td></td></tr><tr><td></td><td><a href="/pages/RRVTMp4vYDCyrwFSnq31">2.3 - ETL Pipelines in Kestra: Google Cloud Platform</a></td><td></td></tr><tr><td><a href="/pages/5n7T1HGNp0KX4AgIkCYG">Bonus: Deploy to the Cloud</a></td><td></td><td></td></tr><tr><td><a href="/pages/reXVLsS13wuJY3Don5aM">Homework</a></td><td></td><td></td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://data-engineering-zoomcamp-2025-t.gitbook.io/tinker0425/module-2/introduction-to-module-2.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.