DE Zoomcamp Notes

Linkedin | Kayla Tinker Github | Tinker0425 Blog | From Clouds to Code BlueSky | Cloudy Blue Wave

Linkedin | Kayla Tinker Github | Tinker0425 Blog | From Clouds to Code BlueSky | Cloudy Blue Wave

Welcome - Data Engineering Zoomcamp 2025 Notes
INTRODUCTION
- Introduction & Set Up
  - Virtual Environments
MODULE 1
Module 2
Module 3
Workshop
- Workshop Week
- Homework
Module 4
Module 5
Module 6
Final Project

Powered by GitBook

Connect

Linkedin | Kayla Tinker
BlueSky | Cloudy Blue Wave
Blog | From Clouds to Code
Github | Tinker0425

On this page

Module 5

5.1 - Install & Intro

5.1.3 - Intro to Spark

Last updated Feb 25, 2025

Previous5.1.2 - Intro to Batch Processing Next5.2 - Spark SQL and DataFrames

Last updated 1 month ago

Estimated time spent on this lesson | ~15 min

Youtube Video | ~7 min

In this video we learn about apache spark, how to use it, and when to use it.

Spark is used to execute batch or streaming. Spark is usually in a wrapper with Java, Scala, Python, R, etc.

The python wrapper is called PySpark

Workflow Example

🕓

✍️

Apache Spark™ - Unified Engine for large-scale data analyticsApacheSpark

Getting Started — PySpark 3.3.0 documentation