---

Jupyter All Spark Notebook

A Docker Compose Jupyter notebook image with Apache Spark, PySpark 4.1.1, JupyterLab, and Rust support.

Based on the quay.io/jupyter/all-spark-notebook image.

TL;DR

docker compose build
docker compose up

To run in the background:

docker-compose up -d

Access JupyterLab at http://localhost:8888

Features

• Python & PySpark: Full data processing and analysis with Apache Spark
• Rust Support: Interactive Rust notebooks and Python-Rust integration via PyO3 and maturin
• JupyterLab: Modern notebook interface with multiple kernels

Rust Integration

This image includes Rust support for building high-performance extensions and working with Rust alongside PySpark. See RUST_SUPPORT.md for detailed documentation.

Contributing

Prerequisites

Install asdf, then add the required plugins and install all tool versions:

asdf plugin add python
asdf plugin add java
asdf plugin add nodejs
asdf plugin add scala
asdf plugin add spark
asdf plugin add sbt
asdf plugin add coursier
asdf install

This will provision the exact versions defined in .tool-versions.

Running Tests

pip install -r requirements.txt
pytest

Running Locally with Docker

docker compose up

Access JupyterLab at http://localhost:8888. Notebooks in ./notebooks are mounted automatically.

Resources

• quay.io/jupyter/all-spark-notebook
• Data science with JupyterLab
• Supercharging AI/ML Development with JupyterLab and Docker
• PySpark Cheat Sheet
• Rust Support Guide