Find trending repositories by name or description.
Imagination Studio by keys+Wasserman powered by TensorFold: an AI film studio for Mac and NVIDIA DGX Spark. Plan a film like a crew (Slate workflow by Sam Wasserman), render Qwen-Image-2.1 / FLUX3-K pictures and MiniMax H3 / FastH3 video with sound on TensorFold.
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow
Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
Fast & opinionated local AI routing: one MCP server that loads local image and video models on demand on your own GPU box (DGX Spark first)
Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/aeon-7/aeon-vllm-ultimate:latest container, tuned for long-context draft acceptance on DGX Spark. 6 HF variants (BF16/NVFP4/MTP/MTP-XS), docker-compose, and QuickStart.
DeepSeek-V4.1-Flash on 2x NVIDIA DGX Spark with the TensorFold engine: 1.8-1.9x code, 2.5x multi-stream, 1.7-1.95x prefill vs the vLLM kit, exact speculative decoding. Work in progress.
Reproducible Docker/vLLM deployment, tuning, and benchmarks for DeepSeek V4.1 Flash on a switchless three-node or four-node DGX Spark fabric.
Qwen3.8 27B on SGLang for DGX Spark
BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.
A better compressed bitset in Java: used by Apache Spark, Netflix Atlas, Apache Pinot, Tablesaw, and many others
Apache Spark - A unified analytics engine for large-scale data processing
Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University
Expressive types for Spark.
Security library for Sparkjava: OpenID Connect, SAML2, CAS, OAuth, LDAP, JWT...
A connector for Spark that allows reading and writing to/from Redis cluster
Arch Linux Provisioning with Ansible
Spark job that aggregates zipkin spans for use in the UI
1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
R interface for Apache Spark
This repository contains the development code for sparkMeasure, an Apache Spark performance analysis and troubleshooting library. It simplifies collecting, aggregating, and exporting Spark task/stage metrics, and is designed for practical use by developers and data engineers in interactive analysis, testing, and production monitoring workflows.
Zeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark
Process Common Crawl data with Python and Spark
TiSpark is built for running Apache Spark on top of TiDB/TiKV
BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
State of the Art Natural Language Processing
The Internals of Spark SQL
Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.