Spice AI Announces Contribution of TableProviders for PostgreSQL, MySQL, DuckDB, and SQLite to the Apache DataFusion Project
Spice AI has contributed new TableProviders for PostgreSQL, MySQL, DuckDB, and SQLite to the Apache DataFusion project. This addition reflects our commitment to building together in the data and AI ecosystem and supporting the open-source community.
What is Apache DataFusion?
Apache DataFusion is a high-performance query engine built on Apache Arrow. It allows you to execute SQL queries quickly and efficiently on data stored in various formats. By using the in-memory columnar format of Apache Arrow, DataFusion speeds up data processing and works natively with other Arrow-based tools.
About the Spice OSS Project
Spice OSS is an open-source project from Spice AI that provides developers with a unified SQL query interface to locally materialize, accelerate, and query datasets from any database, data warehouse, or data lake. Spice OSS incorporates Apache DataFusion as its SQL query engine.
Our goal with Spice OSS is to make data and AI-driven development more accessible. By contributing to projects like DataFusion and Arrow, we can help making accessing and using data better for everyone building in the space.
New TableProviders for DataFusion
We've initially added TableProviders for PostgreSQL, MySQL, DuckDB, and SQLite to DataFusion. This expands the range of data sources you can query using DataFusion. And we plan to add more in the future.
- PostgreSQL: A robust and extensible open-source relational database.
- MySQL: A reliable and user-friendly open-source relational database.
- DuckDB: An in-process SQL OLAP database for analytical queries.
- SQLite: A lightweight, disk-based database commonly used in embedded systems.
These new TableProviders make DataFusion even more versatile, allowing you to work with your existing databases and data lakes more easily.
Our Commitment to Data and AI
At Spice AI, we believe in the power of open-source and the potential of data and AI. By contributing to Apache DataFusion, we're helping to advance data processing technology and make powerful tools available to developers everywhere.
To learn more about Spice AI and our open-source projects, check out our GitHub repository. You can also explore Apache DataFusion at datafusion.apache.org.
Stay tuned for more updates from Spice AI as we continue to contribute to the data and AI ecosystem.
Get the latest insights
Get expert updates on building, deploying, and scaling AI applications
Explore more lorem ipsum sit amet consectetur
Vorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc vulputate libero et velit interdum, ac aliquet odio mattis.
Spice Cloud v1.8.0: Iceberg Write Support, Acceleration Snapshots & More
Spice Cloud & Spice.ai Enterprise 1.8.0 are live! v1.8.0 includes Iceberg write support, acceleration snapshots, partitioned S3 Vector indexes, a new AI SQL function for LLM integration, and an updated Spice.js SDK. V1.8.0 also introduces developer experience upgrades, including a redesigned Spice Cloud dashboard with tabbed navigation: Spice Cloud customers will automatically upgrade to v1.8.0 on deployment, while […]
Spice Cloud v1.7.0: DataFusion v49, Full-Text Search Updates & More
Spice Cloud & Spice.ai Enterprise 1.7.0 are now live, bringing performance upgrades with DataFusion v49, real-time full-text search indexing, EmbeddingGemma support, and improvements across search, embeddings, and API integrations. Spice Cloud customers will automatically upgrade to v1.7.0 on deployment, while Spice.ai Enterprise customers can consume the Enterprise v1.7.0 image from the Spice AWS Marketplace listing. What’s New in v1.7.0 […]
True Hybrid Search: Vector, Full-Text, and SQL in One Runtime
TL;DR Show Me the (Data)! It’s well established (and maybe even trite) to say that enterprises are going all-in on artificial intelligence, with more than $40 billion directed toward generative AI projects in recent years. The initial results have been underwhelming. A recent study from the Massachusetts Institute of Technology’s NANDA initiative concluded that despite the enormous allocation of […]
See Spice in action
Get a guided walkthrough of how development teams use Spice to query, accelerate, and integrate AI for mission-critical workloads.
Get a demo
