$ arrow-data_

Crate

Name: arrow-data New version: 57.0.0 Release date: 2025-10-23T11:00:57Z

Crate readme

Short description Native Rust implementation of Apache Arrow and Apache Parquet

Long description This repository contains the Rust implementation for Apache Arrow, a popular in-memory columnar format, and Apache Parquet, a columnar file format. It includes core functionality for memory layout, arrays, and low-level computations in the arrow crate, support for the Arrow-Flight IPC protocol in arrow-flight, and comprehensive support for the Parquet columnar file format in the parquet crate. Additionally, the parquet_derive crate facilitates deriving RecordWriter/RecordReader for simple structs. The project follows Semantic Versioning and releases approximately monthly, with major versions at most quarterly. It is built and tested with stable Rust, maintaining a rolling MSRV that is at least 6 months old.

Features • arrow crate • arrow-flight crate • parquet crate • parquet_derive crate • MSRV policy

Code Examples Arrow Rust Community Slack Invite

 https://s.apache.org/slack-invite

GitHub Issues

 https://github.com/apache/arrow-rs/issues

GitHub Discussions

 https://github.com/apache/arrow-rs/discussions

Discord channel

 https://discord.gg/YAb2TdazKQ

Links • https://arrow.apache.org/ • https://crates.io/crates/arrow • https://crates.io/crates/parquet • https://crates.io/crates/parquet-derive • https://crates.io/crates/arrow-flight • https://github.com/apache/arrow-rs-object-store • https://docs.rs/arrow/latest • https://docs.rs/arrow-flight/latest • https://docs.rs/parquet/latest • https://docs.rs/parquet-derive/latest • https://arrow.apache.org/rust • https://semver.org/ • https://github.com/apache/arrow-rs/issues/7835 • https://github.com/apache/arrow-rs/milestone/3 • https://github.com/apache/arrow-rs/milestone/5 • https://github.com/apache/arrow-rs/milestone/6 • https://github.com/apache/arrow-rs/issues/5368 • CONTRIBUTING.md#breaking-changes • https://github.com/apache/arrow-rs/issues/6737 • https://crates.io/crates/datafusion • https://crates.io/crates/ballista • https://crates.io/crates/parquet_opendal • https://github.com/apache/opendal/blob/main/integrations/parquet/README.md • https://github.com/apache/arrow-rs-object-store/blob/main/README.md • https://github.com/apache/datafusion/blob/main/README.md • https://github.com/apache/datafusion-ballista/blob/main/README.md • https://arrow.apache.org/community/ • https://s.apache.org/slack-invite • https://github.com/apache/arrow-rs/discussions • https://discord.gg/YAb2TdazKQ • https://github.com/apache/arrow-rs/issues • arrow/README.md • CONTRIBUTING.md • parquet/README.md • arrow-flight/README.md • https://www.rust-lang.org/ • https://crates.io/crates/object-store • https://github.com/apache/arrow-rs/tree/57.0.0 • https://github.com/apache/arrow-rs/compare/56.2.0...57.0.0 • https://github.com/apache/arrow-rs/pull/8626 • https://github.com/apache/arrow-rs/pull/8596 • https://github.com/apache/arrow-rs/pull/8595 • https://github.com/apache/arrow-rs/pull/8587 • https://github.com/apache/arrow-rs/pull/8570 • https://github.com/apache/arrow-rs/pull/8569 • https://github.com/apache/arrow-rs/pull/8530 • https://github.com/apache/arrow-rs/pull/8529 • https://github.com/apache/arrow-rs/pull/8470 • https://github.com/apache/arrow-rs/pull/8447 • https://github.com/apache/arrow-rs/pull/8425 • https://github.com/apache/arrow-rs/pull/8424 • https://github.com/apache/arrow-rs/pull/8405 • https://github.com/apache/arrow-rs/pull/8396 • https://github.com/apache/arrow-rs/pull/8291 • https://github.com/apache/arrow-rs/pull/8290 • https://github.com/apache/arrow-rs/pull/8286

https://api.github.com/repos/apache/arrow-rs/releases/256648414

Release info

Release version: 57.0.0

Release description Version 57.0.0 introduces several breaking changes, primarily focused on internal refactoring and consistency improvements within the Apache Arrow Rust project. A significant change involves the consistent use of Arc<FileEncryptionProperties> across the board to align with FileDecryptionProperties, impacting how file encryption details are handled, particularly within the Parquet module. The project has also enhanced the display formatting for various DataType variants, including RunEndEncoded, Map, ListView, and LargeListView, aiming for clearer, more readable representations. Internally, a 'Thrift remodel' effort has led to refactoring Thrift encryption and storing encodings as a bitmask, alongside the adoption of a custom Thrift parser for Parquet metadata. These changes, while breaking, contribute to a more robust and maintainable codebase. Additionally, the arrow-avro crate now includes more comprehensive type support and roundtrip tests. Memory efficiency has been improved in the Parquet writer for compressed pages, and the use of Arc for decryption properties reduces memory overhead when encryption is enabled. The project also bumps the pyo3 dependency to version 0.26.0.

Code Examples