$ arrow-row_

Crate

Name: arrow-row New version: 57.0.0 Release date: 2025-10-23T11:00:57Z

Crate readme

Short description Native Rust implementation of Apache Arrow and Apache Parquet

Long description This repository contains the Rust implementation of Apache Arrow, a popular in-memory columnar format, and Apache Parquet, a columnar file format. It includes core functionality for memory layout, arrays, and low-level computations (arrow), support for the Arrow-Flight IPC protocol (arrow-flight), and support for the Parquet columnar file format (parquet). Additionally, parquet_derive allows for deriving RecordWriter/RecordReader for simple structs. The crates are released monthly following Semantic Versioning, with major versions no more than once a quarter. The project maintains a rolling MSRV (minimum supported Rust version) policy, ensuring compatibility with stable Rust.

Features • arrow crate • arrow-flight crate • parquet crate • parquet_derive crate • Rolling MSRV policy • Semantic Versioning

Code Examples Add to Cargo.toml

 arrow = "57.0.0"
arrow-flight = "57.0.0"
parquet = "57.0.0"
parquet_derive = "57.0.0"

Use case example

 you can write SQL queries or a `DataFrame` (using the
[`datafusion`] crate) to read a parquet file (using the [`parquet`] crate), 
evaluate it in-memory using Arrow's columnar format (using the [`arrow`] crate), 
and send to another process (using the [`arrow-flight`] crate).

Links • https://arrow.apache.org/ • https://crates.io/crates/arrow • https://crates.io/crates/parquet • https://crates.io/crates/parquet-derive • https://crates.io/crates/arrow-flight • https://github.com/apache/arrow-rs-object-store • https://docs.rs/arrow/latest • https://docs.rs/arrow-flight/latest • https://docs.rs/parquet/latest • https://docs.rs/parquet-derive/latest • https://arrow.apache.org/rust • https://semver.org/ • https://github.com/apache/arrow-rs/issues/7835 • https://github.com/apache/arrow-rs/milestone/3 • https://github.com/apache/arrow-rs/milestone/5 • https://github.com/apache/arrow-rs/milestone/6 • https://github.com/apache/arrow-rs/issues/5368 • https://crates.io/crates/datafusion • https://crates.io/crates/ballista • https://crates.io/crates/parquet_opendal • https://github.com/apache/datafusion/blob/main/README.md • https://github.com/apache/arrow-rs-object-store/blob/main/README.md • https://github.com/apache/datafusion-ballista/blob/main/README.md • https://github.com/apache/opendal/blob/main/integrations/parquet/README.md • https://arrow.apache.org/community/ • https://s.apache.org/slack-invite • https://github.com/apache/arrow-rs/discussions • https://discord.gg/YAb2TdazKQ • https://github.com/apache/arrow-rs/issues • https://www.rust-lang.org/ • https://crates.io/crates/object-store • arrow/README.md • CONTRIBUTING.md • parquet/README.md • arrow-flight/README.md • https://github.com/apache/arrow-rs/issues/6737 • https://github.com/apache/arrow-rs/tree/57.0.0 • https://github.com/apache/arrow-rs/compare/56.2.0...57.0.0 • https://github.com/apache/arrow-rs/pull/8626 • https://github.com/apache/arrow-rs/pull/8596 • https://github.com/apache/arrow-rs/pull/8595 • https://github.com/apache/arrow-rs/pull/8587 • https://github.com/apache/arrow-rs/pull/8570 • https://github.com/apache/arrow-rs/pull/8569 • https://github.com/apache/arrow-rs/pull/8530 • https://github.com/apache/arrow-rs/pull/8529 • https://github.com/apache/arrow-rs/pull/8470 • https://github.com/apache/arrow-rs/pull/8447 • https://github.com/apache/arrow-rs/pull/8425 • https://github.com/apache/arrow-rs/pull/8424 • https://github.com/apache/arrow-rs/pull/8405 • https://github.com/apache/arrow-rs/pull/8396 • https://github.com/apache/arrow-rs/pull/8291 • https://github.com/apache/arrow-rs/pull/8290 • https://github.com/apache/arrow-rs/pull/8286 • https://github.com/apache/arrow-rs/issues/4886 • https://github.com/apache/arrow-rs/issues/8639 • https://github.com/apache/arrow-rs/issues/8614 • https://github.com/apache/arrow-rs/issues/8612 • https://github.com/apache/arrow-rs/issues/8610 • https://github.com/apache/arrow-rs/issues/8609 • https://github.com/apache/arrow-rs/issues/8606 • https://github.com/apache/arrow-rs/issues/8591 • https://github.com/apache/arrow-rs/issues/8559 • https://github.com/apache/arrow-rs/issues/8531 • https://github.com/apache/arrow-rs/issues/8523 • https://github.com/apache/arrow-rs/issues/8517 • https://github.com/apache/arrow-rs/issues/8515 • https://github.com/apache/arrow-rs/issues/8513 • https://github.com/apache/arrow-rs/issues/8503 • https://github.com/apache/arrow-rs/issues/8497 • https://github.com/apache/arrow-rs/issues/8469 • https://github.com/apache/arrow-rs/issues/8461 • https://github.com/apache/arrow-rs/issues/8451 • https://github.com/apache/arrow-rs/issues/8439 • https://github.com/apache/arrow-rs/issues/8412 • https://github.com/apache/arrow-rs/issues/8411 • https://github.com/apache/arrow-rs/issues/8398 • https://github.com/apache/arrow-rs/issues/8389 • https://github.com/apache/arrow-rs/issues/8386 • https://github.com/apache/arrow-rs/issues/8370 • https://github.com/apache/arrow-rs/issues/8361 • https://github.com/apache/arrow-rs/issues/8347 • https://github.com/apache/arrow-rs/issues/8337 • https://github.com/apache/arrow-rs/issues/8336 • https://github.com/apache/arrow-rs/issues/8334 • https://github.com/apache/arrow-rs/issues/8332 • https://github.com/apache/arrow-rs/issues/8331 • https://github.com/apache/arrow-rs/issues/8330 • https://github.com/apache/arrow-rs/issues/8303 • https://github.com/apache/arrow-rs/issues/8285 • https://github.com/apache/arrow-rs/issues/8281 • https://github.com/apache/arrow-rs/issues/8631 • https://github.com/apache/arrow-rs/issues/8579 • https://github.com/apache/arrow-rs/issues/8542 • https://github.com/apache/arrow-rs/issues/8539 • https://github.com/apache/arrow-rs/issues/8534 • https://github.com/apache/arrow-rs/issues/8533 • https://github.com/apache/arrow-rs/issues/8526 • https://github.com/apache/arrow-rs/issues/8505 • https://github.com/apache/arrow-rs/issues/8443 • https://github.com/apache/arrow-rs/issues/8435 • https://github.com/apache/arrow-rs/issues/8406 • https://github.com/apache/arrow-rs/issues/8404 • https://github.com/apache/arrow-rs/issues/8403 • https://github.com/apache/arrow-rs/issues/8382 • https://github.com/apache/arrow-rs/issues/8319 • https://github.com/apache/arrow-rs/issues/8296 • https://github.com/apache/arrow-rs/pull/8383 • https://github.com/apache/arrow-rs/issues/8588 • https://github.com/apache/arrow-rs/pull/8654 • https://github.com/apache/arrow-rs/pull/8583 • https://github.com/apache/arrow-rs/pull/8574 • https://github.com/apache/arrow-rs/pull/8527 • https://github.com/apache/arrow-rs/pull/8388 • https://github.com/apache/arrow-rs/issues/8601 • https://github.com/apache/arrow-rs/issues/8538 • https://github.com/apache/arrow-rs/issues/8504 • https://github.com/apache/arrow-rs/issues/8418 • https://github.com/apache/arrow-rs/issues/8410 • https://github.com/apache/arrow-rs/issues/8380 • https://github.com/apache/arrow-rs/issues/8377 • https://github.com/apache/arrow-rs/issues/8374 • https://github.com/apache/arrow-rs/pull/8642 • https://github.com/apache/arrow-rs/pull/8636 • https://github.com/apache/arrow-rs/pull/8634 • https://github.com/apache/arrow-rs/pull/8633 • https://github.com/apache/arrow-rs/pull/8629 • https://github.com/apache/arrow-rs/pull/8627 • https://github.com/apache/arrow-rs/pull/8625 • https://github.com/apache/arrow-rs/pull/8621 • https://github.com/apache/arrow-rs/pull/8620 • https://github.com/apache/arrow-rs/pull/8617 • https://github.com/apache/arrow-rs/pull/8616 • https://github.com/apache/arrow-rs/pull/8615 • https://github.com/apache/arrow-rs/pull/8613 • https://github.com/apache/arrow-rs/pull/8611 • https://github.com/apache/arrow-rs/pull/8604 • https://github.com/apache/arrow-rs/pull/8603 • https://github.com/apache/arrow-rs/pull/8599 • https://github.com/apache/arrow-rs/pull/8598 • https://github.com/apache/arrow-rs/pull/8597 • https://github.com/apache/arrow-rs/pull/8594 • https://github.com/apache/arrow-rs/pull/8592 • https://github.com/apache/arrow-rs/pull/8586 • https://github.com/apache/arrow-rs/pull/8585 • https://github.com/apache/arrow-rs/pull/8584 • https://github.com/apache/arrow-rs/pull/8582 • https://github.com/apache/arrow-rs/pull/8581 • https://github.com/apache/arrow-rs/pull/8580 • https://github.com/apache/arrow-rs/pull/8578 • https://github.com/apache/arrow-rs/pull/8577 • https://github.com/apache/arrow-rs/pull/8576 • https://github.com/apache/arrow-rs/pull/8575 • https://github.com/apache/arrow-rs/pull/8573 • https://github.com/apache/arrow-rs/pull/8563 • https://github.com/apache/arrow-rs/pull/8562 • https://github.com/apache/arrow-rs/pull/8560 • https://github.com/apache/arrow-rs/pull/8558 • https://github.com/apache/arrow-rs/pull/8557 • https://github.com/apache/arrow-rs/pull/8556 • https://github.com/apache/arrow-rs/pull/8555 • https://github.com/apache/arrow-rs/pull/8554 • https://github.com/apache/arrow-rs/pull/8552 • https://github.com/apache/arrow-rs/pull/8550 • https://github.com/apache/arrow-rs/pull/8546 • https://github.com/apache/arrow-rs/pull/8545 • https://github.com/apache/arrow-rs/pull/8544 • https://github.com/apache/arrow-rs/pull/8543 • https://github.com/apache/arrow-rs/pull/8541 • https://github.com/apache/arrow-rs/pull/8540 • https://github.com/apache/arrow-rs/pull/8536 • https://github.com/apache/arrow-rs/pull/8535 • https://github.com/apache/arrow-rs/pull/8525 • https://github.com/apache/arrow-rs/pull/8524 • https://github.com/apache/arrow-rs/pull/8522 • https://github.com/apache/arrow-rs/pull/8519 • https://github.com/apache/arrow-rs/pull/8516 • https://github.com/apache/arrow-rs/pull/8514 • https://github.com/apache/arrow-rs/pull/8512 • https://github.com/apache/arrow-rs/pull/8511 • https://github.com/apache/arrow-rs/pull/8510 • https://github.com/apache/arrow-rs/pull/8509 • https://github.com/apache/arrow-rs/pull/8508 • https://github.com/apache/arrow-rs/pull/8507 • https://github.com/apache/arrow-rs/pull/8506 • https://github.com/apache/arrow-rs/pull/8502 • https://github.com/apache/arrow-rs/pull/8501 • https://github.com/apache/arrow-rs/pull/8500 • https://github.com/apache/arrow-rs/pull/8499 • https://github.com/apache/arrow-rs/pull/8498 • https://github.com/apache/arrow-rs/pull/8494 • https://github.com/apache/arrow-rs/pull/8493 • https://github.com/apache/arrow-rs/pull/8492 • https://github.com/apache/arrow-rs/pull/8491 • https://github.com/apache/arrow-rs/pull/8490 • https://github.com/apache/arrow-rs/pull/8489 • https://github.com/apache/arrow-rs/pull/8488 • https://github.com/apache/arrow-rs/pull/8487 • https://github.com/apache/arrow-rs/pull/8486 • https://github.com/apache/arrow-rs/pull/8485 • https://github.com/apache/arrow-rs/pull/8484 • https://github.com/apache/arrow-rs/pull/8483 • https://github.com/apache/arrow-rs/pull/8482 • https://github.com/apache/arrow-rs/pull/8481 • https://github.com/apache/arrow-rs/pull/8478 • https://github.com/apache/arrow-rs/pull/8475 • https://github.com/apache/arrow-rs/pull/8473 • https://github.com/apache/arrow-rs/pull/8471 • https://github.com/apache/arrow-rs/pull/8468 • https://github.com/apache/arrow-rs/pull/8462 • https://github.com/apache/arrow-rs/pull/8459 • https://github.com/apache/arrow-rs/pull/8458 • https://github.com/apache/arrow-rs/pull/8457 • https://github.com/apache/arrow-rs/pull/8456 • https://github.com/apache/arrow-rs/pull/8455 • https://github.com/apache/arrow-rs/pull/8454 • https://github.com/apache/arrow-rs/pull/8453 • https://github.com/apache/arrow-rs/pull/8452 • https://github.com/apache/arrow-rs/pull/8450 • https://github.com/apache/arrow-rs/pull/8449 • https://github.com/apache/arrow-rs/pull/8448 • https://github.com/apache/arrow-rs/pull/8444 • https://github.com/apache/arrow-rs/pull/8438 • https://github.com/apache/arrow-rs/pull/8436 • https://github.com/apache/arrow-rs/pull/8434 • https://github.com/apache/arrow-rs/pull/8433 • https://github.com/apache/arrow-rs/pull/8432 • https://github.com/apache/arrow-rs/pull/8431 • https://github.com/apache/arrow-rs/pull/8429 • https://github.com/apache/arrow-rs/pull/8426 • https://github.com/apache/arrow-rs/pull/8421 • https://github.com/apache/arrow-rs/pull/8417 • https://github.com/apache/arrow-rs/pull/8416 • https://github.com/apache/arrow-rs/pull/8415 • https://github.com/apache/arrow-rs/pull/8414 • https://github.com/apache/arrow-rs/pull/8413 • https://github.com/apache/arrow-rs/pull/8409 • https://github.com/apache/arrow-rs/pull/8408 • https://github.com/apache/arrow-rs/pull/8402 • https://github.com/apache/arrow-rs/pull/8401 • https://github.com/apache/arrow-rs/pull/8399 • https://github.com/apache/arrow-rs/pull/8397 • https://github.com/apache/arrow-rs/pull/8395 • https://github.com/apache/arrow-rs/pull/8392 • https://github.com/apache/arrow-rs/pull/8391 • https://github.com/apache/arrow-rs/pull/8390 • https://github.com/apache/arrow-rs/pull/8385 • https://github.com/apache/arrow-rs/pull/8381 • https://github.com/apache/arrow-rs/pull/8379 • https://github.com/apache/arrow-rs/pull/8375 • https://github.com/apache/arrow-rs/pull/8371 • https://github.com/apache/arrow-rs/pull/8366 • https://github.com/apache/arrow-rs/pull/8349 • https://github.com/apache/arrow-rs/pull/8340 • https://github.com/apache/arrow-rs/pull/8282

https://api.github.com/repos/apache/arrow-rs/releases/256648414

Release info

Release version: 57.0.0

Release description This release introduces significant changes and enhancements across the Apache Arrow Rust project. Key among these are breaking changes focused on internal consistency, such as the mandatory use of Arc<FileEncryptionProperties> to align with FileDecryptionProperties, aiming to reduce memory usage and improve efficiency when encryption is enabled in Parquet operations. The project has also made strides in improving the display formatting for various DataType variants, including RunEndEncoded, Map, ListView, LargeListView, Union, and Timestamp, enhancing debugging and readability. A major enhancement is the addition of a new arrow-avro crate, providing support for Avro data formats. Furthermore, the 'thrift-remodel' initiative has refactored Thrift encryption and encoding, with initial phases of using a custom Thrift parser for Parquet metadata now complete. Performance improvements include reusing zstd compression contexts when writing IPC data and optimizing GenericByteBuilder::append_array with SIMD. The Variant data type has seen extensive development, with numerous additions for improved support in shredding, casting, and reading/writing to Parquet, alongside bug fixes addressing panics and incorrect behavior related to VariantArray.

Code Examples