$ arrow-json_

Crate

Name: arrow-json New version: 57.0.0 Release date: 2025-10-23T11:00:57Z

Crate readme

Short description Native Rust implementation of Apache Arrow and Apache Parquet

Long description This repository contains Rust implementations for the Apache Arrow in-memory columnar format and the Apache Parquet columnar file format. It includes core functionality for memory layout, arrays, and low-level computations in the arrow crate, support for the Arrow-Flight IPC protocol in arrow-flight, and support for the Parquet file format in the parquet crate. Additionally, the parquet_derive crate allows deriving RecordWriter/RecordReader for simple Rust structs. The project follows Semantic Versioning and releases approximately monthly, with major versions released at most quarterly. The minimum supported Rust version (MSRV) is updated on a rolling basis, ensuring it is at least six months old and is only updated in major releases.

Features • Arrow in-memory columnar format • Parquet columnar file format • Arrow-Flight IPC protocol • Derive RecordWriter/RecordReader • Monthly releases • Rolling MSRV policy • Apache 2.0 License

Code Examples Core functionality

 [`arrow`]: https://crates.io/crates/arrow

Arrow-Flight IPC protocol

 [`arrow-flight`]: https://crates.io/crates/arrow-flight

Parquet columnar file format

 [`parquet`]: https://crates.io/crates/parquet

Derive RecordWriter/RecordReader

 [`parquet_derive`]: https://crates.io/crates/parquet-derive

Object Storage interface

 [`object_store`]: https://crates.io/crates/object-store

Links • https://arrow.apache.org/ • https://crates.io/crates/arrow • https://crates.io/crates/parquet • https://crates.io/crates/parquet-derive • https://crates.io/crates/arrow-flight • https://github.com/apache/arrow-rs-object-store • https://docs.rs/arrow/latest • https://docs.rs/arrow-flight/latest • https://docs.rs/parquet/latest • https://docs.rs/parquet-derive/latest • https://arrow.apache.org/rust • https://semver.org/ • https://github.com/apache/arrow-rs/issues/7835 • https://github.com/apache/arrow-rs/milestone/3 • https://github.com/apache/arrow-rs/milestone/5 • https://github.com/apache/arrow-rs/milestone/6 • https://github.com/apache/arrow-rs/issues/5368 • https://github.com/apache/arrow-rs/blob/main/CONTRIBUTING.md#breaking-changes • https://github.com/apache/arrow-rs/issues/6737 • https://crates.io/crates/datafusion • https://crates.io/crates/ballista • https://crates.io/crates/parquet_opendal • https://github.com/apache/opendal/blob/main/integrations/parquet/README.md • https://github.com/apache/arrow-rs-object-store/blob/main/README.md • https://github.com/apache/datafusion/blob/main/README.md • https://github.com/apache/datafusion-ballista/blob/main/README.md • https://arrow.apache.org/community/ • https://s.apache.org/slack-invite • https://github.com/apache/arrow-rs/discussions • https://discord.gg/YAb2TdazKQ • https://github.com/apache/arrow-rs/issues • https://www.rust-lang.org/ • https://crates.io/crates/object-store • arrow/README.md • CONTRIBUTING.md • parquet/README.md • arrow-flight/README.md • https://github.com/apache/arrow-rs/tree/57.0.0 • https://github.com/apache/arrow-rs/compare/56.2.0...57.0.0 • https://github.com/apache/arrow-rs/pull/8626 • https://github.com/apache/arrow-rs/pull/8596 • https://github.com/apache/arrow-rs/pull/8595 • https://github.com/apache/arrow-rs/pull/8587 • https://github.com/apache/arrow-rs/pull/8570 • https://github.com/apache/arrow-rs/pull/8569 • https://github.com/apache/arrow-rs/pull/8530 • https://github.com/apache/arrow-rs/pull/8529 • https://github.com/apache/arrow-rs/pull/8470 • https://github.com/apache/arrow-rs/pull/8447 • https://github.com/apache/arrow-rs/pull/8425 • https://github.com/apache/arrow-rs/pull/8424 • https://github.com/apache/arrow-rs/pull/8405 • https://github.com/apache/arrow-rs/pull/8396 • https://github.com/apache/arrow-rs/pull/8291 • https://github.com/apache/arrow-rs/pull/8290 • https://github.com/apache/arrow-rs/pull/8286 • https://github.com/apache/arrow-rs/issues/4886 • https://github.com/apache/arrow-rs/issues/8639 • https://github.com/apache/arrow-rs/issues/8614 • https://github.com/apache/arrow-rs/issues/8612 • https://github.com/apache/arrow-rs/issues/8610 • https://github.com/apache/arrow-rs/issues/8609 • https://github.com/apache/arrow-rs/issues/8606 • https://github.com/apache/arrow-rs/issues/8591 • https://github.com/apache/arrow-rs/issues/8559 • https://github.com/apache/arrow-rs/issues/8531 • https://github.com/apache/arrow-rs/issues/8523 • https://github.com/apache/arrow-rs/issues/8517 • https://github.com/apache/arrow-rs/issues/8515 • https://github.com/apache/arrow-rs/issues/8513 • https://github.com/apache/arrow-rs/issues/8503 • https://github.com/apache/arrow-rs/issues/8497 • https://github.com/apache/arrow-rs/issues/8469 • https://github.com/apache/arrow-rs/issues/8461 • https://github.com/apache/arrow-rs/issues/8451 • https://github.com/apache/arrow-rs/issues/8439 • https://github.com/apache/arrow-rs/issues/8412 • https://github.com/apache/arrow-rs/issues/8411 • https://github.com/apache/arrow-rs/issues/8398 • https://github.com/apache/arrow-rs/issues/8389 • https://github.com/apache/arrow-rs/issues/8386 • https://github.com/apache/arrow-rs/issues/8370 • https://github.com/apache/arrow-rs/issues/8361 • https://github.com/apache/arrow-rs/issues/8347 • https://github.com/apache/arrow-rs/issues/8337 • https://github.com/apache/arrow-rs/issues/8336 • https://github.com/apache/arrow-rs/issues/8334 • https://github.com/apache/arrow-rs/issues/8332 • https://github.com/apache/arrow-rs/issues/8331 • https://github.com/apache/arrow-rs/issues/8330 • https://github.com/apache/arrow-rs/issues/8303 • https://github.com/apache/arrow-rs/issues/8285 • https://github.com/apache/arrow-rs/issues/8281 • https://github.com/apache/arrow-rs/issues/8631 • https://github.com/apache/arrow-rs/issues/8579 • https://github.com/apache/arrow-rs/issues/8542 • https://github.com/apache/arrow-rs/issues/8539 • https://github.com/apache/arrow-rs/issues/8534 • https://github.com/apache/arrow-rs/issues/8533 • https://github.com/apache/arrow-rs/issues/8526 • https://github.com/apache/arrow-rs/issues/8505 • https://github.com/apache/arrow-rs/issues/8443 • https://github.com/apache/arrow-rs/issues/8435 • https://github.com/apache/arrow-rs/issues/8406 • https://github.com/apache/arrow-rs/issues/8404 • https://github.com/apache/arrow-rs/issues/8403 • https://github.com/apache/arrow-rs/issues/8382 • https://github.com/apache/arrow-rs/issues/8319 • https://github.com/apache/arrow-rs/issues/8296 • https://github.com/apache/arrow-rs/pull/8383 • https://github.com/apache/arrow-rs/issues/8588 • https://github.com/apache/arrow-rs/pull/8654 • https://github.com/apache/arrow-rs/pull/8583 • https://github.com/apache/arrow-rs/pull/8574 • https://github.com/apache/arrow-rs/pull/8527 • https://github.com/apache/arrow-rs/pull/8388 • https://github.com/apache/arrow-rs/issues/8601 • https://github.com/apache/arrow-rs/issues/8538 • https://github.com/apache/arrow-rs/issues/8504 • https://github.com/apache/arrow-rs/issues/8418 • https://github.com/apache/arrow-rs/issues/8410 • https://github.com/apache/arrow-rs/issues/8380 • https://github.com/apache/arrow-rs/issues/8377 • https://github.com/apache/arrow-rs/issues/8374 • https://github.com/apache/arrow-rs/pull/8642 • https://github.com/apache/arrow-rs/pull/8636 • https://github.com/apache/arrow-rs/pull/8634 • https://github.com/apache/arrow-rs/pull/8633 • https://github.com/apache/arrow-rs/pull/8629 • https://github.com/apache/arrow-rs/pull/8627 • https://github.com/apache/arrow-rs/pull/8625 • https://github.com/apache/arrow-rs/pull/8621 • https://github.com/apache/arrow-rs/pull/8620 • https://github.com/apache/arrow-rs/pull/8617 • https://github.com/apache/arrow-rs/pull/8616 • https://github.com/apache/arrow-rs/pull/8615 • https://github.com/apache/arrow-rs/pull/8613 • https://github.com/apache/arrow-rs/pull/8611 • https://github.com/apache/arrow-rs/pull/8604 • https://github.com/apache/arrow-rs/pull/8603 • https://github.com/apache/arrow-rs/pull/8599 • https://github.com/apache/arrow-rs/pull/8598 • https://github.com/apache/arrow-rs/pull/8597 • https://github.com/apache/arrow-rs/pull/8594 • https://github.com/apache/arrow-rs/pull/8592 • https://github.com/apache/arrow-rs/pull/8586 • https://github.com/apache/arrow-rs/pull/8585 • https://github.com/apache/arrow-rs/pull/8584 • https://github.com/apache/arrow-rs/pull/8582 • https://github.com/apache/arrow-rs/pull/8581 • https://github.com/apache/arrow-rs/pull/8580 • https://github.com/apache/arrow-rs/pull/8578 • https://github.com/apache/arrow-rs/pull/8577 • https://github.com/apache/arrow-rs/pull/8576 • https://github.com/apache/arrow-rs/pull/8575 • https://github.com/apache/arrow-rs/pull/8573 • https://github.com/apache/arrow-rs/pull/8563 • https://github.com/apache/arrow-rs/pull/8562 • https://github.com/apache/arrow-rs/pull/8560 • https://github.com/apache/arrow-rs/pull/8558 • https://github.com/apache/arrow-rs/pull/8557 • https://github.com/apache/arrow-rs/pull/8556 • https://github.com/apache/arrow-rs/pull/8555 • https://github.com/apache/arrow-rs/pull/8554 • https://github.com/apache/arrow-rs/pull/8552 • https://github.com/apache/arrow-rs/pull/8550 • https://github.com/apache/arrow-rs/pull/8546 • https://github.com/apache/arrow-rs/pull/8545 • https://github.com/apache/arrow-rs/pull/8544 • https://github.com/apache/arrow-rs/pull/8543 • https://github.com/apache/arrow-rs/pull/8541 • https://github.com/apache/arrow-rs/pull/8540 • https://github.com/apache/arrow-rs/pull/8536 • https://github.com/apache/arrow-rs/pull/8535 • https://github.com/apache/arrow-rs/pull/8525 • https://github.com/apache/arrow-rs/pull/8524 • https://github.com/apache/arrow-rs/pull/8522 • https://github.com/apache/arrow-rs/pull/8519 • https://github.com/apache/arrow-rs/pull/8516 • https://github.com/apache/arrow-rs/pull/8514 • https://github.com/apache/arrow-rs/pull/8512 • https://github.com/apache/arrow-rs/pull/8511 • https://github.com/apache/arrow-rs/pull/8510 • https://github.com/apache/arrow-rs/pull/8509 • https://github.com/apache/arrow-rs/pull/8508 • https://github.com/apache/arrow-rs/pull/8507 • https://github.com/apache/arrow-rs/pull/8506 • https://github.com/apache/arrow-rs/pull/8502 • https://github.com/apache/arrow-rs/pull/8501 • https://github.com/apache/arrow-rs/pull/8500 • https://github.com/apache/arrow-rs/pull/8499 • https://github.com/apache/arrow-rs/pull/8498 • https://github.com/apache/arrow-rs/pull/8494 • https://github.com/apache/arrow-rs/pull/8493 • https://github.com/apache/arrow-rs/pull/8492 • https://github.com/apache/arrow-rs/pull/8491 • https://github.com/apache/arrow-rs/pull/8490 • https://github.com/apache/arrow-rs/pull/8489 • https://github.com/apache/arrow-rs/pull/8488 • https://github.com/apache/arrow-rs/pull/8487 • https://github.com/apache/arrow-rs/pull/8486 • https://github.com/apache/arrow-rs/pull/8485 • https://github.com/apache/arrow-rs/pull/8484 • https://github.com/apache/arrow-rs/pull/8483 • https://github.com/apache/arrow-rs/pull/8482 • https://github.com/apache/arrow-rs/pull/8481 • https://github.com/apache/arrow-rs/pull/8478 • https://github.com/apache/arrow-rs/pull/8475 • https://github.com/apache/arrow-rs/pull/8473 • https://github.com/apache/arrow-rs/pull/8471 • https://github.com/apache/arrow-rs/pull/8468 • https://github.com/apache/arrow-rs/pull/8462 • https://github.com/apache/arrow-rs/pull/8459 • https://github.com/apache/arrow-rs/pull/8458 • https://github.com/apache/arrow-rs/pull/8457 • https://github.com/apache/arrow-rs/pull/8456 • https://github.com/apache/arrow-rs/pull/8455 • https://github.com/apache/arrow-rs/pull/8454 • https://github.com/apache/arrow-rs/pull/8453 • https://github.com/apache/arrow-rs/pull/8452 • https://github.com/apache/arrow-rs/pull/8450 • https://github.com/apache/arrow-rs/pull/8449 • https://github.com/apache/arrow-rs/pull/8448 • https://github.com/apache/arrow-rs/pull/8444 • https://github.com/apache/arrow-rs/pull/8438 • https://github.com/apache/arrow-rs/pull/8436 • https://github.com/apache/arrow-rs/pull/8434 • https://github.com/apache/arrow-rs/pull/8433 • https://github.com/apache/arrow-rs/pull/8432 • https://github.com/apache/arrow-rs/pull/8431 • https://github.com/apache/arrow-rs/pull/8429 • https://github.com/apache/arrow-rs/pull/8426 • https://github.com/apache/arrow-rs/pull/8421 • https://github.com/apache/arrow-rs/pull/8417 • https://github.com/apache/arrow-rs/pull/8416 • https://github.com/apache/arrow-rs/pull/8415 • https://github.com/apache/arrow-rs/pull/8414 • https://github.com/apache/arrow-rs/pull/8413 • https://github.com/apache/arrow-rs/pull/8409 • https://github.com/apache/arrow-rs/pull/8408 • https://github.com/apache/arrow-rs/pull/8402 • https://github.com/apache/arrow-rs/pull/8401 • https://github.com/apache/arrow-rs/pull/8399 • https://github.com/apache/arrow-rs/pull/8397 • https://github.com/apache/arrow-rs/pull/8395 • https://github.com/apache/arrow-rs/pull/8392 • https://github.com/apache/arrow-rs/pull/8391 • https://github.com/apache/arrow-rs/pull/8390 • https://github.com/apache/arrow-rs/pull/8385 • https://github.com/apache/arrow-rs/pull/8381 • https://github.com/apache/arrow-rs/pull/8379 • https://github.com/apache/arrow-rs/pull/8375 • https://github.com/apache/arrow-rs/pull/8371 • https://github.com/apache/arrow-rs/pull/8366 • https://github.com/apache/arrow-rs/pull/8349 • https://github.com/apache/arrow-rs/pull/8340 • https://github.com/apache/arrow-rs/pull/8282

https://api.github.com/repos/apache/arrow-rs/releases/256648414

Release info

Release version: 57.0.0

Release description This release introduces several breaking changes and enhancements across the Apache Arrow Rust project. Notably, Arc<FileEncryptionProperties> is now used consistently for file encryption properties, aiming to reduce the size of ParquetMetadata and avoid copying when encryption is enabled. The arrow-avro crate has been added, providing support for Avro data formats, along with new ArrowError::AvroError and roundtrip tests.

Significant refactoring has occurred within the Parquet module, including a 'Thrift remodel' that refactors encryption and stores encodings as a bitmask, along with the use of a custom Thrift parser for Parquet metadata. Display formatting for various DataType variants, such as RunEndEncoded, Map, ListView, LargeListView, Union, and Timestamp, has been improved for better readability.

Performance improvements include reusing the zstd compression context when writing IPC data and optimizing convert_row_groups for Thrift remodeling. Bug fixes address issues with Decimal-to-Decimal casting, Parquet writer flush operations, and memory consumption for compressed Parquet writers.

The release also includes the implementation of GeospatialStatistics creation for Parquet logical types and expands support for the Variant LogicalType in Parquet, enhancing data handling capabilities.

Code Examples