Hdf5JavaLib

Hdf5JavaLib: Read Capabilities (Version 0.1.1)

Hdf5JavaLib is a pure Java library for reading HDF5 files, released as version 0.1.1. This guide helps users read datasets in the root group using examples from the org.hdf5javalib.examples.read package. The library reads HDF5 files generated by the C++ HDF5 library, supporting compound datasets, multi-dimensional data, and various datatypes.

Setup

Add the Dependency

Add Hdf5JavaLib to your project via Maven Central:

<dependency>
    <groupId>org.hdf5javalib</groupId>
    <artifactId>hdf5javalib</artifactId>
    <version>0.1.1</version>
</dependency>

Requirements

• Java 17 or higher.
• No external HDF5 dependencies.

Alternative: Build from Source

1. Clone the repository:

   git clone https://github.com/karlnicholas/Hdf5JavaLib.git
   

2. Build with Maven:

   cd Hdf5JavaLib
   mvn install
   

HDF5 Files

• Example HDF5 files (e.g., compound_example.h5, twenty_datasets.h5, ascii_dataset.h5) are used in the examples below.
• When running examples from source, these files are in src/test/resources.
• When using the Maven Central dependency, provide your own HDF5 files and load them using ResourceLoader or a SeekableByteChannel.

Read Capabilities

Hdf5JavaLib supports reading datasets in the root group (/), including:

• Compound Datasets: Structured data with fields like integers, strings, and floats (e.g., CompoundData with recordId, fixedStr).
• Scalar Datasets: Single values (e.g., fixed-point, floating-point, strings).
• Vector Datasets: 1D arrays (e.g., strings, integers).
• Matrix Datasets: 2D arrays (e.g., weather data).
• Multi-Dimensional Datasets: Tested with arrays up to 4D (e.g., Tic-Tac-Toe game state).
• Variable-Length Datasets: Arrays of varying sizes.
• Datatypes: Fixed-point (BigInteger, BigDecimal), floating-point (Float, Double), strings (ASCII, UTF-8), time, bitfields, compounds, opaque, references, enums, arrays.
• Metadata: Partial support for SNODs (symbol table nodes).

The library supports sequential and parallel streaming for efficient data processing.

Usage

Use the HdfFileReader class (in org.hdf5javalib.file) to read HDF5 files. The TypedDataSource class provides typed data access (scalar, vector, matrix) and streaming, while HdfDisplayUtils simplifies data display. The org.hdf5javalib.examples.read package shows how to read different datasets.

Example 1: Read a Compound Dataset

This example reads a compound dataset (CompoundData) from an HDF5 file, with fields like recordId (Long) and fixedStr (String), and maps it to a custom Java class.

package org.hdf5javalib.examples.read;

import org.hdf5javalib.HdfFileReader;
import org.hdf5javalib.dataclass.HdfCompound;
import org.hdf5javalib.datasource.TypedDataSource;
import org.hdf5javalib.examples.ResourceLoader;
import org.hdf5javalib.file.HdfDataSet;

import java.nio.channels.SeekableByteChannel;

public class HdfCompoundRead {
    public static void main(String[] args) throws Exception {
        try (SeekableByteChannel channel = ResourceLoader.loadResourceAsChannel("compound_example.h5")) {
            HdfFileReader reader = new HdfFileReader(channel).readFile();
            try (HdfDataSet dataSet = reader.getRootGroup().findDataset("CompoundData")) {
                new TypedDataSource<>(channel, reader, dataSet, HdfCompound.class)
                    .streamVector()
                    .limit(2)
                    .forEach(c -> System.out.println("Row: " + c.getMembers()));
            }
        }
    }
}

Output (example):

Row: [recordId=1000, fixedStr="Fixed0", varStr="Variable0", ...]
Row: [recordId=1001, fixedStr="Fixed1", varStr="Variable1", ...]

Note: If using your own HDF5 file, replace "compound_example.h5" with the path to your file relative to the classpath or use a SeekableByteChannel from a file path.

Example 2: Read Scalar Datasets

This example reads scalar datasets from an HDF5 file, displaying values as Long.

package org.hdf5javalib.examples.read;

import org.hdf5javalib.HdfFileReader;
import org.hdf5javalib.examples.ResourceLoader;
import org.hdf5javalib.file.HdfDataSet;
import org.hdf5javalib.utils.HdfDisplayUtils;

import java.nio.channels.SeekableByteChannel;

public class HdfTwentyScalarRead {
    public static void main(String[] args) throws Exception {
        try (SeekableByteChannel channel = ResourceLoader.loadResourceAsChannel("twenty_datasets.h5")) {
            HdfFileReader reader = new HdfFileReader(channel).readFile();
            for (HdfDataSet dataSet : reader.getRootGroup().getDataSets()) {
                try (HdfDataSet ds = dataSet) {
                    HdfDisplayUtils.displayScalarData(channel, ds, Long.class, reader);
                }
            }
        }
    }
}

Output (example):

Dataset0: 123
Dataset1: 456
...

Example 3: Read a String Vector

This example reads a vector of ASCII strings from an HDF5 file.

package org.hdf5javalib.examples.read;

import org.hdf5javalib.HdfFileReader;
import org.hdf5javalib.examples.ResourceLoader;
import org.hdf5javalib.file.HdfDataSet;
import org.hdf5javalib.utils.HdfDisplayUtils;

import java.nio.channels.SeekableByteChannel;

public class HdfStringRead {
    public static void main(String[] args) throws Exception {
        try (SeekableByteChannel channel = ResourceLoader.loadResourceAsChannel("ascii_dataset.h5")) {
            HdfFileReader reader = new HdfFileReader(channel).readFile();
            try (HdfDataSet dataSet = reader.getRootGroup().findDataset("strings")) {
                HdfDisplayUtils.displayVectorData(channel, dataSet, String.class, reader);
            }
        }
    }
}

Output (example):

["Hello", "World", "HDF5"]

Running Examples

1. Using Maven Central:
   • Add the dependency to your pom.xml.
   • Place your HDF5 files in src/main/resources (or another classpath-accessible location).
   • Compile and run:

     mvn compile
     java -cp target/classes org.hdf5javalib.examples.read.HdfCompoundRead
     

2. From Source:
   • Example HDF5 files are in src/test/resources.
   • Run tests to verify:

     mvn test
     

   • Run examples:

     mvn compile
     java -cp target/classes org.hdf5javalib.examples.read.HdfCompoundRead
     

3. Tips:
   • Use Java 17+.
   • Ensure HDF5 files are accessible via the classpath or file system.
   • For custom HDF5 files, use ResourceLoader or create a SeekableByteChannel from a file path.

Limitations

• Version 0.1.1: Reads datasets in the root group only (/).
• No Subgroups: Groups beyond the root group are not supported.
• No Compression or Chunking: Cannot read compressed or chunked datasets.
• Partial Metadata Support: Reads SNODs but not attributes or advanced structures (e.g., fractal heaps).
• Tested Scope: Validated with C++ HDF5 files included in the source repository.

Feedback

Help improve Hdf5JavaLib by reporting issues at GitHub Issues. Please include:

• Sample HDF5 files that fail to read.
• Expected outputs or error messages.
• Suggestions for new features (e.g., subgroup support, chunking).

Visit https://www.hdf5javalib.org for updates and resources.