Skip to content

Fletcher checksum failure when reading netcdf-4 file. #16

@harel

Description

@harel

Hi!

I'm trying to read a .nc file, which is identified as NetCDF-4. I can read it in R, h5dump etc. The file is valid.
The data is grouped, and it does contain Fletcher32 checksum. The dataset i'm interested in is in the /PRODUCT group.

> h5dump -pH filename.nc | grep -A 10 'latitude'

DATASET "latitude" {
         DATATYPE  H5T_IEEE_F32LE
         DATASPACE  SIMPLE { ( 1, 373, 450 ) / ( 1, 373, 450 ) }
         STORAGE_LAYOUT {
            CHUNKED ( 1, 373, 450 )
            SIZE 300032 (2.238:1 COMPRESSION)
         }
         FILTERS {
            CHECKSUM FLETCHER32
            PREPROCESSING SHUFFLE
            COMPRESSION DEFLATE { LEVEL 3 }

Here is a sample code I'm using to try to read the variables off the file.

ncf, err := netcdf.Open(path)
if err != nil {
	panic(err)
}
defer ncf.Close()

grps := ncf.ListSubgroups()
fmt.Println("Groups are", grps)  // lists the groups

nc, _ := ncf.GetGroup("/PRODUCT")

vars := nc.ListVariables()
fmt.Println("Vars are", vars) // lists valid vars

vr, err := nc.GetVariable("latitude")
fmt.Println("LATITUDE IS ", vr, err)   

Here it fails with "fletcher checksum failure". I've tried different ways to access the variable but could not progress beyond that.

I can go around it by cloning the repo, replacing it in my go.mod to use the local copy and commenting out the actual checksum checks, but it feels like a hacky thing to do.

You can find a sample file that exhibits this issue here

Thank you

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions