Skip to content

Saving 3D array into .npy file for large dimension gets corrupted output #100

@sahilhassanzee

Description

@sahilhassanzee

I'm trying to put a 3D array of dimension (10000, 160, 346) in an .npy file using the append mode as shown in the code below:

#include "cnpy.h"
#include <vector>
#include <cstdlib>
constexpr int N = 346*160;

int main(int argc, char* argv[]) {
    int ROWS = 10000;

    // Create a vector
    std::vector<double> vec (N,0);

    // Fill the vector with data
    for(int i=0; i < N; i++) {
        vec[i] = i;
    }

    // Append the vector ROWS number of times into an npz file
    for (int t=0; t<ROWS; t++){
        cnpy::npy_save("sample.npy", &vec[0], {1,160,346}, "a");
    }

    return 0;
}

If I now try to open this file using the following Python code:

import numpy as np
x = np.load('sample.npy')

I get the following error:

ValueError: cannot reshape array of size 553599998 into shape (10000,160,346)

So, the size of array in the saved sample.npy is 2 Bytes smaller than what's expected. I have observed arrays upto 16 Bytes off from the expected size for larger size. Could you help resolve this issue?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions