mask and dicom both？

[MOMOANNIE]

Hello, developer, this project is very meaningful. I am also studying the segmentation of bronchopulmonary. I want to know if your project can be used for segmentation of bronchopulmonary?
If it can be used for bronchopulmonary segmentation, is it necessary to have both the mask and the original dicom data?

[LiquidFun]

Hello Anne! Yes, it segments the bronchopulmonary segments, but you need the mask for this. The original DICOM data set is not necessary. The mask is essentially a 3D numpy array where a 1 is written where there is bronchus, and a 0 elsewhere. More details in the README. Hope this helps!

[MOMOANNIE]

Hello Anne! Yes, it segments the bronchopulmonary segments, but you need the mask for this. The original DICOM data set is not necessary. The mask is essentially a 3D numpy array where a 1 is written where there is bronchus, and a 0 elsewhere. More details in the README. Hope this helps!

Brutenis, thank you very much for answering my questions in such a timely manner. I have a doubt. If I need to segment a patient's bronchopulmonary segment, do I need to correspond to the patient's template image? Still like deep learning, after training this model, I only need to input the original image of the patient to get the segmentation of bronchopulmonary.

[MOMOANNIE]

In the process of reproduction, I want to use stage-01 as the input. When I use the file airway/image_processing/save_images_as_npz.py to convert the original dicom data into a numpy array, the following problem appears. How can I solve it?

ERROR: Insufficient count of input/output paths supplied!
Expected 1 output paths and 1 input paths!
Got sys.argv: []

[LiquidFun]

You can use the script airway-pipeline.py for all the stages, like this: python3 airway-pipeline.py model. This will create stage 1 for you. But you first have to create the defaults, for this copy example_defaults.yaml to defaults.yaml in the root folder of the project and change the line local: /path/to/local_data/ to your data path. Although I highly recommend looking at the diagram in the README which shows the data structure.

The script you used expects two parameters, the input path and the output path, so you could call it with both as well.

You could also create the model with a simple script, but this of course depends on your bronchus mask format. See below for an example how this could be done. In this case all IMG* files are layers in the bronchus mask in the same folder as the script.

from pathlib import Path

import numpy as np
import pydicom

image_files = Path().cwd().glob('IMG*')  # Our data was called IMG1 to (around) IMG700
sort_ascending = sorted(image_files, key=lambda f: int(f.name[3:]))

layers = []
for image_name in sort_ascending:
    # Our DICOM image contained -10000 for empty voxels, and some value around 100 for bronchus
    im = pydicom.dcmread(image_name).pixel_array  # Read the DICOM image
    im = np.add(im, 10000)  # Shift all values to between 0 and around 11000
    im = np.clip(im, 0, 1)  # Clip all values to between 0 and 1
    layers.append(im)  # Append current layer to
model = np.stack(layers)  # Stack all layers into a single array

print(model.shape)  # Should be around 700 x 512 x 512
np.savez_compressed("model", model)  # Save it as a compressed numpy array

I agree that currently this is a bit convoluted, I'll try to simplify the process and add a tutorial for this within the next two weeks or so.

[MOMOANNIE]

You can use the script airway-pipeline.py for all the stages, like this: python3 airway-pipeline.py model. This will create stage 1 for you. But you first have to create the defaults, for this copy example_defaults.yaml to defaults.yaml in the root folder of the project and change the line local: /path/to/local_data/ to your data path. Although I highly recommend looking at the diagram in the README which shows the data structure.

The script you used expects two parameters, the input path and the output path, so you could call it with both as well.

You could also create the model with a simple script, but this of course depends on your bronchus mask format. See below for an example how this could be done. In this case all IMG* files are layers in the bronchus mask in the same folder as the script.

from pathlib import Path

import numpy as np
import pydicom

image_files = Path().cwd().glob('IMG*')  # Our data was called IMG1 to (around) IMG700
sort_ascending = sorted(image_files, key=lambda f: int(f.name[3:]))

layers = []
for image_name in sort_ascending:
    # Our DICOM image contained -10000 for empty voxels, and some value around 100 for bronchus
    im = pydicom.dcmread(image_name).pixel_array  # Read the DICOM image
    im = np.add(im, 10000)  # Shift all values to between 0 and around 11000
    im = np.clip(im, 0, 1)  # Clip all values to between 0 and 1
    layers.append(im)  # Append current layer to
model = np.stack(layers)  # Stack all layers into a single array

print(model.shape)  # Should be around 700 x 512 x 512
np.savez_compressed("model", model)  # Save it as a compressed numpy array

I agree that currently this is a bit convoluted, I'll try to simplify the process and add a tutorial for this within the next two weeks or so.

Thank you very much for your patient answers, my stage-01 has run through, and I am currently debugging stage-02.

I read the readme of the project again, and also read other people’s questions. I need to confirm two points with you:

First: If my mask does not segment the lung lobes, bronchus, arteries, and veins like the mask in your project, but only extracts the bronchus (that is, the bronchus is 1 and other places are 0), Can I use my mask to divide the bronchus into 18 segments?

Second: local: /path/to/local_data/ is the path where my mask is stored, not my original dicom path, isn’t it? (You said that the original dicom data is not necessary, the mask is necessary)

[MOMOANNIE]

Hello, developer, I put the bronchial mask (bronchial area is 1, and the rest is 0) into the stage-01 folder. When stage-02 is run, it passes successfully, but when stage-04 is run, an error message appears. Is the error reported because my slice thickness is not uniform?

You mentioned in the readme that the slicer thickness in all directions must be the same. I am confused about the slice thickness in all directions. Is there any difference between the slice thickness and the spacing? I don’t know how to unify the slice thickness in all directions

[LiquidFun]

First: If my mask does not segment the lung lobes, bronchus, arteries, and veins like the mask in your project, but only extracts the bronchus (that is, the bronchus is 1 and other places are 0), Can I use my mask to divide the bronchus into 18 segments?

Yes, I just tested it with only the bronchus mask and it worked well (encountered a bug in stage 60, but this shouldn't be a problem for now. I'll fix it soon though).

Second: local: /path/to/local_data/ is the path where my mask is stored, not my original dicom path, isn’t it? (You said that the original dicom data is not necessary, the mask is necessary)

Yes, this is the directory where your compressed mask is stored, and looks like this:

    ├── stage-01                    🠔 Each stage now has the same basic format
    │   ├── 3123156
    │   │   └── model.npz
    │   ├── 3123193
    │   │   └── model.npz
    │   └── ...
    ├── stage-02                    🠔 Each stage from here on will be created by the pipeline itself
    │   ├── 3123156                    so you do not need to handle this, each of them have different
    │   │   └── reduced_model.npz
    │   └── ...                   

And no DICOM data is needed at this point, so it should work.

Hello, developer, I put the bronchial mask (bronchial area is 1, and the rest is 0) into the stage-01 folder. When stage-02 is run, it passes successfully, but when stage-04 is run, an error message appears. Is the error reported because my slice thickness is not uniform?

That's unfortunate, could you run it with the -v flag and post the both STDERR and STDOUT output here? Like this python airway-pipeline.py 4 -v. The slice thickness should have no impact for now, as once you convert it to a numpy array it doesn't know that it's not uniform anymore. It might only impact the final results in the classification (stage-11), but the previous stages should still work.

Hope it works for you! Excited to see it work on someone else's data!

[MOMOANNIE]

That's unfortunate, could you run it with the -v flag and post the both STDERR and STDOUT output here? Like this python airway-pipeline.py 4 -v. The slice thickness should have no impact for now, as once you convert it to a numpy array it doesn't know that it's not uniform anymore. It might only impact the final results in the classification (stage-11), but the previous stages should still work.

I just re-run the program after unifying the slice thicknesses, and I still get an error at stage-04. I tried to use python airway-pipeline.py 4 -v, but it didn’t print STDERR and STDOUT. I don’t know which stage cause the error and how to debug

[MOMOANNIE]

I printed out my compressed mask array, and the result showed that the value of most positions is "false", a few positions are "true", not "0" and "1". Is it the data type in the array that caused stage-04 to report an error?

[LiquidFun]

I printed out my compressed mask array, and the result showed that the value of most positions is "false", a few positions are "true", not "0" and "1". Is it the data type in the array that caused stage-04 to report an error?

True and False should work, I just tested it. I think numpy doesn't really differentiate between True/False and 1/0.

Are you sure there is a mistake if it doesn't print an error? Because otherwise some things will definitely be printed.

If your data isn't private, I could take a look at it if you can send me the model.npz (bronchus mask only would be enough).

Also, I've pushed some updates for the later stages.

[MOMOANNIE]

I printed out my compressed mask array, and the result showed that the value of most positions is "false", a few positions are "true", not "0" and "1". Is it the data type in the array that caused stage-04 to report an error?

True and False should work, I just tested it. I think numpy doesn't really differentiate between True/False and 1/0.

Are you sure there is a mistake if it doesn't print an error? Because otherwise some things will definitely be printed.

If your data isn't private, I could take a look at it if you can send me the model.npz (bronchus mask only would be enough).

Also, I've pushed some updates for the later stages.

Can you see the pictures I uploaded? The picture shows that an error occurred when processing stage-04.
How do I send the data to you? It seems that the data format of npz cannot be uploaded on github

[MOMOANNIE]

I printed out my compressed mask array, and the result showed that the value of most positions is "false", a few positions are "true", not "0" and "1". Is it the data type in the array that caused stage-04 to report an error?

True and False should work, I just tested it. I think numpy doesn't really differentiate between True/False and 1/0.

Are you sure there is a mistake if it doesn't print an error? Because otherwise some things will definitely be printed.

If your data isn't private, I could take a look at it if you can send me the model.npz (bronchus mask only would be enough).

Also, I've pushed some updates for the later stages.
model.zip

I put the model.npz file into a folder, compressed it into a zip format and uploaded it, you can see if it can be decompressed normally, and use

[LiquidFun]

I put the model.npz file into a folder, compressed it into a zip format and uploaded it, you can see if it can be decompressed normally, and use

The zip seems to be empty, it only contains a folder. You could send me an E-Mail to brutenis [at] gmail.com with the model as well.

[MOMOANNIE]

I put the model.npz file into a folder, compressed it into a zip format and uploaded it, you can see if it can be decompressed normally, and use

The zip seems to be empty, it only contains a folder. You could send me an E-Mail to brutenis [at] gmail.com with the model as well.

Ok, I have just sent the model.npz data to you via gdutchu@outlook.com, please check it, thank you very much

[LiquidFun]

It works for me without errors (even stage-04). I've included a rendered version of your model:

Although the 0th axis was flipped (i.e. it was upside-down), I flipped it like this:

import numpy as np

model = np.load("original_model.npz")['arr_0']
model = model.astype(np.uint8)
model = np.flip(model, axis=0)
np.savez_compressed("model.npz", model)

No segments were detected because the bronchus mask currently doesn't have enough detail, it does contain a few splits though, so that works.

For the error, maybe you have more folders than necessary? Make sure that each stage folder contains patient folders which contain the models, if there are other folders delete them. And can you try again with the -v, for me it does show what the error message is exactly.