create_n_tuples_dataset.py does not exist.

[yuuho]

README.md says

nbinput=12 && python create_n_tuples_dataset.py --nb_inputs nbinput

but create_n_tuples_dataset.py does not exist.

So I did the following instead:

nbinput=12 && python create_tuples_dataset.py --nb_inputs $nbinput

However, this requires tremendous memory.
Are create_n_tuples_dataset.py and create_tuples_dataset.py the same program?

[Stephlat]

Hi,
sorry for the typo. Yes, it is create_tuples_dataset.py.
I didn't experience memory issues.

[yuuho]

Thanks.

I did this using market-annoatation-train.csv ( in https://yadi.sk/d/suymftBy3S7oKD ) as said in the Deformable GAN repo.
This csv has 36 images of the first person.

Multi-source-Human-Image-Generation/create_tuples_dataset.py

Line 27 in c7a42a9

permu=list(permutations(df[df['person'] == person]['name'], n+1))

If you set nbinput to 12, the calculation will be as follows:
Permutations: nPr = 36P13 = 1.4389334903624E+19
This exceeds 1 million terabytes, I suppose.

Is the dataset used https://yadi.sk/d/suymftBy3S7oKD market-annoatation-train.csv ?

[yuuho]

I also tried nbinput=4 on a machine with 64GB RAM and 50GB swap partition HDD.
However, the program was killed after running for about 20 minutes. ( 260/745 persons were processed. 261st person has 67 images. )

[Jarvisss]

I also tried nbinput=4 on a machine with 64GB RAM and 50GB swap partition HDD.
However, the program was killed after running for about 20 minutes. ( 260/745 persons were processed. 261st person has 67 images. )

Seems it is misleading in the code.
In the paper, it is mentioned that "we initially train E, A and D with Mn = 2".
Which means they set nb_input=2 for training, which gets a much smaller training set

[Jarvisss]

I also tried nbinput=4 on a machine with 64GB RAM and 50GB swap partition HDD.
However, the program was killed after running for about 20 minutes. ( 260/745 persons were processed. 261st person has 67 images. )

Hi, I came accross the same issue, did you solve it?

[yuuho]

I rewrote the code myself, anticipating what they wanted to do.

Below is a piece of code I wrote. ( make_n_tuples function )

    random.seed(1)
    
    for person in tqdm(pd.unique(persons)):
        len_img = len(df[ df['person']==person ]['name'])
        img_names = list( df[ df['person']==person ]['name'] )
        
        if len_img < n+1:
            continue
        elif len_img == n+1:
            for i in range( len_img ):
                listFrTo['to'] += [ img_names[i] ]
                deck = list( set(range(len_img))-set([i]) )
                random.shuffle(deck)
                for idx in range(n):
                    listFrTo['from_'+str(idx)] += [ img_names[deck[idx]] ]
        else:
            for i in range( len_img ):
                for j in set(range(len_img))-set([i]):
                    listFrTo['to'] += [ img_names[i] ]
                    deck = list( set(range(len_img))-set([i,j]) )
                    random.shuffle(deck)
                    for idx in range(n):
                        listFrTo['from_'+str(idx)] += [ img_names[deck[idx]] ]

[Jarvisss]

Thank you very much!