Error when building index from UD data

[bbunzeck]

Hi, I am planning to use decaf for a study, but right now I am experiencing some issues when trying to use it.

Here's what I've done so far:

• created a clean virtual environment (Python 3.12)
• installed decaf via pip install decaffinate[full]
• got some UD-annotated files in .conllu format (specifically the data from https://github.com/UniversalDependencies/UD_English-CHILDES)
• cloned the repo, as indexing seems to depend on this import script

Now I am trying to build an index from the data with the following command (while inside the decaf repo):
python3 scripts/import/ud.py --input /Users/bbunzeck/Downloads/UD_English-CHILDES-master/en_childes-ud-train.conllu --output /Users/bbunzeck/Downloads/decaf-test

I get the following output:

(decaf-test) bbunzeck@ULLW-JP2W13VMC6 decaf-main % python3 scripts/import/ud.py --input /Users/bbunzeck/Downloads/UD_English-CHILDES-master/en_childes-ud-train.conllu --output /Users/bbunzeck/Downloads/decaf-test
=============
📥️ UD Import
=============
Connected to DECAF index at '/Users/bbunzeck/Downloads/decaf-test':
<DecafIndex: 'decaf-test', 0 shard(s)>
Loading UD treebank from '/Users/bbunzeck/Downloads/UD_English-CHILDES-master/en[0/0 | ···] Building index with 0 shard(s)...Traceback (most recent call last):
  File "/Users/bbunzeck/Downloads/decaf-main/scripts/import/ud.py", line 184, in <module>
    main()
  File "/Users/bbunzeck/Downloads/decaf-main/scripts/import/ud.py", line 133, in main
    decaf_index.add_shard()
  File "/Users/bbunzeck/miniconda3/envs/decaf-test/lib/python3.12/site-packages/decaf/index/index.py", line 120, in add_shard
    with resources.open_text('decaf.config', 'schema.sql') as fp:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bbunzeck/miniconda3/envs/decaf-test/lib/python3.12/importlib/resources/_legacy.py", line 44, in open_text
    return (_common.files(package) / normalize_path(resource)).open(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bbunzeck/miniconda3/envs/decaf-test/lib/python3.12/pathlib.py", line 1013, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/Users/bbunzeck/miniconda3/envs/decaf-test/lib/python3.12/site-packages/decaf/config/schema.sql'

I feel like I am missing a step here. I already tried troubleshooting by editing the scripts to point to the schema.sql file in the repo, but I get further, different errors from that. Any help/hint is greatly appreciated! Many thanks in advance! 🙏

[personads]

Hi @bbunzeck,

Thanks a lot for your interest in our work, and for providing a detailed error description!

I was able to reproduce your issue and have pushed DECAF v0.2.1 to address it:
https://pypi.org/project/decaffinate/0.2.1/

Using the updated version, I was able to import the CHILDES UD data as follows:

# set up virtualenv
mkdir decaf-test && cd decaf-test
python3 -m venv venv
source venv/bin/activate
pip install decaffeinate[full]

# clone repo with import scripts
git clone https://github.com/personads/decaf.git decaf-scripts

# clone CHILDES dataset
git clone https://github.com/UniversalDependencies/UD_English-CHILDES.git childes

# create index
mkdir indices
python decaf-scripts/scripts/import/ud.py --input childes/en_childes-ud-train.conllu --output indices/childes

With default settings, this should yield an index with 4 shards:

=============
📥️ UD Import
=============
Connected to DECAF index at 'indices/childes':
<DecafIndex: 'childes', 0 shard(s)>
Built index with 4 shard(s) containing 476792 literals and 1976350 structures with 2348946 hierarchical relations for 34732 sentences from 'childes/en_childes-ud-train.conllu' in 5.30s.
Saved updated DECAF index to 'indices/childes'.

Could you let me know if this works for you? Thanks for your patience!