Switch similarity search to different library

[deepfates]

Annoy takes a long time to build, maybe there's something else out there that doesn't

[igorbrigadir]

Curious to see what you end up with here. I had some good benchmarks from https://github.com/nmslib/hnswlib

[deepfates]

This looks quite useful, thank you! It can do CRUD ops on individual elements in the index which is desirable. It does look like it needs to know the maximum possible number of elements before creating the index, but maybe we can work within that constraint..

…

On Mon, Mar 21, 2022, 12:52 PM Igor Brigadir ***@***.***> wrote: Curious to see what you end up with here. I had some good benchmarks from https://github.com/nmslib/hnswlib — Reply to this email directly, view it on GitHub <#30 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN7DJVCVTQH5XKWWNSYIGQ3VBDANFANCNFSM5RFYAOCA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[igorbrigadir]

Yeah the way I used it last time was just setting an order of magnitude of expected entries, and if you fill up, you can always reindex (which isn't as bad as it sounds, usually)