The app is unbearably slow due to the IO bound nature of multiple network calls in Python. This can be solved a few ways as I see it.
- Concurrent download of feeds using asyncio, multiprocessing, genvent or twisted
- Refactor current front end to javascript and back feed the app db
- Add Celery task queue
- Pre-fetch the feeds with scrape cron job and write the results to a file, parse the file at run time
- Write a Go lang .so file that can be run like C
#5 is super interesting for a variety of reasons. Although it doesn't seem like the most practical approach to this problem, it surely represents something I've never attempted. Also, provides some new learning opportunities. However, Celery and the various concurrent modules are provide learning opportunities as well.
Not ready to make a decision yet.
The app is unbearably slow due to the IO bound nature of multiple network calls in Python. This can be solved a few ways as I see it.
#5 is super interesting for a variety of reasons. Although it doesn't seem like the most practical approach to this problem, it surely represents something I've never attempted. Also, provides some new learning opportunities. However, Celery and the various concurrent modules are provide learning opportunities as well.
Not ready to make a decision yet.