README.md

Python Web Crawler

This script will gather all links in a particular website.

• Origin tutorial is here
• Call python ./python_crawler/main.py
• Enter a website to crawl (Default is [http://www.pythonanywhere.com/])
• Go for coffee