Web Scraping with AgentQL and Playwright

This project demonstrates web scraping using AgentQL and Playwright in Python.

Prerequisites

• Python 3.12 or higher
• Poetry (Python package manager)

Installation

1. Clone this repository:

git clone <repository-url>
cd webscraping

2. Install dependencies using Poetry:

poetry install

3. Install Playwright browsers:

poetry run playwright install

Usage

Run the scraping script:

poetry run python main.py

To Run the tests:

poetry run pytest

Project Structure

• main.py: Main scraping script
• pyproject.toml: Poetry project configuration and dependencies
• README.md: Project documentation (this file)

Dependencies

• agentql: Web scraping framework
• playwright: Browser automation tool