Projet du cours d'Infrastructures et Systèmes Logiciels (S1 3A ENSAE)
The following project is realized as part of the computer science course given by Antoine Chancel during the last year at ENSAE Paris. The main goal is to deploy an app with Docker. This was very challenging and really cool to realize!
In this repository, you will find the following elements:
- A duration_model folder that contain the model that has been trained and the data used for the train. This model is then used for inference in the app.
- A Docker-compose.yml which orchestrates the containers.
- Some Docker Images : each of the following folder corresponds to an Image to build (with Python):
- etl : Scrape data from Youtube with Python
- web_app : Create a user web-app on local host with Dash
This project aims to help YouTube fans or Youtubers, particularly American ones, using data science. To do this, this project includes a Dash web application that provides :
- Insights of current YouTube trends;
- A survival analysis;
- Generate a thumbnail for based on a Stable Diffusion model.
The project is built around 2 containers (available on the docker-compose): the data collection and the web-app.
Our Dash web app allows users to view information about Youtube trends. The app is hosted on the local machine at http://localhost:8000/.
The first tab is an overview of the current YouTube trends. Together with the second tab, they provide statistical information such as the average number of views or likes per top creator, as well as others graphs such as the number of trending videos by category, etc... This format allows for an easy understanding of the selected categories and youtubers.
The third tab is a simple duration model. The project is not focused on using machine learning, so we used a simple duration model described in the "Docs" tab. By estimating survival curves (determined using traditional methods), we predicted the probability of entering the trend. Based on the user's desire for an insight of a recent YouTube video (to be chosen by the user himself!), we provide content.
The fourth tab provides access to a Stable Diffusion model hosted by HuggingFace. First, the user has to create an HuggingFace account to benefit from HuggingFace API access (it's free so don't hesitate to make it!). After having generated its API token and entered it in our app, the user can finally generate an image. The idea is that the user should enter a video title, and the image can be used (or inspire him) as a video thumbnail.
In order to run the code, you will need to follow the following steps:
- Clone the repository
git clone https://github.com/Yedson54/YouTrend.git- Change your current working directory
cd YouTrend- Create a .env file at the root of the working directory with several elements (can be modified). One of them is the Youtube API Key that you can obtain following this procedure:
TRENDING_TYPE_DICT = {'Now': None,'Music': '4gINGgt5dG1hX2NoYXJ0cw%3D%3D','Gaming': '4gIcGhpnYW1pbmdfY29ycHVzX21vc3RfcG9wdWxhcg%3D%3D','Movie': '4gIKGgh0cmFpbGVycw%3D%3D'}
# COUNTRY_CODE_LIST = ['US', 'CA', 'AU']
NATIONALITY_CODE = ['FR']
YOUTUBE_API_KEY =[YOUR_YOUTUBE_API_KEY] - Launch the app Since we have a multi-container Docker application (as defined in the docker-compose.yml), you can run the full project (scraper and web app) by running the following command:
docker-compose upNote: This can take a bit of time.