The Main Application for ABET-Tools

• ThFeature/symfony legacyroutere Main Application for ABET-Tools
  • Overview
  • Getting Started
  • ABET private key setup
  • Docker Installs
    • Linux
    • Windows
  • Important Files
    • .env files
    • .htaccess files
  • Managing PHP dependencies with Composer
    • Installing Composer
    • Using Composer
    • Composer Install (the command)
  • The Symfony Framework
  • Database development
    • Editing the init.sql directly
    • Using Migrations
  • Pulling from the server
  • More information

Overview

This is our application, containerized for easier development. Included are containers for:

• The PHP/Apache server
• Canvas Formatting APIs
• The report generation API
• The Canvas Extraction API
• The MySQL database (to simulate the real one)
• PHPMyAdmin for easy database administration
• A selenium container for E2E testing

Getting Started

This requires:

• Docker engine AND docker compose to be installed. Docker Install
• CPanel This also requires you to have the cPanel server private keys added. ABET Private Key
• A bash command line. (a given)

Once that's done, you can run these commands to view the application

1. cd into docker/ and create a .env. demo.env is a template file that is suitable for development.
2. within docker/, run docker compose up --build
3. you can visit localhost port 8080 to see the interface
4. you can visit localhost port 8081 to use phpMyAdmin (useful to see database state)

Note

Env files are how we configure the containers to run differently on your local machine and on the server. Using the env files correctly will ensure that your code works as intended on the server.

More information about the ENV files are available in this section.

ABET private key setup

Using the CPanel

1. download abet ssh key from the CPanel
2. Set correct permissions: chmod 600 abet
3. Move it to a convenient spot. usually .ssh/
4. eval "$(ssh-agent -s)"
5. ssh-add $PATH_TO_ABET_PRIVATE_KEY
6. Use the ABET ssh key password in the discord.

Docker Installs

You should follow official Docker installation. You got this. Installing Docker Desktop is the easiest way, regardless of OS.

Linux

Docker engine

Docker compose

Windows

Docker desktop (Installs both!)

Important Files

everything in docker/ refers to the current containers that we have in the application. If we want more, we should add a folder with the container name, and a docker-compose.yaml.

app container: a php-apache container that runs both php and apache. Apache config is in docker/apache2, but you need to copy this from the server.

mysql container: the container containing the mysql instance and whose database is in ./docker/mysql/mysql_data

if we want to integrate the python file, we can easily create a fastapi container in the docker-compose.

The rest of the project organization info is HERE in the docs

.env files

This is the MOST important concept of configuration in the project. On first setup, you probably run cp demo.env .env, but you don't really know what that does. docker/.env stores all of the filled environment variables, and is used by docker-compose.yml which then may set the environment variables of the containers to those values.

.env stores ALL of our secret keys, but also our configuratoin information. Changing .env will change how the containers are built. I made demo.env with the purpose of easy setup, but these values should NEVER be exposed or used in the real server.

Note

The project usually has a second .env file called prod.env that is meant to only run on the server and has credentials that may never be exposed. You only interact with this file if you are deploying the application or messing with the server.

More information about .env files

More information about project deployment

.htaccess files

These are apache configuration files that exist only within the src/public directory. They are local and apply to the current directory and all subdirectories. They work hieracically: htaccess files in subdirectories overwrite ones in parent directories.

We use .htaccess files to rewrite important paths and to set apache settings specific to our project only. DO NOT try to update the server's actual apache's config, it will affect EVERY project hosted by this server.

Click here for official docs on the file type

Warning

Using .htaccess for pretty file rewrites is about to be deprecated. We are migrating to, or are already using Symfony for routing. Check the github milestone to see if people should use Symfony.

Managing PHP dependencies with Composer

Installing Composer

To use composer on your system, you need PHP8.3+ installed. (The server has php8.3 installed) To install everything, I did this:

sudo apt install php8.3 && \
sudo apt install php8.3-xml && \
sudo apt install php8.3-mysql && \
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/bin --filename=composer

Using Composer

If the last command was successful, you now have PHP and composer installed on your system! Now, you can install any composer package you want!

cd src/public
composer require pestphp/pest --dev --with-all-dependencies 
# FORMAT: composer require PACKAGE_NAME:PACKAGE_VERSION --with-all-dependencies
# --dev specifies that Pest (testing) is only for development (not deployment)

Note

You might get an error when trying to require your package. This is due to uninstalled PHP extensions.

If this happens, Require the latest version of the package, and then check composer's output.

In this example, ext-dom is not installed. Installing php8.3-xml fixed the errors for me. I trust that you can install this yourself.

All you need to do now is to require autoload.php in your php files

# Note that this is the path RELATIVE to the current file. 
require __DIR__ . '/vendor/autoload.php'; 

Official Composer Usage Docs

Composer Install (the command)

Let's say some file requires autoload and you haven't installed the composer files on your system. You can fix this by running:

cd src/abet_private
composer install

This simply reads the dependencies in composer.json and installs them in the vendor/ folder.

The Symfony Framework

Symfony is a framework to route the website. It is modular and has a lot of built-in security tools that we should use. The version that we are using (7.4) is compatible with PHP 8.3 and has LTS.

It's best practice to use well-known frameworks because their security has been tested throughout time. Since security isn't optional for this app, we are migrating to Symfony to take advantage of these security features.

Symfony may not be fully implemented at the time of reading. To check, see if the Github Milestone is met. If it is, then you should use it when developing.

Note

Migrating to Symfony will change a lot of our normal workflow. While our previous tools will work, creating new pages will follow Symfony's pattern. All of the process changes are listed in the Symfony Docs. Please read it to make sure you're developing correctly.

Database development

There are two ways to work on the database now.

• You may choose to make a migration (newer, easier integration).
• Or you may choose to edit the init.sql file (might get deprecated.)

Don't worry, Migrations are not that much different than making raw sql. In fact, it allows you to prototype stuff quicker.

Editing the init.sql directly

If you're working on the database, the most important file in the project for you is docker/mysql/init.sql, as it defines all of the database tables that you will interface with.

Important

Restarting the docker compose contaienrs will NOT update the mySQL tables. This is because the mysql container is simply restarted, not built again.

You can fix this by running the following commands:

within the docker/ folder

docker compose down     # or docker-compse if you have that
docker compose up --build

Using Migrations

Migrations are how we can set scripts so syncrhonize tables on your end with prod. Please use this when altering tables. Otherwise, during deployment, not updating the tables on the server can cause big problems.

New tables added with IF NOT NULL can stay in init.sql, but we can put ALTER TABLE statements in these migrations to version the database.

Installing dependencies

sudo apt install php8.3-cli
sudo apt install php8.3-xml
sudo apt install php8.3-mysql
sudo apt install composer

Install compser packages

cd src/abet_private
composer install

Generate migrations (will be in src/abet_private/database/migrations) A new file with the dated version will appear. The other migrations can be a template to show you how to implement a migration.

composer doctrine generate

Tip

All migrations are stored in src/abet_private/database/migrations. A simple example of how to use these migrations exist here.

This migration uses small ALTER tables

This migration uses a PHP heredoc to use SQL syntax highlighting

This is how you test migrations: Note that you need your mysql docker container up to test this. Remove the --dry-run argument to actually execute it.

compser doctrine migrate --dry-run

Information on how to link to the database

Pulling from the server

NOTE: copy_from_server.bash does not currently sync everything up, as we have not fully adopted this method. abet_private is copied into src/abet_private/abet_private and abet.asucapstonetools.com is copied into src/public/abet.asucapstonetools.com

1. git clone this repo.
2. cd into scripts/ and run copy_from_server.bash
   • I hardcoded the server IP, so we might need to change this if the step doesn't work
   • The script also assumes you're in scripts/, else it will copy the files to who knows where.
3. git add, commit, and push your changes

More information

More docs are located in the docs directory.

Even more information is found in this master document.