pylocus

This repository contains the reference implementation for the Python 3 package pylocus.

Description

The pylocus package is specialised in creating isochrone maps for public transport networks. Starting from a position of interest, isochrone maps are used to depict visually regions which are accessible by public transport within a certain time range. The current implementation of pylocus is capable of producing high resolution isochrone maps, which can then be explored dynamically as web maps.

Organisation of the pylocus package

• ./mains contains the main routines to generate isochrone maps and build/update the database.
• ./data contains the data required by the package: GTFS timetables, shapefiles and temporary files. Exported maps are also stored there.
• ./pylocus/benchmark contains debugging routines that test specific features of the package as well as benchmark competing implementations of the most computationally intensive functions of the package.
• ./pylocus/classes contains the Python classes used to store input parameters and walking transfers. .* ./pylocus/io contains input/output routines: import data and construct the database, export static or dynamic isochrone maps.
• ./pylocus/routing/csa contains a custom implementation of the CSA routing algorithm. If computational time is critical, precompiled C++ versions of the algorithm obtained via Numba are available in ./pylocus/routing/csa/__pycache__.
• ./pylocus/routing/utils contains various plotting and miscellaneous routines.

Technical Notes

• The underlying routing algorithm leveraged by pylocus is an extension of the Connection Scan Algorithm (CSA) from Dibbelt et al. supporting walking transfers.
• Pylocus is currently restricted to the Swiss public transport network (all modes), but is compatible with any public transport timetable complying with the General Transit Feed Specification (GTFS) standard, a common format for public transportation schedules and associated geographic information used by Google Maps.
• The zoomable web maps are generated using the Mapbox GL JS JavaScript library.

Installation

Note: The following instructions have been tested on a MacBook Pro (16-inch, 2019) with macOS Catalina version 10.15.2 (19C57).

PostgreSQL

Behind the scenes, pylocus relies on the PostgreSQL database system, very close to the SQL standard. We recommend using Homebrew to install and maintain PostgreSQL on macOS. To verify if Homebrew is installed on your system, type brew -v in your terminal. If Homebrew is installed, the version of the software should appear, otherwise an error is raised.

If Homebrew is not installed, install it by running in the terminal the following command:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Follow the installation instructions. If Homebrew is installed on your system but outdated (<2.2.2), you may want to update it (and upgrade the packages already installed on your system) via:

brew update 
brew upgrade

Once Homebrew installed, we can install PostgreSQL with the command:

brew install postgresql

The command brew services list should now list a service named postgresql with status stopped. We can start the service with the command:

brew services start postgresql

Now that the service is started, we are going to create a default user called locus_admin with password isochrone and a database called cff_database. To create the user we execute:

createuser -P -s -e locus_admin

This command issues a prompt for the password of the new user. Type isochrone as a password (twice).

To create the database:

createdb -e cff_database

We will now grant all privileges to the user locus_admin to the newly created database cff_database. To this end we log in cff_database as $USER (default owner of all databases):

psql cff_database $USER

The line should now start with cff_database=#, which means that we have successfully logged into the database. Type now the folowing to grant locus_admin all privileges on cff_database:

GRANT ALL PRIVILEGES ON DATABASE cff_database to locus_admin;

To check that the configuration was successful, type \du to see all users (you should see locus_admin listed) then \list to see the existing databases (you should see cff_database and the privileges column should mention locus_admin among other users). Finally, log out of the database by typing \q.

Maintenance of PostgreSQL

When developing on a database and experiencing a sudden crash of the system, it could happen that the connection to the database is not properly closed, making the database unresponsive and forbidding any user to log to it. In this case, the status of postgresql returned by brew service list will typically be colored in yellow. To fix this issue, it can help to stop the postgresql service with brew service stop postgresql, then delete the file /usr/local/var/postgres/postmaster.pid and start the service again with brew service start postgresql.

Python

Pylocus is a Python 3 package relying on multiple open source packages written by third-partites (you can see all the import calls of pylocus using the command grep -h -R -e "^import" --include '*.py' ./ | sort -h | uniq). To facilitate the management of cross-dependencies, we make use of Miniconda, a light-weight version of the package and environment management open source software Anaconda.

Installing and configuring Miniconda

If you are on macOS Catalina and had Miniconda or Anaconda installed prior to upgrading to the latest macOS, then your installation is probably broken due to new restrictions from Apple. To properly install Miniconda on macOS Catalina, download the binary file at this link. From the terminal, navigate to the location where the file was downloaded and execute

bash Miniconda3-latest-MacOSX-x86_64.sh

Follow the prompts. Then, run source ~/miniconda3/bin/activate followed by conda init zsh. Finally, reopen the terminal and conclude the installation with conda update.

We are now ready to create a conda environment named pylocus with Python 3.7 as the default python version:

conda create -n pylocus python=3.7 

We will set the channel priority to strict, and add conda-forge as channel with higher priority.

conda config --set channel_priority strict
conda config --add channels conda-forge

These settings are stored in the hidden configuration file ~/.condarc. Execute cat ~/.condarc to check that this is indeed the case.

conda Python dependencies

We can now install the relevant packages in the created environment:

conda install -n pylocus -c conda-forge matplotlib numpy pandas requests scipy sqlalchemy basemap basemap-data-hires numba gdal tqdm ipython-sql ipython psycopg2 jupyter

pip Python dependencies

Almost done, we need now to install packages not available in the Anaconda Cloud. To this end execute:

source activate pylocus 

and

pip install chardet geojsoncontour pandarallel

We are done with the installation of dependencies.

Matplotlib custom style

Pylocus relies on a custom matplotlib style for more visually pleasing plots. From the pylocus root folder, type in your terminal:

mkdir ~/.matplotlib/stylelib 
cp data/style/custom_style.mplstyle ~/.matplotlib/stylelib/custom_style.mplstyle

Update PYTHONPATH

To use the custom pylocus package, we need to tell to Python where to look for the pylocus package. To this end, navigate with your terminal to the root of the pylocus project and type:

echo "export PYTHONPATH=$PYTHONPATH:${PWD}" >> ~/.zshrc

Close and reopen you terminal for the changes to take place.

Creating/Updating the Database

To download the Swiss GTFS data and construct/update the database, run in your terminal:

source activate pylocus

Then go to the root of the pylocus folder and execute:

python mains/update_database.py

Wait until completion, this may take a while... Progress bar will monitor the process. This routine downloads the latest GTFS timetable data from https://opentransportdata.swiss/dataset/timetable-2019-gtfs/permalink, sores it as text files in data/gtfs as well as in the database cff_database previously created.

Alternatively, you can also run update_database.py from an IDE like PyCharm (making sure to set the project environment to pylocus).

Create an Isochrone Map

Open mains/isochrone_main.py with your favourite text editor (e.g. Sublime Text) or Python IDE (e.g. PyCharm) and modify the parameters of the map on the first line:

setup = IsochroneSetup(box_height=15, box_width=15, center_lon=8.394694, center_lat=47.121146,
                       tstart='08:00:00', walk_distance=600,
                       tend='08:30:00', verbose=True, query_backend='sql', day='monday', resolution=1024,
                       nb_contours=6)

The help of the IsochroneSetup class gives more details about the role of each parameter:

Initialize setup object for an isochrone experiment.
    :param box_height: float
        Height of delimiting box in km.
    :param box_width: float
        Width of delimiting box in km.
    :param center_lon: float
        Longitude of central point in degrees. Must belong to the bounding box of one of the supported countries
        listed in variable self._country_bboxes.
    :param center_lat: float
        Latitude of central point in degrees. Must belong to the bounding box of one of the supported countries
        listed in variable self._country_bboxes.
    :param tstart: string
        Start of time window in format 'hh:mm:ss'.
    :param tend: string
        End of time window in format 'hh:mm:ss'.
    :param day: string
        Day of the week.
    :param walk_distance: float, optional
        Maximum walking distance for connections in m.
    :param walk_speed: float, optional
        Average walking speed in km/h.
    :param resolution: int, optional
        Number of pixels of the isochrone map in the largest dimension. The resolution of the other dimension is
        set according to the aspect ratio of the delimiting box.
    :param country: string
        Specifies the country.
    :param verbose: bool
        Whether or not to print details of map creation (timing of various steps...)
    :param export: str,
        Desired type of export: web, bokeh or image.
    :param query_backend: str,
        Whether to use SQL or Pandas to query the data.

On the last line of mains/isochrone_main.py you can modify the page_title and legend_title parameters of the function write_html_file which creates the web map. The rest should remain unchanged.

write_html_file(setup, page_title='Locus Map', legend_title='Commute to/from Root',
                access_token='pk.eyJ1IjoibWF0dGhpZXVtZW8iLCJhIjoiY2sybHE0emJwMDh2cDNtb2ZsNTdtZW5laSJ9.VXQYES3RlejhgrttnkPXdw',
                stylesheet='mapbox://styles/matthieumeo/ck2lqph9w0ill1cp84vyafnyc', zoom=11, dot_size=200,
                fill_opacity=0.8, zorder=6)

You can now execute the script mains/isochrone_main.py. With your terminal, navigate to the root of the pylocus folder and type:

source activate pylocus
python mains/isochrone_main.py

In verbose mode, the script will print various messages to let you monitor the process. If the script executes normally, the created web map is located in data/tmp_exports/index.html. Open this file with your favourite navigator to see the result!

Credits

Matthieu Simeoni and Mathias Haug, creators of the Locus project.

Yanis Berkani for his precious contribution in the context of his Bachelor Project at EPFL.