This repo shares example Processing code demonstrating a few of the technical aspects of The River is a Circle (2025) and The Earth Eaters (2025), both currently being shown at The Whitney. Both works were made in collaboration with Marina Zurkow.
These two works use a custom software framework built on top of the open source projects Processing and py5. The custom software is not open source but we decided to open source a few pieces of it to share our solutions to some of the unique technical challenges we faced while building these works.
The River is a Circle (2025) and The Earth Eaters (2025) are both image-heavy works with well over 100K images between the two of them. Some of the image sequences are quite large and would consume several GBs of memory each. This causes a serious problem because the computer's RAM, VRAM, and the Java Virtual Machine's heap size are all limited. It is critical to stay well within memory limits to avoid garbage collection pauses. Memory issues and garbage collection pauses cause visual stutters or brief freezes in the work. Much effort has gone into ensuring visual stutters never happen.
The River is a Circle (2025) runs on a small Intel NUC computer with 32 GB of RAM. The Earth Eaters (2025) runs on a larger NUC computer with 64 GB of RAM and a NVidia 4060 GPU with only 8 GB of VRAM.
We devised clever ideas for how to get the software to fit within these memory constraints. Our solutions are bespoke image compression algorithms that are either lossless or visually indistinguishable from the original images. These are not general purpose image compression algorithms, but they are algorithms that are precisely effective for our needs.
The main idea is to pre-process the source images to compress multiple source images into a single image that can later be used to display all of the source images. We use Python code to generate these compressed images that will need a custom shader to display correctly. This is effectively an on-GPU image decompression algorithm. If we compress four images into one, we can reduce memory usage by 75%.
Necessity is the mother of invention.
These algorithms meet our needs but they might not be useful for you. Most importantly, all of this is a needless complication if you are not facing memory constraints from an image-heavy Processing sketch. And the compression algorithms may be far from lossless for your situation. Please look at this as an example of creative problem solving and also consider the possibility of using an image's RGBA color channels as four data channels that are interpreted by a shader. If you understand the concepts, you may be able to design a different algorithm that works well for you.
Consider a grayscale image. The red, green, and blue channels are identical. We only need one of them. Storing all of them in memory is a waste of precious memory.
The prepare-images/writers/grayscale.py code can compress a source directory of grayscale images into a set of compressed images that each contain up to four of the originals. The code also creates a small data file that contains the number of original images. This is necessary because the number of original images may not be a multiple of four so the last compressed image can have unused channels.
A Processing Sketch can use a shader to display an image using only one of an image's four color channels.
An RGB image can contain up to 16777216 unique colors. However, most images don't use that many. Marina's drawings typically use a smaller color palette. We can use that to our advantage by creating images that use a smaller color palette with color quantization.
The prepare-images/writers/indexed.py code can compress a source directory of images into a set of compressed images that contain 4 sets of color indices that map to a color palette. The color palette is stored in a separate JSON file.
A Processing Sketch can use a shader to display an image by mapping the indices stored in one of an image's four color channels to the colors stored in the color palette.
This repo already contains test images in the src-images directory, but if you want to create them yourself, use this Python script:
python prepare-images/create-test-images.pyYou'll need to have py5 installed.
The compressed images are written to the appropriate data directories in the processing-sketches directory.
Open and run the Processing Sketches in processing-sketches using the Processing Development Environment (PDE). For both you'll see a player class that manages the compressed image data and the shader. Detailed information about how the players work is contained in the source code.
You'll observe the shaders are quite simple. That's actually the point. The interesting part is the Python code. The goal of all of this is to use Python to reorient the image data so that a simple, fast, and efficient shader can render the images while using as little memory as possible. Arcane shader code isn't helpful here.
These algorithms do not support images with transparent pixels. It can be expanded to support transparency, and in fact does so, successfully, in both The River is a Circle and The Earth Eaters. The Python code for that is quite a bit more complicated. If there's sufficient interest I will update this repository to include that as well.
The River is a Circle (2025) and The Earth Eaters (2025) both involve complex interactions between different parts of the code. In particular, some events in The River is a Circle (2025) are triggered by rules that could combine the current weather, tides, time of day, or season. Internally there is a State Manager that keeps track of state and evaluates state expressions. All components in the software can connect to the same State Manager to coordinate activities. All components in the software can set state values that are later consumed by other software components. This coordination vastly simplifies the code and makes it easier to maintain.
A specific example of how this is used in The River is a Circle (2025) is with the weather. One component in the work is responsible for processing weather data. It sets weather states with the State Manager to communicate if it is raining or snowing. Other components in the work will check in with the State Manager to evaluate relevant boolean expressions and may take action based on the results of a boolean expression. For example, a certain actor may only appear on the screen during the summer at nighttime when it is not raining.
The State Manager maintains the true and false values of named states. You define the state by first setting the state to true or false. Any state that is unknown the the State Manager is assumed to be false.
StateManager stateManager = StateManager.getInstance();
stateManager.setState("hot", true);
stateManager.setState("cold", false);
stateManager.setState("raining", false);
stateManager.setState("snowing", false);A valid state name can contain any alphanumeric character or underscores.
Boolean expressions are statements that the State Manager will evaluate to true or false. The expressions use state names and the boolean operators AND, OR, XOR, and NOT. You may also use parentheses.
Below are some possible boolean expressions and what they evaluate to, given the initial states set above.
| Expression | Value |
|---|---|
| hot AND raining | false |
| hot OR cold | true |
| hot AND (raining OR snowing) | false |
| NOT hot AND (raining OR snowing) | false |
| hot XOR raining | true |
| cold OR sleet | false |
Much of the value of the State Manager comes from the builtin time states. The builtin time states are:
daytime: true between sunrise and sunsetnighttime: true between sunset and the next day's sunrisejanuary,february,march, ...,december: true if the current date falls within the given monthaugust11,august12,august13, etc: true when the current date is the given month and daymonday,tuesday,wednesday, ...,sunday: true when the current date falls on the given day of the weeksummer,fall,winter,spring: true when the current season falls in the given season.summer_solstice,winter_solstice,vernal_equinox,autumnal_equinox: true when the current date is on a solstice or equinoxhour0, ...,hour22,hour23: true when the current hour of the day is equal to the given hourhour13_22: true when the current hour of the day is between the two given hourshour22_04: true when the current hour is between the given hours of two days
These can be combined with each other to form complex boolean expressions.
Open and run the test state manager Processing Sketch in processing-sketches using the Processing Development Environment (PDE). Look at the output in the console for the results of the boolean expressions. Experiment by adding new state expressions and see if you can figure out if they are true or false.
The River is a Circle (2025) monitors marine traffic in the immediate vicinity of The Whitney museum. When ships traveling along the Hudson River are viewable from The Whitney's terrace, the work will detect their presence and respond by displaying related animations on the screen.
The work uses data from the free real-time data source aisstream.io to obtain AIS data. To use this data and the example code, you'll need to sign up for an account to get an API key. The API key is a 40 character code that you must place in the file ais-data/main.py.
The example code is written in Python. You'll need to install the Python libraries websockets, numpy, and Pandas.
pip install websockets numpy pandasAdd your API key to the file ais-data/main.py before running it. Update the DATA_DIR directory to a location for AIS message logs.
python ais-data/main.pyInitially you will see log messages about unknown ships; it will take a few minutes for the code to receive enough messages for it to be able to pair static data and position data together and display meaningful statements.
Every message received will be saved to the DATA_DIR directory. You can and should write Python code to analyze these json data files and figure out how to best filter and process the data to achieve your goals.
AIS data is transmitted (or is supposed to be transmitted) from all marine ships. The intent is for ships to communicate with their neighbors to share information about their respective positions, velocities, and ship types. This information is communicated through several kinds of messages, or message types.
The full list of message types is extensive. Some of the message types are rarely used. The most informative messages are static data and position data, which should be combined to get a complete picture of ship traffic.
There are two static message types: ShipStaticData messages and StaticDataReport messages (scroll to the bottom of the page to see information about StaticDataReport messages). These messages can contain information about the ship type, size, and ship name. Be aware that sometimes these fields will be missing or the information will not match what you think it should be based on direct observation of the actual ship. Managing erroneous or misleading data is part of the experience of working with this kind of data. The class B messages are typically used by smaller ships, and those ships are more likely to transmit useless information.
Both of these static messages are transmitted once every 6 minutes. Because they are transmitted once every 6 minutes, you will need to wait that much time before you will have this information for all of the ships in your desired area. You will also need to wait as new ships move into your desired area.
There are two ship position message types: PositionReport messages and StandardClassBPositionReport messages. These messages contain information about each ship's location, heading, and speed. The data for these messages is typically accurate for ships that are underway (actively moving). The data may be inconsistent for ships that are not underway. To manage this I filtered out ships with speeds below a low threshold. Note that ships that are not underway can still move as the water around them moves.
The transmission time for these messages is variable. To get the current position of any ship, the best approach is to store the position data timestamps and estimate using the last position report's location, speed, heading, and the current timestamp.
The stream.py file contains the code that subscribes to the data source aisstream.io.
You will need to edit this file to set the subscription's bounding box. The bounding box limits the messages to ships within a specific region specified by latitude and longitude. Without this, your code will need to process several thousand messages a second. You should make the bounding box larger than the area you are actually interested in so that you will always have both static data and position data for all of the ships in your area of interest.
After updating the latitude and longitude, update the LATITUDE_DEGREES_TO_METERS and LONGITUDE_DEGREES_TO_METERS values with a latitude and longitude distance calculator to enable the code to calculate distances in meters. The code will be able to calculate coordinates in meters relative to the BOUNDING_BOX_CENTER. You should modify this code if those calculations do not meet your needs.
The record_ais_stream() method in the AISStream class will subscribe to the websocket and filter by message type. Comment out that filter if you would like to see all of the messages transmitted by the ships.
All of the received messages are passed to a AISDataState instance that will analyze the current ship data at a predefined interval.
The state.py file contains code that receives messages from the AISStream class and transforms the data into a more manageable form. It will interpret the static and position messages and allow you to pair them together.
The two data classes ShipInfo and PositionReport store the data. The PositionReport data class has two methods current_position() and future_position() needed to calculate current or future ship position. AIS data timestamps are always in the UTC time zone.
Update the process_ship_data() method to perform your desired data processing. You will first want to filter for your area of interest if you set the bounding box to a larger region. You will need to study the data using the recorded messages to understand how to best process the data to achieve your goals.
Here are resources that will be helpful to you as you work with AIS data and write your code.