Foundation

Directories

src/raspi

This folder contains the Python code necessary to run the Raspberry PI.

The folder is divided up by functions. Here are the subfolders:

• Component
• State management
• Samples

Component

The Component folder contains the component code used by Catbot. Contains the following Components:

• Compressor
• Latch
• Motor
• Muscle

EE and ME should look at this code to understand the functions available for each electrical component.

State management

The state management folder contains the code for the centralized store, parsing behaviors, and the utility file used in the Component. Contains the following:

• generic_devices (output device, input device, etc.)
• utils
• device.py

EE could look at what these codes do but should only look at what is exported from these files. Most of which will be explained in the components folder. ME should not be looking at this code. (Trust me, do not waste your time. If you really want to then you are in the wrong field)

Samples

The samples folder contains sample code that EE or ME can run and test out the hardware. Each file should explain what its purpose is and which components it tests. If not, ask the original creator. Ask SE if you don't know how.

Catbot Fundamentals

1. Componentized Modules

One of the biggest issues we face as Catbot developers in Software is that things change often in Hardware. That could be replacement of components, swapping connection of components, or eliminating an entire components, but one thing that doesn't change in all of this is the actual component itself. Meaning component logic or component property do not change that often. This means that if we organize/group things by components there are minimal amount of changes in software necessary when hardware changes. This is the biggest concept we want to preserve as much as possible as a software engineer. This increases reusability of components and shorten development time before the deadline. You will see that as we dig deeper into the Catbot python architecture this is the biggest goal of catbot. To maintain code reusability at the component level and customizability of relations outside of components.

Organizing code by component is the first example of such idea. By separating things by components, we can feed sub components into components that was never written for it to be used as a subcomponent, but if done correctly, everything should behave the way we structured it to be. However, there is one caveat. You have to make sure the component is actually a component or else we will loose the flexibility of such an idea. There are few tricks and tips I (Hiroto) will give you further down this document that will make sure this is done correctly and any issue is caught early on and prevented.

One of the biggest pitfalls is when we use a communication device. Imagine this scenario where a regular brushless motor is hooked up to an ESC, and the ESC takes in a PWM signal and a binary direction signal. The direction signal is fed in by an addressable latch. A latch is a component that can output 8 digital output and the latch takes in 5 inputs: 3 for digital(binary) address pins, 1 for enable, and 1 more for data. We are going to ignore the actual use and logic of these electrical devices here, but my question to you for this scenario is how many components did you have in total? Hint: one of the component is the PWM output device where it can send PWM signal to the motor.

The answer is four with one extra fake component.

1. PWM output device: this was given by the hint. Even though this device does not physically exist as an electrical component, in the code we create this device for several reasons. I will explain one of the reason in the next bullet, but think of this as each GPIO pin on the Raspberry Pi.
2. Digital output device: if we have a PWM output device we have to have a Digital or binary version of it. Even though this is also the same pin on the Raspberry Pi we still consider as another component because when we initialize the GPIO pin we have to specify that one will be used for PWM and another is used for Digital output. One of the biggest benefit we get when we componentize GPIO Pin is that we can swap these for output signals from other devices like the latch.
3. Addressable Latch: the latch in the actual implementation does something odd where it creates a digital output device or the extra fake component, which follows a factory pattern from object oriented design patterns, and the created digital output device follows a proxy pattern. Do not worry if you don't know what any of those patterns are, but the important thing that these patterns allow us to do is now we can treat all of the output pins on the addressable pin as a regular digital output pins on the Raspberry Pi. Now all we have to worry about is inject this new output device into other components and treat it as if it is directly connected to the Raspberry Pi. This further simplify the components connected on top.
4. Motor: The last component that is the most obvious here is the Motor component. The way I listed these components prevented us from falling into the pitfall of this scenario, but the mistake many people would make in this scenario is embedding the latch logic directly into the motor component. The issue with doing so is that now the two completely isolated electrical components are heavily coupled together which means if an EE decide that we need to hook up the latch to a valve then we have to take the latch logic out of motor and inject it into the valve logic. Even though the valve is a single digital output device and the motor is just a PWM and digital output signal we had to change two completely different things from the changes. However, if we break up the latch logic we can now just feed the fake digital output device into the valve and not have to worry about the logic of the motor.

Now that we went through an example of the concept of componentization, I hope this cleared up some of the ways I create these components. Similar structure is used for things like I2C, ADC, and IO Expanders, where they act as a middle communication layer that handles the communication between the RPI and the actual components that we want to work with. If you ever find yourself working with more than 1 hardware documentation then it is a good signal to stop yourself and think is there really just one component? or can you break it down further.

2. Pin Config File

Now that we broke down our code into components wouldn't it be nice if you can just intertwine the inputs and outputs of each of these components in a single file? Well, I have a good news; you absolutely can. The pinconfig.json file exists for this exact reason. When these components are coupled just by output and input device components, you can just inject the hell out of these codes and not have any issues. How is that possible? Well let me show you an example:

Let's say you have the above in the pinconfig file. The top layer of dictionary object represents list of components, so we use the components input_device, output_device, and pwm_output_device. Inside we declare the identifier of the components we want to declare (e.g. left_pressure lat_enb motor_1_speed); we use these identifiers to point directly at the devices we want to use in the code. The value for each key is the values that will be used to declare those devices (e.g. left_pressure is made with pin number 27, motor_1_speed has a pin number of 4 and frequency of 1000Hz). Alright that is just the simple syntax, but what about if you want components within the higher components?

Here we utilizes a higher level component where we throw in a component left_pressure within an attribute of left_muscle. That makes total sense; hopefully. We can reference the names of the components we have declared as the parameter utilizing their identifiers. However, there is a bigger issue and that is the valve parameter. If you have done the exercise above that should immediately trigger a red flag, the valve parameter should take in a digital output device, but it directly takes in a pin number of the output device and creates an output device in the constructor. Well, that is true, but this is something the configurator does when it reads the config file. It finds these parameters that requires other components and automatically creates an instance of them on the spot, if necessary. In the state management section we will talk more about how we achieve this, but for the time being I want you to understand that I gave you the freedom to declare the configuration in two ways.

1. Directly use the name of the components by their identifiers.
2. Directly pass in the sub-component's parameter as the value of the parent component's parameter.

The reason why I gave two ways of declaring the config file is because sometimes it is easier to read if we declare it as 2., but could be hard to track of all of the lower components which is why I give you 1. where all of the lower components are declared together. There are more complex examples that could be important to know, but everything above covers majority of the basic concept of the config file. If you want to see a full example of this config file you can go to the config file.

3. State Management

The last fundamental concept I am going to cover in this page is the state management. The issue we still face, even with all of the work we do above, is that the script file would just couple all of the component back together when we try to control the lower level component in the main script. Although we rarely would do such thing in most of the sample file, it would still be nice if we can prevent such actions from ruining all of the work we did above. This is where the state management comes into play. If we can somehow point directly at the lower components without going through the higher components, we can eliminate ourselves from coupling the entire structure. In order to do that we utilized the identifier of the device names more. Conceptually our code goes to the state management and get the component we want to use just when we need it. If you have already tried running our sample code with higher level components, something you might notice is that the instance of the higher level components does not actually contain the instance of the lower level components. Instead it holds the identifiers of the lower components. That is the effect of the state management where we can achieve "zero" coupling by only holding the identifier of the components we don't actually have to care where our components are and just start calling methods and just point at the device we want to use. Here is an example of the sample code:

Here is a simple sample file using one component and allow us to manually control the latch inputs and learn the latch behavior. Something you may have realized in this script is that there is only one variable, which is the variable i or user input. Everything else is just the constant variable set to the identifiers. Using the identifiers we can achieve something similar to pointers in C where we can point at the location that the device is stored in and not have to worry about scopes and control of the variables. This still doesn't prevent us from just throwing in a pointer that doesn't exist, but now we do not have to worry about where the components live and make the scripts stupidly simple and, more importantly, readable.

But what if you want to just work without the state management? Well, if you really don't like the state management you still can work with these samples. You can just throw in the instance of the devices where the identifiers are and manipulate the devices regularly. Just because the state management is used does not prevent you from just creating the component structure manually and work with that. This is heavily used for us to test each of these methods where we would just directly call these methods with the device instance thrown in as the parameter.

---

Syntax

Now that we have learned a little bit about how these components work fundamentally and used. Let's get down deeper and talk about how these components are actually implemented. The most important thing to understand in the internals system of the architecture is that even though it seems like we have a central location that all of the devices are stored, it in fact does not. We have multiple dictionaries to simulate storage. There are several benefit to breaking it up.

1. Security: This is one of the most important things to have in mind as robotics software that is going to be flexible. If we have a completely flexible software it is very difficult to make sure that all of the object do not get mixed up and used in ways that was not intended to be used. I could be something like throwing identifier for one device that is different from the device that the method was written for.
2. Single responsibility: this comes from the OOP paradigm, but organizing each dictionary to have single responsibility of storying their own respective devices will be cleaner in the long run because we can point to specific dictionary and know confidently that that dictionary will only have that compatible device and we don't have to worry about making sure the type of the device is correct. Also know that when ever we talk about contexts or store, these smaller dictionaries are what we are talking about.
3. Improve space complexity: this is something I did my best to make it happen where each dictionary is only initialized if they are used in the main script file. Ultimately, this is makes a very minuscule difference in the long run. I realized that we can have another identifier for each context, but the reason I did not is because 1. I did not want to confuse context identifier and device identifier and 2. I wanted us to have more control over the initialization of these contexts. If you don't fully understand this do not worry it will be explained in the next section.

Python Imports

A lot of people ignores python imports very often, but understanding how to use python imports are very important to controlling what gets initialized and what doesn't. In the bullet 3 above I mentioned that we can improve space complexity by only initializing contexts that we use. How we control what gets initialized or not is by utilizing the python imports. For example lets say you have the following imports:

# in main.py
from component.motor import raw_motor_action

# component.motor.__init__.py
from . import raw_motor as raw_motor_action
from .pin import *

# component.motor.pin.__init__.py
from . import direction_pin as direction_pin_action
from . import speed_pin as speed_pin_action

You can imagine that all of the modules referenced in the imports are used, so other modules like component.muscle or component.compressor will never run. So contexts initialized within those modules will never exist in the main script. That means devices that would have been stored in those contexts would also never be initialized either.

Initialize Context

We have two main methods to initialize the contexts

1. create_context(generic_device_name: str, device_classes: list[class], parser_func: callable = None, on_exit: callable = None)-> Context: Creates a regular context instance for any device.
2. create_masked_context(ctx: Context, device_name: str)-> Context: Duplicates the context instance passed into the method and creates smaller encapsulation

Note

We have a third method create_generic_context which is identical version of create_context, but I left this one for you guys to use to further specify that certain devices just exists to be masked and further encapsulated and never used as itself. (i.e. input_device_ctx, output_device_ctx)

The create_context method will, as the name suggest, instantiate context for a particular device classes that is passed in.

The create_masked_context method will create context for masked devices. The masked devices are devices like switch, compressor, or speed pin within a motor device, where all of the devices can just use a simple class like digital_input_device, digital_output_device, or pwm_output_device. We also call devices that are used by masked devices to be generic devices where by themselves it is not used as a component, thus do not have an action method utilizing them.

Parsing method

When we parse the config file we need a way to initialize the data stored in the config. We can just pass data straight into the parameter of the device classes, but that restrict us to have config declared in only one way, which is enough for simple structs and parameters for some devices, but for classes that requires more complex values in the constructors are not possible. By having a way to declare your own way to construct the devices outside of class initializers will allow you to have one location to clean up any logic to the parameter passed in before the initialization or even in some complex factory devices can use these parser methods to extract logic for initializing other devices. By having a parsing method declared within the context class we can have a more powerful and flexible ways of declaring devices. This also allows us to use library classes as well and gives us more freedom without having to create an inherited class.

Method:

• device_parser(ctx: Context)-> callable[[callable], None]: decorators for registering parsing method to a context.

Usage:

This is a simple example of how to declare a parser method. The instance returned by the method will be stored into the context and the returned instance will be used in the location of the identifier.

Note

** is a destructor syntax and is equivalent to doing the following:

Warning

Parser decorator is a consumer of the method, which means it returns None and the method is not callable. This may change in the future to add easier ways to debug the method, but currently is not a feature built in.

Action method

This is the final method responsible for declaring all methods used in the main script. One of the main responsibility of these action methods is that they need to be able to take in a device identifier and be able to convert them to actual device instances stored in the state management. This way we can call the method with just some_action("foo", param2) instead of some_action(foo, param2) where we don't want to throw in the instance of the object when calling because that will defeat the purpose of the state management. However, at the same time we also want to be able to pass in instance of the object such that we can easily test the action methods.

Method:

• device_action(ctx: Context)-> callable[[callable], callable]: Decorator responsible for wrapping the method to give the method a behavior to swap the identifiers by its actual instance object.

Usage:

Warning

If you pass in a identifier that is not stored in the context the method will not be called and throws an error. The identifier also have to be passed into the first parameter

This wrapped method will now be usable with identifier passed in as a first parameter. The identifier will only work as the first parameter. The decorator will not look at any other parameter. This could change in the future but this syntax is very similar to how you would declare regular method within a class object with self reference. Also I would recommend declaring the first param to be the type of the actual device of the identifier such that you would get type hinting by the IDE or VSCode.

Struct Object

If we have all of the methods outside of classes to be used to in the main scripts, you might wonder what should be in the actual device objects. So far, we have defined all of the methods that are usually defined within the class to be declared outside of the class which leaves the class with just the attributes of the class. This certain usage of classes and plain objects are most popular in Web dev and unfortunately its not more performant to use this structure of state than just having regular classes, but it does make the rest of the code easier to handle, so this is the structure of the code I recommend everyone to use. The biggest feature we use is the dataclass decorator.

What is dataclass?

The Dataclass decorator is used to wrap the entire class such that they have extra abstracted definitions of the simple class you defined such that each class you defined is more complete. There are even ways to increase performance with just one extra parameter. One of the main feature it has is it adds constructor method.

Note

There are more complex usage of the constructor portion of the class which I will not go over in this doc, but you can learn more about them here.

Lets say you have the following syntax:

slot=true is a feature built in dataclass method that works with python's slot dunder method in class, and the decorator will also add a similar constructor as the following:

You can imagine how simple defining classes for this architecture will be without all of the redundant constructor. That leaves us with just the class definition above to be sufficient for the RawMotor class. You might have also realized that speed and direction attribute of the RawMotor class is another device object, and it couples the class with that certain class. We can actually just store the identifier of the class instead and not have the actual instances to be coupled to each other.

Construct inner device within the higher device component

In the last RawMotor class example, we saw that there was another device class within which we can call a higher level component. For the constructor of the higher component, at least one parameter require the value to be the inner device. We have several ways this can be done:

1. Pass the instance of the inner object (mainly used for testing)
   • Instance is directly passed in with no operation on the value during initialization.
2. Pass the identifier of the inner object
   • Identifier is also directly passed into the class with no operation. This is okay because the device action function will convert the identifier to the stored instance during runtime.
3. Pass the parameter to construct the inner device
   • The parameter of the inner object will be used to construct the object during initialization. This requires us to add extra logic to instantiate the inner device during construction of the outer object. This is done by the @device decorator where the decorator will wrap the __init__ function to add the extra logic. For this to work, the @device decorator needs to know whether a certain value needs an extra logic to construct an instance of the inner object or not. To indicate to the decorator that a certain attribute is a inner device we use the identifier(ctx: Context) function to "mark" the attribute, which will then be picked up by the decorator on runtime to then be converted during execution.

Since all of the logic above is baked into the constructor by the decorator, we can eliminate any extra logic necessary in the constructor function. However, do note that the constructor function will run prior to the logic above. Here is the order of execution: @device_parser function > @device function > Device.__init__() or @dataclass init and postinit

An example can be seem in the Latch factory class below:

Warning

If you use option 2 or 3 to construct the higher level component, within the @device_action methods, we will not be able to access any of the attributes within the inner device. For example, higherDevice.innerDevice.on() would be invalid because .innerDevice is an identifier and does not have such property. A fix for this is to pass the innerDevice as the parameter of the inner device's @device_action function: innerDevice_action.on(higherDevice.innerDevice). Option 1 is allowed to make it easier to quickly change the attribute of the inner device for testing.

---

EDITS

Created by Hiroto Takeuchi 4/4/2024

Edited by Hiroto Takeuchi 4/13/2024 Fixed warning "Construct multiple device classes at once"

Edited by Hiroto Takeuchi 6/9/2024 Added "Construct inner device within the higher device component" and removed "Construct multiple device classes at once"

Edited by Hiroto Takeuchi 6/9/2024 Added warning for using inner device's methods on higher device's device action method.