Static vs. dynamic hooks

[cjdoucette]

I'm looking to use jbpf in an application where the set of hooks is not known at compile-time, as new hooks and codelets can be added at any time. However, I noticed that the hook abstraction is modeled as a function built into the binary.

Is there any issue with creating new struct jbpf_hooks when needed in memory? I had to add some supporting functionality to add, remove, and execute hooks, but it seems to be working in general. I'm just wondering if I missed an important detail of why hooks are implemented the way that they are.

[xfoukas]

Can you please give a few more details or some sample code about what you are trying to do? Do you mean instrumenting the code in the same way as uprobes do in Linux?

[cjdoucette]

Currently to define a hook, you need to use macros like DECLARE_JBPF_HOOK() and DEFINE_JBPF_HOOK(). These macros create a symbol for a function at compile-time that invokes one or more codelets.

However, what if I don't know the number of hooks at compile time?

For example, in a multi-tenant environment, I might want to have a separate hook/codelet for each tenant's traffic. But tenants can be added or removed at any time, so I want to be able to add a new hook at runtime and then attach a codelet to that hook to process that tenant's traffic with custom logic.

So my question is basically why is there no API for dynamically creating hooks at runtime.

[xfoukas]

Can you please elaborate on what you mean by "tenant" in the scenario that you described? What does adding or removing a tenant entail? What kind of network function or application are you trying to instrument? Do you want to add hooks in arbitrary locations of your code, that are not known a priori or do you want to capture specific function calls?

If by dynamic hooks, you are referring to dynamic instrumentation of existing symbols exposed by the application (e.g., similar to what uprobes do), then that would be possible by leveraging instrumentation techniques like function interposition and binary re-writing. However, this is something that we do not currently support, but we are exploring it as a future extension.

You also mentioned that you already have a working implementation of the scenario that you have in mind. If this is something that you think that would benefit the community, then please feel free to create a PR, describing the use case in more details, and we are more than happy to take a look and provide feedback.

[cjdoucette]

Thanks for asking clarifying questions!

The network function I'm building is a middlebox that allows different actors ("tenants") to inject custom packet processing code into the network. A tenant can be added to the system by bringing their own IP prefix and eBPF program. When the middlebox receives a packet destined to an address in the tenant's IP prefix, the eBPF program owned by that tenant is invoked on the packet.

They're called tenants because they exist in a shared system, but are isolated from each other in the sense that they each have their own eBPF program/codelet that only processes their own traffic. So therefore, we have a bunch of separate codelets, and the set of codelets can be added to if more tenants enter the system at runtime.

I believe each of these codelets needs its own hook. But in the current code, hooks are declared statically at compile time using macros. Instead, I'd like for the application to manage a set of struct jbpf_hooks that it can dynamically add to and remove from.

You also mentioned that you already have a working implementation of the scenario that you have in mind. If this is something that you think that would benefit the community, then please feel free to create a PR, describing the use case in more details, and we are more than happy to take a look and provide feedback.

Happy to do this. Just want to make sure I haven't invented a problem that is already addressed in some other way with the existing code.

[xfoukas]

Thanks for the clarifications. I understand your use case better now.

jbpf allows several codelets to be loaded on a single hook. However, from your description I understand that you don't want the other tenants to have any visibility into the traffic of others, right?

One way I can think that you could implement this functionality without making any modifications to the existing code, would be the following:

• Add N statically pre-compiled hooks in a switch statement, where N is the max number of tenants the middlebox can support. The codelets of users can be loaded in those hooks.

• Add a single control hook to capture the incoming packets, and emit a routing decision, about which hook in the switch statement should be called. The owner of the middlebox can dynamically load a codelet to this control hook that inspects the packet and returns a single number, from 0 to N-1, that indicates which static hook of users should be called, based on the IP.

The code logic could look something like the following:

uint64_t routing_decision = ctrl_hook_router(usr_pkt);

switch(routing_decision) {

case 0:
   hook_user1(usr_pkt);
   break;
case 1:
   hook_user2(usr_pkt);
   break;
...
}

You could autogenerate such code at compile time for an arbitrary number of tenants.

When a new tenant joins, your control plane logic could assign a hook to that user (from 0 to N-1) and could update the logic of the codelet loaded at ctrl_hook_router() to emit a routing decision that redirects the packets of that user appropriately. To avoid reloading the ctrl_hook_router() codelet each time a new tenant is added or removed, you could instead use a control input map to update the codelet's parameters.