For instance:
- answer_videos_vlm would be useful for clips as well, or even frames
- transcriptions related operators could also work at both videos and clip levels
- same for transcode_clips_ffmpeg that could be relevant to videos
It's a bit of a conceptual issue.
Option A
Duplicating the operators for different levels would work but doesn't sound like a good idea. It would create some amount of code duplication and increase the list of operators being listed.
Option B
Allowing some operators to accept an argument to specify where they apply makes the interaction a bit more complicated and is inconsistent with other operators. Although we could have a default level.
Option C
Another option, used by command line tools like docker, is to always require which unit the action/operation applies to in the command line. E.g. docker image ls vs docker container ls . So for instance python framesense.py clip scenedetect instead of python framesense.py make_shots_scenedetect. Or python framesense.py clip answer_vlm vs python framesense.py video answer_vlm.
With option C, each operator would have to declare the types of units is accepts as input. And remove any specific unit from its name. The name would only contain the action and method. And FrameSense could list operators by units. e.g. python framesense.py clip would show a list of available operators for clips as input.
Option D
In most cases what's applicable to videos is applicable to clips and vice versa, because of their shared file format. So one option here is to facilitate the creation of a clip that represents the whole video when needed. This way we can just have answer_clips_vlm. One advantage of this option is its simplicity of implementation and the fact that any output on a video will have a consistent location to that of clips. A transcription will always be under /VIDEO/CLIP/transcription.json. Which further simplify the operators taking transcriptions as an input. Some users have already mentioned that the distinction between clip and video is not always useful to them and creates overhead. This option would also address that.
Options C & D are the main contenders.
I like the clearer syntax of option C. And the simplicity of D. So perhaps they could be combined. D first because it's a quick fix. Then C when possible.
For instance:
It's a bit of a conceptual issue.
Option A
Duplicating the operators for different levels would work but doesn't sound like a good idea. It would create some amount of code duplication and increase the list of operators being listed.
Option B
Allowing some operators to accept an argument to specify where they apply makes the interaction a bit more complicated and is inconsistent with other operators. Although we could have a default level.
Option C
Another option, used by command line tools like docker, is to always require which unit the action/operation applies to in the command line. E.g.
docker image lsvsdocker container ls. So for instancepython framesense.py clip scenedetectinstead ofpython framesense.py make_shots_scenedetect. Orpython framesense.py clip answer_vlmvspython framesense.py video answer_vlm.With option C, each operator would have to declare the types of units is accepts as input. And remove any specific unit from its name. The name would only contain the action and method. And FrameSense could list operators by units. e.g.
python framesense.py clipwould show a list of available operators for clips as input.Option D
In most cases what's applicable to videos is applicable to clips and vice versa, because of their shared file format. So one option here is to facilitate the creation of a clip that represents the whole video when needed. This way we can just have
answer_clips_vlm. One advantage of this option is its simplicity of implementation and the fact that any output on a video will have a consistent location to that of clips. A transcription will always be under /VIDEO/CLIP/transcription.json. Which further simplify the operators taking transcriptions as an input. Some users have already mentioned that the distinction between clip and video is not always useful to them and creates overhead. This option would also address that.Options C & D are the main contenders.
I like the clearer syntax of option C. And the simplicity of D. So perhaps they could be combined. D first because it's a quick fix. Then C when possible.