Add pipeline filters bar with advanced search and sorting#1871
Add pipeline filters bar with advanced search and sorting#1871
Conversation
🎩 To tophat this PR:You can add the following URL parameter to your browser to tophat this PR: |
This stack of pull requests is managed by Graphite. Learn more about stacking. |
c5d3881 to
fd1b3c9
Compare
fd1b3c9 to
caa1934
Compare
|
question: Is our pipeline search completely unindexed today? I imagine in the future, if not already, we will have users or groups (project) that have hundreds of pipelines. One thing I believe is worth keeping in mind is how it will scale in such an event. The search and filtering may become slow to process, causing freezes in the UI, when the user has a hundred, or several hundred pipelines. A project I will likely take up in the next month is creating a remote data store for pipelines, and that might be an opportunity to leverage indexing. It would be difficult to optimize though if we have search or filtering elements that involves traversing the graphs. What do you think of splitting out any graph-traversal capabilities, now (like you have), and going forward, so that we have that clear separation of easily-indexable fields from slower operations? |
|
"Example pipelines" Button is now feel like part of the "Search" - I would advise to move it so it is separated. |
| query: string, | ||
| ): boolean { | ||
| const impl = fileEntry.componentRef.spec.implementation; | ||
| if (!("graph" in impl)) return false; |
There was a problem hiding this comment.
Use isGraphImplementation type guard
| ): boolean { | ||
| const impl = fileEntry.componentRef.spec.implementation; | ||
| if (!("graph" in impl)) return false; | ||
| const q = query.toLowerCase(); |
There was a problem hiding this comment.
I dont like one-letter variables if it's not something conventional (e.g. x y coordinates). Maybe normalizesQuery?
| searchQuery, | ||
| setSearchQuery, | ||
| dateRange, | ||
| setDateRange, | ||
| sortField, | ||
| setSortField, | ||
| sortDirection, | ||
| setSortDirection, | ||
| componentQuery, | ||
| setComponentQuery, | ||
| hasActiveFilters, | ||
| activeFilterCount, | ||
| clearFilters, | ||
| totalCount, | ||
| filteredCount, | ||
| actions, |
There was a problem hiding this comment.
That's smells . Do we really need to propagate all these setters/values?
| if (componentQuery && !matchesComponentQuery(fileEntry, componentQuery)) | ||
| return false; | ||
| return true; |
There was a problem hiding this comment.
NIT: apply de-morgan's law to simplify return.
return !componentQuery || matchesComponentQuery(fileEntry, componentQuery);| // Stable string that changes only when filter inputs change — safe for useEffect deps | ||
| const filterKey = `${searchQuery}|${dateRange?.from?.getTime() ?? ""}|${dateRange?.to?.getTime() ?? ""}|${componentQuery}|${sortField}|${sortDirection}`; | ||
|
|
||
| const filterBarProps: FilterBarProps = { |
There was a problem hiding this comment.
I dont know what React Compiler will do with this. Would it create new object each time?
| }); | ||
|
|
||
| // Stable string that changes only when filter inputs change — safe for useEffect deps | ||
| const filterKey = `${searchQuery}|${dateRange?.from?.getTime() ?? ""}|${dateRange?.to?.getTime() ?? ""}|${componentQuery}|${sortField}|${sortDirection}`; |
There was a problem hiding this comment.
NIT: to improve readability
const filterKey = [
searchQuery,
dateRange?.from?.getTime(),
dateRange?.to?.getTime(),
componentQuery,
sortField,
sortDirection,
].join("|");| }; | ||
|
|
||
| const filteredPipelines: PipelineEntry[] = Array.from(pipelines.entries()) | ||
| .filter(([name, fileEntry]) => { |
There was a problem hiding this comment.
I feel this filter with some slight modifications to helper functions can be optimized for reading, e.g.
const filteredPipelines: PipelineEntry[] = Array.from(pipelines.entries())
.filter(
([name, fileEntry]) =>
matchesSearch(name, fileEntry, searchQuery) &&
matchesDateRange(fileEntry, dateRange) &&
matchesComponentQuery(fileEntry, componentQuery),
)
.sort((a, b) =>
sortByNameAndModificationTime(a, b, sortField, sortDirection),
)
.map(([name, fileEntry]) =>
buildPipelineEntry(name, fileEntry, searchQuery, componentQuery),
);| if (dateRange?.from) { | ||
| if (new Date(fileEntry.modificationTime) < dateRange.from) return false; | ||
| } | ||
| if (dateRange?.to) { | ||
| const to = new Date(dateRange.to); | ||
| to.setDate(to.getDate() + 1); | ||
| if (new Date(fileEntry.modificationTime) > to) return false; | ||
| } | ||
| return true; |
There was a problem hiding this comment.
NIT: for better readability use early returns, less if-nesting and variable naming
if (!dateRange) return true;
const modificationTime = new Date(fileEntry.modificationTime);
if (dateRange.from && modificationTime < dateRange.from) return false;
if (dateRange.to) {
// Include the entire "to" day by comparing against the start of the next day
const endOfRange = new Date(dateRange.to);
endOfRange.setDate(endOfRange.getDate() + 1);
if (modificationTime > endOfRange) return false;
}
return true;There was a problem hiding this comment.
Or if introduce a helper normalizeEndOfRange
const endOfRange = normalizeEndOfRange(dateRange.to);
if (endOfRange && modificationTime > endOfRange) return false;| {allBadges.map((badge) => ( | ||
| <Badge key={badge.key} variant="outline"> |
There was a problem hiding this comment.
Thinking outloud: this remind me a TagList component from another PR https://app.graphite.com/github/pr/TangleML/tangle-ui/1848?ref=gt-pasteable-stack
There was a problem hiding this comment.
I can come back to this once that PR is in.
maxy-shpfy
left a comment
There was a problem hiding this comment.
Tophatted -works well! I would love to improve readability in usePipelineFilters
caa1934 to
0d83f2e
Compare
0d83f2e to
11e5ca9
Compare

Description
Added a comprehensive filtering system for the pipeline section with advanced search capabilities. The new
PipelineFiltersBarcomponent provides search by name/description/author/notes, date range filtering for last modified dates, sorting by name or date, and component-based filtering to find pipelines using specific components. The filters display active filter badges with individual removal options and a collapsible advanced section for component queries.Related Issue and Pull requests
Type of Change
Checklist
Screenshots (if applicable)
pipeline_filter_basic.mov (uploaded via Graphite)
Test Instructions
Additional Comments
The filtering logic includes matching against pipeline names, descriptions, author annotations, and notes. Component filtering searches through task component references including URLs, names, and spec names. The UI maintains state for showing/hiding filter badges when there are many active filters.