Skip to content

Add pipeline filters bar with advanced search and sorting#1871

Open
Mbeaulne wants to merge 1 commit intomasterfrom
02-27-pipeline_filters
Open

Add pipeline filters bar with advanced search and sorting#1871
Mbeaulne wants to merge 1 commit intomasterfrom
02-27-pipeline_filters

Conversation

@Mbeaulne
Copy link
Collaborator

@Mbeaulne Mbeaulne commented Feb 27, 2026

Description

Added a comprehensive filtering system for the pipeline section with advanced search capabilities. The new PipelineFiltersBar component provides search by name/description/author/notes, date range filtering for last modified dates, sorting by name or date, and component-based filtering to find pipelines using specific components. The filters display active filter badges with individual removal options and a collapsible advanced section for component queries.

Related Issue and Pull requests

Type of Change

  • Bug fix
  • New feature
  • Improvement
  • Cleanup/Refactor
  • Breaking change
  • Documentation update

Checklist

  • I have tested this does not break current pipelines / runs functionality
  • I have tested the changes on staging

Screenshots (if applicable)

pipeline_filter_basic.mov (uploaded via Graphite)

Test Instructions

  1. Navigate to the pipeline section
  2. Test basic search functionality by typing in the search field
  3. Use the date range picker to filter pipelines by modification date
  4. Toggle between name and date sorting with ascending/descending order
  5. Open the advanced section and test component filtering
  6. Verify that active filter badges appear and can be individually removed
  7. Test the "Clear all" functionality to reset all filters
  8. Confirm pagination resets when filters change

Additional Comments

The filtering logic includes matching against pipeline names, descriptions, author annotations, and notes. Component filtering searches through task component references including URLs, names, and spec names. The UI maintains state for showing/hiding filter badges when there are many active filters.

@github-actions
Copy link

github-actions bot commented Feb 27, 2026

🎩 To tophat this PR:

You can add the following URL parameter to your browser to tophat this PR:

`?tophat_location=02-27-pipeline_filters/caa1934`

Copy link
Collaborator Author

Mbeaulne commented Feb 27, 2026

@Mbeaulne Mbeaulne changed the title Pipeline filters Add pipeline filters bar with advanced search and sorting Feb 27, 2026
@Mbeaulne Mbeaulne force-pushed the 02-27-pipeline_filters branch 5 times, most recently from c5d3881 to fd1b3c9 Compare March 2, 2026 15:06
@Mbeaulne Mbeaulne marked this pull request as ready for review March 2, 2026 15:35
@Mbeaulne Mbeaulne requested a review from a team as a code owner March 2, 2026 15:35
@Mbeaulne Mbeaulne force-pushed the 02-27-pipeline_filters branch from fd1b3c9 to caa1934 Compare March 3, 2026 16:17
@Mbeaulne Mbeaulne mentioned this pull request Mar 3, 2026
8 tasks
Copy link

question: Is our pipeline search completely unindexed today?

I imagine in the future, if not already, we will have users or groups (project) that have hundreds of pipelines. One thing I believe is worth keeping in mind is how it will scale in such an event. The search and filtering may become slow to process, causing freezes in the UI, when the user has a hundred, or several hundred pipelines.

A project I will likely take up in the next month is creating a remote data store for pipelines, and that might be an opportunity to leverage indexing. It would be difficult to optimize though if we have search or filtering elements that involves traversing the graphs. What do you think of splitting out any graph-traversal capabilities, now (like you have), and going forward, so that we have that clear separation of easily-indexable fields from slower operations?

Copy link
Collaborator

"Example pipelines" Button is now feel like part of the "Search" - I would advise to move it so it is separated.

query: string,
): boolean {
const impl = fileEntry.componentRef.spec.implementation;
if (!("graph" in impl)) return false;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use isGraphImplementation type guard

): boolean {
const impl = fileEntry.componentRef.spec.implementation;
if (!("graph" in impl)) return false;
const q = query.toLowerCase();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont like one-letter variables if it's not something conventional (e.g. x y coordinates). Maybe normalizesQuery?

Comment on lines +41 to +56
searchQuery,
setSearchQuery,
dateRange,
setDateRange,
sortField,
setSortField,
sortDirection,
setSortDirection,
componentQuery,
setComponentQuery,
hasActiveFilters,
activeFilterCount,
clearFilters,
totalCount,
filteredCount,
actions,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's smells . Do we really need to propagate all these setters/values?

Comment on lines +101 to +103
if (componentQuery && !matchesComponentQuery(fileEntry, componentQuery))
return false;
return true;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: apply de-morgan's law to simplify return.

return !componentQuery || matchesComponentQuery(fileEntry, componentQuery);

// Stable string that changes only when filter inputs change — safe for useEffect deps
const filterKey = `${searchQuery}|${dateRange?.from?.getTime() ?? ""}|${dateRange?.to?.getTime() ?? ""}|${componentQuery}|${sortField}|${sortDirection}`;

const filterBarProps: FilterBarProps = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont know what React Compiler will do with this. Would it create new object each time?

});

// Stable string that changes only when filter inputs change — safe for useEffect deps
const filterKey = `${searchQuery}|${dateRange?.from?.getTime() ?? ""}|${dateRange?.to?.getTime() ?? ""}|${componentQuery}|${sortField}|${sortDirection}`;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: to improve readability

const filterKey = [
    searchQuery,
    dateRange?.from?.getTime(),
    dateRange?.to?.getTime(),
    componentQuery,
    sortField,
    sortDirection,
  ].join("|");

};

const filteredPipelines: PipelineEntry[] = Array.from(pipelines.entries())
.filter(([name, fileEntry]) => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel this filter with some slight modifications to helper functions can be optimized for reading, e.g.

const filteredPipelines: PipelineEntry[] = Array.from(pipelines.entries())
    .filter(
      ([name, fileEntry]) =>
        matchesSearch(name, fileEntry, searchQuery) &&
        matchesDateRange(fileEntry, dateRange) &&
        matchesComponentQuery(fileEntry, componentQuery),
    )
    .sort((a, b) =>
      sortByNameAndModificationTime(a, b, sortField, sortDirection),
    )
    .map(([name, fileEntry]) =>
      buildPipelineEntry(name, fileEntry, searchQuery, componentQuery),
    );

Comment on lines +66 to +74
if (dateRange?.from) {
if (new Date(fileEntry.modificationTime) < dateRange.from) return false;
}
if (dateRange?.to) {
const to = new Date(dateRange.to);
to.setDate(to.getDate() + 1);
if (new Date(fileEntry.modificationTime) > to) return false;
}
return true;
Copy link
Collaborator

@maxy-shpfy maxy-shpfy Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: for better readability use early returns, less if-nesting and variable naming

  if (!dateRange) return true;

  const modificationTime = new Date(fileEntry.modificationTime);

  if (dateRange.from && modificationTime < dateRange.from) return false;

  if (dateRange.to) {
    // Include the entire "to" day by comparing against the start of the next day
    const endOfRange = new Date(dateRange.to);
    endOfRange.setDate(endOfRange.getDate() + 1);
    if (modificationTime > endOfRange) return false;
  }

  return true;

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or if introduce a helper normalizeEndOfRange

  const endOfRange = normalizeEndOfRange(dateRange.to);

  if (endOfRange && modificationTime > endOfRange) return false;

Comment on lines +231 to +232
{allBadges.map((badge) => (
<Badge key={badge.key} variant="outline">
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking outloud: this remind me a TagList component from another PR https://app.graphite.com/github/pr/TangleML/tangle-ui/1848?ref=gt-pasteable-stack

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can come back to this once that PR is in.

Copy link
Collaborator

@maxy-shpfy maxy-shpfy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tophatted -works well! I would love to improve readability in usePipelineFilters

@Mbeaulne Mbeaulne force-pushed the 02-27-pipeline_filters branch from caa1934 to 0d83f2e Compare March 5, 2026 17:12
@Mbeaulne Mbeaulne force-pushed the 02-27-pipeline_filters branch from 0d83f2e to 11e5ca9 Compare March 5, 2026 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants