diff --git a/docs/loom-etl-work-packages/EXPLORER_LOOM_MAPPING_INVENTORY.md b/docs/loom-etl-work-packages/EXPLORER_LOOM_MAPPING_INVENTORY.md new file mode 100644 index 00000000..e6e3bdfa --- /dev/null +++ b/docs/loom-etl-work-packages/EXPLORER_LOOM_MAPPING_INVENTORY.md @@ -0,0 +1,27 @@ +# Explorer read-path inventory + +The Explorer configuration still uses its established `guppyConfig.dataType` +values. The adapter boundary is `packages/core/src/features/loom/mapping.ts`; +all requests below use the canonical value returned by `toLoomDataType`. + +| Explorer surface | Previous request | Loom request | Existing consumer shape | +| --- | --- | --- | --- | +| `pages/Explorer/data.ts` shared-field discovery | Guppy `_mapping` | `dataframeDataset` column metadata | `SharedFieldMapping` | +| `CohortPanel` facets | Guppy grouped aggregations | `dataframeAggregate` COUNT by field | `AggregationsData` | +| `CohortPanel` total | Guppy count aggregation | `dataframeAggregate` COUNT | `number` | +| `ExplorerTable` rows | Guppy offset rows/count | `dataframeRows` with opaque cursor | `JSONObject[]`, row count | +| `QueryRowDetailsPanel` | Guppy single-row query | `dataframeRows` with an ID filter | one `JSONObject` | +| download actions | Guppy download endpoint | Loom dataframe export | JSON/CSV/TSV blob or JSON rows | + +The canonical mapping is: + +- `file` and `document_reference` -> `DocumentReference` +- `research_subject` -> `ResearchSubject` +- `specimen` -> `Specimen` +- `medication_administration` -> `MedicationAdministration` +- `group_member` -> `GroupMember` + +Loom metadata is also the availability boundary. Explorer distinguishes an +unsupported type, transport/authorization failure, missing dataset, a +non-`READY` dataset, and a `READY` dataset with zero rows. There is no Guppy +runtime fallback. diff --git a/docs/loom-etl-work-packages/EXPLORER_LOOM_MAPPING_PLAN.md b/docs/loom-etl-work-packages/EXPLORER_LOOM_MAPPING_PLAN.md new file mode 100644 index 00000000..0ca3c58b --- /dev/null +++ b/docs/loom-etl-work-packages/EXPLORER_LOOM_MAPPING_PLAN.md @@ -0,0 +1,383 @@ +# Explorer-to-Loom dataset mapping plan + +## Objective + +Migrate Explorer reads from the legacy flat metadata API to Loom's published +dataframe API without changing Loom's canonical recipe output names. + +Explorer may continue to use its established data-type vocabulary in routes, +configuration, labels, and component props. A single frontend-owned adapter +will translate those values into Loom `dataType` values before calling the Loom +fetch hooks. + +This is an Explorer integration change. It is not a missing Loom recipe or +publication feature. + +## Decision + +Loom's default recipe output names remain canonical: + +| Explorer or legacy data type | Loom `dataType` | +| --- | --- | +| `file` | `DocumentReference` | +| `document_reference` | `DocumentReference` | +| `research_subject` | `ResearchSubject` | +| `specimen` | `Specimen` | +| `medication_administration` | `MedicationAdministration` | +| `group_member` | `GroupMember` | + +Both `file` and `document_reference` map to `DocumentReference` because the +legacy ETL index name and Explorer's user-facing concept are not necessarily +the same string. The Explorer checkout must verify which form is used at each +current call site before obsolete forms are removed. + +There is no separate `publication` output in the default recipe. Publication +is the operation that makes the five outputs above available through Loom. No +`Publication` dataset or output alias should be invented. + +## Ownership boundary + +### Explorer owns + +- the legacy-to-canonical data-type map; +- selection of the correct Loom hook for each Explorer read; +- conversion of Explorer filters, sorting, pagination, and facets into Loom + query inputs; +- adaptation of Loom responses into existing Explorer view models; +- user-facing fallback, empty, loading, and error states; +- migration and removal of the old Explorer read path. + +### Loom owns + +- canonical output names from the materialized recipe; +- discovery of published datasets; +- principal-scoped project federation; +- active-generation resolution; +- row authorization; +- ClickHouse row, filter, sort, cursor, and aggregation execution; +- publication readiness and error reporting. + +### The ETL job owns + +- loading complete generations into Loom; +- running and waiting for the default recipe materialization; +- failing the job if publication does not reach `READY`. + +The ETL job does not publish Explorer-specific aliases or configure frontend +data-type mappings. + +## Loom API contract used by Explorer + +Explorer should use the projectless, principal-scoped GraphQL read surface: + +```graphql +dataframeDatasets +dataframeDataset(input: { dataType: $dataType }) +dataframeRows(input: { + dataType: $dataType + columns: $columns + filters: $filters + sort: $sort + first: $first + after: $after +}) +dataframeAggregate(input: { + dataType: $dataType + groupBy: $groupBy + filters: $filters + operation: $operation + column: $column +}) +``` + +Explorer supplies a canonical `dataType` after applying its map. It must not +supply a project or generation. Loom derives authorized projects from the +authenticated principal, resolves active generations, and federates matching +published outputs. + +## Work package 1: inventory the current Explorer contract + +Before editing hooks, trace the current Explorer read path from rendered page +to network request. + +1. Locate every route, configuration value, component prop, selector, and hook + that passes a data-type value. +2. Record the actual values in use, including `file`, `document_reference`, and + `research_subject`. +3. Locate the current hooks for: + - schema or field discovery; + - rows; + - total counts; + - facets or grouped counts; + - filter submission; + - sort submission; + - cursor or offset pagination. +4. Record the response shape consumed by each component. +5. Identify whether the new Loom hooks already expose those operations or need + a thin wrapper. + +Deliverable: a call-site inventory that identifies the one adapter boundary +through which all Explorer data types can pass. + +Acceptance criteria: + +- no Explorer metadata request path remains unaccounted for; +- the real network-emitting functions are identified; +- the inventory distinguishes user-facing labels from API data-type values; +- existing hook consumers and their required return shapes are listed. + +## Work package 2: add the canonical mapping module + +Create one mapping module adjacent to the Loom hooks, not inside individual +pages or components. + +The module should expose a closed input type where practical and one resolver: + +```ts +export const EXPLORER_TO_LOOM_DATA_TYPE = { + file: 'DocumentReference', + document_reference: 'DocumentReference', + research_subject: 'ResearchSubject', + specimen: 'Specimen', + medication_administration: 'MedicationAdministration', + group_member: 'GroupMember', +} as const; + +export function toLoomDataType(dataType: ExplorerDataType): LoomDataType { + return EXPLORER_TO_LOOM_DATA_TYPE[dataType]; +} +``` + +Use the frontend's generated GraphQL types for `LoomDataType` if they provide a +useful type. Otherwise define the canonical values locally from the map rather +than accepting arbitrary strings. + +Unknown legacy values must fail visibly during development. Do not silently +pass an unknown value to Loom and do not default it to `DocumentReference`. + +Acceptance criteria: + +- all mappings live in one module; +- the map contains only real default recipe outputs; +- unknown values produce a deterministic error or explicit unsupported state; +- unit tests cover every entry and the unknown-value behavior; +- no Loom backend change is required. + +## Work package 3: route Loom fetch hooks through the mapping + +Apply `toLoomDataType` at the outermost hook boundary that still receives an +Explorer data type. Components should not know Loom recipe output names unless +they already operate directly on Loom metadata. + +For each hook: + +1. Accept the existing Explorer-facing data type. +2. Resolve it once to the canonical Loom `dataType`. +3. Use that canonical value consistently for dataset metadata, rows, and + aggregate requests. +4. Include the original Explorer value in client-side diagnostics while + avoiding sensitive request data. +5. Keep query-cache keys unambiguous. Prefer the canonical Loom value in cache + keys so `file` and `document_reference` share the same dataset cache entry. + +Do not map column names. Dataset aliases and dataframe column names are +different contracts; column compatibility must be assessed separately from +this plan. + +Acceptance criteria: + +- `file` emits `dataType: "DocumentReference"` for dataset, row, and aggregate + requests; +- `research_subject` emits `dataType: "ResearchSubject"` for all three; +- aliases that map to the same Loom output do not create inconsistent caches; +- components retain their current labels and route vocabulary; +- no request sends a legacy alias to Loom. + +## Work package 4: adapt Explorer operations to Loom + +### Dataset and field discovery + +Use `dataframeDataset` to retrieve publication metadata and columns. Convert +`DataframeColumn` values into the field model expected by Explorer, preserving: + +- name; +- logical type; +- nullability and repeatedness; +- filterable, sortable, and aggregatable capabilities. + +Explorer should disable unsupported controls based on these capabilities +instead of issuing invalid requests. + +### Rows + +Use `dataframeRows` with the canonical data type. Preserve the existing table +contract by adapting: + +- returned column order; +- JSON row values; +- total count, when supplied; +- `pageInfo.hasNextPage`; +- `pageInfo.endCursor`. + +The adapter should treat `endCursor` as opaque. It must not parse it or convert +it into an offset. + +### Filters and sorting + +Create explicit conversion functions from Explorer filter state to +`DataframeFilterInput` and from Explorer sort state to `DataframeSortInput`. +Reject unsupported operators before the request and surface a usable UI error. + +Do not infer filter behavior from display labels. Always use Loom column names +from dataset metadata. + +### Facets and counts + +Use `dataframeAggregate` for grouped counts and other supported aggregations. +Keep the same canonical data type and active filter set used by the row query. +Define whether a facet excludes its own active filter based on current Explorer +behavior, then preserve that behavior explicitly in the adapter. + +Acceptance criteria: + +- the Explorer table can load, filter, sort, and paginate Loom rows; +- facets and displayed totals are calculated over the same authorization scope + as rows; +- unsupported column operations are disabled or rejected before querying; +- cursor pagination works across at least two pages without duplicates or + omissions; +- changing the Explorer data type resets incompatible filters and cursors. + +## Work package 5: handle readiness and partial availability + +Explorer must distinguish these cases: + +1. Loom is unreachable or returns a transport error. +2. The user is unauthenticated or has no authorized projects. +3. No active publication exists for the mapped data type. +4. A publication exists but is not `READY`. +5. The publication is `READY` but contains zero rows. +6. The requested legacy data type is unsupported by the mapping. + +Use `dataframeDatasets` or `dataframeDataset` as the availability source. Do not +fall back silently to the old backend after a Loom error; that could show data +from a different generation or authorization path. + +Acceptance criteria: + +- each state has a deterministic hook result and user-visible behavior; +- an empty ready dataset is not reported as a missing publication; +- an unsupported mapping is distinguishable from a missing Loom dataset; +- retries do not mix cursors or rows from different data types. + +## Work package 6: tests + +### Mapping unit tests + +- assert every table entry; +- assert both `file` and `document_reference` map to `DocumentReference`; +- assert unsupported values fail explicitly; +- assert the map contains the five canonical default recipe outputs. + +### Hook request-contract tests + +Mock the GraphQL boundary and verify exact variables for: + +- dataset discovery; +- first row page; +- next row page; +- filters; +- ascending and descending sort; +- grouped count; +- filtered grouped count. + +At minimum, cover `file`, `research_subject`, and one direct snake_case to +PascalCase mapping. + +### Hook response-adapter tests + +- column capabilities; +- empty rows; +- null and repeated values; +- total count present and absent; +- final and non-final pages; +- Loom GraphQL errors; +- missing publication; +- stale request completion after switching data types. + +### Explorer integration tests + +For one representative Explorer page: + +1. render with the legacy `file` concept; +2. verify the network request uses `DocumentReference`; +3. render returned columns and rows; +4. apply a filter and sort; +5. request a second page; +6. render a facet count; +7. switch to `research_subject` and verify state reset. + +### Loom contract check + +Add or retain a Loom-side test proving that canonical output names can resolve +through `dataframeDataset`, `dataframeRows`, and `dataframeAggregate`. This is a +contract check, not an alias feature. + +## Work package 7: rollout + +1. Land the mapping and hook tests without changing the active Explorer source. +2. Add a temporary frontend data-source switch if Explorer does not already + have one. +3. Enable Loom in a development environment with a known `READY` default + recipe publication. +4. Compare visible rows, totals, facets, filters, sorting, and pagination for + the representative Explorer pages. +5. Validate with users authorized for one project, multiple projects, and no + projects. +6. Make Loom the default after acceptance checks pass. +7. Remove the old fetch path and temporary switch in a follow-up change after a + short observation window. + +The switch controls the frontend read implementation only. It must not change +recipe names, ETL publication behavior, or Loom aliases. + +## Observability + +During rollout, record non-sensitive client telemetry for: + +- original Explorer data type; +- mapped Loom data type; +- operation category: dataset, rows, or aggregate; +- success, unsupported mapping, missing publication, authorization failure, or + transport failure; +- request duration. + +Do not log row contents, filter values, authorization paths, or access tokens. + +## Explicit non-goals + +- adding `file`, `research_subject`, or other legacy aliases to Loom bundle + publication; +- renaming Loom's default recipe outputs; +- creating a `Publication` dataframe output; +- changing ETL generation loading or recipe materialization; +- redesigning Explorer routes, labels, or page layout; +- preserving the old backend as an automatic runtime fallback; +- solving dataframe column-name parity inside the dataset-name mapping layer. + +## Definition of done + +The mapping migration is complete when: + +1. Explorer can request all five default metadata datasets through its existing + concepts while every Loom request uses a canonical recipe output name. +2. The representative Explorer flows load rows, filters, sorting, pagination, + totals, and facets from Loom. +3. Authorization and multi-project federation are left to Loom and verified in + integration testing. +4. Missing, empty, unauthorized, unsupported, and failed states are distinct. +5. Mapping, hook-contract, response-adapter, and representative UI tests pass. +6. No Loom publication alias was added for Explorer compatibility. +7. The old Explorer read path can be removed without changing the Loom recipe + or ETL job. + diff --git a/packages/core/src/constants.ts b/packages/core/src/constants.ts index f9d1035b..499f432d 100644 --- a/packages/core/src/constants.ts +++ b/packages/core/src/constants.ts @@ -9,6 +9,8 @@ export const GUID_PREFIX_PATTERN = /^dg.[a-zA-Z0-9]+\//; */ export const GEN3_GUPPY_API = process.env.NEXT_PUBLIC_GEN3_GUPPY_API || `${GEN3_API}/guppy`; +export const GEN3_LOOM_API = + process.env.NEXT_PUBLIC_GEN3_LOOM_API || `${GEN3_API}/loom`; export const GEN3_MDS_API = process.env.NEXT_PUBLIC_GEN3_MDS_API || `${GEN3_API}/mds`; export const GEN3_DOWNLOADS_ENDPOINT = diff --git a/packages/core/src/features/authz/authzMappingSlice.ts b/packages/core/src/features/authz/authzMappingSlice.ts index ae5fbdde..192bace9 100644 --- a/packages/core/src/features/authz/authzMappingSlice.ts +++ b/packages/core/src/features/authz/authzMappingSlice.ts @@ -1,5 +1,7 @@ import { gen3Api } from '../gen3'; import { createSelector } from '@reduxjs/toolkit'; +import type { FetchBaseQueryError } from '@reduxjs/toolkit/query'; +import { QueryStatus } from '@reduxjs/toolkit/query'; import { type AuthzMapping, AuthzOwnerMutationRequest, @@ -14,17 +16,12 @@ import { DeleteAuthzResourceRequest, type ServiceAndMethod, } from './types'; -import { GEN3_AUTHZ_API, GEN3_FENCE_API } from '../../constants'; +import { GEN3_AUTHZ_API } from '../../constants'; import { userAuthApi, selectUserDetailsFromState } from '../user/userSliceRTK'; import type { CoreState } from '../../reducers'; const TAGS = 'authz'; -interface FenceUserAuthzResponse { - readonly authz?: AuthzMapping; - readonly project_access?: Record>; -} - const isServiceAndMethod = (value: unknown): value is ServiceAndMethod => { if (typeof value !== 'object' || value === null) return false; @@ -72,7 +69,7 @@ export const authzApi = authzTags.injectEndpoints({ endpoints: (builder) => ({ getAuthzMappings: builder.query({ providesTags: [TAGS], - async queryFn(_arg, api, _extraOptions, baseQuery) { + async queryFn(_arg, api) { const userDetailsState = selectUserDetailsFromState( api.getState() as CoreState, ); @@ -86,6 +83,20 @@ export const authzApi = authzTags.injectEndpoints({ }; } + const cachedUserError = userDetailsState?.error; + if ( + userDetailsState?.status === QueryStatus.rejected && + cachedUserError && + typeof cachedUserError === 'object' && + 'status' in cachedUserError && + cachedUserError.status === 401 + ) { + // An unauthenticated user has no authz mapping. Treat the cached 401 + // as a terminal empty mapping instead of repeatedly dispatching the + // same Fence /user request from every authz consumer. + return { data: {} }; + } + const userResult = await api.dispatch( userAuthApi.endpoints.fetchUserDetails.initiate(undefined, { forceRefetch: false, @@ -109,22 +120,25 @@ export const authzApi = authzTags.injectEndpoints({ } } - const response = await baseQuery({ - url: `${GEN3_FENCE_API}/user`, - method: 'GET', - credentials: 'include', - }); - - if (response.error) { - return { error: response.error }; - } + const requestError = + 'error' in userResult ? userResult.error : undefined; + const error: FetchBaseQueryError = + requestError && + typeof requestError === 'object' && + 'status' in requestError + ? (requestError as FetchBaseQueryError) + : { + status: 'CUSTOM_ERROR', + error: + requestError && + typeof requestError === 'object' && + 'message' in requestError && + typeof requestError.message === 'string' + ? requestError.message + : 'Fence user details were unavailable', + }; - const fenceResponse = response.data as FenceUserAuthzResponse; - return { - data: normalizeFenceAuthzMapping( - fenceResponse.authz ?? fenceResponse.project_access, - ), - }; + return { error }; }, }), getAuthzResources: builder.query({ diff --git a/packages/core/src/features/fence/fenceApi.ts b/packages/core/src/features/fence/fenceApi.ts index 9594dbd0..bdcdf9fb 100644 --- a/packages/core/src/features/fence/fenceApi.ts +++ b/packages/core/src/features/fence/fenceApi.ts @@ -63,6 +63,8 @@ export interface FetchRequest { readonly body?: object; readonly headers?: Record; readonly isJSON?: boolean; + readonly signal?: AbortSignal; + readonly timeoutMs?: number; } /** diff --git a/packages/core/src/features/fence/index.ts b/packages/core/src/features/fence/index.ts index 6f82f3d6..b1395e2d 100644 --- a/packages/core/src/features/fence/index.ts +++ b/packages/core/src/features/fence/index.ts @@ -18,12 +18,17 @@ import { } from './credentialsApi'; import { useGetJWKKeysQuery } from './jwtApi'; -import { FetchError, Gen3FenceResponse } from './types'; +import { + FetchError, + FenceRequestFailure, + Gen3FenceResponse, +} from './types'; import { fetchFence, isFetchError } from './utils'; export { type Gen3FenceResponse, type FetchError, + type FenceRequestFailure, type FetchRequest, type APIKey, type Gen3FenceCredentials, diff --git a/packages/core/src/features/fence/types.ts b/packages/core/src/features/fence/types.ts index 2d8a0d2d..223fa746 100644 --- a/packages/core/src/features/fence/types.ts +++ b/packages/core/src/features/fence/types.ts @@ -8,6 +8,10 @@ export interface FetchError { readonly request?: T; } +export interface FenceRequestFailure extends FetchError { + readonly kind: 'http' | 'timeout' | 'network' | 'aborted'; +} + export interface Gen3FenceResponse { readonly data: H; readonly status: number; // HTTP Status code diff --git a/packages/core/src/features/fence/utils.ts b/packages/core/src/features/fence/utils.ts index d579ea6a..ad2963cb 100644 --- a/packages/core/src/features/fence/utils.ts +++ b/packages/core/src/features/fence/utils.ts @@ -1,4 +1,8 @@ -import { FetchError, Gen3FenceResponse } from './types'; +import { + FetchError, + FenceRequestFailure, + Gen3FenceResponse, +} from './types'; import { GEN3_FENCE_API } from '../../constants'; import { FetchRequest } from './fenceApi'; @@ -43,13 +47,47 @@ export const fetchFence = async ({ body = {}, method = 'GET', isJSON = true, + signal, + timeoutMs = 12_000, }: FetchRequest): Promise> => { - const res = await fetch(`${GEN3_FENCE_API}${endpoint}`, { - method: method, - credentials: 'include', - headers: headers, - body: 'POST' === method ? JSON.stringify(body) : null, - }); + const controller = new AbortController(); + let timedOut = false; + const abortFromCaller = () => controller.abort(signal?.reason); + signal?.addEventListener('abort', abortFromCaller, { once: true }); + const timeout = setTimeout(() => { + timedOut = true; + controller.abort(); + }, timeoutMs); + + let res: Response; + try { + res = await fetch(`${GEN3_FENCE_API}${endpoint}`, { + method: method, + credentials: 'include', + headers: headers, + body: 'POST' === method ? JSON.stringify(body) : null, + cache: 'no-store', + signal: controller.signal, + }); + } catch (error: unknown) { + const aborted = controller.signal.aborted; + const failure: FenceRequestFailure = { + kind: timedOut ? 'timeout' : aborted ? 'aborted' : 'network', + url: `${GEN3_FENCE_API}${endpoint}`, + status: timedOut ? 408 : 0, + statusText: timedOut + ? 'Fence request timed out' + : aborted + ? 'Fence request was aborted' + : 'Fence request failed', + text: error instanceof Error ? error.message : String(error), + request: { endpoint, method, headers, body, isJSON, timeoutMs }, + }; + throw failure; + } finally { + clearTimeout(timeout); + signal?.removeEventListener('abort', abortFromCaller); + } if (res.ok) return { @@ -57,10 +95,11 @@ export const fetchFence = async ({ status: res.status, }; - throw await buildFetchError(res, { + const failure = await buildFetchError(res, { endpoint, method, headers, body, }); + throw { ...failure, kind: 'http' } satisfies FenceRequestFailure; }; diff --git a/packages/core/src/features/fence/utils.unit.test.ts b/packages/core/src/features/fence/utils.unit.test.ts new file mode 100644 index 00000000..9e0fd041 --- /dev/null +++ b/packages/core/src/features/fence/utils.unit.test.ts @@ -0,0 +1,63 @@ +import { fetchFence } from './utils'; + +describe('fetchFence request guardrails', () => { + afterEach(() => { + jest.useRealTimers(); + jest.restoreAllMocks(); + }); + + it('aborts and classifies a request that exceeds its deadline', async () => { + jest.useFakeTimers(); + jest.spyOn(global, 'fetch').mockImplementation((_input, init) => { + return new Promise((_resolve, reject) => { + init?.signal?.addEventListener('abort', () => { + reject(new DOMException('aborted', 'AbortError')); + }); + }); + }); + + const request = fetchFence({ endpoint: '/user', timeoutMs: 25 }); + await jest.advanceTimersByTimeAsync(25); + + await expect(request).rejects.toMatchObject({ + kind: 'timeout', + status: 408, + statusText: 'Fence request timed out', + }); + }); + + it('returns structured HTTP failures', async () => { + jest.spyOn(global, 'fetch').mockResolvedValue( + new Response('Please login', { + status: 401, + statusText: 'Unauthorized', + }), + ); + + await expect(fetchFence({ endpoint: '/user' })).rejects.toMatchObject({ + kind: 'http', + status: 401, + statusText: 'Unauthorized', + text: 'Please login', + }); + }); + + it('forwards caller cancellation without reporting a timeout', async () => { + const caller = new AbortController(); + jest.spyOn(global, 'fetch').mockImplementation((_input, init) => { + return new Promise((_resolve, reject) => { + init?.signal?.addEventListener('abort', () => { + reject(new DOMException('aborted', 'AbortError')); + }); + }); + }); + + const request = fetchFence({ endpoint: '/user', signal: caller.signal }); + caller.abort(); + + await expect(request).rejects.toMatchObject({ + kind: 'aborted', + status: 0, + }); + }); +}); diff --git a/packages/core/src/features/loom/filters.ts b/packages/core/src/features/loom/filters.ts new file mode 100644 index 00000000..2fdcc529 --- /dev/null +++ b/packages/core/src/features/loom/filters.ts @@ -0,0 +1,96 @@ +import type { + Excludes, + ExcludeIfAny, + FilterSet, + Includes, + Intersection, + Operation, +} from '../filters'; +import type { LoomFilter } from './types'; + +const scalarFilter = ( + column: string, + op: string, + value: unknown, +): LoomFilter => ({ column, op, value }); + +const convertOperation = ( + column: string, + operation: Operation, +): Array => { + switch (operation.operator) { + case '=': + return [scalarFilter(column, 'EQ', operation.operand)]; + case '!=': + return [scalarFilter(column, 'NEQ', operation.operand)]; + case '<': + case '<=': + case '>': + case '>=': + return [scalarFilter(column, operation.operator, operation.operand)]; + case 'in': + case 'includes': + return [scalarFilter(column, 'IN', (operation as Includes).operands)]; + case 'excludes': + return [scalarFilter(column, 'NOT_IN', (operation as Excludes).operands)]; + case 'excludeifany': + return [ + scalarFilter(column, 'NOT_IN', (operation as ExcludeIfAny).operands), + ]; + case 'missing': + return [scalarFilter(column, 'IS_NULL', null)]; + case 'exists': + return [scalarFilter(column, 'IS_NOT_NULL', null)]; + case 'and': + return (operation as Intersection).operands.flatMap((child) => + convertOperation(column, child), + ); + case 'or': + throw new Error( + `Unsupported Loom filter union for column ${column}; use a supported scalar or IN filter`, + ); + case 'nested': { + let nestedOperation: Operation = operation.operand; + const nestedPath = [operation.path]; + while (nestedOperation.operator === 'nested') { + nestedPath.push(nestedOperation.path); + nestedOperation = nestedOperation.operand; + } + const leafField = 'field' in nestedOperation ? nestedOperation.field : ''; + const flattenedColumn = column.includes('.') + ? column + : [...nestedPath, leafField].filter(Boolean).join('.'); + return convertOperation(flattenedColumn, nestedOperation); + } + default: + throw new Error('Unsupported Loom filter operation'); + } +}; + +export const convertFilterSetToLoomFilters = ( + filters?: FilterSet, +): Array => { + if (!filters) return []; + if (filters.mode === 'or') { + throw new Error( + 'Unsupported Loom filter union at the filter-set level; use a supported scalar or IN filter', + ); + } + return Object.entries(filters.root).flatMap(([column, operation]) => + convertOperation(column, operation), + ); +}; + +export const assertLoomFilterable = ( + filters: ReadonlyArray, + columns: ReadonlyArray<{ name: string; filterable: boolean }>, +) => { + const columnMap = new Map(columns.map((column) => [column.name, column])); + for (const filter of filters) { + const column = columnMap.get(filter.column); + if (!column) throw new Error(`Unknown Loom filter column: ${filter.column}`); + if (!column.filterable) { + throw new Error(`Loom column is not filterable: ${filter.column}`); + } + } +}; diff --git a/packages/core/src/features/loom/index.ts b/packages/core/src/features/loom/index.ts new file mode 100644 index 00000000..2777a32d --- /dev/null +++ b/packages/core/src/features/loom/index.ts @@ -0,0 +1,7 @@ +export * from './loomApi'; +export * from './loomSlice'; +export * from './loomDownload'; +export * from './mapping'; +export * from './filters'; +export * from './processing'; +export * from './types'; diff --git a/packages/core/src/features/loom/loomApi.ts b/packages/core/src/features/loom/loomApi.ts new file mode 100644 index 00000000..c84c1d60 --- /dev/null +++ b/packages/core/src/features/loom/loomApi.ts @@ -0,0 +1,83 @@ +import type { BaseQueryFn } from '@reduxjs/toolkit/query'; +import { createApi } from '@reduxjs/toolkit/query/react'; +import { getCookie } from 'cookies-next'; +import { GEN3_LOOM_API } from '../../constants'; +import { selectCSRFToken } from '../user/userSliceRTK'; +import type { CoreState } from '../../reducers'; +import type { LoomApiError, LoomQueryArgs } from './types'; + +interface GraphQLResponse { + readonly data?: T; + readonly errors?: ReadonlyArray<{ + readonly message: string; + readonly extensions?: { readonly code?: string }; + }>; +} + +const loomBaseQuery: BaseQueryFn< + LoomQueryArgs, + unknown, + LoomApiError +> = async ({ query, variables }, api) => { + const csrfToken = selectCSRFToken(api.getState() as CoreState); + const headers: Record = { + Accept: 'application/json', + 'Content-Type': 'application/json', + ...(csrfToken ? { 'X-CSRF-Token': csrfToken } : {}), + }; + if (process.env.NODE_ENV === 'development') { + const accessToken = getCookie('credentials_token'); + if (accessToken) headers.Authorization = `Bearer ${accessToken}`; + } + + try { + const response = await fetch(`${GEN3_LOOM_API}/graphql/flat`, { + method: 'POST', + credentials: 'include', + headers, + body: JSON.stringify({ query, variables }), + signal: api.signal, + cache: 'no-store', + }); + const payload = (await response.json()) as GraphQLResponse; + if (!response.ok) { + return { + error: { + status: response.status, + data: payload, + code: payload.errors?.[0]?.extensions?.code, + }, + }; + } + if (payload.errors?.length) { + const first = payload.errors[0]; + return { + error: { + status: 'CUSTOM_ERROR', + error: first.message, + data: payload, + code: first.extensions?.code, + }, + }; + } + return { data: payload.data }; + } catch (error: unknown) { + return { + error: { + status: 'FETCH_ERROR', + error: error instanceof Error ? error.message : String(error), + }, + }; + } +}; + +export const loomApi = createApi({ + reducerPath: 'loom', + baseQuery: loomBaseQuery, + tagTypes: ['LOOM_DATASET', 'LOOM_ROWS', 'LOOM_AGGREGATE'], + endpoints: () => ({}), +}); + +export const loomApiSliceMiddleware = loomApi.middleware; +export const loomApiSliceReducerPath = loomApi.reducerPath; +export const loomApiReducer = loomApi.reducer; diff --git a/packages/core/src/features/loom/loomDownload.ts b/packages/core/src/features/loom/loomDownload.ts new file mode 100644 index 00000000..3ea4560a --- /dev/null +++ b/packages/core/src/features/loom/loomDownload.ts @@ -0,0 +1,121 @@ +import { getCookie } from 'cookies-next'; +import { GEN3_LOOM_API } from '../../constants'; +import { coreStore } from '../../store'; +import { selectCSRFToken } from '../user'; +import { isJSONObject, JSONObject } from '../../types'; +import { convertFilterSetToLoomFilters } from './filters'; +import { toLoomDataType } from './mapping'; +import type { FilterSet } from '../filters'; +import type { LoomSort } from './types'; + +export interface LoomDownloadParams { + readonly type: string; + readonly fields: ReadonlyArray; + readonly filter?: FilterSet; + readonly sort?: unknown; + readonly format: 'json' | 'csv' | 'tsv' | 'jsonl'; + readonly filename?: string; +} + +export interface DownloadFromLoomParams { + readonly parameters: LoomDownloadParams; + readonly onStart?: () => void; + readonly onDone?: (blob: Blob) => void; + readonly onError?: (error: Error) => void; + readonly onAbort?: () => void; + readonly signal?: AbortSignal; +} + +const normalizeSort = (sort: unknown): LoomSort | undefined => { + if (!sort) return undefined; + if (Array.isArray(sort)) { + const first = sort[0]; + if (typeof first === 'object' && first !== null) { + const [column, direction] = Object.entries(first as Record)[0] ?? []; + if (column) return { column, desc: direction === 'desc' }; + } + return undefined; + } + if (typeof sort === 'object') { + const candidate = sort as Partial; + if (typeof candidate.column === 'string') return candidate as LoomSort; + } + return undefined; +}; + +const buildRequest = (parameters: LoomDownloadParams) => ({ + dataType: toLoomDataType(parameters.type), + columns: [...parameters.fields], + filters: convertFilterSetToLoomFilters(parameters.filter), + sort: normalizeSort(parameters.sort), + format: parameters.format.toUpperCase(), + filename: parameters.filename, +}); + +const fetchLoomExport = async ( + parameters: LoomDownloadParams, + signal?: AbortSignal, +): Promise => { + const csrfToken = selectCSRFToken(coreStore.getState()); + const headers: Record = { + Accept: 'application/octet-stream', + 'Content-Type': 'application/json', + ...(csrfToken ? { 'X-CSRF-Token': csrfToken } : {}), + }; + if (process.env.NODE_ENV === 'development') { + const accessToken = getCookie('credentials_token'); + if (accessToken) headers.Authorization = `Bearer ${accessToken}`; + } + return fetch(`${GEN3_LOOM_API}/api/v1/dataframe/export`, { + method: 'POST', + credentials: 'include', + headers, + body: JSON.stringify(buildRequest(parameters)), + signal, + }); +}; + +export const downloadFromLoomToBlob = async ({ + parameters, + onStart = () => null, + onDone = (_: Blob) => null, + onError = (_: Error) => null, + onAbort = () => null, + signal, +}: DownloadFromLoomParams) => { + onStart(); + try { + const response = await fetchLoomExport(parameters, signal); + if (!response.ok) { + throw new Error((await response.text()) || response.statusText); + } + onDone(await response.blob()); + } catch (error: unknown) { + if (error instanceof Error && error.name === 'AbortError') { + onAbort(); + return; + } + onError(error instanceof Error ? error : new Error(String(error))); + } +}; + +export const downloadJSONDataFromLoom = async ({ + parameters, + onAbort = () => null, + signal, +}: Omit): Promise => { + try { + const response = await fetchLoomExport( + { ...parameters, format: 'json' }, + signal, + ); + if (!response.ok) { + throw new Error((await response.text()) || response.statusText); + } + const value = await response.json(); + return Array.isArray(value) ? value.filter(isJSONObject) : []; + } catch (error: unknown) { + if (error instanceof Error && error.name === 'AbortError') onAbort(); + throw error instanceof Error ? error : new Error(String(error)); + } +}; diff --git a/packages/core/src/features/loom/loomSlice.ts b/packages/core/src/features/loom/loomSlice.ts new file mode 100644 index 00000000..58f2d8b6 --- /dev/null +++ b/packages/core/src/features/loom/loomSlice.ts @@ -0,0 +1,208 @@ +import type { AggregationsData } from '../../types'; +import { loomApi } from './loomApi'; +import { shapeLoomRows, toHistogramKey } from './processing'; +import type { + LoomAggregateRequest, + LoomAggregateResponse, + LoomAggregationsRequest, + LoomDataset, + LoomRowsRequest, + LoomRowsResponse, + LoomQueryArgs, +} from './types'; + +const datasetFields = ` + id name revision state rowCount createdAt readyAt error + columns { name clickhouseType logicalType nullable repeated filterable sortable aggregatable } +`; + +const aggregateFields = ` + materialization { ${datasetFields} } + columns + rows +`; + +const rowsFields = ` + materialization { ${datasetFields} } + columns + rows + totalCount + pageInfo { hasNextPage endCursor } +`; + +export const buildLoomDatasetQuery = (dataType: string): LoomQueryArgs => ({ + query: `query LoomDataset($dataType: String!) { dataframeDataset(input: { dataType: $dataType }) { ${datasetFields} } }`, + variables: { dataType }, +}); + +export const buildLoomRowsQuery = (input: LoomRowsRequest): LoomQueryArgs => ({ + query: `query LoomRows($input: DataframeRowsInput!) { dataframeRows(input: $input) { ${rowsFields} } }`, + variables: { + input: { + ...input, + columns: input.columns ? [...input.columns] : undefined, + filters: input.filters ? [...input.filters] : undefined, + }, + }, +}); + +export const buildLoomAggregateQuery = ( + input: LoomAggregateRequest, +): LoomQueryArgs => ({ + query: `query LoomAggregate($input: DataframeAggregateInput!) { dataframeAggregate(input: $input) { ${aggregateFields} } }`, + variables: { + input: { + ...input, + groupBy: input.groupBy ? [...input.groupBy] : undefined, + filters: input.filters ? [...input.filters] : undefined, + }, + }, +}); + +export const buildLoomCountQuery = ( + input: LoomAggregateRequest, +): LoomQueryArgs => ({ + query: `query LoomCount($input: DataframeAggregateInput!) { dataframeAggregate(input: $input) { columns rows } }`, + variables: { + input: { + dataType: input.dataType, + filters: input.filters ? [...input.filters] : undefined, + operation: 'COUNT', + }, + }, +}); + +export const buildLoomAggregationsQuery = ( + input: LoomAggregationsRequest, +): LoomQueryArgs => { + const selections = input.fields + .map( + (field, index) => + `a${index}: dataframeAggregate(input: { dataType: $dataType, groupBy: [${JSON.stringify(field)}], filters: $filters, operation: "COUNT", column: ${JSON.stringify(field)} }) { columns rows }`, + ) + .join('\n'); + return { + query: `query LoomAggregations($dataType: String!, $filters: [DataframeFilterInput!]) { ${selections} }`, + variables: { + dataType: input.dataType, + filters: input.filters ? [...input.filters] : [], + }, + }; +}; + +const normalizeRowsResponse = ( + response: Omit & { rows: unknown }, +): LoomRowsResponse => ({ + ...response, + rows: shapeLoomRows(response.rows, response.columns), +}); + +const normalizeAggregateResponse = ( + response: Omit & { rows: unknown }, +): LoomAggregateResponse => ({ + ...response, + rows: shapeLoomRows(response.rows, response.columns), +}); + +export const loomTags = loomApi.enhanceEndpoints({ + addTagTypes: ['LOOM_DATASET', 'LOOM_ROWS', 'LOOM_AGGREGATE'], +}); + +export const loomSlice = loomTags.injectEndpoints({ + endpoints: (builder) => ({ + getLoomDatasets: builder.query, void>({ + query: () => ({ + query: `query LoomDatasets { dataframeDatasets { ${datasetFields} } }`, + }), + transformResponse: (response: { dataframeDatasets: Array }) => + response.dataframeDatasets ?? [], + providesTags: ['LOOM_DATASET'], + }), + getLoomDataset: builder.query({ + query: buildLoomDatasetQuery, + transformResponse: (response: { dataframeDataset: LoomDataset | null }) => + response.dataframeDataset, + providesTags: (_result, _error, dataType) => [ + { type: 'LOOM_DATASET', id: dataType }, + ], + }), + getLoomRows: builder.query({ + query: buildLoomRowsQuery, + transformResponse: normalizeRowsResponse, + providesTags: (_result, _error, input) => [ + { type: 'LOOM_ROWS', id: input.dataType }, + ], + }), + getLoomAggregate: builder.query< + LoomAggregateResponse, + LoomAggregateRequest + >({ + query: buildLoomAggregateQuery, + transformResponse: normalizeAggregateResponse, + providesTags: (_result, _error, input) => [ + { type: 'LOOM_AGGREGATE', id: input.dataType }, + ], + }), + getLoomCount: builder.query({ + query: buildLoomCountQuery, + transformResponse: (response: { dataframeAggregate: { rows: unknown } }) => { + const rows = Array.isArray(response.dataframeAggregate?.rows) + ? response.dataframeAggregate.rows + : []; + const row = rows[0]; + if (typeof row === 'number') return row; + if (row && typeof row === 'object') { + const values = Object.values(row as Record); + return Number(values[0]) || 0; + } + return 0; + }, + providesTags: (_result, _error, input) => [ + { type: 'LOOM_AGGREGATE', id: input.dataType }, + ], + }), + getLoomAggregations: builder.query< + Record, + LoomAggregationsRequest + >({ + query: buildLoomAggregationsQuery, + transformResponse: ( + response: Record, + _meta, + args, + ): AggregationsData => { + const aggregations: AggregationsData = {}; + + args.fields.forEach((field, index) => { + const value = response[`a${index}`]; + const rows = shapeLoomRows(value?.rows, value?.columns ?? []); + aggregations[field] = rows.map((row) => ({ + key: toHistogramKey( + row.key ?? row[field] ?? row[value?.columns?.[0] ?? ''], + ), + count: + Number( + row.doc_count ?? + row.count ?? + row[value?.columns?.[1] ?? ''], + ) || 0, + })); + }); + + return aggregations; + }, + providesTags: (_result, _error, input) => [ + { type: 'LOOM_AGGREGATE', id: input.dataType }, + ], + }), + }), +}); + +export const { + useGetLoomDatasetsQuery, + useGetLoomDatasetQuery, + useGetLoomRowsQuery, + useGetLoomAggregateQuery, + useGetLoomCountQuery, + useGetLoomAggregationsQuery, +} = loomSlice; diff --git a/packages/core/src/features/loom/mapping.ts b/packages/core/src/features/loom/mapping.ts new file mode 100644 index 00000000..c9dae1c2 --- /dev/null +++ b/packages/core/src/features/loom/mapping.ts @@ -0,0 +1,30 @@ +import { + ExplorerDataType, + LoomDataType, + LOOM_DATA_TYPES, +} from './types'; + +export const EXPLORER_TO_LOOM_DATA_TYPE: Readonly< + Record +> = { + file: 'DocumentReference', + document_reference: 'DocumentReference', + research_subject: 'ResearchSubject', + specimen: 'Specimen', + medication_administration: 'MedicationAdministration', + group_member: 'GroupMember', +}; + +export const isExplorerDataType = ( + value: string, +): value is ExplorerDataType => value in EXPLORER_TO_LOOM_DATA_TYPE; + +export const isLoomDataType = (value: string): value is LoomDataType => + (LOOM_DATA_TYPES as ReadonlyArray).includes(value); + +export const toLoomDataType = (dataType: string): LoomDataType => { + if (!isExplorerDataType(dataType)) { + throw new Error(`Unsupported Explorer data type: ${dataType}`); + } + return EXPLORER_TO_LOOM_DATA_TYPE[dataType]; +}; diff --git a/packages/core/src/features/loom/processing.ts b/packages/core/src/features/loom/processing.ts new file mode 100644 index 00000000..a6077f7c --- /dev/null +++ b/packages/core/src/features/loom/processing.ts @@ -0,0 +1,95 @@ +import type { AggregationsData, JSONObject, StatsData } from '../../types'; +import type { LoomAggregateResponse, LoomColumn } from './types'; + +const isObject = (value: unknown): value is JSONObject => + typeof value === 'object' && value !== null && !Array.isArray(value); + +const assignDottedValue = (row: JSONObject, path: string, value: unknown) => { + const parts = path.split('.').filter(Boolean); + if (parts.length < 2) { + row[path] = value as never; + return; + } + + let cursor = row; + for (const part of parts.slice(0, -1)) { + const existing = cursor[part]; + if (!isObject(existing)) cursor[part] = {}; + cursor = cursor[part] as JSONObject; + } + cursor[parts[parts.length - 1]] = value as never; +}; + +export const shapeLoomRow = ( + row: unknown, + columns: ReadonlyArray, +): JSONObject => { + const source = Array.isArray(row) + ? Object.fromEntries(columns.map((column, index) => [column, row[index]])) + : isObject(row) + ? row + : {}; + const shaped: JSONObject = {}; + Object.entries(source).forEach(([key, value]) => + assignDottedValue(shaped, key, value), + ); + return shaped; +}; + +export const shapeLoomRows = ( + rows: unknown, + columns: ReadonlyArray, +): Array => + Array.isArray(rows) ? rows.map((row) => shapeLoomRow(row, columns)) : []; + +export const columnsToFieldMapping = ( + columns: ReadonlyArray, +): Record => + Object.fromEntries(columns.map((column) => [column.name, column])); + +const numericValue = (value: unknown): number => { + const result = typeof value === 'number' ? value : Number(value); + return Number.isFinite(result) ? result : 0; +}; + +export const toHistogramKey = (value: unknown): string | [number, number] => { + if ( + Array.isArray(value) && + value.length === 2 && + value.every((item) => typeof item === 'number') + ) { + return [value[0], value[1]]; + } + + return typeof value === 'string' ? value : String(value ?? ''); +}; + +export const aggregateToHistogram = ( + response: LoomAggregateResponse, + field: string, +): AggregationsData => { + const rows = response.rows.map((row) => { + const key = row.key ?? row[field] ?? row[response.columns[0]]; + const count = row.doc_count ?? row.count ?? row[response.columns[1]]; + return { key: toHistogramKey(key), count: numericValue(count) }; + }); + return { [field]: rows }; +}; + +export const aggregateToStats = ( + response: LoomAggregateResponse, + field: string, +): StatsData => { + const row = response.rows[0] ?? {}; + return { + [field]: [ + { + count: numericValue(row.count), + min: numericValue(row.min), + max: numericValue(row.max), + avg: numericValue(row.avg), + sum: numericValue(row.sum), + }, + ], + }; +}; diff --git a/packages/core/src/features/loom/tests/mapping.unit.test.ts b/packages/core/src/features/loom/tests/mapping.unit.test.ts new file mode 100644 index 00000000..b304f0c0 --- /dev/null +++ b/packages/core/src/features/loom/tests/mapping.unit.test.ts @@ -0,0 +1,64 @@ +import { + EXPLORER_TO_LOOM_DATA_TYPE, + isExplorerDataType, + toLoomDataType, +} from '../mapping'; +import { convertFilterSetToLoomFilters } from '../filters'; + +describe('Explorer to Loom mapping', () => { + it.each([ + ['file', 'DocumentReference'], + ['document_reference', 'DocumentReference'], + ['research_subject', 'ResearchSubject'], + ['specimen', 'Specimen'], + ['medication_administration', 'MedicationAdministration'], + ['group_member', 'GroupMember'], + ])('maps %s to %s', (explorerType, loomType) => { + expect(toLoomDataType(explorerType)).toBe(loomType); + }); + + it('exposes a complete canonical mapping', () => { + expect(Object.keys(EXPLORER_TO_LOOM_DATA_TYPE)).toHaveLength(6); + expect(isExplorerDataType('unknown')).toBe(false); + expect(() => toLoomDataType('unknown')).toThrow( + 'Unsupported Explorer data type: unknown', + ); + }); + + it('translates supported filter operations without using Guppy JSON', () => { + expect( + convertFilterSetToLoomFilters({ + mode: 'and', + root: { + status: { operator: '=', field: 'status', operand: 'active' }, + category: { + operator: 'in', + field: 'category', + operands: ['a', 'b'], + }, + }, + }), + ).toEqual([ + { column: 'status', op: 'EQ', value: 'active' }, + { column: 'category', op: 'IN', value: ['a', 'b'] }, + ]); + }); + + it('flattens Explorer nested field paths and rejects unsupported unions', () => { + expect(() => + convertFilterSetToLoomFilters({ mode: 'or', root: {} }), + ).toThrow('Unsupported Loom filter union'); + expect( + convertFilterSetToLoomFilters({ + mode: 'and', + root: { + 'subject.id': { + operator: 'nested', + path: 'subject', + operand: { operator: '=', field: 'id', operand: '1' }, + }, + }, + }), + ).toEqual([{ column: 'subject.id', op: 'EQ', value: '1' }]); + }); +}); diff --git a/packages/core/src/features/loom/tests/processing.unit.test.ts b/packages/core/src/features/loom/tests/processing.unit.test.ts new file mode 100644 index 00000000..24ba2bbe --- /dev/null +++ b/packages/core/src/features/loom/tests/processing.unit.test.ts @@ -0,0 +1,31 @@ +import { shapeLoomRows, aggregateToHistogram } from '../processing'; + +describe('Loom response adapters', () => { + it('shapes flat dotted columns while preserving null and repeated values', () => { + expect( + shapeLoomRows( + [['r1', null, ['a', 'b']]], + ['subject.id', 'subject.birthDate', 'tags'], + ), + ).toEqual([ + { + subject: { id: 'r1', birthDate: null }, + tags: ['a', 'b'], + }, + ]); + }); + + it('adapts empty pages and grouped counts without inventing rows', () => { + expect(shapeLoomRows([], ['id'])).toEqual([]); + expect( + aggregateToHistogram( + { + materialization: {} as any, + columns: ['status', 'count'], + rows: [{ status: 'active', count: 3 }], + }, + 'status', + ), + ).toEqual({ status: [{ key: 'active', count: 3 }] }); + }); +}); diff --git a/packages/core/src/features/loom/tests/query-contract.unit.test.ts b/packages/core/src/features/loom/tests/query-contract.unit.test.ts new file mode 100644 index 00000000..247a2105 --- /dev/null +++ b/packages/core/src/features/loom/tests/query-contract.unit.test.ts @@ -0,0 +1,63 @@ +import { + buildLoomAggregateQuery, + buildLoomAggregationsQuery, + buildLoomDatasetQuery, + buildLoomRowsQuery, +} from '../loomSlice'; + +describe('Loom GraphQL request contracts', () => { + it('maps a legacy file request at the caller boundary', () => { + const request = buildLoomDatasetQuery('DocumentReference'); + expect(request.variables).toEqual({ dataType: 'DocumentReference' }); + expect(request.query).toContain('dataframeDataset'); + }); + + it('preserves opaque cursor pagination and sort variables', () => { + const request = buildLoomRowsQuery({ + dataType: 'ResearchSubject', + columns: ['id', 'status'], + filters: [{ column: 'status', op: 'EQ', value: 'active' }], + sort: { column: 'id', desc: true }, + first: 10, + after: 'opaque-next-cursor', + }); + expect(request.variables).toEqual({ + input: { + dataType: 'ResearchSubject', + columns: ['id', 'status'], + filters: [{ column: 'status', op: 'EQ', value: 'active' }], + sort: { column: 'id', desc: true }, + first: 10, + after: 'opaque-next-cursor', + }, + }); + }); + + it('builds grouped and filtered aggregate requests', () => { + const aggregate = buildLoomAggregateQuery({ + dataType: 'Specimen', + groupBy: ['status'], + filters: [{ column: 'project_id', op: 'EQ', value: 'p1' }], + operation: 'COUNT', + column: 'status', + }); + expect(aggregate.variables).toMatchObject({ + input: { + dataType: 'Specimen', + groupBy: ['status'], + operation: 'COUNT', + }, + }); + + const aggregations = buildLoomAggregationsQuery({ + dataType: 'MedicationAdministration', + fields: ['status'], + filters: [{ column: 'status', op: 'IN', value: ['active', 'stopped'] }], + }); + expect(aggregations.query).toContain('a0: dataframeAggregate'); + expect(aggregations.variables).toEqual({ + dataType: 'MedicationAdministration', + filters: [{ column: 'status', op: 'IN', value: ['active', 'stopped'] }], + }); + }); +}); diff --git a/packages/core/src/features/loom/types.ts b/packages/core/src/features/loom/types.ts new file mode 100644 index 00000000..1ed7da28 --- /dev/null +++ b/packages/core/src/features/loom/types.ts @@ -0,0 +1,120 @@ +import type { AggregationsData, JSONObject } from '../../types'; +import type { FilterSet } from '../filters'; + +export const LOOM_DATA_TYPES = [ + 'DocumentReference', + 'ResearchSubject', + 'Specimen', + 'MedicationAdministration', + 'GroupMember', +] as const; + +export type LoomDataType = (typeof LOOM_DATA_TYPES)[number]; + +export type ExplorerDataType = + | 'file' + | 'document_reference' + | 'research_subject' + | 'specimen' + | 'medication_administration' + | 'group_member'; + +export interface LoomDatasetRef { + readonly dataType: LoomDataType; +} + +export interface LoomColumn { + readonly name: string; + readonly clickhouseType: string; + readonly logicalType: string; + readonly nullable: boolean; + readonly repeated: boolean; + readonly filterable: boolean; + readonly sortable: boolean; + readonly aggregatable: boolean; +} + +export interface LoomDataset extends LoomDatasetRef { + readonly id: string; + readonly name: string; + readonly revision: string; + readonly state: 'PENDING' | 'LOADING' | 'READY' | 'FAILED' | string; + readonly columns: ReadonlyArray; + readonly rowCount: number; + readonly createdAt: string; + readonly readyAt?: string | null; + readonly error?: string | null; +} + +export interface LoomFilter { + readonly column: string; + readonly op: string; + readonly value: unknown; +} + +export interface LoomSort { + readonly column: string; + readonly desc?: boolean; +} + +export interface LoomRowsRequest { + readonly dataType: LoomDataType; + readonly columns?: ReadonlyArray; + readonly filters?: ReadonlyArray; + readonly sort?: LoomSort; + readonly first?: number; + readonly after?: string | null; +} + +export interface LoomRowsResponse { + readonly materialization: LoomDataset; + readonly columns: ReadonlyArray; + readonly rows: ReadonlyArray; + readonly totalCount?: number | null; + readonly pageInfo: { + readonly hasNextPage: boolean; + readonly endCursor?: string | null; + }; +} + +export interface LoomAggregateRequest { + readonly dataType: LoomDataType; + readonly groupBy?: ReadonlyArray; + readonly filters?: ReadonlyArray; + readonly operation: string; + readonly column?: string; +} + +export interface LoomAggregateResponse { + readonly materialization: LoomDataset; + readonly columns: ReadonlyArray; + readonly rows: ReadonlyArray; +} + +export interface LoomAggregationsRequest { + readonly dataType: LoomDataType; + readonly fields: ReadonlyArray; + readonly filters?: ReadonlyArray; +} + +export interface LoomAggregationsResponse { + readonly materialization: LoomDataset | null; + readonly data: AggregationsData; +} + +export interface LoomApiError { + readonly status: number | 'CUSTOM_ERROR' | 'FETCH_ERROR'; + readonly data?: unknown; + readonly error?: string; + readonly code?: string; +} + +export interface LoomQueryArgs { + readonly query: string; + readonly variables?: Record; +} + +export interface LoomFilterRequest { + readonly dataType: ExplorerDataType; + readonly filters?: FilterSet; +} diff --git a/packages/core/src/features/user/userSliceRTK.ts b/packages/core/src/features/user/userSliceRTK.ts index 340b2bad..2b665c7a 100644 --- a/packages/core/src/features/user/userSliceRTK.ts +++ b/packages/core/src/features/user/userSliceRTK.ts @@ -21,7 +21,7 @@ export const userAuthApi = createApi({ reducerPath: 'userAuthApi', refetchOnMountOrArgChange: 1800, refetchOnReconnect: true, - baseQuery: async ({ endpoint }, { getState }) => { + baseQuery: async ({ endpoint }, { getState, signal }) => { let results; const csrfToken = selectCSRFToken(getState() as CoreState); let accessToken = undefined; @@ -33,17 +33,17 @@ export const userAuthApi = createApi({ 'Content-Type': 'application/json', ...(csrfToken ? { 'X-CSRF-Token': csrfToken } : {}), ...(accessToken ? { Authorization: `Bearer ${accessToken}` } : {}), - credentials: 'include', }; try { - results = await fetchFence({ endpoint, headers }); - } catch (_e: unknown) { - /* - Because an "error" response is valid for the auth requests we don't want to - put the request in an error state, or it will attempt the request over and over again - */ - return { data: {} }; + results = await fetchFence({ endpoint, headers, signal }); + } catch (error: any) { + return { + error: { + status: error?.status || 0, + data: error, + }, + }; } return { data: results }; @@ -64,14 +64,24 @@ export const userAuthApi = createApi({ }, }), getCSRF: builder.query({ - queryFn: async () => { + queryFn: async (_arg, { signal }) => { const headers: Record = { Accept: 'application/json', 'Content-Type': 'application/json', }; + const controller = new AbortController(); + let timedOut = false; + const abortFromCaller = () => controller.abort(signal.reason); + signal.addEventListener('abort', abortFromCaller, { once: true }); + const timeout = setTimeout(() => { + timedOut = true; + controller.abort(); + }, 12_000); try { const res = await fetch(`${GEN3_API}/_status`, { headers: headers, + cache: 'no-store', + signal: controller.signal, }); if (res.ok) { @@ -84,16 +94,22 @@ export const userAuthApi = createApi({ } catch (error: unknown) { if (error instanceof Error) { return { - error: error.message, + error: { + status: timedOut ? 408 : 0, + data: timedOut ? 'Commons status request timed out' : error.message, + }, }; } else { return { - error: 'Unknown Error', + error: { status: 0, data: 'Unknown Error' }, }; } + } finally { + clearTimeout(timeout); + signal.removeEventListener('abort', abortFromCaller); } return { - error: 'Unknown Error', + error: { status: 0, data: 'Unknown Error' }, }; }, }), diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index ae542f7c..67ff4386 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -10,6 +10,7 @@ import { GEN3_GRIP_API, GEN3_GECKO_API, GEN3_GUPPY_API, + GEN3_LOOM_API, GEN3_MANIFEST_API, GEN3_MDS_API, GEN3_REDIRECT_URL, @@ -43,6 +44,7 @@ export * from './features/gen3Apps'; export * from './features/gecko'; export * from './features/graphQL'; export * from './features/guppy'; +export * from './features/loom'; export * from './features/manifest'; export * from './features/metadata'; export * from './features/modals'; @@ -62,6 +64,7 @@ export { GEN3_API, GEN3_DOWNLOADS_ENDPOINT, GEN3_GUPPY_API, + GEN3_LOOM_API, GEN3_GRIP_API, GEN3_GECKO_API, GEN3_FENCE_API, diff --git a/packages/core/src/reducers.ts b/packages/core/src/reducers.ts index a238e9c3..4aa8b56c 100644 --- a/packages/core/src/reducers.ts +++ b/packages/core/src/reducers.ts @@ -19,6 +19,10 @@ import { gripApiReducer, gripApiSliceReducerPath, } from './features/grip/gripApi'; +import { + loomApiReducer, + loomApiSliceReducerPath, +} from './features/loom/loomApi'; export const rootReducer = combineReducers({ gen3Services: gen3ServicesReducer, @@ -30,6 +34,7 @@ export const rootReducer = combineReducers({ activeWorkspace: activeWorkspaceReducer, [guppyApiSliceReducerPath]: guppyApiReducer, [gripApiSliceReducerPath]: gripApiReducer, + [loomApiSliceReducerPath]: loomApiReducer, [userAuthApiReducerPath]: userAuthApiReducer, }); diff --git a/packages/core/src/store.ts b/packages/core/src/store.ts index 0e9cd6d4..1a797a6b 100644 --- a/packages/core/src/store.ts +++ b/packages/core/src/store.ts @@ -5,6 +5,7 @@ import { CoreState, rootReducer } from './reducers'; import { gen3ServicesReducerMiddleware } from './features/gen3/gen3Api'; import { guppyAPISliceMiddleware } from './features/guppy/guppyApi'; import { userAuthApiMiddleware } from './features/user/userSliceRTK'; +import { loomApiSliceMiddleware } from './features/loom/loomApi'; import { coreStoreListenerMiddleware } from './listeners'; import { persistReducer, @@ -50,6 +51,7 @@ export const setupCoreStore = (preloadedState?: Partial) => .concat( gen3ServicesReducerMiddleware, guppyAPISliceMiddleware, + loomApiSliceMiddleware, userAuthApiMiddleware, ) .prepend(coreStoreListenerMiddleware.middleware), // needs to be prepended, diff --git a/packages/frontend/src/api/auth/credentialsLogout.ts b/packages/frontend/src/api/auth/credentialsLogout.ts index 7dcf6643..4d83d619 100644 --- a/packages/frontend/src/api/auth/credentialsLogout.ts +++ b/packages/frontend/src/api/auth/credentialsLogout.ts @@ -9,5 +9,5 @@ export default async function (req: NextApiRequest, res: NextApiResponse) { httpOnly: process.env.NODE_ENV === 'production', secure: process.env.NODE_ENV === 'production', }); - res.redirect(307, '/'); + res.status(204).end(); } diff --git a/packages/frontend/src/components/Login/LoginPanel.tsx b/packages/frontend/src/components/Login/LoginPanel.tsx index 19e056b6..1b25280f 100644 --- a/packages/frontend/src/components/Login/LoginPanel.tsx +++ b/packages/frontend/src/components/Login/LoginPanel.tsx @@ -1,6 +1,5 @@ import React, { useCallback } from 'react'; import { useRouter } from 'next/router'; -import { showNotification } from '@mantine/notifications'; import { Center, Stack } from '@mantine/core'; import TexturedSidePanel from '../Layout/TexturedSidePanel'; import LoginProvidersPanel from './LoginProvidersPanel'; @@ -37,30 +36,18 @@ const LoginPanel = (loginConfig: LoginConfig) => { } = router; const handleFenceLoginSelected = useCallback( - async (loginURL: string) => { - router - .push( - `${appendParameterToUrl(loginURL, 'redirect', filterRedirect(referer))}`, - ) - .catch((e) => { - showNotification({ - title: 'Login Error', - message: `error logging in ${e.message}`, - }); - }); + (loginURL: string) => { + window.location.assign( + appendParameterToUrl(loginURL, 'redirect', filterRedirect(referer)), + ); }, - [referer, router], + [referer], ); const handleCredentialsLogin = useCallback(async () => { const redirect = filterRedirect(referer); - router.push(redirect).catch((e) => { - showNotification({ - title: 'Login Error', - message: `error logging in ${e.message}`, - }); - }); - }, [referer, router]); + window.location.assign(redirect); + }, [referer]); return (
diff --git a/packages/frontend/src/components/Modals/Gen3ModalsProvider.tsx b/packages/frontend/src/components/Modals/Gen3ModalsProvider.tsx index 8d3a062f..4aba7f71 100644 --- a/packages/frontend/src/components/Modals/Gen3ModalsProvider.tsx +++ b/packages/frontend/src/components/Modals/Gen3ModalsProvider.tsx @@ -58,9 +58,10 @@ const Gen3ModalsProvider = ({ config, children, }: Gen3StandardModalsProviderProps) => { + const { isAuthenticated } = useIsAuthenticated(); // TODO: move this to another const { isError } = useGetCSRFQuery(undefined, { refetchOnFocus: true }); - useGetAuthzMappingsQuery(); + useGetAuthzMappingsQuery(undefined, { skip: !isAuthenticated }); const [cookie] = useCookies(['Gen3-first-time-use']); const dispatch = useCoreDispatch(); @@ -72,8 +73,6 @@ const Gen3ModalsProvider = ({ () => defaultComposer(defaultConfig, config), [config], ); - const { isAuthenticated } = useIsAuthenticated(); - useDeepCompareEffect(() => { if ( !cookie['Gen3-first-time-use'] && diff --git a/packages/frontend/src/components/Protected/NoAccessOverlay.tsx b/packages/frontend/src/components/Protected/NoAccessOverlay.tsx index deb25c94..d09ac80a 100644 --- a/packages/frontend/src/components/Protected/NoAccessOverlay.tsx +++ b/packages/frontend/src/components/Protected/NoAccessOverlay.tsx @@ -1,11 +1,13 @@ import React, { useEffect } from 'react'; import { Button } from '@mantine/core'; -import { useFileTotalCountQuery } from '../../pages/Apps/fetchFileCounts'; import { useSession } from '../../lib/session/session'; -export function useHasAccess(authz: any) { - const { data, isLoading, isError } = useFileTotalCountQuery(); - const authzMap = authz ?? {}; +type AccessPermission = { method?: string; service?: string }; +type FenceAuthzMapping = Record; + +export function hasFenceAccess(authz: unknown) { + const authzMap: FenceAuthzMapping = + authz && typeof authz === 'object' ? (authz as FenceAuthzMapping) : {}; const isProgramScopedResource = (resource: string): boolean => { const parts = String(resource).split('/').filter(Boolean); @@ -14,7 +16,7 @@ export function useHasAccess(authz: any) { const hasMeaningfulArboristAccess = ( resource: string, - perms: Array<{ method?: string; service?: string }>, + perms: AccessPermission[], ): boolean => { const parts = String(resource).split('/').filter(Boolean); if (parts.length < 2 || parts[0] !== 'programs') return false; @@ -62,7 +64,9 @@ export function useHasAccess(authz: any) { if (parts[0] !== 'programs' || parts[2] !== 'projects') return false; if (!Array.isArray(perms)) return false; - return perms.some((p: any) => p?.method === 'read' && p?.service === '*'); + return perms.some((permission) => + permission?.method === 'read' && permission?.service === '*', + ); }, ).length; @@ -72,16 +76,11 @@ export function useHasAccess(authz: any) { Array.isArray(perms) && hasMeaningfulArboristAccess( resource, - perms as Array<{ method?: string; service?: string }>, + perms as AccessPermission[], ), ); - const hasAccess = - len_access_projects > 0 || - hasProgramScopedAccess || - (data !== undefined && data > 0); - - return { len_access_projects, fileCount: data, isLoading, isError, hasAccess }; + return len_access_projects > 0 || hasProgramScopedAccess; } diff --git a/packages/frontend/src/components/Protected/ProtectedContent.tsx b/packages/frontend/src/components/Protected/ProtectedContent.tsx index 3957d7fd..f0800229 100644 --- a/packages/frontend/src/components/Protected/ProtectedContent.tsx +++ b/packages/frontend/src/components/Protected/ProtectedContent.tsx @@ -1,7 +1,7 @@ import React, { ReactNode, useEffect, useState } from 'react'; import { useRouter } from 'next/router'; import { useSession } from '../../lib/session/session'; -import { Loader, Text } from '@mantine/core'; +import { Text } from '@mantine/core'; import { type JWTSessionStatus } from '@gen3/core'; import { LoginView } from '../Modals/LoginModal'; @@ -13,9 +13,10 @@ export interface ProtectedContentProps { } import { useGetAuthzMappingsQuery } from '@gen3/core'; -import { useHasAccess, NoAccessOverlay } from './NoAccessOverlay'; +import { hasFenceAccess, NoAccessOverlay } from './NoAccessOverlay'; import { VerifyingAccessLoader } from './VerifyingAccessLoader'; +import SessionFailureView from './SessionFailureView'; export { VerifyingAccessLoader }; const isAppHomePath = (path?: string): boolean => @@ -27,28 +28,40 @@ const AccessGate = ({ onBlocked, }: ProtectedContentProps & { onBlocked: () => void }) => { const router = useRouter(); - const { data: authzMapping = {}, isLoading: isAuthZLoading } = - useGetAuthzMappingsQuery(); - const { hasAccess, isLoading: isFileCountLoading } = - useHasAccess(authzMapping); + const { + data: authzMapping = {}, + isLoading: isAuthZLoading, + isError: isAuthZError, + refetch: refetchAuthz, + } = useGetAuthzMappingsQuery(); + const hasAccess = hasFenceAccess(authzMapping); useEffect(() => { - if (!isAuthZLoading && !isFileCountLoading) { + if (!isAuthZLoading && !isAuthZError) { if (!hasAccess) { onBlocked(); } else { sessionStorage.setItem('hasVerifiedAccess', 'true'); } } - }, [hasAccess, isAuthZLoading, isFileCountLoading, onBlocked]); + }, [hasAccess, isAuthZError, isAuthZLoading, onBlocked]); - if (isAuthZLoading || isFileCountLoading) { + if (isAuthZLoading) { if (isAppHomePath(router.pathname)) { return ; } return null; } + if (isAuthZError) { + return ( + void refetchAuthz()} + /> + ); + } + if (!hasAccess) { return null; // Will unmount shortly because parent will pick up the blocked state } diff --git a/packages/frontend/src/components/Protected/SessionFailureView.tsx b/packages/frontend/src/components/Protected/SessionFailureView.tsx new file mode 100644 index 00000000..3dd8f476 --- /dev/null +++ b/packages/frontend/src/components/Protected/SessionFailureView.tsx @@ -0,0 +1,51 @@ +import React from 'react'; +import { Alert, Button, Group, Stack, Text, Title } from '@mantine/core'; +import { IconAlertTriangle, IconRefresh } from '@tabler/icons-react'; + +type SessionFailureViewProps = { + detail?: string; + onRetry: () => void; + onRestartLogin?: () => void; +}; + +const SessionFailureView = ({ + detail, + onRetry, + onRestartLogin, +}: SessionFailureViewProps) => ( +
+ + } + title="We could not verify your session" + color="orange" + variant="light" + > + + + The authentication service did not respond in time. Your browser is + not stuck, and this page will not keep waiting indefinitely. + + {detail && ( + + {detail} + + )} + + + Try the request again or start a clean sign-in. + + + {onRestartLogin && ( + + )} + + +
+); + +export default SessionFailureView; diff --git a/packages/frontend/src/components/Protected/VerifyingAccessLoader.tsx b/packages/frontend/src/components/Protected/VerifyingAccessLoader.tsx index 2238d94f..b54e9917 100644 --- a/packages/frontend/src/components/Protected/VerifyingAccessLoader.tsx +++ b/packages/frontend/src/components/Protected/VerifyingAccessLoader.tsx @@ -1,11 +1,15 @@ import React from 'react'; import { Loader, Text } from '@mantine/core'; -export const VerifyingAccessLoader = () => ( +export const VerifyingAccessLoader = ({ + message = 'Verifying account access...', +}: { + message?: string; +}) => (
- Verifying account access... + {message}
); diff --git a/packages/frontend/src/components/Protected/index.ts b/packages/frontend/src/components/Protected/index.ts index c267a00c..75a87bb8 100644 --- a/packages/frontend/src/components/Protected/index.ts +++ b/packages/frontend/src/components/Protected/index.ts @@ -1,2 +1,3 @@ import ProtectedContent, { VerifyingAccessLoader } from './ProtectedContent'; +export { default as SessionFailureView } from './SessionFailureView'; export { ProtectedContent, VerifyingAccessLoader }; diff --git a/packages/frontend/src/features/CohortBuilder/CohortPanel.tsx b/packages/frontend/src/features/CohortBuilder/CohortPanel.tsx index 748bed11..c0a08220 100644 --- a/packages/frontend/src/features/CohortBuilder/CohortPanel.tsx +++ b/packages/frontend/src/features/CohortBuilder/CohortPanel.tsx @@ -4,17 +4,20 @@ import { Accessibility, AggregationsData, CombineMode, + convertFilterSetToLoomFilters, CoreState, extractEnumFilterValue, FacetDefinition, FacetType, + isExplorerDataType, isIntersection, - selectCurrentCohortId, selectIndexFilters, selectSharedFilters, useCoreSelector, - useGetAggsQuery, - useGetCountsQuery, + toLoomDataType, + useGetLoomAggregationsQuery, + useGetLoomCountQuery, + useGetLoomDatasetQuery, } from '@gen3/core'; import { type CohortPanelConfiguration, type FileActionsConfig } from './types'; import { type SummaryChart } from '../../components/charts/types'; @@ -103,6 +106,9 @@ export const CohortPanel = ({ }, [isSm, isMd, isXl]); const index = guppyConfig.dataType; + const loomDataType = isExplorerDataType(index) + ? toLoomDataType(index) + : null; const fields = useMemo( () => getAllFieldsFromFilterConfigs(filters?.tabs ?? []), [filters?.tabs], @@ -118,22 +124,40 @@ export const CohortPanel = ({ const cohortFilters = useCoreSelector((state: CoreState) => selectIndexFilters(state, index), ); - const cohortId = useCoreSelector((state: CoreState) => - selectCurrentCohortId(state), - ); + const loomFilters = useMemo(() => { + try { + return { + filters: convertFilterSetToLoomFilters(cohortFilters), + error: null, + }; + } catch (error) { + return { + filters: [], + error: error instanceof Error ? error.message : 'Unsupported Loom filter', + }; + } + }, [cohortFilters]); + const { + data: dataset, + isError: isDatasetError, + isLoading: isDatasetLoading, + } = useGetLoomDatasetQuery(loomDataType ?? 'DocumentReference', { + skip: !loomDataType, + }); const { data, isSuccess, isFetching: isAggsQueryFetching, isError: isAggsQueryError, - } = useGetAggsQuery({ - type: index, + } = useGetLoomAggregationsQuery( + { + dataType: loomDataType ?? 'DocumentReference', fields, - filters: cohortFilters, - accessibility: accessLevel, - queryId: cohortId, - }); + filters: loomFilters.filters, + }, + { skip: !loomDataType || !!loomFilters.error }, + ); const chartKeys = useDeepCompareMemo( () => [...Object.keys(chartsSection?.charts ?? {}), ...Object.keys(charts)], @@ -145,16 +169,13 @@ export const CohortPanel = ({ isSuccess: isChartSuccess, isFetching: isChartFetching, isError: isChartError, - } = useGetAggsQuery( + } = useGetLoomAggregationsQuery( { - type: index, + dataType: loomDataType ?? 'DocumentReference', fields: chartKeys, - filters: cohortFilters, - accessibility: accessLevel, - filterSelf: true, - queryId: cohortId, + filters: loomFilters.filters, }, - { skip: chartKeys.length === 0 }, + { skip: chartKeys.length === 0 || !loomDataType || !!loomFilters.error }, ); const cleanChartData = useDeepCompareMemo(() => { @@ -328,13 +349,34 @@ export const CohortPanel = ({ isFetching: isCountsFetching, isSuccess: isCountSuccess, isError: isCountsError, - } = useGetCountsQuery({ - type: index, - filters: cohortFilters, - accessibility: accessLevel, - queryId: cohortId, - }); + } = useGetLoomCountQuery({ + dataType: loomDataType ?? 'DocumentReference', + filters: loomFilters.filters, + operation: 'COUNT', + }, { skip: !loomDataType || !!loomFilters.error }); + if (!loomDataType) { + return ; + } + if (loomFilters.error) { + return ; + } + if (isDatasetError) { + return ; + } + if (isDatasetLoading) { + return ( +
+
+
+ ); + } + if (!dataset) { + return ; + } + if (dataset.state !== 'READY') { + return ; + } if (isCountsError || isAggsQueryError) { return ; } diff --git a/packages/frontend/src/features/CohortBuilder/ExplorerTable/ExploreTableDetails/QueryRowDetailsPanel.tsx b/packages/frontend/src/features/CohortBuilder/ExplorerTable/ExploreTableDetails/QueryRowDetailsPanel.tsx index b3b26561..6ca27f43 100644 --- a/packages/frontend/src/features/CohortBuilder/ExplorerTable/ExploreTableDetails/QueryRowDetailsPanel.tsx +++ b/packages/frontend/src/features/CohortBuilder/ExplorerTable/ExploreTableDetails/QueryRowDetailsPanel.tsx @@ -1,7 +1,13 @@ import React, { useEffect, useMemo } from 'react'; import { LoadingOverlay, Stack, Table, Text } from '@mantine/core'; import { useDisclosure } from '@mantine/hooks'; -import { useGetRawDataAndTotalCountsQuery } from '@gen3/core'; +import { + convertFilterSetToLoomFilters, + isExplorerDataType, + toLoomDataType, + useGetLoomDatasetQuery, + useGetLoomRowsQuery, +} from '@gen3/core'; import { MdKeyboardDoubleArrowLeft as BackIcon } from 'react-icons/md'; import ErrorCard from '../../../../components/MessageCards/ErrorCard'; import { TableDetailsPanelProps } from './types'; @@ -11,31 +17,15 @@ import { isArray } from 'lodash'; import { useStudyContext } from '../../../Study/StudyProvider'; import { SinglePageStudyDetailsPanel } from '../../../Study'; -interface QueryResponse { - data?: Record>; -} - -function isQueryResponse(obj: any): obj is QueryResponse { - // Considering that the data property can be optional - return ( - typeof obj === 'object' && - (obj.data === undefined || typeof obj.data === 'object') - ); -} - const ExtractData = ( - data: QueryResponse, - index: string, + row: Record | undefined, path?: string, ): Record => { - if (data === undefined || data === null) return {}; - if (data.data === undefined || data.data === null) return {}; - - if (!isArray(data.data[index])) return {}; + if (!row) return {}; - let rowData = data.data[index][0]; + let rowData = row; if (path) { - const tmp = JSONPath({ path: path, json: data.data[index][0] }); + const tmp = JSONPath({ path: path, json: row }); if (!isArray(tmp)) { return {}; } @@ -59,35 +49,55 @@ export const QueryRowDetailsPanel = ({ const { setStudyDetails } = useStudyContext(); const [opened, { open, close }] = useDisclosure(false); - const { data, isError, isFetching } = useGetRawDataAndTotalCountsQuery( + const loomDataType = isExplorerDataType(index) + ? toLoomDataType(index) + : null; + const loomFilters = useMemo(() => { + if (!idField || !id) return { filters: [], error: null }; + try { + return { + filters: convertFilterSetToLoomFilters({ + mode: 'and', + root: { + [idField]: buildNested(idField, { + operator: '=', + field: idField, + operand: id, + }), + }, + }), + error: null, + }; + } catch (error) { + return { + filters: [], + error: error instanceof Error ? error.message : 'Unsupported Loom filter', + }; + } + }, [id, idField]); + const { + data: dataset, + isError: isDatasetError, + isLoading: isDatasetLoading, + } = useGetLoomDatasetQuery(loomDataType ?? 'DocumentReference', { + skip: !loomDataType, + }); + const { data, isError: isRowsError, isFetching } = useGetLoomRowsQuery( { - type: index, - fields: tableConfig.fields as string[], - filters: { - mode: 'and', - root: { - [idField as string]: buildNested(idField as string, { - operator: '=', - field: idField as string, - operand: id as string, - }), - }, - }, - offset: 0, - size: 1, - accessibility: accessibility, + dataType: loomDataType ?? 'DocumentReference', + columns: tableConfig.fields as string[], + filters: loomFilters.filters, + first: 1, }, { - skip: !idField || !id, // if no ide do not send request + skip: !loomDataType || !idField || !id || !!loomFilters.error, }, ); const queryData = useMemo( () => - isQueryResponse(data) - ? ExtractData(data, index, tableConfig?.detailsConfig?.dataPath) - : {}, - [data, index, tableConfig?.detailsConfig?.dataPath], + ExtractData(data?.rows?.[0], tableConfig?.detailsConfig?.dataPath), + [data, tableConfig?.detailsConfig?.dataPath], ); useEffect(() => { @@ -100,9 +110,21 @@ export const QueryRowDetailsPanel = ({ ); } - if (isError) { + if (!loomDataType) { + return ; + } + if (loomFilters.error) { + return ; + } + if (isDatasetError || isRowsError) { return ; } + if (isDatasetLoading) { + return ; + } + if (!dataset || dataset.state !== 'READY') { + return ; + } // Inital attempt at using Study Details component diff --git a/packages/frontend/src/features/CohortBuilder/ExplorerTable/ExplorerTable.tsx b/packages/frontend/src/features/CohortBuilder/ExplorerTable/ExplorerTable.tsx index 48715b2e..9dbf2fe6 100644 --- a/packages/frontend/src/features/CohortBuilder/ExplorerTable/ExplorerTable.tsx +++ b/packages/frontend/src/features/CohortBuilder/ExplorerTable/ExplorerTable.tsx @@ -1,12 +1,16 @@ -import React, { useCallback, useMemo, useState } from 'react'; +import React, { useCallback, useEffect, useMemo, useState } from 'react'; import { useDeepCompareMemo } from 'use-deep-compare'; import { CoreState, + convertFilterSetToLoomFilters, + isExplorerDataType, isJSONValue, JSONObject, selectIndexFilters, + toLoomDataType, useCoreSelector, - useGetRawDataAndTotalCountsQuery, + useGetLoomDatasetQuery, + useGetLoomRowsQuery, } from '@gen3/core'; import { MantineReactTable, @@ -25,13 +29,14 @@ import SubtableStack from './SubTables/SubtableStack'; import { JSONPath } from 'jsonpath-plus'; import { StudyProvider } from '../../Study'; import QueryRowDetailsPanel from './ExploreTableDetails/QueryRowDetailsPanel'; +import { ErrorCard } from '../../../components/MessageCards'; const DEFAULT_PAGE_LIMIT_LABEL = 'Rows per Page (Limited to 10,0000):'; const DEFAULT_PAGE_LIMIT = 10000; /** - * Main table component for the explorer page. Fetches data from guppy using - * useGetRawDataAndTotalCountsQuery() hook that leverages guppy core API slices + * Main table component for the Explorer page. Fetches canonical Loom rows and + * adapts them to the existing table contract. * * @param index - Offset to use for fetching/displaying pages of rows * @param tableConfig - Inherited from ExplorerPageGetServerSideProps @@ -150,21 +155,98 @@ const ExplorerTable = ({ selectIndexFilters(state, index), ); - const { data, isLoading, isError, isFetching } = - useGetRawDataAndTotalCountsQuery({ - type: index, - fields: fields, - filters: cohortFilters, - offset: pagination.pageIndex * pagination.pageSize, - size: pagination.pageSize, - sort: - sorting.length > 0 - ? (sorting.map((x) => { - return { [x.id]: x.desc ? 'desc' : 'asc' }; - }) as Record[]) - : undefined, - accessibility: accessibility, - }); + const loomDataType = isExplorerDataType(index) + ? toLoomDataType(index) + : null; + const loomFilters = useMemo(() => { + try { + return { + filters: convertFilterSetToLoomFilters(cohortFilters), + error: null, + }; + } catch (error) { + return { + filters: [], + error: error instanceof Error ? error.message : 'Unsupported Loom filter', + }; + } + }, [cohortFilters]); + const { + data: dataset, + isError: isDatasetError, + isLoading: isDatasetLoading, + } = useGetLoomDatasetQuery(loomDataType ?? 'DocumentReference', { + skip: !loomDataType, + }); + const [cursorLedger, setCursorLedger] = useState>({ + 0: null, + }); + const querySignature = useMemo( + () => + JSON.stringify({ + loomDataType, + loomFilters: loomFilters.filters, + sorting, + pageSize: pagination.pageSize, + }), + [loomDataType, loomFilters.filters, sorting, pagination.pageSize], + ); + useEffect(() => { + setCursorLedger({ 0: null }); + setPagination((current) => ({ ...current, pageIndex: 0 })); + }, [querySignature]); + + const { + data: loomRows, + isLoading, + isError: isRowsError, + isFetching, + } = useGetLoomRowsQuery( + { + dataType: loomDataType ?? 'DocumentReference', + columns: fields, + filters: loomFilters.filters, + first: pagination.pageSize, + after: cursorLedger[pagination.pageIndex] ?? null, + sort: sorting[0] + ? { column: sorting[0].id, desc: sorting[0].desc } + : undefined, + }, + { skip: !loomDataType || !!loomFilters.error }, + ); + useEffect(() => { + const nextCursor = loomRows?.pageInfo?.endCursor; + if (nextCursor) { + setCursorLedger((current) => + current[pagination.pageIndex + 1] === nextCursor + ? current + : { ...current, [pagination.pageIndex + 1]: nextCursor }, + ); + } + }, [loomRows, pagination.pageIndex]); + + const setTablePagination = useCallback( + ( + updater: + | MRT_PaginationState + | ((current: MRT_PaginationState) => MRT_PaginationState), + ) => { + setPagination((current) => { + const next = + typeof updater === 'function' ? updater(current) : updater; + return next.pageIndex === 0 || next.pageIndex in cursorLedger + ? next + : current; + }); + }, + [cursorLedger], + ); + + const data = useMemo( + () => [...(loomRows?.rows ?? [])], + [loomRows?.rows], + ); + const isError = isRowsError || isDatasetError; const { totalRowCount, limitLabel } = useDeepCompareMemo(() => { const pageLimit = @@ -173,9 +255,9 @@ const ExplorerTable = ({ const totalRowCount = tableConfig?.pageLimit ? Math.min( pageLimit, - data?.data?._aggregation?.[index]._totalCount ?? pagination.pageSize, + loomRows?.totalCount ?? dataset?.rowCount ?? pagination.pageSize, ) - : (data?.data?._aggregation?.[index]._totalCount ?? pagination.pageSize); + : (loomRows?.totalCount ?? dataset?.rowCount ?? pagination.pageSize); const limitLabel = tableConfig?.pageLimit ? (tableConfig?.pageLimit?.label ?? DEFAULT_PAGE_LIMIT_LABEL) : 'Rows per Page:'; @@ -186,7 +268,7 @@ const ExplorerTable = ({ * @see https://www.mantine-react-table.com/docs/api/table-options * @param columns - column options table config * @see https://www.mantine-react-table.com/docs/api/column-options - * @param data - data array, from useGetRawDataAndTotalCountsQuery() + * @param data - data array, from the Loom row adapter * @param manualSorting - If this is true, you will be expected to sort your data before it is passed to the table. * @param manualPagination - If this is true, you will be expected to manually paginate the rows before passing them to the table 0. @@ -202,13 +284,13 @@ const ExplorerTable = ({ const table = useMantineReactTable({ columns: tableColumns as any[], //TODO: fix this - data: data?.data?.[index] ?? [], + data, enableColumnFilters: false, manualSorting: true, manualPagination: true, enableStickyHeader: true, paginateExpandedRows: false, - onPaginationChange: setPagination, + onPaginationChange: setTablePagination, onSortingChange: setSorting, enableTopToolbar: false, enableExpandAll: false, @@ -315,6 +397,33 @@ const ExplorerTable = ({ } : undefined, }); + if (!loomDataType) { + return ; + } + if (loomFilters.error) { + return ; + } + if (isDatasetError) { + return ; + } + if (isDatasetLoading) { + return ( +
+
+
+ ); + } + if (!dataset) { + return ; + } + if (dataset.state !== 'READY') { + return ( + + ); + } + return ( @@ -323,12 +432,12 @@ const ExplorerTable = ({ title={`${String(tableConfig?.detailsConfig?.nodeType).charAt(0).toUpperCase() + String(tableConfig?.detailsConfig?.nodeType).slice(1)} / ${getFieldValue( tableConfig, rowSelection, - data?.data?.[index] ?? [], + data, 'project_id', )} / ${getFieldValue( tableConfig, rowSelection, - data?.data?.[index] ?? [], + data, tableConfig?.detailsConfig?.title as string, )}`} id={ diff --git a/packages/frontend/src/features/CohortBuilder/TabbedCohortBuilder.tsx b/packages/frontend/src/features/CohortBuilder/TabbedCohortBuilder.tsx index 2c1cad0a..25f9b341 100644 --- a/packages/frontend/src/features/CohortBuilder/TabbedCohortBuilder.tsx +++ b/packages/frontend/src/features/CohortBuilder/TabbedCohortBuilder.tsx @@ -1,22 +1,26 @@ -import React, { useEffect, useState } from 'react'; +import React, { useEffect, useMemo, useState } from 'react'; import { useRouter } from 'next/router'; import { Stack } from '@mantine/core'; import { Accessibility, CombineMode, + convertFilterSetToLoomFilters, CoreState, extractEnumFilterValue, FacetDefinition, FacetType, isIntersection, - selectCurrentCohortId, + isExplorerDataType, selectIndexFilters, + toLoomDataType, useCoreSelector, - useGetAggsQuery, - useGetCountsQuery, + useGetLoomAggregationsQuery, + useGetLoomCountQuery, + useGetLoomDatasetQuery, usePrevious, } from '@gen3/core'; import FacetTabs from '../../components/facets/FacetTabs'; +import { ErrorCard } from '../../components/MessageCards'; import { classifyFacets, extractRangeValues, @@ -103,37 +107,59 @@ const TabbedCohortBuilder = ({ Accessibility.ALL, ); - const cohortId = useCoreSelector((state: CoreState) => - selectCurrentCohortId(state), - ); - const cohortFilters = useCoreSelector((state: CoreState) => selectIndexFilters(state, index), ); + const loomDataType = isExplorerDataType(index) + ? toLoomDataType(index) + : null; + const loomFilters = useMemo(() => { + try { + return { + filters: convertFilterSetToLoomFilters(cohortFilters), + error: null, + }; + } catch (error) { + return { + filters: [], + error: error instanceof Error ? error.message : 'Unsupported Loom filter', + }; + } + }, [cohortFilters]); + const { + data: dataset, + isError: isDatasetError, + isLoading: isDatasetLoading, + } = useGetLoomDatasetQuery(loomDataType ?? 'DocumentReference', { + skip: !loomDataType, + }); const { data, isSuccess, isFetching: isAggsQueryFetching, isError: isAggsQueryError, - } = useGetAggsQuery({ - type: index, + } = useGetLoomAggregationsQuery( + { + dataType: loomDataType ?? 'DocumentReference', fields: cohortBuilderFilters, - filters: cohortFilters, - accessibility: accessLevel, - queryId: cohortId, - }); + filters: loomFilters.filters, + }, + { skip: !loomDataType || !!loomFilters.error }, + ); const { data: counts, isSuccess: isCountSuccess, isError, - } = useGetCountsQuery({ - type: index, - filters: cohortFilters, - accessibility: accessLevel, - queryId: cohortId, - }); + } = useGetLoomCountQuery( + { + dataType: loomDataType ?? 'DocumentReference', + filters: loomFilters.filters, + operation: 'COUNT', + }, + { skip: !loomDataType || !!loomFilters.error }, + ); const [facetDefinitions, setFacetDefinitions] = useState< Record @@ -251,6 +277,25 @@ const TabbedCohortBuilder = ({ }; }, [getEnumFacetData, getRangeFacetData, index]); + if (!loomDataType) { + return ; + } + if (loomFilters.error) { + return ; + } + if (isDatasetError) { + return ; + } + if (isDatasetLoading) { + return Loading Loom dataset…; + } + if (!dataset) { + return ; + } + if (dataset.state !== 'READY') { + return ; + } + return ( item[referenceIdFieldInDataIndex], - ); + const refIDList = referenceRows + .map((item) => item[referenceIdFieldInDataIndex]) + .filter( + (value): value is string | number => + typeof value === 'string' || typeof value === 'number', + ); // create a filter of the ids to use in the resource index const refIdsFilter: FilterSet = { mode: 'and', root: { manifest_ids: { operator: 'in', - operands: refIDList as string[], + operands: refIDList, field: referenceIdFieldInResourceIndex, } as Includes, ...(dataFormat @@ -139,7 +141,7 @@ export const downloadToManifestAction = async ( }, }; - let resultManifest = await downloadJSONDataFromGuppy({ + let resultManifest = await downloadJSONDataFromLoom({ parameters: { ...cohortFilterParams, type: resourceIndexType, diff --git a/packages/frontend/src/features/CohortBuilder/downloads/actions/downloadTabular.ts b/packages/frontend/src/features/CohortBuilder/downloads/actions/downloadTabular.ts index 19cf5db5..b0cfe5b0 100644 --- a/packages/frontend/src/features/CohortBuilder/downloads/actions/downloadTabular.ts +++ b/packages/frontend/src/features/CohortBuilder/downloads/actions/downloadTabular.ts @@ -1,4 +1,4 @@ -import { downloadJSONDataFromGuppy, GuppyDownloadDataParams } from '@gen3/core'; +import { downloadJSONDataFromLoom, LoomDownloadParams } from '@gen3/core'; import { handleDownload } from './utils'; import { jsonToCsv } from '../utils/jsonToCsv'; import { ActionButtonWithArgsFunction } from '../../types'; @@ -26,23 +26,21 @@ export const downloadTabularAction: ActionButtonWithArgsFunction = async ( fileFields, type, filter, - accessibility, sort, filename, } = params as DownloadTabularParams; const downloadFilename = filename ?? `${type}_export.csv`; - const cohortFilterParams: GuppyDownloadDataParams = { + const cohortFilterParams: LoomDownloadParams = { filter, type: resourceIndexType || type, fields: fileFields, - accessibility, sort, format: 'json', }; try { - const data = await downloadJSONDataFromGuppy({ + const data = await downloadJSONDataFromLoom({ parameters: cohortFilterParams, onAbort, signal, diff --git a/packages/frontend/src/features/CohortBuilder/downloads/actions/downloadToFile.tsx b/packages/frontend/src/features/CohortBuilder/downloads/actions/downloadToFile.tsx index 59a044e6..402b0aff 100644 --- a/packages/frontend/src/features/CohortBuilder/downloads/actions/downloadToFile.tsx +++ b/packages/frontend/src/features/CohortBuilder/downloads/actions/downloadToFile.tsx @@ -1,4 +1,4 @@ -import { downloadFromGuppyToBlob, GuppyDownloadDataParams } from '@gen3/core'; +import { downloadFromLoomToBlob, LoomDownloadParams } from '@gen3/core'; import { handleDownload } from './utils'; export const downloadToFileAction = async ( @@ -8,9 +8,9 @@ export const downloadToFileAction = async ( onAbort?: () => void, signal?: AbortSignal, ): Promise => { - // call the downloadFromGuppy function - await downloadFromGuppyToBlob({ - parameters: params as GuppyDownloadDataParams, + // Call the principal-scoped Loom export endpoint. + await downloadFromLoomToBlob({ + parameters: params as LoomDownloadParams, onDone: (data: Blob) => { handleDownload(data, params.filename); if (done) done(); diff --git a/packages/frontend/src/features/CohortBuilder/downloads/actions/tests/downloadManifest.unit.test.ts b/packages/frontend/src/features/CohortBuilder/downloads/actions/tests/downloadManifest.unit.test.ts index efb78947..73c3f432 100644 --- a/packages/frontend/src/features/CohortBuilder/downloads/actions/tests/downloadManifest.unit.test.ts +++ b/packages/frontend/src/features/CohortBuilder/downloads/actions/tests/downloadManifest.unit.test.ts @@ -1,6 +1,6 @@ -import { downloadToManifestAction, } from '../downloadManifest'; +import { downloadToManifestAction } from '../downloadManifest'; import { - downloadJSONDataFromGuppy, + downloadJSONDataFromLoom, } from '@gen3/core'; import { handleDownload } from '../utils'; @@ -31,7 +31,7 @@ describe('downloadToManifestAction function', () => { const onAbort = jest.fn(); const signal = {} as AbortSignal; - (downloadJSONDataFromGuppy as jest.Mock).mockReturnValue([ + (downloadJSONDataFromLoom as jest.Mock).mockReturnValue([ { 'object_id': 'mocked-object-id-1', 'md5sum': 'mocked-md5sum-1', @@ -63,14 +63,13 @@ describe('downloadToManifestAction function', () => { expect(handleDownload).toHaveBeenCalled(); expect(done).toHaveBeenCalled(); expect(onError).not.toHaveBeenCalled(); - expect(downloadJSONDataFromGuppy).toHaveBeenCalledWith({ + expect(downloadJSONDataFromLoom).toHaveBeenCalledWith({ onAbort: onAbort, signal: signal, parameters: { filter: params.filter, type: params.type, fields: [params.referenceIdFieldInDataIndex, ...params.fileFields], - accessibility: params.accessibility, sort: params.sort, format: 'json', }, @@ -98,7 +97,7 @@ describe('downloadToManifestAction function', () => { const onAbort = jest.fn(); const signal = {} as AbortSignal; - (downloadJSONDataFromGuppy as jest.Mock).mockResolvedValue([]); + (downloadJSONDataFromLoom as jest.Mock).mockResolvedValue([]); await downloadToManifestAction(params, done, onError, onAbort, signal); diff --git a/packages/frontend/src/lib/session/hooks.ts b/packages/frontend/src/lib/session/hooks.ts index ba33ee1c..c1dde932 100644 --- a/packages/frontend/src/lib/session/hooks.ts +++ b/packages/frontend/src/lib/session/hooks.ts @@ -1,5 +1,3 @@ -import { useState } from 'react'; -import { useDeepCompareEffect } from 'use-deep-compare'; import useSWR from 'swr'; import { AuthTokenData } from './types'; import type { IncomingMessage } from 'http'; @@ -95,43 +93,31 @@ export type SessionContextValue = { export const useManageSession = ( userStatus: LoginStatus, ): SessionContextValue => { - const [session, setSession] = useState({ - status: 'not present', - pending: true, - }); - - useDeepCompareEffect(() => { - if (userStatus === 'authenticated') { - setSession((prev) => ({ - ...prev, - userStatus: 'authenticated', + switch (userStatus) { + case 'authenticated': + return { + userStatus, status: 'issued', pending: false, - })); - } else if (userStatus === 'not present') { - // if (isSuccess && existingSession && existingSession.status === 'issued') { - setSession((prev) => ({ - ...prev, - userStatus: 'not present', + }; + case 'pending': + return { + userStatus, status: 'not present', - pending: false, - })); - } else if (userStatus === 'pending') { - setSession((prev) => ({ - ...prev, pending: true, + }; + case 'unauthenticated': + return { + userStatus, + status: 'invalid', + pending: false, + }; + case 'not present': + default: + return { + userStatus: 'not present', status: 'not present', - userStatus: 'pending', - })); - } else if (userStatus === 'unauthenticated') { - setSession((prev) => ({ - ...prev, pending: false, - status: 'invalid', - userStatus: 'unauthenticated', - })); - } - }, [userStatus]); - - return session; + }; + } }; diff --git a/packages/frontend/src/lib/session/session.tsx b/packages/frontend/src/lib/session/session.tsx index 405c7d5f..cd8a3a62 100644 --- a/packages/frontend/src/lib/session/session.tsx +++ b/packages/frontend/src/lib/session/session.tsx @@ -22,11 +22,12 @@ import { useLazyFetchUserDetailsQuery, } from '@gen3/core'; -import { Center, Loader, Text } from '@mantine/core'; +import { Center } from '@mantine/core'; import { MinutesToMilliseconds } from '../../utils'; import { useWorkspaceResourceMonitor } from '../../components/Providers/ResourceMonitor'; import { VerifyingAccessLoader } from '../../components/Protected/VerifyingAccessLoader'; +import SessionFailureView from '../../components/Protected/SessionFailureView'; import { WORKSPACES_ENABLED } from '../../features/Workspace/config'; const ACTIVITY_CHANNEL = 'gen3-user-activity'; @@ -34,6 +35,20 @@ const FORCE_LOGOUT_EVENT = 'gen3-force-logout'; const isAppHomePath = (path?: string): boolean => path === '/' || Boolean(path?.startsWith('/Apps')); +const getRequestErrorStatus = (error: unknown): number | undefined => { + if (!error || typeof error !== 'object' || !('status' in error)) return; + const status = (error as { status?: unknown }).status; + return typeof status === 'number' ? status : undefined; +}; + +const getRequestErrorDetail = (error: unknown): string => { + if (!error || typeof error !== 'object') return 'The request failed.'; + if ('data' in error && typeof error.data === 'string') return error.data; + if ('error' in error && typeof error.error === 'string') return error.error; + const status = getRequestErrorStatus(error); + return status ? `Fence returned HTTP ${status}.` : 'The request failed.'; +}; + export const requestSessionLogout = ({ showLoginModal = false, }: { @@ -52,16 +67,34 @@ export const requestSessionLogout = ({ ); }; +const fetchWithDeadline = async ( + input: RequestInfo | URL, + init: RequestInit = {}, + timeoutMs = 12_000, +) => { + const controller = new AbortController(); + const timeout = setTimeout(() => controller.abort(), timeoutMs); + try { + return await fetch(input, { ...init, signal: controller.signal }); + } finally { + clearTimeout(timeout); + } +}; + export const logoutSession = async () => { // logged in using credentials then execute credentials logout first const accessToken = getCookie('credentials_token'); if (accessToken) { - await fetch('/api/auth/credentialsLogout'); + await fetchWithDeadline('/api/auth/credentialsLogout'); } - await fetch(`${GEN3_FENCE_API}/logout?next=${GEN3_REDIRECT_URL}/`, { - cache: 'no-store', - }); + await fetchWithDeadline( + `${GEN3_FENCE_API}/logout?next=${GEN3_REDIRECT_URL}/`, + { + cache: 'no-store', + redirect: 'manual', + }, + ); }; function useOnline() { @@ -200,12 +233,23 @@ export const SessionProvider = ({ }: SessionProviderProps) => { const router = useRouter(); - const { isSuccess: isGetCSRFSuccess, isError: isGetCSRFError } = - useGetCSRFQuery(); + const { + isSuccess: isGetCSRFSuccess, + isError: isGetCSRFError, + error: getCSRFError, + refetch: refetchCSRF, + } = useGetCSRFQuery(); useWorkspaceResourceMonitor(monitorWorkspace && WORKSPACES_ENABLED); // monitor workspaces if explicitly enabled - const [getUserDetails, { isLoading: isUserDetailsLoading }] = - useLazyFetchUserDetailsQuery(); // Fetch user details + const [ + getUserDetails, + { + isLoading: isUserDetailsLoading, + isFetching: isUserDetailsFetching, + isError: isUserDetailsError, + error: userDetailsError, + }, + ] = useLazyFetchUserDetailsQuery(); // Fetch user details const userStatus = useCoreSelector((state: CoreState) => selectUserAuthStatus(state), ); @@ -213,6 +257,15 @@ export const SessionProvider = ({ const [mostRecentActivityTimestamp, setMostRecentActivityTimestamp] = useState(Date.now()); const forcedLogoutInFlightRef = useRef(false); + const userVerificationPromiseRef = useRef | null>(null); + const homeUnauthorizedRef = useRef(false); + const [isUserVerificationPending, setIsUserVerificationPending] = + useState(true); + const [isHomeRouteTransitionPending, setIsHomeRouteTransitionPending] = + useState(false); + const [isLogoutTransitionPending, setIsLogoutTransitionPending] = + useState(false); + const homeNavigationVerificationRef = useRef | null>(null); const broadcastChannelRef = useRef(null); @@ -220,7 +273,8 @@ export const SessionProvider = ({ // any user event on one tab or window will update mostRecentActivityTimestamp useEffect(() => { if (typeof window !== 'undefined') { - broadcastChannelRef.current = new BroadcastChannel(ACTIVITY_CHANNEL); + const channel = new BroadcastChannel(ACTIVITY_CHANNEL); + broadcastChannelRef.current = channel; // Listen for activity updates from other tabs const handleActivityMessage = (event: MessageEvent) => { @@ -229,18 +283,13 @@ export const SessionProvider = ({ } }; - broadcastChannelRef.current.addEventListener( - 'message', - handleActivityMessage, - ); + channel.addEventListener('message', handleActivityMessage); return () => { - if (broadcastChannelRef.current) { - broadcastChannelRef.current.removeEventListener( - 'message', - handleActivityMessage, - ); - broadcastChannelRef.current.close(); + channel.removeEventListener('message', handleActivityMessage); + channel.close(); + if (broadcastChannelRef.current === channel) { + broadcastChannelRef.current = null; } }; } @@ -267,21 +316,43 @@ export const SessionProvider = ({ // for now, we are using the user status to determine if the user is logged in const endSession = useCallback( async (shouldRedirect = true) => { - logoutSession() - .then(() => { - getUserDetails(); - }) - .catch((e: unknown) => { - showNotification({ - title: 'Logout Error', - message: `error logging in ${e instanceof Error ? e.message : String(e)}`, - }); - }) - .finally(() => { - if (shouldRedirect) router.push('/'); // TODO replace with config option + if (shouldRedirect && typeof window !== 'undefined') { + setIsLogoutTransitionPending(true); + const accessToken = getCookie('credentials_token'); + if (accessToken) { + try { + await fetchWithDeadline('/api/auth/credentialsLogout'); + } catch (e: unknown) { + showNotification({ + title: 'Logout Error', + message: `error logging out ${e instanceof Error ? e.message : String(e)}`, + }); + } + } + + const next = `${GEN3_REDIRECT_URL}/`; + window.location.assign( + `${GEN3_FENCE_API}/logout?next=${encodeURIComponent(next)}`, + ); + return; + } + + try { + await logoutSession(); + } catch (e: unknown) { + showNotification({ + title: 'Logout Error', + message: `error logging out ${e instanceof Error ? e.message : String(e)}`, }); + } finally { + try { + await getUserDetails().unwrap(); + } catch { + // The post-logout user request is expected to be unauthorized. + } + } }, - [getUserDetails, router], + [getUserDetails], ); useEffect(() => { @@ -295,9 +366,11 @@ export const SessionProvider = ({ } forcedLogoutInFlightRef.current = true; + setIsUserVerificationPending(true); void endSession(false).finally(() => { forcedLogoutInFlightRef.current = false; + setIsUserVerificationPending(false); }); }; @@ -308,24 +381,113 @@ export const SessionProvider = ({ }; }, [endSession]); - const updateSession = useCallback(() => { - const updateSessionWithUserStatus = async () => { + const updateSession = useCallback((): Promise => { + if (isAppHomePath(router.pathname) && homeUnauthorizedRef.current) { + setIsUserVerificationPending(false); + return Promise.resolve(); + } + + if (userVerificationPromiseRef.current) { + return userVerificationPromiseRef.current; + } + + setIsUserVerificationPending(true); + + const verification = (async () => { + const hasBearerCredential = Boolean(getCookie('credentials_token')); + try { await getUserDetails().unwrap(); - } catch (err: any) { - if (err?.status === 401) { - endSession(false); + homeUnauthorizedRef.current = false; + } catch (error: unknown) { + const isUnauthorized = getRequestErrorStatus(error) === 401; + if (isUnauthorized && isAppHomePath(router.pathname)) { + homeUnauthorizedRef.current = true; + } + + if ( + hasBearerCredential && + isUnauthorized && + !forcedLogoutInFlightRef.current + ) { + // The 401 has already resolved authentication as logged out. Clear + // the rejected bearer session without issuing another /user request. + forcedLogoutInFlightRef.current = true; + void logoutSession() + .catch((logoutError: unknown) => { + showNotification({ + title: 'Logout Error', + message: `error logging out ${ + logoutError instanceof Error + ? logoutError.message + : String(logoutError) + }`, + }); + }) + .finally(() => { + forcedLogoutInFlightRef.current = false; + }); } } + })(); + + userVerificationPromiseRef.current = verification; + void verification.finally(() => { + if (userVerificationPromiseRef.current === verification) { + userVerificationPromiseRef.current = null; + setIsUserVerificationPending(false); + } + }); + return verification; + }, [getUserDetails, router.pathname]); + + useEffect(() => { + const routePath = (url: string) => url.split(/[?#]/, 1)[0]; + + const handleRouteChangeStart = (url: string) => { + if (!isAppHomePath(routePath(url))) return; + + setIsHomeRouteTransitionPending(true); + homeNavigationVerificationRef.current = updateSession(); + }; + + const handleRouteChangeComplete = (url: string) => { + if (!isAppHomePath(routePath(url))) { + homeNavigationVerificationRef.current = null; + setIsHomeRouteTransitionPending(false); + return; + } + + const verification = + homeNavigationVerificationRef.current ?? updateSession(); + void verification.finally(() => { + if (homeNavigationVerificationRef.current === verification) { + homeNavigationVerificationRef.current = null; + setIsHomeRouteTransitionPending(false); + } + }); + }; + + const handleRouteChangeError = () => { + homeNavigationVerificationRef.current = null; + setIsHomeRouteTransitionPending(false); }; - updateSessionWithUserStatus(); - }, [getUserDetails, endSession]); + router.events.on('routeChangeStart', handleRouteChangeStart); + router.events.on('routeChangeComplete', handleRouteChangeComplete); + router.events.on('routeChangeError', handleRouteChangeError); + + return () => { + router.events.off('routeChangeStart', handleRouteChangeStart); + router.events.off('routeChangeComplete', handleRouteChangeComplete); + router.events.off('routeChangeError', handleRouteChangeError); + }; + }, [router.events, updateSession]); /** * Update session value every updateSessionInterval seconds */ useEffect(() => { - updateSession(); + void updateSession(); if (updateSessionIntervalMilliseconds <= 0) return; // do not poll if updateSessionInterval is 0 @@ -333,11 +495,25 @@ export const SessionProvider = ({ const timestamp = Date.now(); setMostRecentActivityTimestamp(timestamp); - if (broadcastChannelRef.current) { - broadcastChannelRef.current.postMessage({ - type: 'activity-update', - timestamp, - }); + const channel = broadcastChannelRef.current; + if (channel) { + try { + channel.postMessage({ + type: 'activity-update', + timestamp, + }); + } catch (error) { + if ( + error instanceof DOMException && + error.name === 'InvalidStateError' + ) { + if (broadcastChannelRef.current === channel) { + broadcastChannelRef.current = null; + } + } else { + throw error; + } + } } }; @@ -388,7 +564,7 @@ export const SessionProvider = ({ mostRecentSessionRefreshTimestamp, (ts: number) => setMostRecentSessionRefreshTimestamp(ts), ); - updateSession(); + void updateSession(); }, updateSessionIntervalMilliseconds > 0 ? updateSessionIntervalMilliseconds @@ -398,29 +574,73 @@ export const SessionProvider = ({ const value: Session = useDeepCompareMemo(() => { return { ...sessionInfo, - pending: sessionInfo.pending || isUserDetailsLoading, + pending: + sessionInfo.pending || + isUserVerificationPending || + isUserDetailsLoading || + isUserDetailsFetching, updateSession, endSession, }; - }, [sessionInfo, isUserDetailsLoading, updateSession, endSession]); + }, [ + sessionInfo, + isUserVerificationPending, + isUserDetailsLoading, + isUserDetailsFetching, + updateSession, + endSession, + ]); + + const restartLogin = useCallback(() => { + const next = `${GEN3_REDIRECT_URL}/Login`; + window.location.assign( + `${GEN3_FENCE_API}/logout?next=${encodeURIComponent(next)}`, + ); + }, []); if (isGetCSRFError) { return ( -
- {`Error from the commons services. They do not seem to be running`} -
+ void refetchCSRF()} + onRestartLogin={restartLogin} + /> ); } + if (isUserDetailsError && getRequestErrorStatus(userDetailsError) !== 401) { + return ( + void getUserDetails()} + onRestartLogin={restartLogin} + /> + ); + } + + if (isGetCSRFSuccess && isAppHomePath(router.pathname) && value.pending) { + return ; + } + if (isGetCSRFSuccess) return ( - {children} + {isHomeRouteTransitionPending || isLogoutTransitionPending ? ( + + ) : ( + children + )} ); if (isAppHomePath(router.pathname)) { - return ; + return ; } return
; diff --git a/packages/frontend/src/lib/session/types.ts b/packages/frontend/src/lib/session/types.ts index 53f3b30d..e1fbaed8 100644 --- a/packages/frontend/src/lib/session/types.ts +++ b/packages/frontend/src/lib/session/types.ts @@ -11,7 +11,7 @@ export interface AuthTokenData { export interface Session extends AuthTokenData { userStatus?: LoginStatus; user?: Gen3User; - updateSession: () => void; + updateSession: () => Promise; endSession: (shouldRedirect?: boolean) => void; pending: boolean; } diff --git a/packages/frontend/src/pages/Apps/fetchFileCounts.tsx b/packages/frontend/src/pages/Apps/fetchFileCounts.tsx deleted file mode 100644 index ff005a64..00000000 --- a/packages/frontend/src/pages/Apps/fetchFileCounts.tsx +++ /dev/null @@ -1,30 +0,0 @@ -import { useGeneralGQLQuery } from '@gen3/core'; -import { useMemo } from 'react'; - -export const useFileTotalCountQuery = () => { - const { data, isLoading, isError } = useGeneralGQLQuery({ - query: `query{ - _aggregation { - document_reference{ - _totalCount - } - } - }`, - }); - - const cachedCounts = useMemo(() => { - if (data) { - const typedData = data as { - data?: { - _aggregation?: { document_reference?: { _totalCount?: number } }; - }; - }; - return ( - typedData.data?._aggregation?.document_reference?._totalCount ?? -1 - ); - } - return -1; - }, [data]); - - return { data: cachedCounts, isLoading: isLoading, isError: isError }; -}; diff --git a/packages/frontend/src/pages/CALYPR/CALYPR.tsx b/packages/frontend/src/pages/CALYPR/CALYPR.tsx index b329f68e..c0bde59e 100644 --- a/packages/frontend/src/pages/CALYPR/CALYPR.tsx +++ b/packages/frontend/src/pages/CALYPR/CALYPR.tsx @@ -6,8 +6,6 @@ import { Button, Divider, Image, - Center, - Loader, } from '@mantine/core'; import { CalyprLandingPageProps } from './types'; import { LandingPageProps } from '../../components/Content/LandingPageContent'; @@ -50,33 +48,10 @@ const BannerPanel = ({
); }; -const CalyprPage = ({ - headerProps, - footerProps, - hasAuthenticatedSession = false, -}: Props) => { - const session = useSession(false); - const shouldShowAuthenticatedHome = - hasAuthenticatedSession || session.status === 'issued'; +const CalyprPage = ({ headerProps, footerProps }: Props) => { + const { status } = useSession(false); - if (session.pending && !shouldShowAuthenticatedHome) { - return ( - -
- -
- - ); - } - - if (shouldShowAuthenticatedHome) { + if (status === 'issued') { return ( (Array.isArray(value) ? value[0] : value); + +const requestOrigin = (context: Parameters[0]): string => { + const forwardedProtocol = firstHeaderValue( + context.req.headers['x-forwarded-proto'], + ); + const forwardedHost = firstHeaderValue( + context.req.headers['x-forwarded-host'], + ); + const host = forwardedHost ?? context.req.headers.host; + + return host ? `${forwardedProtocol ?? 'http'}://${host}` : ''; +}; + +const sessionEndpoint = ( + context: Parameters[0], +): string | undefined => { + const fenceBase = GEN3_FENCE_API.replace(/\/$/, ''); + if (/^https?:\/\//i.test(fenceBase)) { + return `${fenceBase}/user`; + } + + const origin = requestOrigin(context); + return origin + ? new URL(`${fenceBase}/user`, `${origin}/`).toString() + : undefined; +}; + +export const verifyAuthenticatedSession = async ( + context: Parameters[0], + headers: Record, +): Promise => { + const endpoint = sessionEndpoint(context); + if (!endpoint) return null; + + const controller = new AbortController(); + const timeout = setTimeout( + () => controller.abort(), + SESSION_CHECK_TIMEOUT_MS, + ); + + try { + const response = await fetch(endpoint, { + headers, + cache: 'no-store', + signal: controller.signal, + }); + + if (response.status === 401 || response.status === 403) return false; + if (!response.ok) return null; + + const user = (await response.json()) as { username?: unknown }; + return typeof user.username === 'string' && user.username.length > 0; + } catch (error: unknown) { + console.warn('Failed to verify the server-side user session:', error); + return null; + } finally { + clearTimeout(timeout); + } +}; + export const CalyprPageGetServerSideProps: GetServerSideProps< NavPageLayoutProps > = async (context) => { const requestHeaders: Record = {}; const cookieHeader = context.req.headers.cookie; const authorizationHeader = context.req.headers.authorization; - const hasAuthenticatedSession = - (typeof authorizationHeader === 'string' && - authorizationHeader.length > 0) || - (typeof cookieHeader === 'string' && - /(?:^|;\s*)(?:access_token|credentials_token)=/i.test(cookieHeader)); - if (typeof cookieHeader === 'string' && cookieHeader) { requestHeaders.Cookie = cookieHeader; } @@ -24,14 +83,22 @@ export const CalyprPageGetServerSideProps: GetServerSideProps< requestHeaders.Authorization = authorizationHeader; } - const calyprConfig: CalyprProps = - await ContentSource.getContentDatabase().get( - `${GEN3_COMMONS_NAME}/calyprLandingPage.json`, - ); + const navigationRequestHeaders = { + ...requestHeaders, + ...(context.req.headers.host ? { Host: context.req.headers.host } : {}), + }; + + const [hasAuthenticatedSession, calyprConfig, initialNavPageLayoutProps] = + await Promise.all([ + verifyAuthenticatedSession(context, requestHeaders), + ContentSource.getContentDatabase().get( + `${GEN3_COMMONS_NAME}/calyprLandingPage.json`, + ), + getNavPageLayoutPropsFromConfig(navigationRequestHeaders), + ]); - let navPageLayoutProps = - await getNavPageLayoutPropsFromConfig(requestHeaders); - if (!hasAuthenticatedSession) { + let navPageLayoutProps = initialNavPageLayoutProps; + if (hasAuthenticatedSession !== true) { const filteredItems = navPageLayoutProps.headerProps.topBar.items.filter( (item: { href: string }) => item.href !== '/git', ); diff --git a/packages/frontend/src/pages/CALYPR/data.unit.test.ts b/packages/frontend/src/pages/CALYPR/data.unit.test.ts new file mode 100644 index 00000000..e1f25b52 --- /dev/null +++ b/packages/frontend/src/pages/CALYPR/data.unit.test.ts @@ -0,0 +1,67 @@ +import type { GetServerSidePropsContext } from 'next'; + +jest.mock('@gen3/core', () => ({ + GEN3_COMMONS_NAME: 'cbds', + GEN3_FENCE_API: '/user', +})); +jest.mock('../../lib/common/staticProps', () => ({ + getNavPageLayoutPropsFromConfig: jest.fn(), +})); +jest.mock('../../lib/content', () => ({ + __esModule: true, + default: {}, +})); + +import { verifyAuthenticatedSession } from './data'; + +const context = { + req: { + headers: { + host: 'commons.example', + 'x-forwarded-proto': 'https', + }, + }, +} as unknown as GetServerSidePropsContext; + +describe('verifyAuthenticatedSession', () => { + const originalFetch = global.fetch; + + afterEach(() => { + global.fetch = originalFetch; + jest.restoreAllMocks(); + }); + + it('uses the Fence user response rather than cookie presence', async () => { + global.fetch = jest.fn().mockResolvedValue({ + ok: false, + status: 401, + }); + + await expect( + verifyAuthenticatedSession(context, { Cookie: 'access_token=stale' }), + ).resolves.toBe(false); + }); + + it('accepts a user response with a username', async () => { + global.fetch = jest.fn().mockResolvedValue({ + ok: true, + status: 200, + json: async () => ({ username: 'researcher@example.org' }), + }); + + await expect(verifyAuthenticatedSession(context, {})).resolves.toBe(true); + expect(global.fetch).toHaveBeenCalledWith( + 'https://commons.example/user/user', + expect.objectContaining({ cache: 'no-store' }), + ); + }); + + it('leaves the result unresolved when Fence is unavailable', async () => { + global.fetch = jest.fn().mockResolvedValue({ + ok: false, + status: 502, + }); + + await expect(verifyAuthenticatedSession(context, {})).resolves.toBeNull(); + }); +}); diff --git a/packages/frontend/src/pages/CALYPR/types.ts b/packages/frontend/src/pages/CALYPR/types.ts index f7288d71..03caf0ef 100644 --- a/packages/frontend/src/pages/CALYPR/types.ts +++ b/packages/frontend/src/pages/CALYPR/types.ts @@ -8,7 +8,7 @@ interface CalyprConfig { export interface CalyprProps { calyprConfig?: CalyprConfig; - hasAuthenticatedSession?: boolean; + hasAuthenticatedSession?: boolean | null; } export type CalyprLandingPageProps = NavPageLayoutProps & CalyprProps; diff --git a/packages/frontend/src/pages/Explorer/data.ts b/packages/frontend/src/pages/Explorer/data.ts index c11c8145..2068f2bb 100644 --- a/packages/frontend/src/pages/Explorer/data.ts +++ b/packages/frontend/src/pages/Explorer/data.ts @@ -9,10 +9,11 @@ import { import { fetchJSONDataFromURL, GEN3_COMMONS_NAME, - GEN3_GUPPY_API, + GEN3_LOOM_API, groupSharedFields, HttpMethod, SharedFieldMapping, + toLoomDataType, } from '@gen3/core'; import { isArray } from 'lodash'; import type { NavPageLayoutProps } from '../../features/Navigation'; @@ -35,23 +36,49 @@ const GetSharedFieldMapping = async ( if (cohortBuilderConfiguration?.sharedFilters) { if (cohortBuilderConfiguration?.sharedFilters?.autoCreate) { - const indices = cohortBuilderConfiguration?.explorerConfig.map( - (tab) => tab.guppyConfig.dataType, - ); + const tabs = cohortBuilderConfiguration?.explorerConfig ?? []; try { - const data = await fetchJSONDataFromURL( - `${GEN3_GUPPY_API}/graphql`, + const dataTypes = Array.from( + new Set(tabs.map((tab) => toLoomDataType(tab.guppyConfig.dataType))), + ); + const selections = dataTypes + .map( + (dataType, index) => + `d${index}: dataframeDataset(input: { dataType: ${JSON.stringify(dataType)} }) { name columns { name } }`, + ) + .join(' '); + const response = await fetchJSONDataFromURL<{ + data?: Record< + string, + { name: string; columns: Array<{ name: string }> } | null + >; + }>( + `${GEN3_LOOM_API}/graphql/flat`, true, HttpMethod.POST, - { query: `{ _mapping { ${indices.join(' ')} }}`, variables: {} }, + JSON.stringify({ + query: `query ExplorerDatasets { ${selections} }`, + variables: {}, + }), + ); + const fieldsByExplorerType = Object.fromEntries( + tabs.map((tab) => { + const loomType = toLoomDataType(tab.guppyConfig.dataType); + const datasetIndex = dataTypes.indexOf(loomType); + const dataset = response?.data?.[`d${datasetIndex}`]; + return [ + tab.guppyConfig.dataType, + dataset?.columns.map((column) => column.name) ?? [], + ]; + }), ); - if ('_mapping' in data.data) { - sharedFiltersMap = groupSharedFields(data.data['_mapping']); + if (response?.data && Object.keys(fieldsByExplorerType).length > 0) { + sharedFiltersMap = groupSharedFields(fieldsByExplorerType); } } catch (err: unknown) { if (err instanceof Error) { - console.warn('Unable to get mapping data from guppy:', err); + console.warn('Unable to get Explorer field mapping from Loom:', err); } } } diff --git a/packages/frontend/src/pages/GitExplorer/GitLanding.tsx b/packages/frontend/src/pages/GitExplorer/GitLanding.tsx index 4ebf0923..d1c65120 100644 --- a/packages/frontend/src/pages/GitExplorer/GitLanding.tsx +++ b/packages/frontend/src/pages/GitExplorer/GitLanding.tsx @@ -83,6 +83,7 @@ import { repositoryFullNamesEqual, } from './githubConnectState'; import type { GitExplorerPageProps } from './types'; +import { hardNavigate } from './navigation'; const pluralize = (count: number, singular: string, plural = `${singular}s`) => `${count} ${count === 1 ? singular : plural}`; @@ -1835,12 +1836,12 @@ const CompactProjectRow = ({
{ - void router.push(localProjectHref); + hardNavigate(router, localProjectHref); }} onKeyDown={(event) => { if (event.key === 'Enter' || event.key === ' ') { event.preventDefault(); - void router.push(localProjectHref); + hardNavigate(router, localProjectHref); } }} role="link" @@ -2631,15 +2632,12 @@ const GitLandingPage = ({
{searchResults.length > 0 ? ( searchResults.map((result) => ( - setSearchQuery('')} > - setSearchQuery('')} - > {result.kind === 'organization' ? (
- - + )) ) : ( { if (!Number.isFinite(size) || size < 0) { @@ -549,7 +550,8 @@ const GitProjectPage = ({ .map((segment) => encodeURIComponent(segment)) .join('/'); const query = selectedRef ? `?ref=${encodeURIComponent(selectedRef)}` : ''; - void router.push( + hardNavigate( + router, `/org/${encodeURIComponent(organization)}/project/${encodeURIComponent(project)}/blob/${encodedPath}${query}`, ); }; @@ -582,17 +584,10 @@ const GitProjectPage = ({ nextQuery.path = pathSegments.join('/'); } - void router.push( - { - pathname: '/org/[org]/project/[project]', - query: { - org: organization, - project, - ...nextQuery, - }, - }, - undefined, - { shallow: true }, + const queryString = new URLSearchParams(nextQuery).toString(); + hardNavigate( + router, + `/org/${encodeURIComponent(organization)}/project/${encodeURIComponent(project)}${queryString ? `?${queryString}` : ''}`, ); }; @@ -920,15 +915,13 @@ const GitProjectPage = ({ disabled={areRefsLoading || refOptions.length === 0} leftSection={} onChange={(value) => { - void router.push({ - pathname: `/org/${encodeURIComponent(organization)}/project/${encodeURIComponent(project)}`, - query: { - ...(value ? { ref: value } : {}), - ...(currentPath - ? { path: currentPath } - : {}), - }, - }); + const query = new URLSearchParams(); + if (value) query.set('ref', value); + if (currentPath) query.set('path', currentPath); + hardNavigate( + router, + `/org/${encodeURIComponent(organization)}/project/${encodeURIComponent(project)}${query.toString() ? `?${query.toString()}` : ''}`, + ); }} placeholder={ areRefsLoading ? 'Loading refs...' : 'Default ref' @@ -947,19 +940,17 @@ const GitProjectPage = ({ value={effectiveRef} />
- - - - {project} - - - + + {project} + + {breadcrumbSegments.map((breadcrumb) => ( @@ -1142,12 +1133,12 @@ const GitProjectPage = ({ disabled={areRefsLoading || refOptions.length === 0} leftSection={} onChange={(value) => { - void router.push({ - pathname: `/org/${encodeURIComponent(organization)}/project/${encodeURIComponent(project)}`, - query: { - ...(value ? { ref: value } : {}), - }, - }); + const query = new URLSearchParams(); + if (value) query.set('ref', value); + hardNavigate( + router, + `/org/${encodeURIComponent(organization)}/project/${encodeURIComponent(project)}${query.toString() ? `?${query.toString()}` : ''}`, + ); }} placeholder={ areRefsLoading ? 'Loading refs...' : 'Default ref' diff --git a/packages/frontend/src/pages/GitExplorer/GitProjectFile.tsx b/packages/frontend/src/pages/GitExplorer/GitProjectFile.tsx index 2154b7da..e20848fc 100644 --- a/packages/frontend/src/pages/GitExplorer/GitProjectFile.tsx +++ b/packages/frontend/src/pages/GitExplorer/GitProjectFile.tsx @@ -1,5 +1,4 @@ import React, { useEffect, useMemo, useState } from 'react'; -import Link from 'next/link'; import { useRouter } from 'next/router'; import { Alert, @@ -26,6 +25,7 @@ import ProtectedContent from '../../components/Protected/ProtectedContent'; import { NavPageLayout } from '../../features/Navigation'; import { useIsEmbedded } from '../../utils'; import type { GitExplorerPageProps } from './types'; +import { hardNavigate } from './navigation'; const formatBytes = (size: number): string => { if (!Number.isFinite(size) || size < 0) { @@ -230,14 +230,12 @@ const GitProjectFilePage = ({
- - - Back to repository tree - - + Back to repository tree + {breadcrumbSegments[breadcrumbSegments.length - 1] || filePath} @@ -276,12 +274,10 @@ const GitProjectFilePage = ({ const encodedPath = pathSegments .map((segment) => encodeURIComponent(segment)) .join('/'); - void router.push({ - pathname: `/org/${encodeURIComponent(organization)}/project/${encodeURIComponent(project)}/blob/${encodedPath}`, - query: { - ...(value ? { ref: value } : {}), - }, - }); + hardNavigate( + router, + `/org/${encodeURIComponent(organization)}/project/${encodeURIComponent(project)}/blob/${encodedPath}${value ? `?ref=${encodeURIComponent(value)}` : ''}`, + ); }} placeholder={ areRefsLoading ? 'Loading refs...' : 'Select a ref' @@ -292,12 +288,11 @@ const GitProjectFilePage = ({ </Group> <Breadcrumbs> - <Link - href={`/org/${encodeURIComponent(organization)}/project/${encodeURIComponent(project)}`} - legacyBehavior + <a + href={`${router.basePath ?? ''}/org/${encodeURIComponent(organization)}/project/${encodeURIComponent(project)}`} > - <a>root</a> - </Link> + root + </a> {breadcrumbSegments.map((segment, index) => ( <Text key={`${segment}-${index}`} size="sm"> {segment} diff --git a/packages/frontend/src/pages/GitExplorer/navigation.ts b/packages/frontend/src/pages/GitExplorer/navigation.ts new file mode 100644 index 00000000..56868f93 --- /dev/null +++ b/packages/frontend/src/pages/GitExplorer/navigation.ts @@ -0,0 +1,13 @@ +import type { NextRouter } from 'next/router'; + +/** + * Git explorer pages intentionally use a full document navigation. The tree + * path is part of the page URL, and remounting the page keeps browser history, + * SSR, and query-derived state consistent across browsers. + */ +export const hardNavigate = ( + router: Pick<NextRouter, 'basePath'>, + href: string, +) => { + window.location.assign(`${router.basePath ?? ''}${href}`); +};