Skip to content

feat: advanced job filters #41

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AHMED-GAMAL-AG wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat: advanced job filters #41

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions api/upwork.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ export type Job = {
paymentVerificationStatus: number | null
totalFeedback: number
totalSpent: number
// Returned by all feeds but only surfaced for filtering (optional per feed).
totalReviews?: number
totalHires?: number

location?: {
country: string | null
Expand All @@ -65,6 +68,10 @@ export type Job = {
renewedOn: Date | null
createdOn: Date | string

// Surfaced for filtering (present on all feeds via the raw response).
isApplied?: boolean
premium?: boolean

// internal attribute
__isSeen: boolean
}
Expand Down
22 changes: 17 additions & 5 deletions components/StorageProvider.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import jobStorage from '@/utils/jobs'
import promptStorage from '@/utils/prompt'
import { GlobalState } from '@/utils/globalState'
import globalState from '@/utils/globalState'
import jobFiltersStorage, { JobFiltersConfig } from '@/utils/jobFilters'
import { ReactNode, useEffect, useState } from 'react'
import Storage, { StorageInterface } from '@/contexts/storage'
type Props = {
Expand All @@ -14,33 +15,44 @@ const StorageProvider = (props: Props) => {
const [prompt, setPrompt] = useState<string>('')
const [initialized, setInitialized] = useState(false)
const [state, setState] = useState<GlobalState>(globalState.getDefaultState)
const [jobFilters, setJobFilters] = useState<JobFiltersConfig>(
jobFiltersStorage.getDefaultConfig
)

const exposedApi: StorageInterface = {
jobs,
prompt,
initialized,
globalState: state,
jobFilters,
setJobs: jobStorage.save,
setState: globalState.save,
setPrompt: promptStorage.save,
setJobFilters: jobFiltersStorage.save,
}

useEffect(() => {
const initializeStorage = async () => {
const [freshState, freshJobs, freshPrompt] = await Promise.all([
globalState.get(),
jobStorage.getAll(),
promptStorage.get(),
])
const [freshState, freshJobs, freshPrompt, freshFilters] =
await Promise.all([
globalState.get(),
jobStorage.getAll(),
promptStorage.get(),
jobFiltersStorage.get(),
])

setJobs(freshJobs ?? [])
setState(freshState)
setPrompt(freshPrompt)
setJobFilters(freshFilters)

jobStorage.addEventListener((newJobs) => setJobs(newJobs ?? []))
globalState.addEventListener((newState) =>
setState(newState ?? globalState.getDefaultState())
)
jobFiltersStorage.addEventListener((newFilters) =>
setJobFilters(newFilters ?? jobFiltersStorage.getDefaultConfig())
)

setInitialized(true)
}
Expand Down
5 changes: 5 additions & 0 deletions contexts/storage.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,27 +3,32 @@ import { createContext } from 'react'
import jobStorage from '@/utils/jobs'
import promptStorage from '@/utils/prompt'
import globalState, { GlobalState } from '@/utils/globalState'
import jobFiltersStorage, { JobFiltersConfig } from '@/utils/jobFilters'

export type StorageInterface = {
initialized: boolean
jobs: Job[]
prompt: string
globalState: GlobalState
jobFilters: JobFiltersConfig

setJobs: typeof jobStorage.save
setState: typeof globalState.save
setPrompt: typeof promptStorage.save
setJobFilters: typeof jobFiltersStorage.save
}

const StorageContext = createContext<StorageInterface>({
initialized: false,
jobs: [],
prompt: '',
globalState: globalState.getDefaultState(),
jobFilters: jobFiltersStorage.getDefaultConfig(),

setJobs: Promise.resolve,
setState: Promise.resolve,
setPrompt: Promise.resolve,
setJobFilters: Promise.resolve,
})

export default StorageContext
20 changes: 13 additions & 7 deletions entrypoints/background/fetchJobs.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ import { ErrorType } from '@/utils/errors'
import extension from '@/utils/extension'
import stateStorage from '@/utils/globalState'
import jobStorage from '@/utils/jobs'
import jobFiltersStorage from '@/utils/jobFilters'
import { filterJobs } from '@/utils/matchJob'
import logger from '@/utils/logger'
import notifications from '@/utils/notifications'
import { captureEvent, captureException } from '@/utils/sentry'
Expand Down Expand Up @@ -104,19 +106,23 @@ const fetchJobs = async () => {
(job) => job.ciphertext
)

const newJobs = newBatch.filter(
(job) => !oldBatchIds.includes(job.ciphertext)
)

const newProcessedBatch = [
...newBatch
.filter((job) => !oldBatchIds.includes(job.ciphertext))
.map((job) => ({ ...job, __isSeen: false })),
...newJobs.map((job) => ({ ...job, __isSeen: false })),
...(oldBatch ?? []),
].slice(0, 50)
Comment on lines +109 to 116

@coderabbitai coderabbitai Bot Jul 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

api_file="$(fd -t f -a 'upwork\.ts$' . | head -n 1)"
test -n "$api_file"

ast-grep outline "$api_file" --items all --match 'getJobs' --view expanded
rg -n -C 10 'getJobs|ciphertext|isApplied|isPremium|review|hire|spend' "$api_file"

Repository: neeilya/upwork-toolkit

Length of output: 2830

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Candidate files:"
fd -t f '(fetchJobs|upwork)\.ts$|types|interface|job' . | sed -n '1,120p'

echo
echo "background fetchJobs outline:"
ast-grep outline entrypoints/background/fetchJobs.ts --view expanded || true

echo
echo "fetchJobs relevant content:"
cat -n entrypoints/background/fetchJobs.ts | sed -n '1,190p'

echo
echo "upwork requestJobs relevant content:"
cat -n api/upwork.ts | sed -n '220,310p'

echo
echo "Job/Ciphertext usages:"
rg -n -C 4 'type Job|interface Job|ciphertext|oldBatch|newProcessedBatch|newBatch|upwork\.getJobs|fetchJobs' entrypoints api tests . --glob '!node_modules' --glob '!dist' --glob '!build' | sed -n '1,240p'

Repository: neeilya/upwork-toolkit

Length of output: 23123

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "upwork query and type mappings:"
cat -n api/upwork.ts | sed -n '37,78p;200,250p;309,330p'

echo
echo "GQL queries around ciphertext and related fields:"
cat -n api/gqlQueries.ts | sed -n '85,130p;175,210p'

echo
echo "jobFilter relevant data access:"
cat -n utils/matchJob.ts | sed -n '1,230p'
cat -n utils/jobFilters.ts | sed -n '1,200p'

Repository: neeilya/upwork-toolkit

Length of output: 16764

🌐 Web query:

Upwork jobs feed API ciphertext job id response schema updated job

💡 Result:

The Upwork API uses a ciphertext-based identifier (e.g., ~01abc...) as the stable, public-facing unique identifier for job postings [1][2][3]. This format is the standard for both the modern GraphQL API and historical data references [1][4][2]. Key details regarding the ciphertext job ID and API schema: 1. Identification: The ciphertext ID is an alias for the job's unique identifier and is typically formatted with a leading tilde (e.g., ~01... or ~02...) [1][5][3]. It serves as the primary key when interacting with job-related endpoints, such as retrieving job details [1][4][2]. 2. API Schema Context: Upwork transitioned from a legacy REST API to a GraphQL-based API, which is now the primary interface [6][7]. In the GraphQL schema, job postings are queried using this ID (often labeled as jobPostingId or similar in specific query arguments) [4]. 3. Response Structure: When fetching job postings—such as through marketplace queries—the response schema typically includes the ciphertext alongside other fields like title, description, and publication date [4][8]. While specific internal schemas may evolve, the ciphertext remains the consistent, required identifier for referencing a specific job posting [9][4]. 4. Usage: Developers should use the ciphertext ID for operations such as fetching full job details, applying to jobs, or identifying job events in webhooks [2][3]. When using these identifiers in URL parameters or API requests, ensure they are correctly handled (e.g., URL-encoding the leading tilde if necessary) [3]. For official integration, developers are directed to the Upwork Developer documentation, which provides the current GraphQL schema definitions and query examples [4][7].

Citations:

Merge refreshed job snapshots before trimming the batch.

upworkApi.getJobs fetches feed snapshots, not deltas, and ciphertext is the job identifier. Jobs whose updated fields appear in newBatch get dropped by the old-duplicate filter, so filters and storage can use stale client metrics, isApplied, or premium. Preserve __isSeen, then update matching existing jobs with refreshed values before .slice(0, 50).

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@entrypoints/background/fetchJobs.ts` around lines 109 - 116, The
newProcessedBatch construction currently removes refreshed jobs instead of
merging them, leaving stale fields in existing entries. Update the batch
processing around newJobs and newProcessedBatch to match jobs by ciphertext,
merge refreshed newBatch values into existing oldBatch entries while preserving
__isSeen, retain genuinely new jobs with __isSeen false, and only then apply the
50-item slice.

Comment on lines 113 to 116

@coderabbitai coderabbitai Bot Jul 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Use the normalized oldBatch value.

The code detects invalid oldBatch values and normalizes them for ID lookup. Line 115 still spreads the raw value. A truthy non-iterable value then throws during the fetch cycle instead of recovering from corrupt storage. Reuse one normalized array for both operations.

Proposed fix 
+  const validOldBatch = Array.isArray(oldBatch) ? oldBatch : []
+
-  const oldBatchIds = (Array.isArray(oldBatch) ? oldBatch : []).map(
+  const oldBatchIds = validOldBatch.map(
     (job) => job.ciphertext
   )
...
-    ...(oldBatch ?? []),
+    ...validOldBatch,
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@entrypoints/background/fetchJobs.ts` around lines 113 - 116, Update the batch
construction near newProcessedBatch to reuse the normalized oldBatch array
already produced for ID lookup, rather than spreading the raw oldBatch value.
Ensure invalid or corrupt stored values fall back to the normalized empty/array
result without throwing, while preserving the existing merge and slice behavior.


// Advanced filters refine what notifies and what's counted; every job is
// still stored so the list can be un-filtered without re-fetching.
const filters = await jobFiltersStorage.get()

const unseenJobs = newProcessedBatch.filter((job) => !job.__isSeen)
const unseenCount = unseenJobs.length
const unseenCount = filterJobs(unseenJobs, filters).length

const hasNewUnseenJobs =
unseenJobs.length > 0 &&
unseenJobs.some((job) => !oldBatchIds.includes(job.ciphertext))
const hasNewUnseenJobs = filterJobs(newJobs, filters).length > 0
Comment on lines 113 to +125

@coderabbitai coderabbitai Bot Jul 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

api_file="$(fd -t f -a 'upwork\.ts$' . | head -n 1)"
test -n "$api_file"

ast-grep outline "$api_file" --items all --match 'getJobs' --view expanded
rg -n -C 10 'getJobs|limit|pageSize|slice|take|MAX' "$api_file"

Repository: neeilya/upwork-toolkit

Length of output: 2843

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== fetchJobs outline =="
ast-grep outline entrypoints/background/fetchJobs.ts --view expanded || true

echo "== fetchJobs relevant lines =="
cat -n entrypoints/background/fetchJobs.ts | sed -n '80,140p'

echo "== requestJobs relevant lines =="
cat -n api/upwork.ts | sed -n '240,308p'

echo "== search for fetchJobs usages/tests =="
rg -n -C 5 'fetchJobs|getJobs|newProcessedBatch|hasNewUnseenJobs|unseenCount' .

Repository: neeilya/upwork-toolkit

Length of output: 16948

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== feedOptions and response types =="
cat -n api/upwork.ts | sed -n '1,140p'

echo "== graphql best matches response type search =="
rg -n -C 4 'type BestMatches|interface BestMatches|const Best|query.*Best|bestMatchJobsFeed' api/upwork.ts entrypoints . --glob '!node_modules' --glob '!dist'

Repository: neeilya/upwork-toolkit

Length of output: 9714

Align notification eligibility with the retained batch.

newProcessedBatch stores only the first 50 new jobs, but hasNewUnseenJobs checks all newJobs. If a fetch returns more than 50 new jobs, a filtered match outside storage can trigger a notification without increasing unseenCount. Compute hasNewUnseenJobs from the retained new jobs instead, or enforce a clear max feed size that matches the 50-item storage contract.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@entrypoints/background/fetchJobs.ts` around lines 113 - 125, Update
hasNewUnseenJobs in the fetchJobs batch-processing flow to evaluate only the new
jobs retained in newProcessedBatch’s 50-item storage limit, ensuring
notification eligibility matches unseenCount and stored feed contents. Preserve
the existing filterJobs behavior and avoid considering newJobs entries that were
truncated.


await Promise.all([
jobStorage.save(newProcessedBatch),
Expand Down
2 changes: 2 additions & 0 deletions entrypoints/options/App.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { SnackbarProvider } from 'notistack'
import analytics from '@/utils/analytics'
import Settings from './pages/Settings'
import CoverLetter from './pages/CoverLetter'
import Filters from './pages/Filters'
import { useContext, useEffect, useState } from 'react'
import { LocalizationProvider } from '@mui/x-date-pickers'
import { Route, Routes, HashRouter } from 'react-router-dom'
Expand Down Expand Up @@ -50,6 +51,7 @@ const App = () => {
<Route path="debug" element={<Debug />} />
<Route path="settings" element={<Settings />} />
<Route path="cover-letter" element={<CoverLetter />} />
<Route path="filters" element={<Filters />} />
</Route>
</Routes>
</HashRouter>
Expand Down
Loading