Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 21 additions & 6 deletions .claude/skills/ai-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ List responses from `/api/v1/ai-review/` and `/api/v1/ai-review/proposed/`
include compact submission fields plus the active proposal summary:

- `assigned_to`, `assigned_to_name`
- `has_more_info_request`, `is_more_info_resubmitted`
- `has_proposal`
- `proposed_action`, `proposed_points`, `proposed_staff_reply`
- `proposed_contribution_type`
Expand Down Expand Up @@ -176,11 +177,16 @@ already makes the filter meaningless. Send backend parameter names exactly.
| `proposed_template` | template ID | Active proposal template |
| `has_appeal` | `true`, `false` | Submitter appeal flag |
| `is_interesting` | `true`, `false` | Internal interesting flag |
| `has_more_info_request` | `true`, `false` | Whether a non-proposal structured note records a `more_info` request; this is the same history rendered as a more-information block on submission cards |
| `is_more_info_resubmitted` | `true`, `false` | Whether the append-only transition log records an edited transition from `more_info_needed` to `pending` |
| `min_accepted_contributions` | positive integer | Submitter has at least this many accepted submissions |

The old `resubmitted_more_info` filter was removed (re-open paths clear
`reviewed_at`, so it could never match current data). Do not send it; unknown
params are silently ignored and you would get the ENTIRE unfiltered queue.
`reviewed_at`, so it could never match current data). Use
`is_more_info_resubmitted` for a confirmed reopen transition or
`has_more_info_request` for recorded request history. Do not send the old name;
unknown params are silently ignored and you would get the ENTIRE unfiltered
queue.

## More-Info Filters

Expand All @@ -189,14 +195,15 @@ Use the exact filter for the question being asked:
| Need | Endpoint | Filter |
|---|---|---|
| Submissions currently waiting on submitter info | `/api/v1/ai-review/reviewed/` or steward search | `state=more_info_needed` |
| Pending submissions resubmitted after more info was requested | `/api/v1/ai-review/{id}/` | No query param exists; inspect `internal_notes` for a prior `more_info` decision note |
| Pending submissions with a recorded more-info request block | `/api/v1/ai-review/` | `has_more_info_request=true` |
| Pending submissions confirmed as resubmitted after a more-info request | `/api/v1/ai-review/` | `is_more_info_resubmitted=true` |
| Active proposals recommending more info | `/api/v1/ai-review/proposed/` | `proposed_action=more_info` |
| Reviewed submissions where final steward decision was more info | `/api/v1/ai-review/reviewed/` | `state=more_info_needed` |
| Exclude current more-info submissions | Any list endpoint where state is not fixed | `exclude_state=more_info_needed` |

There is no direct query parameter for "ever had a more-info request but later
became accepted or rejected". For that, use `/api/v1/ai-review/reviewed/` and
inspect `internal_notes`.
To find reviewed submissions that have ever had a recorded more-info request,
use `/api/v1/ai-review/reviewed/?has_more_info_request=true`; the reviewed
endpoint's calibration-data scope still applies.

## Common Filter Examples

Expand All @@ -205,6 +212,14 @@ inspect `internal_notes`.
curl -s -H "Authorization: Bearer $TOKEN" \
"$BASE_URL/api/v1/ai-review/?category=builder&exclude_empty_evidence=true"

# Unassigned pending submissions with a recorded more-info request block.
curl -s -H "Authorization: Bearer $TOKEN" \
"$BASE_URL/api/v1/ai-review/?assigned_to=unassigned&has_more_info_request=true"

# Pending submissions with an audited more-info-needed to pending transition.
curl -s -H "Authorization: Bearer $TOKEN" \
"$BASE_URL/api/v1/ai-review/?is_more_info_resubmitted=true"

# Active more-info proposals from AI or human stewards.
curl -s -H "Authorization: Bearer $TOKEN" \
"$BASE_URL/api/v1/ai-review/proposed/?proposed_action=more_info"
Expand Down
3 changes: 2 additions & 1 deletion backend/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ backend/
- ProjectMetric - Admin-managed title/value/detail metric rows for project pages
- ProjectPageRevision - Owner-submitted ordered page blocks rendered through whitelisted portal components
- **AI Review**: `contributions/ai_review/views.py`
- `/api/v1/ai-review/` - List pending unproposed submissions for the external AI review agent; proposal filters such as `has_proposal=true` opt into active proposals
- `/api/v1/ai-review/` - List pending unproposed submissions for the external AI review agent; `has_more_info_request=true` selects submissions with recorded request blocks, while `is_more_info_resubmitted=true` selects audited resubmissions
- `/api/v1/ai-review/{id}/` - Retrieve a pending submission with evidence and user history
- `/api/v1/ai-review/{id}/propose/` - Submit an AI proposal for human approval
- `/api/v1/ai-review/proposed/` - List pending submissions with active proposals awaiting steward review; use `proposed_by=ai` for AI-created proposals only
Expand Down Expand Up @@ -351,6 +351,7 @@ backend/
- **Audit**: `DiscordEarnedRoleAssignment` stores each successful automatic grant with the Discord member, role, CP/POAP snapshot, and creation time; records are read-only in admin.
- **Service**: `DiscordRoleSyncService.add_member_role(discord_user_id, role_id)` PUTs to Discord; 404 (member left) returns False.
- **Trigger**: `POST /api/v1/users/discord/assign-earned-roles/` (cron-protected, background thread + `DiscordRoleSyncLock` row `discord_earned_role_assign`), called daily at 00:30 UTC by `.github/workflows/assign-discord-roles.yml` (after the 00:00 MEE6 XP sync). Manual/backfill: `python manage.py assign_earned_discord_roles [--dry-run]`; dry run prints the would-assign list for review.
- **Admin**: superusers can start the same locked background assignment from the `DiscordEarnedRoleAssignment` changelist after confirming the operation.
- **Ops**: the bot needs Manage Roles and its role above Synapse/Brain in the guild hierarchy.

### Database & Migrations
Expand Down
42 changes: 40 additions & 2 deletions backend/contributions/ai_review/serializers.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
Evidence,
ProjectMilestoneReview,
SubmissionNote,
SubmissionStateTransition,
SubmittedContribution,
)
from contributions.rubric_review import (
Expand Down Expand Up @@ -36,7 +37,35 @@ def get_is_active(self, obj):
return obj.is_active()


class AIReviewProposalFieldsMixin(serializers.Serializer):
class AIReviewLifecycleFieldsMixin(serializers.Serializer):
"""Recorded request history and durable lifecycle facts for DAI."""

has_more_info_request = serializers.SerializerMethodField()
is_more_info_resubmitted = serializers.SerializerMethodField()

def get_has_more_info_request(self, obj):
annotated = getattr(obj, 'has_more_info_request_flag', None)
if annotated is not None:
return bool(annotated)
return SubmissionNote.objects.filter(
submitted_contribution=obj,
is_proposal=False,
data__action='more_info',
).exists()

def get_is_more_info_resubmitted(self, obj):
annotated = getattr(obj, 'more_info_resubmitted_flag', None)
if annotated is not None:
return bool(annotated)
return SubmissionStateTransition.objects.filter(
submitted_contribution=obj,
event=SubmissionStateTransition.EVENT_EDITED,
from_state='more_info_needed',
to_state='pending',
).exists()


class AIReviewProposalFieldsMixin(AIReviewLifecycleFieldsMixin):
"""Shared active-proposal fields for list and detail AI review payloads."""

proposed_by_name = serializers.SerializerMethodField()
Expand Down Expand Up @@ -112,6 +141,8 @@ class Meta:
'gate_reviewed',
'has_appeal',
'appeal_reason',
'has_more_info_request',
'is_more_info_resubmitted',
'has_proposal',
'assigned_to',
'assigned_to_name',
Expand Down Expand Up @@ -224,6 +255,8 @@ class Meta:
'gate_reviewed',
'has_appeal',
'appeal_reason',
'has_more_info_request',
'is_more_info_resubmitted',
'evidence_items',
'internal_notes',
'user_history',
Expand Down Expand Up @@ -277,7 +310,10 @@ def get_has_proposal(self, obj):
return obj.proposed_action is not None


class AIReviewReviewedSubmissionSerializer(serializers.ModelSerializer):
class AIReviewReviewedSubmissionSerializer(
AIReviewLifecycleFieldsMixin,
serializers.ModelSerializer,
):
"""Serializer for reviewed submissions — includes review outcome and notes."""

contribution_type_name = serializers.CharField(
Expand Down Expand Up @@ -311,6 +347,8 @@ class Meta:
'gate_reviewed',
'has_appeal',
'appeal_reason',
'has_more_info_request',
'is_more_info_resubmitted',
'reviewed_at',
'evidence_items',
'internal_notes',
Expand Down
18 changes: 14 additions & 4 deletions backend/contributions/ai_review/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@
SubmissionNote,
SubmittedContribution,
)
from contributions.lifecycle_filters import (
SubmissionLifecycleFilterMixin,
annotate_more_info_flags,
)
from contributions.proposal_filters import ProposalReviewStatusFilterMixin
from contributions.rubric_review import rubric_summary_text, uses_project_rubric
from service_accounts.authentication import ServiceAccountAuthentication
Expand All @@ -51,7 +55,11 @@ class AIReviewPagination(PageNumberPagination):

# ─── FilterSet ────────────────────────────────────────────────────────────────

class AIReviewFilterSet(ProposalReviewStatusFilterMixin, FilterSet):
class AIReviewFilterSet(
SubmissionLifecycleFilterMixin,
ProposalReviewStatusFilterMixin,
FilterSet,
):
"""Filterset for AI review agent submission queries."""

contribution_type = NumberFilter(field_name='contribution_type_id')
Expand Down Expand Up @@ -82,6 +90,8 @@ class AIReviewFilterSet(ProposalReviewStatusFilterMixin, FilterSet):
search = CharFilter(method='filter_search')
mission = CharFilter(method='filter_mission')
exclude_mission = CharFilter(method='filter_exclude_mission')
has_more_info_request = BooleanFilter(method='filter_has_more_info_request')
is_more_info_resubmitted = BooleanFilter(method='filter_is_more_info_resubmitted')

class Meta:
model = SubmittedContribution
Expand Down Expand Up @@ -441,7 +451,7 @@ def get_queryset(self):
if self.action != 'list':
prefetches.append('internal_notes')

return (
return annotate_more_info_flags(
qs.select_related(
'contribution_type',
'contribution_type__category',
Expand Down Expand Up @@ -653,7 +663,7 @@ def proposed(self, request):
Use GET /ai-review/{id}/ to retrieve full proposal details for any
submission returned here.
"""
queryset = (
queryset = annotate_more_info_flags(
SubmittedContribution.objects.filter(
state='pending',
proposed_action__isnull=False,
Expand Down Expand Up @@ -706,7 +716,7 @@ def reviewed(self, request):
user__email=AI_STEWARD_EMAIL,
)

queryset = (
queryset = annotate_more_info_flags(
SubmittedContribution.objects
.filter(
state__in=['accepted', 'rejected', 'more_info_needed'],
Expand Down
50 changes: 50 additions & 0 deletions backend/contributions/lifecycle_filters.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
from django.db.models import Exists, OuterRef

from .models import SubmissionNote, SubmissionStateTransition


def more_info_request_notes():
"""Recorded requests rendered by the submission card's history blocks."""
return SubmissionNote.objects.filter(
submitted_contribution_id=OuterRef('pk'),
is_proposal=False,
data__action='more_info',
)


def more_info_resubmission_transitions():
"""Transitions that prove a submitter reopened a more-info request."""
return SubmissionStateTransition.objects.filter(
submitted_contribution_id=OuterRef('pk'),
event=SubmissionStateTransition.EVENT_EDITED,
from_state='more_info_needed',
to_state='pending',
)


def annotate_more_info_flags(queryset):
"""Expose request-history and durable transition flags without N+1 queries."""
return queryset.annotate(
has_more_info_request_flag=Exists(more_info_request_notes()),
more_info_resubmitted_flag=Exists(more_info_resubmission_transitions()),
)


class SubmissionLifecycleFilterMixin:
"""Shared history filters for steward and DAI submission queues."""

def filter_has_more_info_request(self, queryset, name, value):
request_exists = Exists(more_info_request_notes())
if value is True:
return queryset.filter(request_exists)
if value is False:
return queryset.exclude(request_exists)
return queryset

def filter_is_more_info_resubmitted(self, queryset, name, value):
transition_exists = Exists(more_info_resubmission_transitions())
if value is True:
return queryset.filter(transition_exists)
if value is False:
return queryset.exclude(transition_exists)
return queryset
87 changes: 87 additions & 0 deletions backend/contributions/tests/test_calibration_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
ContributionType,
Evidence,
SubmissionNote,
SubmissionStateTransition,
SubmittedContribution,
)
from leaderboard.models import GlobalLeaderboardMultiplier
Expand Down Expand Up @@ -443,6 +444,92 @@ def test_ai_review_list_supports_compound_assignment_filters(self):
self.assertNotIn(str(assigned_to_steward.id), exclude_ids)
self.assertIn(str(assigned_to_ai.id), exclude_ids)

def test_ai_review_can_filter_and_identify_more_info_resubmissions(self):
resubmitted = self.fixtures['submission']
SubmissionStateTransition.record(
resubmitted,
SubmissionStateTransition.EVENT_EDITED,
from_state='more_info_needed',
actor=self.fixtures['submitter'],
)
ordinary_pending = SubmittedContribution.objects.create(
user=self.fixtures['submitter'],
contribution_type=self.fixtures['ct'],
contribution_date=timezone.now(),
notes='Ordinary pending submission.',
state='pending',
)

response = self.client.get(
'/api/v1/ai-review/',
data={
'assigned_to': 'unassigned',
'is_more_info_resubmitted': 'true',
},
**self.ai_auth,
)

self.assertEqual(response.status_code, 200)
self.assertEqual(len(response.data['results']), 1)
self.assertEqual(str(response.data['results'][0]['id']), str(resubmitted.id))
self.assertTrue(response.data['results'][0]['is_more_info_resubmitted'])

response = self.client.get(
'/api/v1/ai-review/',
data={'is_more_info_resubmitted': 'false'},
**self.ai_auth,
)
result_by_id = {str(item['id']): item for item in response.data['results']}
self.assertIn(str(ordinary_pending.id), result_by_id)
self.assertFalse(result_by_id[str(ordinary_pending.id)]['is_more_info_resubmitted'])
self.assertNotIn(str(resubmitted.id), result_by_id)

def test_ai_review_can_filter_pending_submissions_with_more_info_request_blocks(self):
requested = self.fixtures['submission']
SubmissionNote.objects.create(
submitted_contribution=requested,
user=self.fixtures['steward_user'],
message='Reviewed: more_info',
is_proposal=False,
data={
'action': 'more_info',
'staff_reply': 'Please add the missing evidence.',
},
)
ordinary_pending = SubmittedContribution.objects.create(
user=self.fixtures['submitter'],
contribution_type=self.fixtures['ct'],
contribution_date=timezone.now(),
notes='Ordinary pending submission.',
state='pending',
)

response = self.client.get(
'/api/v1/ai-review/',
data={
'assigned_to': 'unassigned',
'has_more_info_request': 'true',
},
**self.ai_auth,
)

self.assertEqual(response.status_code, 200)
self.assertEqual(len(response.data['results']), 1)
payload = response.data['results'][0]
self.assertEqual(str(payload['id']), str(requested.id))
self.assertTrue(payload['has_more_info_request'])
self.assertFalse(payload['is_more_info_resubmitted'])

response = self.client.get(
'/api/v1/ai-review/',
data={'has_more_info_request': 'false'},
**self.ai_auth,
)
result_by_id = {str(item['id']): item for item in response.data['results']}
self.assertIn(str(ordinary_pending.id), result_by_id)
self.assertFalse(result_by_id[str(ordinary_pending.id)]['has_more_info_request'])
self.assertNotIn(str(requested.id), result_by_id)

def test_ai_review_invalid_steward_id_filters_do_not_error(self):
submission = self.fixtures['submission']
submission.assigned_to = self.fixtures['steward_user']
Expand Down
Loading
Loading