Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions api/stovepipe/proto/stovepipe.proto
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,43 @@ message ListOpenIssuesResponse {
string next_page_token = 2;
}

// RequestIssueSummary describes one accepted issue update, not the queue's open-issue set.
message RequestIssueSummary {
// Logical validation task.
string task_id = 1;
// Validation unit; empty identifies a task-wide failure.
string unit_id = 2;
// "new" (including reopened), "existing" (reported and already open), or "closed".
string kind = 3;
// Build whose accepted update produced this result.
string build_id = 4;
// Build where this occurrence first opened.
string first_seen_build_id = 5;
}

message GetIssuesByRequestIDRequest {
// Required configured queue.
string queue = 1;
// Required canonical positive decimal request ID within the queue.
string request_id = 2;
// Maximum results to return. Zero selects 50; otherwise 1 through 200.
int32 page_size = 3;
// Opaque continuation bound to this queue and request.
string page_token = 4;
}

// Completed and abandoned results are stable; in-progress pages are not a snapshot.
message GetIssuesByRequestIDResponse {
// Results in ascending bytewise task/unit order, classified as new, existing, or closed.
repeated RequestIssueSummary issues = 1;
// "in_progress", "complete", "incomplete" (abandoned), or "unavailable".
// Unavailable means recording has not started or predates retained coverage.
// Complete describes reconciliation, not build success or absence of open issues.
string status = 2;
// Empty ends this scan. Restart in-progress scans after completion to include late results.
string next_page_token = 3;
}

// Stovepipe provides the Stovepipe API.
service Stovepipe {
// Ping returns a response indicating the service is alive
Expand All @@ -396,4 +433,6 @@ service Stovepipe {
rpc GetQueueStatus(GetQueueStatusRequest) returns (GetQueueStatusResponse) {}
// ListOpenIssues returns current open issues validated against their summaries.
rpc ListOpenIssues(ListOpenIssuesRequest) returns (ListOpenIssuesResponse) {}
// GetIssuesByRequestID returns accepted issue results for one request.
rpc GetIssuesByRequestID(GetIssuesByRequestIDRequest) returns (GetIssuesByRequestIDResponse) {}
}
300 changes: 272 additions & 28 deletions api/stovepipe/protopb/stovepipe.pb.go

Large diffs are not rendered by default.

286 changes: 171 additions & 115 deletions api/stovepipe/protopb/stovepipe.pb.yarpc.go

Large diffs are not rendered by default.

40 changes: 40 additions & 0 deletions api/stovepipe/protopb/stovepipe_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions doc/rfc/stovepipe/request-issues-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Request issues API

The request-issues lookup returns accepted results for a queue and request ID, not the queue's full open-issue set. Each result is new (opened or reopened), existing (reported but already open), or closed (actually closed by this request). Older issues retained because closure was suppressed are not existing results unless this request reported them.

The issue log retains openings, repeat reports, and closures. Its materializer resolves the source build to its request and derives New, Existing, or Closed results in the request-issue summary. Repeat reports leave lifecycle status and timestamps unchanged. The retained opening reference supplies historical provenance without consulting current issue state; legacy events without this reference do not create request summaries.

Results are read from the request-issue summary under the queue/request primary-key prefix, in task/unit order. The default page size is 50 and the maximum is 200. A queue- and request-bound opaque continuation resumes after the last returned key; one extra row establishes whether another page exists. No joins or secondary indexes are required. A closure wins over earlier reports or openings from the same request.

The response reports in-progress recording, complete reconciliation, incomplete abandoned work, or unavailable coverage. Completion is read before the result page and is written only after all accepted issue events and their projections are durable. Completed or abandoned results remain stable as later requests update the queue. Complete does not mean the build passed or all queue issues closed; canceled builds and task-wide failures can suppress closure.

In-progress pages are not a snapshot: concurrent recording can add keys behind the cursor. Restart from the first page once recording finishes. Unavailable means recording has not started or the request predates the retained start marker; an empty page then does not establish that the request had no issues. Historical results are not backfilled.

Recording coverage assumes every issue-tracking worker writes request results. Drain older workers before admitting requests whose summaries need complete coverage; requests started before that deployment boundary are not backfilled.
2 changes: 2 additions & 0 deletions service/stovepipe/server/handler/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ go_library(
"//stovepipe/controller:go_default_library",
"//stovepipe/core/queuelog:go_default_library",
"//stovepipe/core/queuestate:go_default_library",
"//stovepipe/extension/storage:go_default_library",
"@org_golang_google_grpc//codes:go_default_library",
"@org_golang_google_grpc//status:go_default_library",
],
Expand All @@ -19,6 +20,7 @@ go_library(
go_test(
name = "go_default_test",
srcs = [
"get_issues_by_request_id_test.go",
"list_open_issues_test.go",
"list_test.go",
"queue_status_test.go",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
// Copyright (c) 2026 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package handler

import (
"context"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
pb "github.com/uber/submitqueue/api/stovepipe/protopb"
"github.com/uber/submitqueue/stovepipe/controller"
controllermock "github.com/uber/submitqueue/stovepipe/controller/mock"
"github.com/uber/submitqueue/stovepipe/entity"
"github.com/uber/submitqueue/stovepipe/extension/storage"
"go.uber.org/mock/gomock"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)

func TestGetIssuesByRequestID(t *testing.T) {
for _, tt := range []struct {
name string
err error
code codes.Code
}{
{"success", nil, codes.OK}, {"invalid", controller.ErrInvalidRequest, codes.InvalidArgument},
{"missing request", storage.ErrNotFound, codes.NotFound}, {"canceled", context.Canceled, codes.Canceled},
{"deadline", context.DeadlineExceeded, codes.DeadlineExceeded}, {"dependency", assert.AnError, codes.Unknown},
} {
t.Run(tt.name, func(t *testing.T) {
c := controllermock.NewMockGetIssuesByRequestIDController(gomock.NewController(t))
c.EXPECT().GetIssuesByRequestID(gomock.Any(), entity.GetIssuesByRequestIDRequest{Queue: "queue", RequestID: "12", PageSize: 10, PageToken: "token"}).Return(entity.GetIssuesByRequestIDResult{Status: entity.RequestIssuesStatusIncomplete, NextPageToken: "next"}, tt.err)
server := NewStovepipeServer(nil, nil, nil, nil, nil, nil, nil, c)
response, err := server.GetIssuesByRequestID(context.Background(), &pb.GetIssuesByRequestIDRequest{Queue: "queue", RequestId: "12", PageSize: 10, PageToken: "token"})
require.Equal(t, tt.code, status.Code(err))
if tt.err == nil {
require.Equal(t, "incomplete", response.Status)
require.Equal(t, "next", response.NextPageToken)
} else {
require.Nil(t, response)
}
})
}
}
2 changes: 1 addition & 1 deletion service/stovepipe/server/handler/list_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ func TestList(t *testing.T) {
}, req)
return tt.result, tt.err
}}
srv := NewStovepipeServer(nil, nil, nil, nil, fake, nil, nil)
srv := NewStovepipeServer(nil, nil, nil, nil, fake, nil, nil, nil)
response, err := srv.List(ctx, &pb.ListRequest{
Queue: "queue", PageSize: 25, PageToken: "token",
AcceptedLowerBound: &pb.ListRequest_AcceptedAtOrAfterMs{},
Expand Down
2 changes: 1 addition & 1 deletion service/stovepipe/server/handler/queue_status_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ func TestGetQueueStatus(t *testing.T) {
require.Equal(t, entity.GetQueueStatusRequest{Queue: "repo/main", ChangeURI: "C"}, req)
return entity.GetQueueStatusResult{Queue: "repo/main", Enablement: entity.QueueState{Version: 1, State: entity.QueueEnablementDisabled}}, wantErr
}}
server := NewStovepipeServer(nil, nil, nil, nil, nil, fake, nil)
server := NewStovepipeServer(nil, nil, nil, nil, nil, fake, nil, nil)
response, err := server.GetQueueStatus(context.Background(), &pb.GetQueueStatusRequest{Queue: "repo/main", ChangeUri: "C"})
if wantErr != nil {
require.Equal(t, tt.code, status.Code(err))
Expand Down
22 changes: 22 additions & 0 deletions service/stovepipe/server/handler/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import (

"github.com/uber/submitqueue/stovepipe/core/queuelog"
"github.com/uber/submitqueue/stovepipe/core/queuestate"
"github.com/uber/submitqueue/stovepipe/extension/storage"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"

Expand All @@ -38,6 +39,7 @@ type StovepipeServer struct {
projectStatusController controller.ProjectStatusController
listController controller.ListController
openIssuesController controller.ListOpenIssuesController
requestIssuesController controller.GetIssuesByRequestIDController
queueStatusController controller.GetQueueStatusController
}

Expand All @@ -50,6 +52,7 @@ func NewStovepipeServer(
listController controller.ListController,
queueStatusController controller.GetQueueStatusController,
openIssuesController controller.ListOpenIssuesController,
requestIssuesController controller.GetIssuesByRequestIDController,
) *StovepipeServer {
return &StovepipeServer{
pingController: pingController,
Expand All @@ -58,6 +61,7 @@ func NewStovepipeServer(
projectStatusController: projectStatusController,
listController: listController,
openIssuesController: openIssuesController,
requestIssuesController: requestIssuesController,
queueStatusController: queueStatusController,
}
}
Expand Down Expand Up @@ -161,3 +165,21 @@ func (s *StovepipeServer) ListOpenIssues(ctx context.Context, req *pb.ListOpenIs
}
return mapper.ListOpenIssuesResultToProto(result), nil
}

// GetIssuesByRequestID returns accepted results and their recording status.
func (s *StovepipeServer) GetIssuesByRequestID(ctx context.Context, req *pb.GetIssuesByRequestIDRequest) (*pb.GetIssuesByRequestIDResponse, error) {
result, err := s.requestIssuesController.GetIssuesByRequestID(ctx, mapper.ProtoToGetIssuesByRequestIDRequest(req))
if err != nil {
switch {
case controller.IsInvalidRequest(err):
return nil, status.Error(codes.InvalidArgument, err.Error())
case errors.Is(err, storage.ErrNotFound):
return nil, status.Error(codes.NotFound, err.Error())
case errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded):
return nil, status.FromContextError(err).Err()
default:
return nil, err
}
}
return mapper.GetIssuesByRequestIDResultToProto(result), nil
}
2 changes: 2 additions & 0 deletions service/stovepipe/server/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -304,9 +304,11 @@ func run() error {
listController := controller.NewListController(logger.Sugar(), scope, storageFty, tenants)
queueStatusController := controller.NewGetQueueStatusController(scope, storageFty, sourceControl)
openIssuesController := controller.NewListOpenIssuesController(logger.Sugar(), scope, storageFty, tenants)
requestIssuesController := controller.NewGetIssuesByRequestIDController(logger.Sugar(), scope, storageFty, tenants)
srv := handler.NewStovepipeServer(
pingController, ingestController, requestHistoryController, projectStatusController,
listController, queueStatusController, openIssuesController,
requestIssuesController,
)
pb.RegisterStovepipeServer(grpcServer, srv)

Expand Down
2 changes: 2 additions & 0 deletions service/stovepipe/server/mapper/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ load("@rules_go//go:def.bzl", "go_library", "go_test")
go_library(
name = "go_default_library",
srcs = [
"get_issues_by_request_id.go",
"ingest.go",
"list.go",
"list_open_issues.go",
Expand All @@ -22,6 +23,7 @@ go_library(
go_test(
name = "go_default_test",
srcs = [
"get_issues_by_request_id_test.go",
"ingest_test.go",
"list_open_issues_test.go",
"list_test.go",
Expand Down
34 changes: 34 additions & 0 deletions service/stovepipe/server/mapper/get_issues_by_request_id.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
// Copyright (c) 2026 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package mapper

import (
pb "github.com/uber/submitqueue/api/stovepipe/protopb"
"github.com/uber/submitqueue/stovepipe/entity"
)

// ProtoToGetIssuesByRequestIDRequest leaves defaults and validation to the controller.
func ProtoToGetIssuesByRequestIDRequest(req *pb.GetIssuesByRequestIDRequest) entity.GetIssuesByRequestIDRequest {
return entity.GetIssuesByRequestIDRequest{Queue: req.GetQueue(), RequestID: req.GetRequestId(), PageSize: req.GetPageSize(), PageToken: req.GetPageToken()}
}

// GetIssuesByRequestIDResultToProto preserves request classifications and completion status.
func GetIssuesByRequestIDResultToProto(result entity.GetIssuesByRequestIDResult) *pb.GetIssuesByRequestIDResponse {
issues := make([]*pb.RequestIssueSummary, 0, len(result.Issues))
for _, issue := range result.Issues {
issues = append(issues, &pb.RequestIssueSummary{TaskId: issue.TaskID, UnitId: issue.UnitID, Kind: string(issue.Kind), BuildId: issue.BuildID, FirstSeenBuildId: issue.FirstSeenBuildID})
}
return &pb.GetIssuesByRequestIDResponse{Issues: issues, Status: string(result.Status), NextPageToken: result.NextPageToken}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
// Copyright (c) 2026 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package mapper

import (
"testing"

"github.com/stretchr/testify/require"
pb "github.com/uber/submitqueue/api/stovepipe/protopb"
"github.com/uber/submitqueue/stovepipe/entity"
)

func TestRequestIssueMappings(t *testing.T) {
require.Empty(t, ProtoToGetIssuesByRequestIDRequest(nil))
require.Equal(t, entity.GetIssuesByRequestIDRequest{Queue: "queue", RequestID: "12", PageSize: 10, PageToken: "token"}, ProtoToGetIssuesByRequestIDRequest(&pb.GetIssuesByRequestIDRequest{Queue: "queue", RequestId: "12", PageSize: 10, PageToken: "token"}))
result := entity.GetIssuesByRequestIDResult{Issues: []entity.RequestIssueSummary{{RequestID: "12", TaskID: "task", UnitID: "unit", Kind: entity.RequestIssueKindExisting, BuildID: "b/12", FirstSeenBuildID: "b/10"}}, Status: entity.RequestIssuesStatusComplete, NextPageToken: "next"}
require.Equal(t, &pb.GetIssuesByRequestIDResponse{Issues: []*pb.RequestIssueSummary{{TaskId: "task", UnitId: "unit", Kind: "existing", BuildId: "b/12", FirstSeenBuildId: "b/10"}}, Status: "complete", NextPageToken: "next"}, GetIssuesByRequestIDResultToProto(result))
require.Empty(t, GetIssuesByRequestIDResultToProto(entity.GetIssuesByRequestIDResult{}).Issues)
}
Loading
Loading