Skip to content

Get mouse position, GPU mode browsers #115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sjmiller609 merged 8 commits into from
Feb 19, 2026
+111 −14
Merged

Get mouse position, GPU mode browsers #115

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
6336f16
CLI: Update hypeman SDK to a897d4c032e104d1f60b1f29415b8f691cd818bb a…
kernel-internal[bot] Feb 11, 2026
3f54109
Merge main into cli-coverage-update
kernel-internal[bot] Feb 18, 2026
1d5b939
feat(browsers): add computer get-mouse-position and batch commands
kernel-internal[bot] Feb 18, 2026
fe7b346
chore: remove all hypeman commands and dependency
cursoragent Feb 18, 2026
ef63bc4
Merge main into cli-coverage-update
kernel-internal[bot] Feb 19, 2026
1730338
Update Go SDK to v0.35.0
kernel-internal[bot] Feb 19, 2026
7f6d156
Add support for gpu
sjmiller609 Feb 19, 2026
b9dd123
fix: correct --actions flag help text to match expected JSON shape
masnwilliams Feb 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 91 additions & 1 deletion cmd/browsers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,9 +92,11 @@ type BrowserPlaywrightService interface {

// BrowserComputerService defines the subset we use for OS-level mouse & screen.
type BrowserComputerService interface {
Batch(ctx context.Context, id string, body kernel.BrowserComputerBatchParams, opts ...option.RequestOption) (err error)
CaptureScreenshot(ctx context.Context, id string, body kernel.BrowserComputerCaptureScreenshotParams, opts ...option.RequestOption) (res *http.Response, err error)
ClickMouse(ctx context.Context, id string, body kernel.BrowserComputerClickMouseParams, opts ...option.RequestOption) (err error)
DragMouse(ctx context.Context, id string, body kernel.BrowserComputerDragMouseParams, opts ...option.RequestOption) (err error)
GetMousePosition(ctx context.Context, id string, opts ...option.RequestOption) (res *kernel.BrowserComputerGetMousePositionResponse, err error)
MoveMouse(ctx context.Context, id string, body kernel.BrowserComputerMoveMouseParams, opts ...option.RequestOption) (err error)
PressKey(ctx context.Context, id string, body kernel.BrowserComputerPressKeyParams, opts ...option.RequestOption) (err error)
Scroll(ctx context.Context, id string, body kernel.BrowserComputerScrollParams, opts ...option.RequestOption) (err error)
Expand Down Expand Up @@ -172,6 +174,7 @@ type BrowsersCreateInput struct {
TimeoutSeconds int
Stealth BoolFlag
Headless BoolFlag
GPU BoolFlag
Kiosk BoolFlag
ProfileID string
ProfileName string
Expand Down Expand Up @@ -340,6 +343,9 @@ func (b BrowsersCmd) Create(ctx context.Context, in BrowsersCreateInput) error {
if in.Headless.Set {
params.Headless = kernel.Opt(in.Headless.Value)
}
if in.GPU.Set {
params.GPU = kernel.Opt(in.GPU.Value)
}
if in.Kiosk.Set {
params.KioskMode = kernel.Opt(in.Kiosk.Value)
}
Expand Down Expand Up @@ -527,6 +533,7 @@ func (b BrowsersCmd) Get(ctx context.Context, in BrowsersGetInput) error {
tableData = append(tableData, []string{"Timeout (seconds)", fmt.Sprintf("%d", browser.TimeoutSeconds)})
tableData = append(tableData, []string{"Headless", fmt.Sprintf("%t", browser.Headless)})
tableData = append(tableData, []string{"Stealth", fmt.Sprintf("%t", browser.Stealth)})
tableData = append(tableData, []string{"GPU", fmt.Sprintf("%t", browser.GPU)})
tableData = append(tableData, []string{"Kiosk Mode", fmt.Sprintf("%t", browser.KioskMode)})
if browser.Viewport.Width > 0 && browser.Viewport.Height > 0 {
viewportStr := fmt.Sprintf("%dx%d", browser.Viewport.Width, browser.Viewport.Height)
Expand Down Expand Up @@ -740,6 +747,16 @@ type BrowsersComputerSetCursorInput struct {
Hidden bool
}

type BrowsersComputerGetMousePositionInput struct {
Identifier string
Output string
}

type BrowsersComputerBatchInput struct {
Identifier string
ActionsJSON string
}

func (b BrowsersCmd) ComputerClickMouse(ctx context.Context, in BrowsersComputerClickMouseInput) error {
if b.computer == nil {
pterm.Error.Println("computer service not available")
Expand Down Expand Up @@ -956,6 +973,49 @@ func (b BrowsersCmd) ComputerSetCursor(ctx context.Context, in BrowsersComputerS
return nil
}

func (b BrowsersCmd) ComputerGetMousePosition(ctx context.Context, in BrowsersComputerGetMousePositionInput) error {
if b.computer == nil {
pterm.Error.Println("computer service not available")
return nil
}
br, err := b.browsers.Get(ctx, in.Identifier, kernel.BrowserGetParams{})
if err != nil {
return util.CleanedUpSdkError{Err: err}
}
res, err := b.computer.GetMousePosition(ctx, br.SessionID)
if err != nil {
return util.CleanedUpSdkError{Err: err}
}
if in.Output == "json" {
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
return enc.Encode(res)
Copy link

@cursor cursor bot Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSON output re-marshals struct instead of using raw API response

Low Severity

ComputerGetMousePosition uses json.NewEncoder/enc.Encode(res) for its --output json path, while every other command in the file consistently uses util.PrintPrettyJSON. That utility explicitly calls RawJSON() on the SDK response to preserve the original API response and avoid zero-value fields that appear when re-marshaling the Go struct (as noted in its own doc comment). Re-marshaling via enc.Encode may produce different output — extra zero-value fields, missing unmapped fields — breaking consistency with all other --output json commands.

Fix in Cursor Fix in Web

}
fmt.Printf("x: %d\ny: %d\n", res.X, res.Y)
return nil
}

func (b BrowsersCmd) ComputerBatch(ctx context.Context, in BrowsersComputerBatchInput) error {
if b.computer == nil {
pterm.Error.Println("computer service not available")
return nil
}
br, err := b.browsers.Get(ctx, in.Identifier, kernel.BrowserGetParams{})
if err != nil {
return util.CleanedUpSdkError{Err: err}
}
var body kernel.BrowserComputerBatchParams
if err := json.Unmarshal([]byte(in.ActionsJSON), &body); err != nil {
pterm.Error.Printf("Invalid JSON: %v\n", err)
return nil
}
if err := b.computer.Batch(ctx, br.SessionID, body); err != nil {
return util.CleanedUpSdkError{Err: err}
}
pterm.Success.Println("Batch actions executed")
return nil
}

// Replays
type BrowsersReplaysListInput struct {
Identifier string
Expand Down Expand Up @@ -2300,7 +2360,16 @@ func init() {
computerSetCursor.Flags().String("hidden", "", "Whether to hide the cursor: true or false")
_ = computerSetCursor.MarkFlagRequired("hidden")

computerRoot.AddCommand(computerClick, computerMove, computerScreenshot, computerType, computerPressKey, computerScroll, computerDrag, computerSetCursor)
// computer get-mouse-position
computerGetMousePosition := &cobra.Command{Use: "get-mouse-position <id>", Short: "Get current mouse cursor position", Args: cobra.ExactArgs(1), RunE: runBrowsersComputerGetMousePosition}
computerGetMousePosition.Flags().StringP("output", "o", "", "Output format: json for raw API response")

// computer batch
computerBatch := &cobra.Command{Use: "batch <id>", Short: "Execute a batch of computer actions from JSON", Args: cobra.ExactArgs(1), RunE: runBrowsersComputerBatch}
computerBatch.Flags().String("actions", "", "JSON object with actions array (e.g., {\"actions\":[{\"type\":\"click_mouse\",...}]})")
_ = computerBatch.MarkFlagRequired("actions")

computerRoot.AddCommand(computerClick, computerMove, computerScreenshot, computerType, computerPressKey, computerScroll, computerDrag, computerSetCursor, computerGetMousePosition, computerBatch)
browsersCmd.AddCommand(computerRoot)

// playwright
Expand All @@ -2316,6 +2385,7 @@ func init() {
_ = browsersCreateCmd.Flags().MarkDeprecated("persistent-id", "use --timeout (up to 72 hours) and profiles instead")
browsersCreateCmd.Flags().BoolP("stealth", "s", false, "Launch browser in stealth mode to avoid detection")
browsersCreateCmd.Flags().BoolP("headless", "H", false, "Launch browser without GUI access")
browsersCreateCmd.Flags().Bool("gpu", false, "Launch browser with hardware-accelerated GPU rendering")
browsersCreateCmd.Flags().Bool("kiosk", false, "Launch browser in kiosk mode")
browsersCreateCmd.Flags().IntP("timeout", "t", 60, "Timeout in seconds for the browser session")
browsersCreateCmd.Flags().String("profile-id", "", "Profile ID to load into the browser session (mutually exclusive with --profile-name)")
Expand Down Expand Up @@ -2359,6 +2429,7 @@ func runBrowsersCreate(cmd *cobra.Command, args []string) error {
}
stealthVal, _ := cmd.Flags().GetBool("stealth")
headlessVal, _ := cmd.Flags().GetBool("headless")
gpuVal, _ := cmd.Flags().GetBool("gpu")
kioskVal, _ := cmd.Flags().GetBool("kiosk")
timeout, _ := cmd.Flags().GetInt("timeout")
profileID, _ := cmd.Flags().GetString("profile-id")
Expand Down Expand Up @@ -2470,6 +2541,7 @@ func runBrowsersCreate(cmd *cobra.Command, args []string) error {
TimeoutSeconds: timeout,
Stealth: BoolFlag{Set: cmd.Flags().Changed("stealth"), Value: stealthVal},
Headless: BoolFlag{Set: cmd.Flags().Changed("headless"), Value: headlessVal},
GPU: BoolFlag{Set: cmd.Flags().Changed("gpu"), Value: gpuVal},
Kiosk: BoolFlag{Set: cmd.Flags().Changed("kiosk"), Value: kioskVal},
ProfileID: profileID,
ProfileName: profileName,
Expand Down Expand Up @@ -3001,6 +3073,24 @@ func runBrowsersComputerSetCursor(cmd *cobra.Command, args []string) error {
return b.ComputerSetCursor(cmd.Context(), BrowsersComputerSetCursorInput{Identifier: args[0], Hidden: hidden})
}

func runBrowsersComputerGetMousePosition(cmd *cobra.Command, args []string) error {
client := getKernelClient(cmd)
svc := client.Browsers
output, _ := cmd.Flags().GetString("output")

b := BrowsersCmd{browsers: &svc, computer: &svc.Computer}
return b.ComputerGetMousePosition(cmd.Context(), BrowsersComputerGetMousePositionInput{Identifier: args[0], Output: output})
}

func runBrowsersComputerBatch(cmd *cobra.Command, args []string) error {
client := getKernelClient(cmd)
svc := client.Browsers
actionsJSON, _ := cmd.Flags().GetString("actions")

b := BrowsersCmd{browsers: &svc, computer: &svc.Computer}
return b.ComputerBatch(cmd.Context(), BrowsersComputerBatchInput{Identifier: args[0], ActionsJSON: actionsJSON})
}

func truncateURL(url string, maxLen int) string {
if len(url) <= maxLen {
return url
Expand Down
14 changes: 14 additions & 0 deletions cmd/browsers_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -671,7 +671,9 @@ func makeStream[T any](vals []T) *ssestream.Stream[T] {
// --- Fake for Computer ---

type FakeComputerService struct {
BatchFunc func(ctx context.Context, id string, body kernel.BrowserComputerBatchParams, opts ...option.RequestOption) error
ClickMouseFunc func(ctx context.Context, id string, body kernel.BrowserComputerClickMouseParams, opts ...option.RequestOption) error
GetMousePositionFunc func(ctx context.Context, id string, opts ...option.RequestOption) (*kernel.BrowserComputerGetMousePositionResponse, error)
MoveMouseFunc func(ctx context.Context, id string, body kernel.BrowserComputerMoveMouseParams, opts ...option.RequestOption) error
CaptureScreenshotFunc func(ctx context.Context, id string, body kernel.BrowserComputerCaptureScreenshotParams, opts ...option.RequestOption) (*http.Response, error)
PressKeyFunc func(ctx context.Context, id string, body kernel.BrowserComputerPressKeyParams, opts ...option.RequestOption) error
Expand All @@ -681,12 +683,24 @@ type FakeComputerService struct {
SetCursorVisibilityFunc func(ctx context.Context, id string, body kernel.BrowserComputerSetCursorVisibilityParams, opts ...option.RequestOption) (*kernel.BrowserComputerSetCursorVisibilityResponse, error)
}

func (f *FakeComputerService) Batch(ctx context.Context, id string, body kernel.BrowserComputerBatchParams, opts ...option.RequestOption) error {
if f.BatchFunc != nil {
return f.BatchFunc(ctx, id, body, opts...)
}
return nil
}
func (f *FakeComputerService) ClickMouse(ctx context.Context, id string, body kernel.BrowserComputerClickMouseParams, opts ...option.RequestOption) error {
if f.ClickMouseFunc != nil {
return f.ClickMouseFunc(ctx, id, body, opts...)
}
return nil
}
func (f *FakeComputerService) GetMousePosition(ctx context.Context, id string, opts ...option.RequestOption) (*kernel.BrowserComputerGetMousePositionResponse, error) {
if f.GetMousePositionFunc != nil {
return f.GetMousePositionFunc(ctx, id, opts...)
}
return &kernel.BrowserComputerGetMousePositionResponse{X: 100, Y: 200}, nil
}
func (f *FakeComputerService) MoveMouse(ctx context.Context, id string, body kernel.BrowserComputerMoveMouseParams, opts ...option.RequestOption) error {
if f.MoveMouseFunc != nil {
return f.MoveMouseFunc(ctx, id, body, opts...)
Expand Down
6 changes: 2 additions & 4 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,13 @@ require (
github.com/charmbracelet/lipgloss/v2 v2.0.0-beta.1
github.com/golang-jwt/jwt/v5 v5.2.2
github.com/joho/godotenv v1.5.1
github.com/kernel/kernel-go-sdk v0.33.0
github.com/kernel/kernel-go-sdk v0.35.0
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c
github.com/pquerna/otp v1.5.0
github.com/pterm/pterm v0.12.80
github.com/samber/lo v1.51.0
github.com/spf13/cobra v1.9.1
github.com/spf13/pflag v1.0.6
github.com/stretchr/testify v1.11.0
github.com/stretchr/testify v1.11.1
github.com/zalando/go-keyring v0.2.6
golang.org/x/crypto v0.47.0
golang.org/x/oauth2 v0.30.0
Expand All @@ -27,7 +26,6 @@ require (
atomicgo.dev/cursor v0.2.0 // indirect
atomicgo.dev/keyboard v0.2.9 // indirect
atomicgo.dev/schedule v0.1.0 // indirect
github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc // indirect
github.com/charmbracelet/colorprofile v0.3.0 // indirect
github.com/charmbracelet/x/ansi v0.8.0 // indirect
github.com/charmbracelet/x/cellbuf v0.0.13 // indirect
Expand Down
13 changes: 4 additions & 9 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,6 @@ github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lpr
github.com/atomicgo/cursor v0.0.1/go.mod h1:cBON2QmmrysudxNBFthvMtN32r3jxVRIvzkUiF/RuIk=
github.com/aymanbagabas/go-udiff v0.2.0 h1:TK0fH4MteXUDspT88n8CKzvK0X9O2xu9yQjWpi6yML8=
github.com/aymanbagabas/go-udiff v0.2.0/go.mod h1:RE4Ex0qsGkTAJoQdQQCA0uG+nAzJO/pI/QwceO5fgrA=
github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc h1:biVzkmvwrH8WK8raXaxBx6fRVTlJILwEwQGL1I/ByEI=
github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
github.com/boyter/gocodewalker v1.4.0 h1:fVmFeQxKpj5tlpjPcyTtJ96btgaHYd9yn6m+T/66et4=
github.com/boyter/gocodewalker v1.4.0/go.mod h1:hXG8xzR1uURS+99P5/3xh3uWHjaV2XfoMMmvPyhrCDg=
github.com/charmbracelet/colorprofile v0.3.0 h1:KtLh9uuu1RCt+Hml4s6Hz+kB1PfV3wi++1h5ia65yKQ=
Expand Down Expand Up @@ -66,8 +64,8 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/kernel/kernel-go-sdk v0.33.0 h1:kfk2bwrw3mbR4IW3JMnOj6Tecxor44YjM8YV153xDTY=
github.com/kernel/kernel-go-sdk v0.33.0/go.mod h1:EeZzSuHZVeHKxKCPUzxou2bovNGhXaz0RXrSqKNf1AQ=
github.com/kernel/kernel-go-sdk v0.35.0 h1:zQcDPxq7N1njnNVoFmxvi3XMKoqemOVlnkVYuYPqAE0=
github.com/kernel/kernel-go-sdk v0.35.0/go.mod h1:EeZzSuHZVeHKxKCPUzxou2bovNGhXaz0RXrSqKNf1AQ=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.0.10/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/klauspost/cpuid/v2 v2.0.12/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
Expand Down Expand Up @@ -99,8 +97,6 @@ github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmd
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pquerna/otp v1.5.0 h1:NMMR+WrmaqXU4EzdGJEE1aUUI0AMRzsp96fFFWNPwxs=
github.com/pquerna/otp v1.5.0/go.mod h1:dkJfzwRKNiegxyNb54X/3fLwhCynbMspSyWKnvi1AEg=
github.com/pterm/pterm v0.12.27/go.mod h1:PhQ89w4i95rhgE+xedAoqous6K9X+r6aSOI2eFF7DZI=
github.com/pterm/pterm v0.12.29/go.mod h1:WI3qxgvoQFFGKGjGnJR849gU0TsEOvKn5Q8LlY1U7lg=
github.com/pterm/pterm v0.12.30/go.mod h1:MOqLIyMOgmTDz9yorcYbcw+HsgoZo3BQfg2wtl3HEFE=
Expand All @@ -125,12 +121,11 @@ github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.0 h1:ib4sjIrwZKxE5u/Japgo/7SJV3PvgjGiRNAvTVGqQl8=
github.com/stretchr/testify v1.11.0/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
Expand Down