-
Notifications
You must be signed in to change notification settings - Fork 3
All issues
Issue creation is restricted in this repository
Issues
is:issue state:open
is:issue state:open
Search results
L4: end-to-end agent evaluation on live pages
agentAgent-in-the-loop evaluationAgent-in-the-loop evaluationL4End-to-end agent evaluation layerEnd-to-end agent evaluation layerresearchOpen question, not yet scheduledOpen question, not yet scheduledStatus: Open.#24 In lexmount/Lexbench-Headless-Browser;L3-b: frozen real sites
datasetTask set, fixtures and subsetsTask set, fixtures and subsetsL3Frozen interactive environments layerFrozen interactive environments layerlegalLicensing and redistributionLicensing and redistributionStatus: Open.#23 In lexmount/Lexbench-Headless-Browser;L3-a: a frozen interactive environment set
datasetTask set, fixtures and subsetsTask set, fixtures and subsetsL3Frozen interactive environments layerFrozen interactive environments layerRLReinforcement learning environmentsReinforcement learning environmentsStatus: Open.#22 In lexmount/Lexbench-Headless-Browser;Two measurement gaps beyond L1 and L2
architectureStructure of the benchmarkStructure of the benchmarkdocsDocumentationDocumentationscoringScoring rules and denominatorsScoring rules and denominatorsStatus: Open.#21 In lexmount/Lexbench-Headless-Browser;Raster is not measured on any layer
datasetTask set, fixtures and subsetsTask set, fixtures and subsetsL1Protocol and driver compatibility layerProtocol and driver compatibility layerL2Web platform semantics layerWeb platform semantics layerStatus: Open.#20 In lexmount/Lexbench-Headless-Browser;Eight capabilities produce no score
L2Web platform semantics layerWeb platform semantics layerscoringScoring rules and denominatorsScoring rules and denominatorsStatus: Open.#19 In lexmount/Lexbench-Headless-Browser;