Skip to content

Conversation

@rlundeen2
Copy link
Contributor

@rlundeen2 rlundeen2 commented Jan 3, 2026

Refactor Seed Models, allowing Seeds to configure simulating conversation in Attacks.

  • Separated SeedGroup and SeedAttackGroup into distinct files, where SeedAttackGroup is a subclass - specifically a SeedGroup with exactly one objective. I imagine in the future there will be more groups (e.g. SeedBenchmarkGroup).
  • Added SeedSimulatedConversation to be able to configure simulating conversations in datasets. In the future I expect there will be more seed types (e.g. SeedExpectedResponse that might have an expected return value and required for benchmarks)
  • Added code in AttackExecutor and Attack's FromSeedGroups to generate simulated conversations based on SeedSimulatedConversation

Multiple small refactors to clearly separate responsibilities along the way. In the future

To document, I added examples in 2_seed_programming that shows how Seeds translate to attack parameters, and included the new SeedSimulatedConversation

DB Migration Notes

The following failed before migration:

pytest .\tests\integration\memory\test_azure_sql_memory_integration.py::test_get_seeds_with_metadata_filter

Migration in test and prod:

ALTER TABLE [SeedPromptEntries] ADD [seed_type] NVARCHAR(50) NULL;

Then the test passes

@rlundeen2 rlundeen2 changed the title FEAT: SeedSimulatedConversation to generate simulated conversations in attacks Draft FEAT: SeedSimulatedConversation to generate simulated conversations in attacks Jan 4, 2026
@rlundeen2 rlundeen2 changed the title Draft FEAT: SeedSimulatedConversation to generate simulated conversations in attacks FEAT: SeedSimulatedConversation to generate simulated conversations in attacks Jan 6, 2026
@rlundeen2 rlundeen2 force-pushed the users/rlundeen/2026_01_02_simulated_attack_param branch from 4737f02 to 17541d8 Compare January 7, 2026 00:15
@rlundeen2 rlundeen2 merged commit 520257e into Azure:main Jan 8, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants