Skip to content

Results about ScienceQA #2

@zhangbaijin

Description

@zhangbaijin

你好,我将你的配置替换到了llava以后,跑llava的时候,发现prompt似乎对输出无法规范化,之前llava跑SQA,可以直接输出Answer,比如ABCD,但是换了这个baseline以后,原始的prompt无法使得模型直接输出选项,例如,我的结果例子如下:
id": "5", "prompt": "<image>\nContext: People can use the engineering-design process to develop solutions to problems. One step in the process is testing if a potential solution meets the requirements of the design.\nQuestion: Which of the following could Gordon's test show?\nOptions: (A) if the spacecraft was damaged when using a parachute with a 1 m vent going 200 km per hour (B) how steady a parachute with a 1 m vent was at 200 km per hour (C) whether a parachute with a 1 m vent would swing too much at 400 km per hour\nAnswer with the option's letter from the given choices directly.", "text": "(B) how steady a parachute with a 1 m vent was at 200 km per hour".

id": "11", "prompt": "<image>\nContext: N/A\nQuestion: What is the name of the colony shown?\nOptions: (A) Maryland (B) New Hampshire (C) Rhode Island (D) Vermont\nAnswer with the option's letter from the given choices directly.", "text": "The colony shown in the image is New Hampshire. This can be determined by the distinctive shape of the state, which is elongated with a jagged eastern border that follows the coastline, a characteristic feature of New Hampshire's geography. The color green is used to highlight this particular state on the map, distinguishing it from the surrounding areas. Therefore, the correct answer to the question is (B) New Hampshire.",
之前llava的tex{}里面是直接选项例如ABCD,现在是各种都有,这是什么问题?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions