Lazy Teacher

Is a library that allows for automatic question generation and evaluation using the help of llm, built on langchain.

Installation

Example Usage

Create Question Manually

To create questions manually, use the ShortAnswer, Essay, or MultipleChoice classes from "lazyteacher/questions":

• Example of ShortAnswer

import { ShortAnswer } from "lazyteacher/questions";

const short_answer = new ShortAnswer(
    "What is the capital of France?", // Question
    "Paris" // Correct answer (optional)
);

• Example of Essay

import { Essay } from "lazyteacher/questions";

const essay = new Essay(
    "Explain the concept of RESTful API?",
    // Correct Answer is Optional
);

• Example of MultipleChoice

import { MultipleChoice } from "lazyteacher/questions";

const multi_choice = new MultipleChoice(
    "What is the capital of France?", // Question
    ["Paris", "London", "Berlin"], // Options
    0 // Correct answer, this is the index of choices
);

Evaluate Question

Use QuestionEvaluator to evaluate user answers.

• Example of evaluation without criteria

import { QuestionEvaluator } from "lazyteacher/evaluator";
import { ShortAnswer } from "lazyteacher/questions";

const short_answer = new ShortAnswer(
    "What is the capital of France?", // Question
    "Paris" // Correct answer, optional
);

const evaluator = new QuestionEvaluator();

const evaluation = await evaluator.evaluate(
    short_answer, // Question Object
    "the capital of france is paris" // User answer
);

• Example of evaluation with criteria

const evaluation_with_criteria = await evaluator.evaluate(
    short_answer, // Question Object
    "pharis", // User answer
    "one letter typo is still acceptable as correct" // Criteria
);

Example Evaluation Results:

ShortAnswer {
    question: 'What is the capital of France?',
    correct_answer: 'Paris',
    type: 'short_answer',
    answer: 'pharis',
    criteria: 'one letter typo is still acceptable as correct',
    score: 10,
    reason: "The user answer 'pharis' contains a one-letter typo, which is acceptable according to the criteria provided. Therefore, the answer is considered correct despite the misspelling."
}

Essay {
    question: 'Explain the concept of RESTful API?',
    correct_answer: 'A RESTful API follows the Representational State Transfer (REST) principles, using HTTP methods (GET, POST, PUT, DELETE) to interact with resources, typically in JSON or XML format. It is stateless, meaning each request contains all necessary information without relying on previous server sessions. REST is widely used for web applications due to its simplicity, scalability, and ease of integration.',
    type: 'essay',
    answer: 'A RESTful API allows applications to communicate with a server using HTTP methods like GET (fetch data) and POST (send data), usually in JSON format.',
    criteria: undefined,
    score: 5,
    reason: 'The user answer provides a basic understanding of RESTful APIs by mentioning HTTP methods and data formats, but it lacks details about the stateless nature, the full range of HTTP methods, and the principles of REST. Therefore, it is incomplete compared to the correct answer.'
}

MultipleChoice {
    question: 'What is the capital of France?',
    correct_answer: 0,
    type: 'multiple_choice',
    choices: [ 'Paris', 'London', 'Berlin' ],
    answer: 1,
    score: 0
}

Change the Evaluation Score Range

The default score range is 0-10. You can change the score range by setting the low_score and high_score attributes of QuestionEvaluator.

import { QuestionEvaluator } from "lazyteacher/evaluator";

const evaluator = new QuestionEvaluator();
evaluator.low_score = 0;
evaluator.high_score = 1;

Generate Question

The generated questions can also be evaluated using QuestionEvaluator. The generated result contains a list of question objects (Essay, ShortAnswer, or MultipleChoice).

• Generate Questions from Prompt

import { QuestionGenerator } from 'lazyteacher/generator';

const questionGenerator = new QuestionGenerator();
const user_prompt = "Question about AI Agent for future";
const total_questions = 3;
const question_type = "multiple_choice";

const generated_questions = await questionGenerator.generateFromPrompt({ question_type: question_type, user_prompt: user_prompt, total_questions: total_questions });

console.dir(generated_questions);

Output:

[
    MultipleChoice {
        question: 'What is one potential benefit of AI agents mentioned in the text?',
        correct_answer: 0,
        type: 'multiple_choice',
        choices: [
        'Enhancing productivity',
        'Increasing job displacement',
        'Reducing data privacy',
        'Limiting personalized experiences'
        ],
        answer: undefined,
        score: undefined
    },
    MultipleChoice {
        question: 'What is a significant challenge associated with the rise of AI agents?',
        correct_answer: 1,
        type: 'multiple_choice',
        choices: [
        'Streamlining operations',
        'Data privacy concerns',
        'Improving healthcare',
        'Enhancing customer service'
        ],
        answer: undefined,
        score: undefined
    },
    MultipleChoice {
        question: 'According to the text, what is crucial as we integrate AI agents into our lives?',
        correct_answer: 2,
        type: 'multiple_choice',
        choices: [
        'Maximizing job displacement',
        'Ignoring ethical considerations',
        'Striking a balance between benefits and societal implications',
        'Focusing solely on technological advancements'
        ],
        answer: undefined,
        score: undefined
    }
]

• Generate Questions from Document

import { QuestionGenerator } from 'lazyteacher/generator';
import * as fs from "fs";

const total_questions = 7;
const content = fs.readFileSync("example.txt", "utf-8");

const questionGenerator = new QuestionGenerator();

const question_generated = await questionGenerator.generateFromDocument({
    question_type: "short_answer",
    document_text: content, // in string
    total_questions: total_questions,
    chunk_size: 1000,
    chunk_overlap: 50,
    document_query: "Jakarta", // query relevant document (optional)
    top_query: 5 // top query relevant (optional)
});