-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathtesting.html
More file actions
97 lines (78 loc) · 7.84 KB
/
Copy pathtesting.html
File metadata and controls
97 lines (78 loc) · 7.84 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Testing </title>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@5.2.3/dist/css/bootstrap.min.css">
<style>
body{
background-color: #000042 ;
color: #fff;
}
</style>
</head>
<body>
<nav class="navbar navbar-expand-lg navbar-light bg-light">
<div class="container-fluid">
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarNav" aria-controls="navbarNav" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
<div class="collapse navbar-collapse justify-content-end" id="navbarNav">
<ul class="navbar-nav">
<li class="nav-item">
<a class="nav-link " href="home.html">Home</a>
</li>
<li class="nav-item">
<a class="nav-link " href="team.html">Meet The Team</a>
</li>
<li class="nav-item">
<a class="nav-link" href="research.html">Research</a>
</li>
<li class="nav-item">
<a class="nav-link active" href="testing.html">Testing</a>
</li>
<li class="nav-item">
<a class="nav-link" aria-current="page" href="analysis.html">Analysis</a>
</li>
<li class="nav-item">
<a class="nav-link" href="futureworks.html">Future Works & Conclusion</a>
</li>
<li class="nav-item">
<a class="nav-link" href="citations.html">Citations</a>
</li>
</ul>
</div>
</div>
</nav>
</nav>
<button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarNav" aria-controls="navbarNav" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
<div class="container">
<div class="container py-5">
<h1 class="text-center mb-5">Testing</h1>
<div class="container py-5" style="border: 1px solid white; margin-bottom: 20px;;">
<h4 class="text-center mb-5">Pretest During Semester 1<h4>
<h6><u>Questions for the pretest</u></h6>
<p>Once we had identified our target audience, which consisted of computer science students at different levels of education (freshmen, sophomore, junior, and senior), we developed specific questions for each category. These questions were designed to assess the participants' knowledge and expertise in programming, with the aim of understanding how the use of AI autocomplete tools would impact their ability to answer them.</p>
<p>Each category of participants was given three different questions, and they were instructed to answer them without using any AI tools, which would take approximately 5-10 minutes. We then compared the results of their responses with and without the use of the AI tool to evaluate the effectiveness of the tool in enhancing their ability to answer the questions.</p>
<p class="mb-5">This approach allowed us to assess the effectiveness of the AI autocomplete tool in improving the participants' performance in answering the questions, and to determine whether the tool had a significant impact on their ability to complete the tasks. By comparing the results with and without the AI tool, we aimed to gain insights into the potential benefits and limitations of using such tools in daily programming tasks for different levels of expertise.</p>
<h6><u>Pretest runs</u></h6>
<p>To ensure the smooth execution of our research, we conducted a pretest with a small group of approximately 10-15 students from our target audience pool. The purpose of the pretest was to familiarize ourselves with the research procedures and to assess the time it would take for participants to complete the tasks </p>
</div>
<div class="container py-5" style="border: 1px solid white; margin-bottom: 20px;;">
<h4 class="text-center mb-5">Semester 2<h4>
<h6><u>Testing</u></h6>
<p>To recruit participants for our project, we initiated the process by seeking permission from relevant course professors to conduct testing during their classes. Prior to receiving their consent, we developed new questions to comprehensively evaluate the capabilities of ChatGPT-3. Once we obtained approval, we introduced the project to the students and provided a detailed explanation of the testing procedures. Interested students were then provided with further information and invited to log in for the testing session. Through this collaborative effort with the professors and conducting testing during class time, we were able to access a diverse pool of participants with varying levels of expertise and knowledge. This approach ensured that our research was conducted in an organized and controlled environment, allowing us to obtain valuable insights into the impact of AI autocomplete tools on programming proficiency among students at different educational levels. </p>
<p >Our study aimed to evaluate the efficacy of ChatGPT-3's autocomplete feature in answering questions accurately, compared to personal knowledge and visual perception. We postulated that ChatGPT-3's autocomplete feature had the potential to enhance accuracy and efficiency in text-based questions by offering pertinent and precise responses based on its extensive knowledge base. To examine this hypothesis, we presented participants with a text-based question that required them to provide an answer based on their personal knowledge, while having unrestricted access to the internet. We then compared ChatGPT-3's autocomplete feature's accuracy and efficiency with participants' personal knowledge to determine its effectiveness in various settings. </p>
<p >Additionally, we assessed ChatGPT-3's proficiency in processing visual information and generating accurate responses based on visual cues, using picture-based questions. The processing of visual information is an essential aspect of natural language processing, and we aimed to ascertain ChatGPT-3's effectiveness in this domain. To achieve this, we presented participants with a picture-based question that necessitated an answer solely based on the visual information provided in the image. We then compared ChatGPT-3's accuracy and efficiency with participants' visual perception to evaluate its efficacy in processing visual information. </p>
<p>Apart from assessing the efficacy of ChatGPT-3, we were also interested in investigating the potential advantages and disadvantages associated with its use in question-answering tasks. We compared ChatGPT-3's accuracy and efficiency with participants' personal knowledge and unrestricted internet access to determine its usefulness in various situations. We also evaluated ChatGPT-3's performance in processing visual information and identified any potential biases or limitations in the model that could affect its effectiveness in diverse settings. </p>
<p class="mb-5">In summary, our testing approach aimed to collect more accurate data and insights into ChatGPT-3's effectiveness in different scenarios and explore its potential applications in natural language processing and related fields. By incorporating two different question formats and considering various factors such as participants' prior knowledge and experience, we obtained a comprehensive evaluation of ChatGPT-3's performance and potential benefits and drawbacks associated with its use in question-answering tasks. </p>
</div>
</div>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.2.3/dist/js/bootstrap.bundle.min.js"></script>
</div>
</body>
</html>