XData

Home

Installation

Docker Installation

Administrator Usage

Instructor Mode

• Basic setup
  
• Assignment creation
  
• Evaluation
  
• Grades and partial marking
  
• Learning Tool Integration
  

Student Usage

Developer Setup

Instructor Usage

New Assignment:

1. The Create Assignment' link allows the instructor to setup a new assignment for the students.
2. The instructor needs to provide the name and description of the assignment, along with the default schema to be used for this assignment.
3. The Interactive check box allows instructors to set whether the assignment is an interactive assignment or not. If assignment is marked interactive, the student query will be evaluated against the instructor's query when the students submit their answer. Students will be able to see the datasets for which their query failed (if the query is incorrect) along with the expect result and the result of the their query. Students can resubmit their queries and will thus be able to learn from their mistakes. A link to view instructor's answer is also provided and students can see the correct answer if they wish to do so.
4. If Interactive option is not checked, the assignment is a graded assignment. When students submit their query it is be checked against a basic test case and saved. The instructor can evaluate the queries once the deadline of the assignment is over.
5. The instructor also needs to select the database connection for the assignment. This connection will be used for testing the student query. We use the test database and test password provided during "New database connection".
6. The instructor selects the Default schema to be used for this assignment. On selection, if the schema has more than one data file, then the UI shows all existing data files for selection.
   Instructor can select the Default datasets for evaluation of assignment (Additional datasets provided by instructor using “upload sample data” UI on which student queries will be tested.)
7. The Starts at and Ends at can be used to specify the start and end time for the assignment. Students can submit their queries only within this time-frame.



1. View Assignments:

   1. The View Assignment link allows instructors to view a list of assignments created by the them for the course. Once the assignment is selected the instructor can edit questions, grade questions, view results and scores once the questions are evaluated.
   2. The assignment details also provides option to edit the assignment details like start date, end date, name, description, etc., The assignment can be deleted (which internally deletes all questions, student answers, marks details and anything related to that assignment) using Edit assignment UI
   3. The Copy Link to Clipboard button can be used to copy the url to clipboard (This feature uses Adobe Flash. Please enable Flash if it is not enabled in your browser. You can also manually copy the URL manually by selecting the text shown). This URL can be used to connect to that assignment in XData directly from any Learning Management System.
      The consumer key and secret key set by the admin for that LMS instance url will be required by instructor for connecting to XData from Learning Management System.
   4. The options given to the instructor are as follows :
      1. View/Edit question
      2. Result summary by Question
      3. View Scores

View/Edit Questions

• This allows the instructor to add / view / modify the questions that are existing for the assignment with options for editing each question,evaluating each question, generation of dataset, see evaluation results, delete question,etc,. On click of Edit link, the question can be edited – partial mark parameters, marks allotted for the question, default datasets for evaluation for that particular question, question text, SQL answer,etc., can be edited and changes can be saved.
• The instructor can provide more than one query, including some that may be partially correct, for a question by using the Add Another Query button. The instructor needs to choose, using radio buttons, whether the students answer will be evaluated on datasets for all correct queries (for more rigorous correction) or against dataset generated for any correct query (if the question was ambiguous).
• Marks:The maximum marks to be awarded for the question can be specified a Marks scroll bar just above question text.
  If there are multiple queries to one question, the maximum marks awarded for each question may be chosen for partially correct queries using the mark sliders given above each answer box.
• Partial Marking Parameters: There is a View / Set Partial Marking Parameters link above each answer box. This is used for setting weightage for partial marks.
  The partial marking feature in the grading tool assigns marks to student queries based on a heuristic of how close the student is to to the instructor's query. It ensures to ensure a standard and consistent evaluation scheme across all students.
  Partial marking is done only if the student query fails one or more datasets during evaluation. For these student submissions the instructor can click on the 'Details' link in Evaluation Status page to see how the weightage of partial marks were awarded. Based on this, the student query is evaluated and partial marks are awarded if the parameters match.
• If a question uses a schema other than the default schema specified during assignment creation, the instructor can set the schema for the question using a dropdown select.
• If the schema holds more than one data set, all the datasets for the schema will be listed. The instructor can select the default data sets for evaluation for that question alone.
• Once the question and queries are added, the instructor can click the Update Query button to update the question. The assignment page will now show the list of questions and the SQL query answers set by the instructor along with options for editing, deleting and generating data for questions.

Data Generation

• The instructor can generate datasets for each query using Generate Data link. Data generation starts as a background process.
• If data generation fails because of any error an appropriate message is displayed. Logs can be checked in the tomcat server log file (/logs/catalina.out) for details. The instructor may edit the query and try generating datasets again. If datasets are not generated the sample dataset provided with the assignment will be used to grade the student query.
• If the dataset is successfully generated, a Show Dataset link is displayed for the SQL query. The link displays the generated datasets as shown in the figure below. Each dataset has a tag that shows the type of error the dataset is meant to catch.
  The data generated for satisfying the primary and foreign key references are shown on click of "View reference tables" link.

Evaluation

• Once the deadline for the assignment is over, the instructor can grade the student queries if the assignment was not set to interactive mode. The instructor can follow the steps given below to grade student queries.
• The instructor can use Evaluate assignment link above all questions to evaluate all questions in the assignment.
• The instructor can also evaluate each question one by one using Evaluate Questionlink.
• The Evaluate Question link evaluates students queries for the question against the instructor queries. For evaluation, each student query result is compared with the instructor query result on the generated datasets and default data sets.
  The student is awarded full marks if the student query passes all datasets .
  If the student query fails one or more datasets, partial marks are awarded based on a heuristic that determines how close the student query is to the instructor query.
• In case the instructor has provided multiple answers, the student query must pass datasets for all queries or any one query as set by the instructor. If the student query does not pass the test cases, partial marks are evaluated with each instructor query and the highest marks are awarded.
• When the evaluation starts, Evaluating student answers status is shown for the question. This goes away once the evaluation is over. and the status of evaluation can be viewed by the instructor using Evaluation status.
• This Evaluation status link allows the instructor to see whether the student queries were marked correct or not and the marks obtained by the students. If student query is incorrect, instructor can view the failed test cases and student result and expected result on those test case datasets using Test Cases link.
  If a student query is incorrect, the instructor can view details to check the weightage of partial marking parameters. The instructor can also change the total marks obtained by the student from the Mark Detail UI. If the student has not attempted the question, Not Answered status is shown.

Result summary by Question

1. This Result summary by Question link gives the result of evaluation per question. It gives status on how many students answered the questions correctly and how many answered incorrectly and the correct answer for each assignment question.

View Scores

1. After evaluation of all questions in the assignment, the View Scores link can be used to see the total score of each student for that assignment across all questions.
   
2. The table shows all student's name, mail id, marks obtained and a Details link. This link opens a modal to show the answers given by student for each assignment question, marks obtained for each question, status, if incorrect - it shows link for viewing mark details (partial mark details) and failed test cases.

Copyright © 2015 XData