Autograder

Autograder

Overview

This feature enables JuliaBox group admins to create/fetch/grade assignment submitted by the students belonging to the same group. it involves three kinds of actors and communication between these actors forms the autograder workflow. Following are the actors involved:

  1. Admin

  2. Members

  3. JuliaBox Autograder

Admin: This account will have the ability to create assignment notebooks(tutor notebook) using Autograder tab provided to them. All notebooks will have a access to a new cell toolbar called Create Assignment, which will be used to create assignment notebooks. This notebook then will be converted to a student notebook and finally released to members.

Member: Once Admin release an assignment, all the members belonging to Admin's group will have assignments in their home directory inside folder named after their group. Member will now complete the assignment and submit assignment by clicking Submit button provided in the notebook. All the submitted notebooks will be automatically graded by JuliaBox Autograded in parallel. Students can view the result of the submisson by clicking Feedback button provided in the notebook.

JuliaBox Autograder: uses the tutor notebook to grade the student's submission and stores the feedback. Admins will have access to member's submission notebook as well the feedback generated by JuliaBox. JuliaBox will also provide a button for Admins to export the scores as CSV file.

Creating tutor notebook

  1. Click on the Autograder tab

  2. Now you can see the table containing list of all the created assignments if they exist.

  3. Click on Add new assignment link at the bottom.

  4. Give a name to the assignment. Name should not contain any whitespace or should be unique within the course. (Due date and Timezone are not supported yet)

  5. Click on Save

  6. Wait for your Assignment to be listed in the table.

  7. Now Click on the Assignment name which is displayed as a hyper-link. This should open a notebook directory. This will be the assignment directory. All the contents are this directory will be distributed to students.

  8. Upload your assignment notebook.

Assignment notebook will typically contain exercises in Julia code. Exercises supported by JuliaBoxGrader involves implementing function defination, Assigning values to variables only. For eg: Instructor wants students to implement following function

function fact(n)
    if n == 1
        n
    else
        n * fact(n-1)
    end
end

Typically assignment notebook will contain such exercises.

  1. Once the notebook is uploaded, which typically has all the solutions, we need to create a student version, which won't have the solutions in it.

  2. Go to notebook menu, Click on View->Cell Toolbar-> Create Assignment.

  3. Click on the dropdown, which has the values (-, Read Only, Autograded Answer, Autograded Test, Manually Graded Answer). Manually Graded Answer is not supported yet.

  4. Click on the the notebook cell which contains the function to be implemented. Select Autograded Answer in the dropdown.

  5. Add the delimter ### BEGIN SOLUTION and ### END SOLUTION inside the function body all the contents inside the delimiter will be removed and replaced by # Your Code Here from the student's notebook.

  6. Implemented function needs to be tested now. Create a new cell right below the exercise cell.

  7. Select Autograded Tests from the dropdown.

  8. Now you can write assert statements inside the cell to test the function implementation. if this cells block upon run, doesnt throw any exception, We consider that exercise Tests passed. You need to assign point to test cell block which will be awarded to student if the tests pass.

  9. You can remove some or all of tests cases from student's notebook, which will of course be added again automatically while grading. You can do this by placing the tests inside delimiter ### BEGIN HIDDEN TESTS and ### END HIDDEN TESTS.

  10. Once Step 13-17 is done for all the exercises in a notebook. Please save the notebook and return to Autograder dashboard.

  11. Now we need a student version of tutor notebook which has all the delimiter applied to them.

  12. Click on Generate link next to the assignment name to generates student notebook.

  13. Click on Preview to view the student version.

  14. Click Release to distribute the student version to all the students.

Accessing student's submission and scores

  1. You can get all students scorelist as CSV, by clicking Get Students Score button in Autograder dashboard