Unitgrade-private

Do not distribute this repository, or files from this repository, to students

This repository contains the secret parts of the unitgrade framework.

What it looks like to a student

Homework is broken down into reports. A report is a collection of questions which are individually scored, and each question may in turn involve multiple tests. Each report is therefore given an overall score based on a weighted average of how many tests are passed.

In practice, a report consist of an ordinary python file which they simply run. It looks like this:

python cs101report1.py

The file cs101report1.py is just an ordinary, non-obfuscated file which they can navigate and debug using a debugger. The file may contain the homework, or it may call functions the students have written. Running the file creates console output which tells the students their current score for each test:

Starting on 02/12/2020 14:57:06
Evaluating CS 101 Report 1

Question 1: Reversal of list
================================================================================
*** q1.1) ListReversalItem..................................................PASS
*** q1.2) ListReversalWordsItem.............................................PASS
*** Question q1............................................................. 5/5

Question 2: Linear regression and Boston dataset
================================================================================
*** q2.1) CoefficientsItem..................................................PASS
*** q2.2) RMSEItem..........................................................PASS
*** Question q2........................................................... 13/13

Finished at 14:57:06
Provisional evaluation
-----------  -----
Question q1  5/5
Question q2  13/13
Total        18/18
-----------  -----

Note your results have not yet been registered.
To register your results, please run the file:
>>> cs101report1_grade.py
In the same manner as you ran this file.

Once students are happy with the result, they run an alternative, not-easy-to-tamper-with script called cs101report1_grade.py:

python report1_grade.py

This runs the same tests, and generates a file cs101report1.token which they upload to campusnet. This file contains the results of the report evaluation, the script output, and so on.

How to develop tests

The framework is build around the build-in unittest framework in python. Using the framework therefore also familiarizes students with the use of automatic testing.

A unittest normally consist of three things:

• The result of the users code,
• The expected result,
• A comparison operation of the two which may either fail or succeed.

The comparisons are build on top of pythons unittest framework to obtain a wide variety of well-documented comparisons, and it is easy to write your own.

To get the expected result, one option (which is certainly possible) is to specify it yourself, however the recommended (and much easier option) is to maintain a working branch of the code with all the funtionality the students must implement and then automatically compute the expected output. In other words, we save the step of specifying the expected result, which also has the benefit we don't need to update expected output when the scripts change. The video contains a full working example of how a test is developed.