Newer
Older
Unitgrade is an automatic report and exam evaluation framework that enables instructors to offer automatically evaluated programming assignments.
Unitgrade is build on pythons `unittest` framework so that the tests can be specified in a familiar syntax and will integrate with any modern IDE. What it offers beyond `unittest` is the ability to collect tests in reports (for automatic evaluation) and an easy and 100% safe mechanism for verifying the students results and creating additional, hidden tests. A powerful cache system allows instructors to automatically create test-answers based on a working solution.
- 100% Python `unittest` compatible
- No external configuration files: Just write a `unittest`
- No unnatural limitations: Use any package or framework. If you can `unittest` it, it works.
- Granular security model:
- Students get public `unittests` for easy development of solutions
- Students get a tamper-resistant file to create submissions which are uploaded
- Instructors can automatically verify the students solution using a Docker VM and run hidden tests
- Tests are quick to run and will integrate with your IDE
** Note: This is the development version of unitgrade. If you are a student, please see http://gitlab.compute.dtu.dk/tuhe/unitgrade. **
## At a glance
Unitgrade makes the following assumptions:
- Your code is in python
- Whatever you want to do can be specified as a `unittest`
Although not required, it is recommended you maintain two version of the code:
- A fully-working version (i.e. all tests pass)
- A public version distributed to students (some code removed))
In this example, I will use `snipper` (see http://gitlab.compute.dtu.dk/tuhe/snipper) to synchronize the two versions automatically.
Let's look at an example. You need three files
instructor/cs101/homework.py # This contains the students homework
instructor/cs101/report1.py # This contains the tests
instructor/cs101/deploy.py # A private file to deploy the tests
The homework is just any old python code you would give to the students. For instance:
def reverse_list(mylist): #!f
"""
Given a list 'mylist' returns a list consisting of the same elements in reverse order. E.g.
reverse_list([1,2,3]) should return [3,2,1] (as a list).
"""
return list(reversed(mylist))
def add(a,b): #!f
""" Given two numbers `a` and `b` this function should simply return their sum:
> add(a,b) = a+b """
return a+b
if __name__ == "__main__":
# Problem 1: Write a function which add two numbers
print(f"Your result of 2 + 2 = {add(2,2)}")
print(f"Reversing a small list", reverse_list([2,3,5,7]))
The test consists of individual problems and a report-class. The tests themselves are just regular Unittest (we will see a slightly smarter idea in a moment). For instance:
```python
from homework1 import reverse_list, add
import unittest
class Week1(unittest.TestCase):
def test_add(self):
self.assertEqual(add(2,2), 4)
self.assertEqual(add(-100, 5), -95)
def test_reverse(self):
self.assertEqual(reverse_list([1,2,3]), [3,2,1])
```
A number of tests can be collected into a `Report`, which will allow us to assign points to the tests and use the more advanced features of the framework later. A complete, minimal example:
from unitgrade2.unitgrade2 import Report
from unitgrade2.unitgrade_helpers2 import evaluate_report_student
from homework1 import reverse_list, add
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
class Week1(unittest.TestCase):
def test_add(self):
self.assertEqual(add(2,2), 4)
self.assertEqual(add(-100, 5), -95)
def test_reverse(self):
self.assertEqual(reverse_list([1,2,3]), [3,2,1])
import cs101
class Report1(Report):
title = "CS 101 Report 1"
questions = [(Week1, 10)] # Include a single question for 10 credits.
pack_imports = [cs101]
if __name__ == "__main__":
# Uncomment to simply run everything as a unittest:
# unittest.main(verbosity=2)
evaluate_report_student(Report1())
```
## Deployment
The above is all you need if you simply want to use the framework as a self-check: Students can run the code and see how well they did.
In order to begin using the framework for evaluation we need to create a bit more structure. We do that by deploying the report class as follows:
```python
from report1 import Report1
from unitgrade_private2.hidden_create_files import setup_grade_file_report
from snipper import snip_dir
import shutil
if __name__ == "__main__":
setup_grade_file_report(Report1, minify=False, obfuscate=False, execute=False)
# Deploy the files using snipper: https://gitlab.compute.dtu.dk/tuhe/snipper
snip_dir.snip_dir(source_dir="../cs101", dest_dir="../../students/cs101", clean_destination_dir=True, exclude=['__pycache__', '*.token', 'deploy.py'])
- The first line creates the `report1_grade.py` script and any additional data files needed by the tests (none in this case)
- The second line set up the students directory (remember, we have included the solutions!) and remove the students solutions. You can check the results in the students folder.