updates

README.md

 # Unitgrade-devel
 **Note: This is the development version of unitgrade. If you are a student, please see http://gitlab.compute.dtu.dk/tuhe/unitgrade.**
 
-Unitgrade is an automatic report and exam evaluation framework that enables instructors to offer automatically evaluated programming assignments. It is currently used in 02465: https://gitlab.gbar.dtu.dk/02465material/02465students
-
-## Why not (alternative online automatic evaluation framework)?
-I think the most important thing to ask from an automatic evaluation framework is that it helps the students become better programmers. 
-Automatic evaluation frameworks are fundamentally about testing code. We know testing works, but we also know that not all ways of doing tests are equally good. Online testing has several clear disadvantages:
-
- - You need to upload code and often press to see output
- - No debugger
- - Often encourage/requires coding in the browser, i.e. without IDE support
- - They are often constraining, meaning problems must be phrased in a less than optimal way to suit the evaluation framework
-
-For these reasons, we would never accept an online evaluation tool as a testing tool -- so why should we ask that of students?
-
-Unitgrade is different because it is build on pythons `unittest` framework. This means tests are specified simply as unittests and will integrate with any modern IDE. What it offers beyond `unittest` is the ability to collect tests in reports (for automatic evaluation) and an easy and 100% safe mechanism for verifying the students results and creating additional, hidden tests. A cache system allows instructors to automatically create test-answers based on a working solution. 
+Unitgrade is an automatic report and exam evaluation framework that enables instructors to offer automatically evaluated programming assignments. 
+ Unitgrade is build on pythons `unittest` framework so that the tests can be specified in a familiar syntax and will integrate with any modern IDE. What it offers beyond `unittest` is the ability to collect tests in reports (for automatic evaluation) and an easy and 100% safe mechanism for verifying the students results and creating additional, hidden tests. A powerful cache system allows instructors to automatically create test-answers based on a working solution. 
 
  - 100% Python `unittest` compatible
  - No configuration files
@@ -35,6 +23,14 @@ pip install unitgrade-devel
 ```
 This will install `unitgrade-devel` (this package) and all dependencies to get you started.
 
+## Overview
+![alt text|small](https://gitlab.compute.dtu.dk/tuhe/unitgrade_private/-/raw/master/docs/images/process.png)
+
+The figure shows an overview of the workflow. 
+ - You write exercises and a suite of unittests. 
+ - They are then compiled to a version of the exercises without solutions. 
+ - The students solve the exercises using the tests and when they are happy, they run an automatically generated `_grade.py`-script to produce a `.token`-file with the number of points they obtain. This file is then uploaded for further verification/evaluation.
+
 ### Videos
 Videos where I try to talk and code my way through the examples can be found on youtube:
 
@@ -46,7 +42,7 @@ Videos where I try to talk and code my way through the examples can be found on
   - Autolab: https://youtu.be/h5mqR8iNMwM
 
 # Instructions and examples of use
-The examples can be found in the `/examples` directory: https://gitlab.compute.dtu.dk/tuhe/unitgrade_private/-/tree/master/examples . 
+The examples can be found in the `/examples` directory: https://gitlab.compute.dtu.dk/tuhe/unitgrade_private/-/tree/master/examples
 
 ## A simple example
 Unitgrade makes the following assumptions:
@@ -240,7 +236,7 @@ When this is run, the titles are shown as follows:
 | | | |_ __  _| |_| |  \/_ __ __ _  __| | ___ 
 | | | | '_ \| | __| | __| '__/ _` |/ _` |/ _ \
 | |_| | | | | | |_| |_\ \ | | (_| | (_| |  __/
- \___/|_| |_|_|\__|\____/_|  \__,_|\__,_|\___| v0.1.17, started: 21/09/2021 11:57:05
+ \___/|_| |_|_|\__|\____/_|  \__,_|\__,_|\___| v0.1.17, started: 19/05/2022 15:14:09
 
 CS 102 Report 2 
 Question 1: Week1                                                                                                       
@@ -254,7 +250,7 @@ Question 2: The same problem as before with nicer titles
  * q2.2) Checking if reverse_list([1, 2, 3]) = [3, 2, 1]............................................................PASS
  * q2)   Total...................................................................................................... 6/6
  
-Total points at 11:57:05 (0 minutes, 0 seconds)....................................................................16/16
+Total points at 15:14:09 (0 minutes, 0 seconds)....................................................................16/16
 
 Including files in upload...
  * cs102
@@ -270,7 +266,7 @@ What happens behind the scenes when we set `self.title` is that the result is pr
 ### Caching computations
 The `@cache`-decorator offers a direct ways to compute the correct result on an instructors computer and submit it to the student. For instance:
 ```python
-# example_framework/instructor/cs102/report2_test.py
+# example_framework/instructor/cs102/report2.py
 class Question2(UTestCase): 
     @cache
     def my_reversal(self, ls):
@@ -507,7 +503,7 @@ The code for the example can be found in `examples/autolab_example`. It consists
 
 Concretely, the following code will download and build the image (note this code must be run on the same machine that you have installed Autolab on)
 ```python
-# autolab_example/deploy_autolab.py
+# autolab_token_upload/deploy_autolab.py
     # Step 1: Download and compile docker grading image. You only need to do this once.  
     download_docker_images("./docker") # Download docker images from gitlab (only do this once.
     dockerfile = f"./docker/docker_tango_python/Dockerfile"
@@ -517,7 +513,7 @@ Concretely, the following code will download and build the image (note this code
 Next, simply call the framework to compile any `_grade.py`-file into an Autolab-compatible `.tar` file that can be imported from the web interface. The script requires you to specify 
 both the instructor-directory and the directory with the files the student have been handed out (i.e., the same file-system format we have seen earlier). 
 ```python
-# autolab_example/deploy_autolab.py
+# autolab_token_upload/deploy_autolab.py
     # Step 2: Create the cs102.tar file from the grade scripts. 
     instructor_base = f"../example_framework/instructor"
     student_base = f"../example_framework/students"
@@ -552,9 +548,9 @@ and TAs can choose to annotate the students code directly in Autolab -- we are h
 # Citing
 ```bibtex
 @online{unitgrade_devel,
-	title={Unitgrade-devel (0.1.35): \texttt{pip install unitgrade-devel}},
+	title={Unitgrade-devel (0.1.39): \texttt{pip install unitgrade-devel}},
 	url={https://lab.compute.dtu.dk/tuhe/unitgrade_private},
-	urldate = {2022-05-19}, 
+	urldate = {2022-06-15}, 
 	month={9},
 	publisher={Technical University of Denmark (DTU)},
 	author={Tue Herlau},

docs/README.jinja.md

@@ -23,6 +23,14 @@ pip install unitgrade-devel
 ```
 This will install `unitgrade-devel` (this package) and all dependencies to get you started.
 
+## Overview
+![alt text|small]({{resources}}/docs/images/process.png)
+
+The figure shows an overview of the workflow. 
+ - You write exercises and a suite of unittests. 
+ - They are then compiled to a version of the exercises without solutions. 
+ - The students solve the exercises using the tests and when they are happy, they run an automatically generated `_grade.py`-script to produce a `.token`-file with the number of points they obtain. This file is then uploaded for further verification/evaluation.
+
 ### Videos
 Videos where I try to talk and code my way through the examples can be found on youtube:
 

docs/snips/deploy.txt

@@ -3,7 +3,7 @@
 | | | |_ __  _| |_| |  \/_ __ __ _  __| | ___ 
 | | | | '_ \| | __| | __| '__/ _` |/ _` |/ _ \
 | |_| | | | | | |_| |_\ \ | | (_| | (_| |  __/
- \___/|_| |_|_|\__|\____/_|  \__,_|\__,_|\___| v0.1.17, started: 19/05/2022 15:14:09
+ \___/|_| |_|_|\__|\____/_|  \__,_|\__,_|\___| v0.1.22, started: 15/06/2022 09:18:15
 
 CS 102 Report 2 
 Question 1: Week1                                                                                                       
@@ -17,9 +17,10 @@ Question 2: The same problem as before with nicer titles
  * q2.2) Checking if reverse_list([1, 2, 3]) = [3, 2, 1]............................................................PASS
  * q2)   Total...................................................................................................... 6/6
  
-Total points at 15:14:09 (0 minutes, 0 seconds)....................................................................16/16
+Total points at 09:18:16 (0 minutes, 0 seconds)....................................................................16/16
 
 Including files in upload...
+path.: _NamespacePath(['C:\\Users\\tuhe\\Documents\\unitgrade_private\\examples\\example_framework\\instructor\\cs102', 'C:\\Users\\tuhe\\Documents\\unitgrade_private\\examples\\example_framework\\instructor\\cs102'])
  * cs102
 > Testing token file integrity...
 Done!

docs/snips/deploy_autolab_a.py

-# autolab_example/deploy_autolab.py
+# autolab_token_upload/deploy_autolab.py
     # Step 1: Download and compile docker grading image. You only need to do this once.  
     download_docker_images("./docker") # Download docker images from gitlab (only do this once.
     dockerfile = f"./docker/docker_tango_python/Dockerfile"

docs/snips/deploy_autolab_b.py

-# autolab_example/deploy_autolab.py
+# autolab_token_upload/deploy_autolab.py
     # Step 2: Create the cs102.tar file from the grade scripts. 
     instructor_base = f"../example_framework/instructor"
     student_base = f"../example_framework/students"

docs/snips/report2.py

-# example_framework/instructor/cs102/report2_test.py
+# example_framework/instructor/cs102/report2.py
 from unitgrade import UTestCase, cache  
 
 class Week1(UTestCase):

docs/snips/report2_b.py

-# example_framework/instructor/cs102/report2_test.py
+# example_framework/instructor/cs102/report2.py
 class Week1Titles(UTestCase): 
     """ The same problem as before with nicer titles """
     def test_add(self):

docs/snips/report2_c.py

-# example_framework/instructor/cs102/report2_test.py
+# example_framework/instructor/cs102/report2.py
 class Question2(UTestCase): 
     @cache
     def my_reversal(self, ls):

docs/snips/report2_test.py

+# autolab_example_py_upload/instructor/cs102_autolab/report2_test.py
+from unitgrade import UTestCase, cache  
+import homework1
+import unittest
+
+
+class Week1(UTestCase):
+    def test_add(self):
+        self.assertEqualC(add(2,2))
+        self.assertEqualC(add(-100, 5))
+
+    def test_reverse(self):
+        self.assertEqualC(reverse_list([1, 2, 3])) 
\ No newline at end of file

docs/snips/report2_test_b.py

+# autolab_example_py_upload/instructor/cs102_autolab/report2_test.py
+class Week1Titles(UTestCase): 
+    """ The same problem as before with nicer titles """
+    def test_add(self):
+        """ Test the addition method add(a,b) """
+        self.assertEqualC(add(2,2))
+        print("output generated by test")
+        self.assertEqualC(add(-100, 5))
+        # self.assertEqual(2,3, msg="This test automatically fails.")
+
+    def test_reverse(self):
+        ls = [1, 2, 3]
+        reverse = reverse_list(ls)
+        self.assertEqualC(reverse)
+        # Although the title is set after the test potentially fails, it will *always* show correctly for the student.
+        self.title = f"Checking if reverse_list({ls}) = {reverse}"  # Programmatically set the title 
\ No newline at end of file

docs/snips/report2_test_c.py

+# autolab_example_py_upload/instructor/cs102_autolab/report2_test.py
+class Question2(UTestCase): 
+    @cache
+    def my_reversal(self, ls):
+        # The '@cache' decorator ensures the function is not run on the *students* computer
+        # Instead the code is run on the teachers computer and the result is passed on with the
+        # other pre-computed results -- i.e. this function will run regardless of how the student happens to have
+        # implemented reverse_list.
+        return reverse_list(ls)
+
+    def test_reverse_tricky(self):
+        ls = (2,4,8)
+        ls2 = self.my_reversal(tuple(ls))                   # This will always produce the right result, [8, 4, 2]
+        print("The correct answer is supposed to be", ls2)  # Show students the correct answer
+        self.assertEqualC(reverse_list(ls))                 # This will actually test the students code.
+        return "Buy world!"                                 # This value will be stored in the .token file  
\ No newline at end of file

docs/unitgrade_devel.bib

 @online{unitgrade_devel,
-	title={Unitgrade-devel (0.1.35): \texttt{pip install unitgrade-devel}},
+	title={Unitgrade-devel (0.1.39): \texttt{pip install unitgrade-devel}},
 	url={https://lab.compute.dtu.dk/tuhe/unitgrade_private},
-	urldate = {2022-05-19}, 
+	urldate = {2022-06-15}, 
 	month={9},
 	publisher={Technical University of Denmark (DTU)},
 	author={Tue Herlau},

examples/02105/students/week2/stones_tests.py

@@ -6,10 +6,7 @@ from stones import maximum_stones
 # A fancy helper function to generate nicer-looking titles.
 def trlist(x):
     s = str(list(x))
-    if len(s) > 30:
-        s = s[:30] + "...]"
-    return s
-
+    return s[:30] + "...]" if len(s) > 30 else s
 
 class Stones(UTestCase):
     """ Test of the Stones function """
@@ -21,7 +18,6 @@ class Stones(UTestCase):
     def test_basecase(self):
         """ Test the stones-example given in the homework """
         N = maximum_stones(15, [2, 5, 3, 1, 8, 4, 5, 7])
-        print("How many stones could we pick up?", N)
         self.assertEqual(N, 5) # Test that we can collect 5 stones.
 
     def test_stones1(self):

examples/example_autolab_deploy/autolab_courses.py

@@ -19,11 +19,11 @@ if __name__ == "__main__":
     snip_dir("../02105/instructor/week2", "../02105/students/week2", clean_destination_dir=True, exclude=['*.token', 'deploy*.py', '*_grade.py', 'tmp', '*.tar'])
     snip_dir("../02631/instructor/week5", "../02631/students/week5", clean_destination_dir=True, exclude=['*.token', 'deploy*.py', '*_grade.py', 'tmp', '*.tar'])
 
-    # Step 1: Download and compile docker grading image. You only need to do this once.  #!s=a
+    # Step 1: Download and compile docker grading image. You only need to do this once.
     download_docker_images("./docker") # Download docker images from gitlab (only do this once).
     dockerfile = f"./docker/docker_tango_python/Dockerfile"
     autograde_image = 'tango_python_tue'  # Tag given to the image in case you have multiple images.
-    compile_docker_image(Dockerfile=dockerfile, tag=autograde_image, no_cache=False)  # Compile docker image. #!s
+    compile_docker_image(Dockerfile=dockerfile, tag=autograde_image, no_cache=False)  # Compile docker image.
 
     from unitgrade_private import load_token
     data, _ = load_token("../02105/instructor/week2/StoneReport_handin_10_of_10.token")
@@ -31,7 +31,7 @@ if __name__ == "__main__":
     format_autolab_json(data, indent=2)
 
 
-    # Step 2: Create the cs102.tar file from the grade scripts. #!s=b
+    # Step 2: Create the cs102.tar file from the grade scripts. 
     instructor_base = f"../02105/instructor/week2"
     student_base = "../02105/students/week2"