Automated exercise checking at Machon Lev
Pull requests in, tests and drafted feedback out, the lecturer decides the grade.
Illustration: an exercise submissions queue — pull requests come in, tests and drafted feedback go out, and a submission whose test timed out is flagged for the lecturer, who decides the grade.
The course
A Linux/C++ programming course (with some Python) at Machon Lev, the Jerusalem College of Technology. In the spring 2026 semester I worked weekly with the course staff on a system that checks exercises and drafts feedback, so their time goes to teaching rather than to running each submission by hand.
The problem
Checking every submission by hand is slow, and the feedback a student gets depends on who checked and when. Code that passed on one laptop failed on another. The staff wanted checking that is fast, consistent and the same for everyone, without giving up their own judgement on the grade.
What was built
- Submissions as pull requests
- Each exercise is a GitHub pull request via a GitHub App, so every submission has a history and a place for comments.
- One environment for everyone
- Tests run in Docker containers built the same way every time, so "works on my machine" stops being an argument.
- Tests, linting and learning links
- Functional tests per exercise and pylint checks, with feedback that links to the learning material behind each finding.
- Feedback drafted with a rubric
- AI drafts feedback against the course rubric. The lecturer reads it, edits it and decides the grade.
- Reports and score views
- A data model linking submissions, pull requests and exercises feeds grading reports, score views and a web view for loading submissions.
- Who submitted, who went quiet
- Submission tracking shows who has handed in and who hasn't, and sends automated reminders to the quiet ones.
My part
I built the system and worked with the course staff week by week while it ran in the live course: submission flow, containers, tests, the data model, reports and the AI feedback. The configuration for checking each exercise was recorded and handed to the staff, so they can run it without me.
Supported outcomes
- Every submission runs in the same environment, so results are comparable
- Students get feedback that is consistent and points to what to read
- The lecturer's time goes to judgement, not to running code
- The staff can configure checks for new exercises themselves
Tools
GitHub, GitHub Apps, Docker, Python, pylint, C/C++, AI feedback with a rubric