BeyondGrader evaluates multiple facets of code quality—style and formatting, cyclomatic complexity, execution efficiency, source line counts, memory usage, and dead code detection, with more coming soon. We can help you teach students to create code that is not just correct, but also beautiful, efficient, and idiomatic.
Let’s see how!
The examples below use the following example problem. Toward the end, we’ll show how easy it is to write this question!
Feel free to give it a try! We won’t repeat the instructions below.
Array Squares Sorted
Create a class
SquaredSorter that provides a single class method named
maxOfSortedSquares accepts an array of
If the passed array is
null or empty, throw an
Otherwise, proceed as follows.
First, square every value of the array.
Next, sort the array in ascending order.
Finally, return the maximum of the squared, sorted array.
Modify the array in place, and do not create a copy.
Sort the array using a helper method from
Before we discuss code quality, we don’t want you to miss the fact that BeyondGrader is also a fantastic autograder. We have designed the feedback to be clear, concise, and useful to beginning programmers.
Check out the walkthrough below to find out more!
What use is correct code if it’s unreadable? In our experience, teaching students to naturally write code to a style guide is extremely valuable. Certain aspects of style—such as whitespace and indentation—have a huge impact on intelligibility and a student’s ability to understand their own code. And, when all your students are creating consistently-formatted code, it’s much easier for staff to read their code when helping them.
BeyondGrader implements the Sun Java style guide using
checkstyle. This is the most commonly-used set of formatting rules for Java, and
a superset of Google’s guidelines. For Kotlin we use
ktlint and it’s default settings.
checkstyle provides helpful
error messages, and students quickly become used to its conventions.
The walkthrough below discusses various aspects of style and formatting examined by BeyondGrader.
Cyclomatic complexity measures how many different ways control can flow through a program. In our experience, extra code paths in student code typically reflect a misunderstanding about how to correctly approach the problem. BeyondGrader will flag submissions that add too many code paths when compared to the reference solution.
Let’s examine an example of overly-complicated code in the walkthrough below.
Sometimes it’s appropriate to stress performance when working with students. Sometimes not. Regardless, extreme inefficiency almost always points to deeper problems with a submission.
BeyondGrader uses bytecode instrumentation to measure the number of lines executed by a submission. This is a stable metric—unlike wall clock time—and lets you establish tight efficiency bounds when appropriate(1). When the execution line count exceeds the limit, students are prompted to improve their code.
In the walkthrough below, we examine an example of how evaluating this aspect of code quality produces better submissions.
Setting reasonable limits on the number of lines in a submission can flag common programming errors and student misconceptions(2). Note that this is possible in BeyondGrader given brace-based languages like Java and Kotlin, due to the fact that our linting rules prevent students from collapsing everything down to a single line.
The example below exhibits a common student programming misconception, and the interactive walkthrough discusses how we can detect it using source line counts.
Portions of student submissions not covered during testing are very likely to indicate a mistake or a misunderstanding of how to solve the problem. So, while BeyondGrader by default will accept submissions that have cyclomatic and runtime complexity greater than the reference solution, it always flags any instance of unexecuted code.
Regardless of whether you stress performance in your course, student submissions that allocate way too much memory can probably be improved. BeyondGrader combines bytecode rewriting with a custom Java agent to track and report memory allocation totals by student submissions, and provide feedback when memory usage has gotten out of hand.
We have several new exciting code quality features planned for upcoming releases:
- Syntactic feature comparison: Should the submission use a loop, or maybe not if recursion was called for? Is the student declaring a bunch of extra variables? We’ll compare the language features used by the student submission to the reference solution to guide students towards more idiomatic solutions.
- Complexity categorization: If an O(n) algorithm was expected, and the student submits something that’s O(n2), we should probably talk.
- Variable name analysis: Tired of seeing
baz? So are we! We’ll help verify that student submissions are using meaningful variable names appropriate to the problem they are solving.
BeyondGrader goes beyond just providing code quality feedback. It also provides a framework for question development that makes writing accurate new problems fast, easy, and—maybe, just maybe—even fun! Notably, there is no need and no way to write test cases!
In the video we demonstrate the process of writing the question we’ve been experimenting with above. Check it out!