Home
beyondgrader.com Logo

Better Feedback. Better Students.
Better Feedback. Better Students.

BeyondGrader evaluates multiple facets of code quality—style and formatting, cyclomatic complexity, execution efficiency, source line counts, and dead code detection, with more coming soon. We can help you teach students to create code that is not just correct, but also beautiful, efficient, and idiomatic.

Let’s see how!

Example Problem
Example Problem

The examples below use the following example problem. Toward the end, we’ll show how easy it is to write this question!

Feel free to give it a try! We won’t repeat the instructions below.

Array Squares Sorted

Geoffrey Challen // 2022.3.0

Create a class SquaredSorter that provides a single class method named maxOfSortedSquares. maxOfSortedSquares accepts an array of ints. If the passed array is null or empty, throw an IllegalArgumentException.

Otherwise, proceed as follows. First, square every value of the array. Next, sort the array in ascending order. Finally, return the maximum of the squared, sorted array. Modify the array in place, and do not create a copy. Sort the array using a helper method from java.util.Arrays.

// Write your solution here...

Correctness
Correctness

Before we discuss code quality, we don’t want you to miss the fact that BeyondGrader is also a fantastic autograder. We have designed the feedback to be clear, concise, and useful to beginning programmers.

Check out the walkthrough below to find out more!

public class SquaredSorter {
public static int maxOfSortedSquares(final int[] values) {
for (int i = 0; i < values.length; i++) {
values[i] = values[i] * values[i];
}
return values[values.length - 1];
}
}

Style and Formatting
Style and Formatting

What use is correct code if it’s unreadable? In our experience, teaching students to naturally write code to a style guide is extremely valuable. Certain aspects of style—such as whitespace and indentation—have a huge impact on intelligibility and a student’s ability to understand their own code. And, when all your students are creating consistently-formatted code, it’s much easier for staff to read their code when helping them.

BeyondGrader implements the Sun Java style guide using checkstyle. This is the most commonly-used set of formatting rules for Java, and a superset of Google’s guidelines. For Kotlin we use ktlint and it’s default settings. checkstyle provides helpful error messages, and students quickly become used to its conventions.

The walkthrough below discusses various aspects of style and formatting examined by BeyondGrader.

import java.util.Arrays;
public class SquaredSorter {
static public int maxOfSortedSquares (int[] values)
{
if (values == null || values.length==0) {
throw
new IllegalArgumentException();
}
for (
int i = 0; i < values.length; i++
) {
values[i] = values[i]*values[i];
}
Arrays.sort(values); return values[values.length - 1];
}}

Cyclomatic Complexity
Cyclomatic Complexity

Cyclomatic complexity measures how many different ways control can flow through a program. In our experience, extra code paths in student code typically reflect a misunderstanding about how to correctly approach the problem. BeyondGrader will flag submissions that add too many code paths when compared to the reference solution.

Let’s examine an example of overly-complicated code in the walkthrough below.

import java.util.Arrays;
public class SquaredSorter {
public static int maxOfSortedSquares(final int[] values) {
if (values == null || values.length == 0) {
throw new IllegalArgumentException();
}
for (int i = 0; i < values.length; i++) {
if (values[i] > 0) {
values[i] = values[i] * values[i];
} else if (values[i] == 0) {
values[i] = 0;
} else {
values[i] = (-1 * values[i]) * (-1 * values[i]);
}
}
Arrays.sort(values);
return values[values.length - 1];
}
}

Execution Efficiency
Execution Efficiency

Sometimes it’s appropriate to stress performance when working with students. Sometimes not. Regardless, extreme inefficiency almost always points to deeper problems with a submission.

BeyondGrader uses bytecode instrumentation to measure the number of lines executed by a submission. This is a stable metric—unlike wall clock time—and lets you establish tight efficiency bounds when appropriate(1). When the execution line count exceeds the limit, students are prompted to improve their code.

In the walkthrough below, we examine an example of how evaluating this aspect of code quality produces better submissions.

import java.util.Arrays;
public class SquaredSorter {
public static int maxOfSortedSquares(final int[] values) {
if (values == null || values.length == 0) {
throw new IllegalArgumentException();
}
for (int i = 0; i < values.length; i++) {
for (int j = 0; j < 2; j++) {
int value = values[i] * values[i];
}
values[i] = values[i] * values[i];
}
Arrays.sort(values);
return values[values.length - 1];
}
}

Source Line Counts
Source Line Counts

Setting reasonable limits on the number of lines in a submission can flag common programming errors and student misconceptions(2). Note that this is possible in BeyondGrader given brace-based languages like Java and Kotlin, due to the fact that our linting rules prevent students from collapsing everything down to a single line.

The example below exhibits a common student programming misconception, and the interactive walkthrough discusses how we can detect it using source line counts.

import java.util.Arrays;
public class SquaredSorter {
public static int maxOfSortedSquares(final int[] values) {
if (values == null || values.length == 0) {
IllegalArgumentException e = new IllegalArgumentException();
throw e;
}
for (int i = 0; i < values.length; i++) {
int value = values[i];
int valueSquared = value * value;
values[i] = valueSquared;
}
Arrays.sort(values);
int max = values[values.length - 1];
return max;
}
}

Dead Code Detection
Dead Code Detection

Portions of student submissions not covered during testing are very likely to indicate a mistake or a misunderstanding of how to solve the problem. So, while BeyondGrader by default will accept submissions that have cyclomatic and runtime complexity greater than the reference solution, it always flags any instance of unexecuted code.

import java.util.Arrays;
public class SquaredSorter {
public static int maxOfSortedSquares(final int[] values) {
if (values == null || values.length == 0) {
throw new IllegalArgumentException();
}
for (int i = 0; i < values.length; i++) {
values[i] = values[i] * values[i];
}
Arrays.sort(values);
if (values.length == 0) {
return 0;
}
return values[values.length - 1];
}
}

Memory Allocation Measurement
Memory Allocation Measurement

Regardless of whether you stress performance in your course, student submissions that allocate way too much memory can probably be improved. BeyondGrader combines bytecode rewriting with a custom Java agent to track and report memory allocation totals by student submissions, and provide feedback when memory usage has gotten out of hand.

import java.util.Arrays;
public class SquaredSorter {
public static int maxOfSortedSquares(final int[] values) {
if (values == null || values.length == 0) {
throw new IllegalArgumentException();
}
for (int i = 0; i < values.length; i++) {
values[i] = values[i] * values[i];
}
int[] valuesCopy = Arrays.copyOf(values, values.length);
Arrays.sort(values);
return values[values.length - 1];
}
}

Coming Soon
Coming Soon

We have several new exciting code quality features planned for upcoming releases:

Problem Authoring
Problem Authoring

BeyondGrader goes beyond just providing code quality feedback. It also provides a framework for question development that makes writing accurate new problems fast, easy, and—maybe, just maybe—even fun! Notably, there is no need and no way to write test cases!

In the video we demonstrate the process of writing the question we’ve been experimenting with above. Check it out!