Competency-Based Grading

Abstract

We propose a browser extension which will allow for Gradescope to more expediently be used for competency-based grading. Competency-based grading is an approach to academic grading which ties learning outcomes to specific competencies. Competencies can be evaluated in a variety of assignments and later grades on a specific competency supersede earlier grades automatically, allowing for students to be rewarded for demonstrating growth. Current native support for competency-based grading in enterprise-level learning management software (LMS) is slim, and any LMS extensions would need to be specifically approved by technical administrators, adding a barrier to adoption. Thus, we propose to use a browser extension which scrapes the Gradescope page for data and uses existing site functionality to run competency-based grading as an instructor. As the extension doesn’t transmit any data to remote servers, but instead only acts as an intermediate layer between the user and Gradescope, the extension could be used in a class without violating FERPA.

Goals

Core

These are our core goals for the project, roughly ordered by importance:

1) Develop a tool which allows for the management of competency-based grades and assignments when used with Gradescope. The tool should support basic functions such as posting assignments and checking grades.

2) Develop a tool which is accessible, which can be used with a variety of ATs, and which complies with WCAG guidelines to at least an AA level.

3) Develop a tool which can be easily used with little to no instruction. The tool should not have noticeable delays or inconsistencies. Develop a tool which can be used in an enterprise educational setting without violating FERPA or otherwise requiring an audit. Functionally, this means no transmitting of data to remote servers.

Stretch

Features we aspire to implement but likely won’t have the time for:

1) Test the tool with users outside of our project group to evaluate usability for end users.

2) Develop a tool which is visually appealing and stylistic.

3) Develop a tool which conforms to the higher level of WCAG specification, AAA.

4) Develop a tool which works on a variety of modern browsers.

5) Develop a tool which is resilient against minor changes in the underlying Gradescope page structure.

Requirements

Basic Functions

In order to meet the first core goal of the project, we need to develop an extension which operates on top of the underlying Gradescope page and supports basic competency-based grading functionality. Some items that our interface must support include:

  • The ability for instructors to set up their class through the interface. This should allow instructors to add competencies to the class. Each competency consists of, at minimum, a title and a description.

  • The ability for instructors to post assignments through the interface. Assignments should consist of, at minimum, a title, a description, and a section to select competencies which are included in that assignment. There must also be some system of associating competencies to specific portions of the assignment.

  • The ability for instructors to grade assignments through the interface, scoring students on each included competency, and writing comments where relevant.

  • The ability for students to view their grades through the interface. This must support both viewing the student’s calculated competency scores, as well as a conversion from the competency-based scale to the standard four-point scale.

  • The ability for students to fill out assignments through the interface, submitting text or other media. The interface must clearly indicate which parts or aspects of the assignment are relevant to which competencies.

WCAG Conformance

In order to meet our second core goal, WCAG AA conformance, our tool will:

  • Support keyboard-only access Label all forms clearly

  • Be compatible with AT such as screen readers and switch control

  • Provide information on how to adjust the website to the user’s need e.g. size or color

  • Have a minimum color contrast ratio of 4.5 : 1 between text and background

  • Organize all content clearly with heading levels

  • Have consistent navigation icons throughout, including always having the ability to navigate to the previous page

Usability

In order to meet our third goal, to be usable with little to no instruction, our tool will:

  • Include an about page with instructions on how to use the tool

  • Use standard symbology for navigation e.g. back arrow for back navigation, hamburger menu symbol for menu

  • Include tooltips on buttons

  • Surface all menus and navigation in expected places such as at the top or the bottom of the page

FERPA Compliance

In order to meet our fourth goal, FERPA compliance, our tool will:

  • Never send grades to a remote server as part of its operation.

    • The tool can achieve this by reading grades from the Canvas and Gradescope pages each time it is activated in the user’s browser, so that grades never need to be stored by our tool across sessions.

Potential Changes

Our tool is still very much in its infancy, and much is subject to change. Generally our core goals are exempt from this, but some considerations:

  • We haven’t yet decided exactly how we want to link competencies to assignment minutiae. For an open-ended approach, we could make all assignments linked to a rubric for each relevant competency, which would allow for assignments which blended multiple competencies. Alternatively, we could separate assignments by competency, which would streamline the creation of certain assignments (as you wouldn’t need to enter ‘got [n]+ questions correct’ in every rubric entry for a multiple choice quiz, for example), but would also somewhat constrain the format of assignments.

  • We may want to leave competency descriptions as a simple blank text box to allow for maximal instructor freedom. However, it may prove more beneficial for users if we included certain prompts with their own inputs to be used in describing the competency, such as “Summary” or “How to Demonstrate”.

  • For students filling out assignments, the underlying Gradescope interface may prove sufficient. There’s no specific functionality that our assignment submission interface would require that Gradescope doesn’t support— while we do need some indicator of what parts of the assignment are relevant to what competencies, this can be accomplished via text. Using the Gradescope submission interface would streamline development in that it would prevent us from needing to implement a file-upload functionality and similar components for our interface. However, navigating between our tool and the gradescope interface may be confusing, and present accessibility barriers. So, we might need to ease the transition or avoid this altogether and implement our own submission interface from scratch. A potential way to ease the transition is to have an ‘assignments’ page in our interface which prompts the user to follow provided links to the correct Gradescope page.

Implementation and Validation

Extensions

This section details potential extensions to the project, any and all of which could be either folded into our plan if important enough or added later, past the scope of this class.

  • We could add the ability for instructors to add arbitrary points to their classes, which could factor into the grading scheme of the class. This would also require these points to be factored into the grade view on both the student side and the teacher side.

  • We could add the ability for instructors to weigh competencies differently. It may prove convenient for certain course settings to have some competencies contribute to a student’s grade more than others.

  • Create an example class and walkthrough video to showcase how to use the tool. This would improve the outcome of our third goal to make the tool highly approachable.

Alternatives

Our approach is not absolute, and while developing this project we’ve considered some alternative formats:

  • A Gradescope plugin. We avoided this because Gradescope extensions seemed to be developed in-house for certain LMSs, and Gradescope is closed source, meaning an officially approved plugin would be very difficult to develop.

  • A Canvas plugin. Canvas is the LMS used by the University of Washington and has support for third-party plugins. However, explicit administrator approval is needed to add any Canvas plugin to a course, and there are FERPA concerns when handling student data such as grades. While Canvas is open source, we would still need to figure out how to use its plugin system, and how to interact with Gradescope so that grades are stored in some format in Gradescope but still retrieved in a competency-based format through the plugin interface. Ultimately the FERPA concerns combined with the adoption cost meant we decided against this option.

Disability Justice Principles

RECOGNIZING WHOLENESS

The DJ principle recognizing wholeness is about the inherent worth people have outside of all the systems we have created that claim to measure it. Sins Invalid specifically mentions “commodity relations and capitalist notions of productivity”. A traditional 0-100 grading system very much falls into that trap. It allows for little recognition of a student’s ability beyond their performance at very rigid checkpoints, like homework assignments or quizzes. While students can demonstrate understanding through projects and homework sets, there is little room for improvement, and no incentive to encourage continued learning. By creating a tool to increase the ease with which competency-based grading can be implemented in the classroom, we can hopefully allow for students’ skills to be measured in a more holistic way. According to “A Narrative Review of Selected Studies in Competency”, a report by Dr. Paul Wright of Marzano Academies, competency-based grading measures demonstration and understanding rather than rote learning, which on the whole serves as a more meaningful measure of a student’s mastery of a subject. With this in mind, a tool that encourages the use of competency-based grading relates to the DJ Principle of recognizing wholeness.

SUSTAINABILITY

The DJ principle sustainability is about prioritizing ongoing change by pacing ourselves. In a traditional 100-point grading scale, you have only one chance to succeed, and one chance to fail. This encourages a very troubling mindset of “if this is not on the test, I will not need to know it”. As students who are in college specifically to build skills that will sustain us once we leave, this isn’t exactly a mindset that is conducive to truly mastering anything. Competency based grading encourages sustained learning because it has chances for multiple redos; there is an emphasis on showing mastery instead of relying on just one limited metric. Another way we embody sustainability is because building tools like this encourages long-term behavior change in our target audience. If implementing competency based grading becomes simpler—and we also have other tools to do this, like some kind of research-backed guide—then our tool could potentially create long-term change in the Allen School.