Publications using OK

Instructors often use their course data to research the effectiveness of assignments.
Here's are some of things instructors have published with the help of OK

Course Delivery as a Software Problem

Talk: How the OK Software Suite has helped CS 61A scale to 1600 students in a single semester.

John DeNero, EECS Colloqium Invited Talk - October 19, 2016

Automated Test Generation

Fuzz-testing projects discovered errors in 50% of students that instructor created tests missed.

Sumukh Sridhara, Brian Hou, Jeffrey Lu, and John DeNero
Fuzz Testing Projects in Massive Courses. In Proceedings of the Third (2016) ACM Conference on Learning @ Scale (L@S '16).

Identifying Misunderstandings

A technique to identify conceptual misunderstandings from constructed response questions.

Kristin Stephens-Marintez, An Ju, Colin Schoen, John DeNero, Armando Fox
Identifying Student Misunderstandings using Constructed Responses. Extended Abstract at Learning At Scale 2016. ACM L@S ’16

Hints from Similiar Submissions

Using program synthesis to create hints using past submissions from others.

Rolim, R. et al. (2016)
Learning Syntactic Program Transformations from Examples.

Feedback at Scale

Combining program synthesis and teacher feedback to give hints for incorrect programs.

Andrew Head, Elena Glassman, Gustavo Soares, Ryo Suzuki, Lucas Figueredo, Loris D’Antonik, Bjorn Hartmann
Writing Reusable Code Feedback at Scale with Mixed-Initiative Program Synthesis. To appear at ACM L@S ’17