title: "Codemotion: Expanding the Design Space of Learner Interactions with Computer Programming Tutorial Videos" authors: Kandarp Khandwala and Philip J. Guo venue: ACM Conference on Learning at Scale year: 2018 links: - Blog post tweet: Codemotion automatically extracts the source code within computer programming screencast videos abstract: > Love them or hate them, videos are a pervasive format for delivering online education at scale. They are especially popular for computer programming tutorials since videos convey expert narration alongside the dynamic effects of editing and running code. However, these screencast videos simply consist of raw pixels, so there is no way to interact with the code embedded inside of them. To expand the design space of learner interactions with programming videos, we developed Codemotion, a computer vision algorithm that automatically extracts source code and dynamic edits from existing videos. Codemotion segments a video into regions that likely contain code, performs OCR on those segments, recognizes source code, and merges together related code edits into contiguous intervals. We used Codemotion to build a novel video player and then elicited interaction design ideas from potential users by running an elicitation study with 10 students followed by four participatory design workshops with 12 additional students. Participants collectively generated ideas for 28 kinds of interactions such as inline code editing, code-based skimming, pop-up video search, and in-video coding exercises. bibtex: > @inproceedings{KhandwalaLAS2018, author = {Khandwala, Kandarp and Guo, Philip J.}, title = {Codemotion: Expanding the Design Space of Learner Interactions with Computer Programming Tutorial Videos}, booktitle = {Proceedings of the Fifth Annual ACM Conference on Learning at Scale}, series = {L@S '18}, year = {2018}, isbn = {978-1-4503-5886-6}, location = {London, United Kingdom}, pages = {57:1--57:10}, articleno = {57}, numpages = {10}, url = {http://doi.acm.org/10.1145/3231644.3231652}, doi = {10.1145/3231644.3231652}, acmid = {3231652}, publisher = {ACM}, address = {New York, NY, USA}, keywords = {computer programming, screencasts, tutorial videos}, }