Creating Interactive Behaviors in Early Sketch by Recording and Remixing Crowd Demonstrations

Abstract: In the early stages of designing graphical user interfaces (GUIs), the look (appearance) can be easily presented by sketching, but the feel (interactive behaviors) cannot, and often requires an accompanying description of how it works (Myers et al. 2008). We propose to use crowdsourcing to augment early sketches with interactive behaviors generated, used, and reused by collective "wizards-of-oz" as opposed to a single wizard as in prior work (Davis et al. 2007). This demo presents an extension of Apparition (Lasecki et al. 2015), a crowd-powered prototyping tool that allows end users to create functional GUIs using speech and sketch. In Apparition, crowd workers collaborate in real-time on a shared canvas to refine the user-requested sketch interactively, and with the assistance of the end users. Our demo extends this functionality to let crowd workers "demonstrate" the canvas changes that are needed for a behavior and refine their demonstrations to improve the fidelity of interactive behaviors. The system then lets workers "remix" these behaviors to make creating future behaviors more efficient.