title: "Mallard: Turn the Web into a Contextualized Prototyping Environment for Machine Learning"
authors: Xiong Zhang and Philip J. Guo
venue: ACM Symposium on User Interface Software and Technology (UIST)
year: 2019
links:
- <a href="https://bearzx.github.io/Mallard/">Webpage</a>

tweet: Mallard turns the browser into an environment for prototyping machine learning apps using web data

abstract: >
  Machine learning (ML) can be hard to master, but what first trips up
  novices is something much more mundane: the incidental complexities of
  installing and configuring software development environments. Everyone
  has a web browser, so can we let people experiment with ML within the
  context of any webpage they visit? This paper's contribution is the
  idea that the web can serve as a contextualized prototyping
  environment for ML by enabling analyses to occur within the context of
  data on actual webpages rather than in isolated silos. We realized
  this idea by building Mallard, a browser extension that scaffolds
  acquiring and parsing web data, prototyping with pretrained ML models,
  and augmenting webpages with ML-driven results and interactions. To
  demonstrate the versatility of Mallard, we performed a case study
  where we used it to prototype nine ML-based browser apps, including
  augmenting Amazon and Twitter websites with sentiment analysis,
  augmenting restaurant menu websites with OCR-based search, using
  real-time face tracking to control a Pac-Man game, and style transfer
  on Google image search results. These case studies show that Mallard
  is capable of supporting a diverse range of hobbyist-level ML
  prototyping projects.

bibtex: >
  @inproceedings{ZhangUIST2019,
    author = {Zhang, Xiong and Guo, Philip J.},
    title = {Mallard: Turn the Web into a Contextualized Prototyping Environment for Machine Learning},
    year = {2019},
    isbn = {9781450368162},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3332165.3347936},
    doi = {10.1145/3332165.3347936},
    booktitle = {Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology},
    pages = {605--618},
    numpages = {14},
    keywords = {contextualized machine learning, ml prototyping},
    location = {New Orleans, LA, USA},
    series = {UIST '19}
  }