September 2019 - May 2019

Santa Clara University, CA

For my college Senior Design Project, I built a co-creative web app for musicians to make music in their browser with assistance from ML algorithms. I teamed up with two classmates and we set off to follow a thorough system design process. We formally defined functional, non-functional, and security requirements. Then we examined use cases and created architecture with standard UML diagramming. I learned how intentional planning, thoughtful problem statements, and architecture alignment can help guide a project from start to finish.

We built a Single Page Web app (SPA) with React, a music generation engine and backend in Django, storage with SQLite, and several custom co-creative algorithms to add harmonies and drums. The harmonization algorithm was based on a Bayesian Network built with a python library called Pomegranate. The network was trained on a corpus of midi music scraped from various open source datasets. The drum line algorithm, used a naive pattern matching approach with no learning.

Finally I administered a production Digital Ocean web server with a firewall, GUnicorn WSGI for Django, Nginx Reverse Proxy, and Let’s Encrypt certificates for TLS traffic. The system was used by several classmates to learn about Midi music, upload melodies, and input piano tunes in real time. The web app was operational until May 2019 when I shut the servers down to focus on new projects.

I learned a lot about team leadership and problem solving. I learned the difficulties of benchmarking generative AI especially in the Audio domain. I learned the importance of dependency management, simplicity of architecture, and the importance of clean code.

Link to paper: https://scholarcommons.scu.edu/cseng_senior/152/

Technology:

Python, Django, React, MIDI, Generative AI, Bayesian Networks, Decision Trees, Digital Ocean Droplets, NGINX, SQLite