With version 0.9, Raveberry received a major overhaul of its architecture, and a bunch of other improvements.
Before, Raveberry initialized itself by instantiating a single Base object, which in turn created objects related to Raveberry's different functionalities (Settings, Musiq, Lights etc.). I initially decided to use this naive object-oriented approach because responsibilities are clearly encapsulated. Each object deals with a different aspect of the project and they can communicate with each other by accessing variables or calling methods. While this makes it comfortable to design the different aspects of Raveberry and their interactions, it comes with a big drawback: Relying on a single object to handle all requests limits Raveberry to a single process.
The single object performs one-time initializations and contains locks to ensure correct behavior. Additionally, it creates background threads to control playback and compute the visualization. Creating a second instance would break all of this functionality, as for example two threads would simultaneously try to play songs. Thus, the old architecture could not be expanded beyond a single process.
Even if threads allow executing different tasks seemingly in parallel in a single process, this is not the case in Python. Due to Python's Global Interpreter Lock only one thread can execute python code at once. If most threads are IO-bound, this is not a problem, because other threads can execute during wait times. If a thread does a lot of computation though, it impacts the performance of other threads. Particularly the thread rendering the visualization at ~30fps falls into this category.
This architectural limitation has bothered me for a long time, and I finally came around to implement a cleaner solution. Long running tasks are now queued via Celery, where they run in separate processes, benefiting from full parallel performance. Interactions between those tasks and request handlers are coordinated through Redis and the database.
Since one task cannot simply access attributes of the respective python object anymore, I needed to revisit the interactions between the different components. For each one, I looked into which database keys, Redis values or locks it requires. This improved the general code quality, because it lead to clearly defined interfaces and reduced coupling between components.
Now, all state is kept in centralized locations (Redis, db) independent of the tasks or requests. More expensive requests (such as enqueuing new songs) are processed as celery tasks, moving out work from the main request/response cycle. This means that a single instance of daphne, the server used by Raveberry, is able to handle all incoming requests, even though now it would be possible to start a second instance.
Aside from being a cleaner implementation, this new architecture also has some practical side effects:
A little explanation on the last point: Before, I needed a place to initialize the object. I had problems with django's ready function, as it was called for management commands as well. Thus, I put the initialization into the urls.py file, where the mappings from url to handler function are defined. This file is only parsed when the first request is made and thus the object is not initialized for management commands. It also means that playback only started when the first page loaded. With the new architecture, this is not the case anymore. You can enjoy your music as soon as the server boots up.
Since I needed to touch almost all of Raveberry's code during this transition, I improved some other stuff while I was at it:
DJANGO_MOCK and DJANGO_POSTGRES are not used anymore to tell Raveberry how to behave. It only initializes the server when called with daphne or runserver, management commands are possible without any environment variables.daphne logs into its own log file instead of cluttering syslog.I meant to do this rewrite for a long time, always postponing it due to the seemingly daunting amount of work. Now I am very happy that I can finally present this change. I want to thank r/django for their valuable input to the problem in this post. This definitely helped me make this architecture change possible.
Another factor is that recently things went back to being a little bit more normal again (at least over here), allowing for some uses of Raveberry in real life. Seeing this project in action always makes me happy and motivates me to improve it.
With this change finally implemented, I'm starting to feel confident enough to call Raveberry "stable" not too far in the future. Are there things you would like to see in Raveberry 1.0?