How I reduced Raveberry's transferred frontend code by 90%

April 3, 2021 - Reading time: 10 minutes

tl;dr: Only include what's necessary, minification, gzip.

For those that don't know, my side project Raveberry is a multi user music server that allows democratic selection of songs. People can download it from PyPi and setup their own instance.

In this post I want to share with you the journey of how I significantly reduced the size of the frontend code used in Raveberry. For a long time, I tried to avoid the frontend ecosystem as much as possible. Stories like a transitive dependencies breaking thousands of projects or npm rendering your system unusable by changing a bunch of permissions don't really help. That's why, for a long time, Raveberry's frontend was structured pretty straightforward.

The Dark Ages

The external dependencies were downloaded into the static/ folder containing all frontend assets using yarn and a package.json. Other than that, yarn was not used. Next I ran a bash script that removed all files I did not need, keeping the javascript, css and font files that should be served. In html, these files where directly included. I knew that loading many files is bad, so I used django-compressor to combine them into one big file. Conveniently, this tool also provided me with a way to compile scss.

For the most part this worked quite well, except of some hindrances. For example, the exact javascript file required by the compressor had to be bundled in the package, or else the compressor would not serve it, making its inclusion redundant. However, the biggest drawback becomes visible when examining the size of the code that was included in Raveberry:

We see that version 0.7.8 contained 1.7MB worth of frontend code, which is more than 50% of the whole package size. Almost half of that is taken up by external javascript libraries. Most of what is contained in the package is also loaded on a page visit, the big example being the scss code (I don't even know why that was included in addition to the compiled css). For a page that basically only acts as a remote jukebox control, this seems quite excessive. Eventually (starting with 0.7.9) I gave in and tried to find better solutions.

Better Solutions

Let's go through each of the components and talk about the tools used to improve this situation.

Javascript

The biggest offender. In the process of restructuring the frontend, I migrated to Typescript, a long overdue task. Typescript made me strictly define all external dependencies, which made it clearer which parts get used where.

Analyzing these dependencies, I found that some of them could be reduced or replaced. For example, jquery-ui is used for autocompletion and reordering. All additional widgets provided by jquery-ui (e.g. sliders, datepickers) are dead weight and can be excluded from the final bundle. Another example is marked, which was used to render the changelog. However, the changelog has a very simple structure and does not require a \~300KB library to be parsed. So instead, I use snarkdown, a lightweight alternative which is fully sufficient for this application.

To bring everything together, I use webpack to create the final bundle. It takes care of including the necessary libraries and minifies the output. I tried to use rollup but there were issues when importing only parts of jquery-ui. The final bundle.js has a size of 272KB. This is still not ideal, but since the functionality is basically identical, I consider it quite an achievement. The next step would probably be to replace jquery and jquery-ui with more lightweight alternatives.

css

Bootstrap provides css rules for a lot of html elements. Instead of serving all of it, purgecss is used to only include the necessary rules. The tool sifts through every html template and all javascript files to detect used identifiers. Everything else is removed from the resulting file, drastically reducing its size. I only had to manually prevent a few dynamically created identifiers from being removed.

Afterwards, a postcss pass minifies the css code, resulting in 21,8KB per light/dark style. In comparison to the naive approach, this amounts to almost 90% reduction.

Fonts

Similar to the approach used for the css code, the fonts were also modified to only contain used symbols. A script parses html and javascript files to find used identifiers. Then, fonttools is used to create a subtyped font file with only these symbols. Since Raveberry only uses a small set of icons and Font Awesome provides a lot of them, this approach eliminates the vast majority. The resulting font files are 6.7KB and 2.3KB, much smaller than the original ones.

While writing this article I also noticed that I unnecessarily include the regular font file. Since no icon from this file is used, it is never loaded by the browser, but it does not need to be packaged and will be removed in the next update.

gzip

Another thing I realized was that compression wasn't even enabled in nginx. Since virtually every browser supports it, enabling it was a very effective method of lowering the amount transferred bytes.

Summary

Taking all of this together, we end up with this result:

Each of the components was significantly reduced in size. In total, 81% less frontend code is contained in the package, reducing its total size by 42%. The amount of transferred bytes was even cut down by 90%.

During the whole process I learned a lot about frontend development. It was fun wrestling with the code and trying to remove unnecessary elements. I became familiar with quite a few tools, making everything seem less intimidating. And as a result, Raveberry installs and loads faster.

Thank you for reading and have a great day.