Automating Musescore 3.6 PDF and MP3 Generation Using Github Actions
Back in the day, I used to use Sibelius and Finale for music composition, and engraving. MuseScore 3.6 is an open-source, cross-platform music composition tool that fits right in there with the other two. Considering that it is free it punches above its weight class. I first found out about MuseScore when I watched a review of its interface by Tantacrul who ripped the UI to shreds. I thought that this would be the end of it for me, however, I later found out that after this video was made Tantacrul joined the MuseScore team as a project manager on the UI team, and with 3.6 it seems like they have already started to fix a lot of the underlying issues (and I found it to be a joy to work in).
So when I wanted to write down some transcriptions and exercises for mandolin I decided to use MuseScore. Since I am a massive nerd I made this into a git repository using their .xml
based .mscx
(.musicxml
) files to track changes over time. Continuing on the massive nerd path I thought it would be interesting to use the command-line arguments and job files to create pdfs and mp3s on push, build-server style, then add the resulting files to the repository to share easily.
Why would you want to do this? If you are tracking versions and you want distributable files to be continuously updated and created for you. An alternative would be to tag releases and upload the results to those tags.
To do this I used Github Actions, made a Makefile which I tested locally for scanning for all .mscx
and .mscz
files to make .mp3
and .pdf
files. After that, I decided to add basic support for using .json
job files as input.
Installing MuseScore 3.6
To install MuseScore 3.6 I used the flatpak release under Ubuntu 20.04:
sudo apt install flatpak
sudo flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
sudo flatpak install -y --noninteractive flathub org.musescore.MuseScore
This will keep you up to date and also match you with what is being used on the build server, otherwise, just make sure you are using MuseScore 3.6 and not an earlier version.
The Makefile
So we need to be able to take in 3 different types of files: .mscx
, .mscz
and .json
. The .mscx
and .mscz
files we want to create .pdf
and .mp3
files. The .json
files will create a bunch of intermediate files, so we will never know if it has been executed or when. As a result, we will create a .job
file that will contain all of the files created by that job.
The resulting makefile is reasonably straight forward:
MSCORE := "mscore"
MSCX_FILES = $(shell find . -name "*.mscx")
MSCZ_FILES = $(shell find . -name "*.mscz")
JOB_FILES = $(shell find . -name "*.json")
PDFS = $(MSCX_FILES:.mscx=.pdf) $(MSCZ_FILES:.mscz=.pdf)
MP3S = $(MSCX_FILES:.mscx=.mp3) $(MSCZ_FILES:.mscz=.mp3)
JOB_OUTS = $(JOB_FILES:.json=.job)
.PHONY: all
all: $(PDFS) $(MP3S) $(JOB_OUTS)
%.pdf: %.mscx
$(MSCORE) -o $@ $<
%.mp3: %.mscx
$(MSCORE) -o $@ $<
%.pdf: %.mscz
$(MSCORE) -o $@ $<
%.mp3: %.mscz
$(MSCORE) -o $@ $<
# mark the output of this job as secondary so make does not delete the intermediate files
.SECONDARY:%.job
%.job: %.json
# Create a file at the beginning of the job
# Add to it a list of any files which were created in this
# directory after the job is done running. This allows for
# a full clean of the directory
#
# CAVEAT: This does not work well for parrallel builds using make
touch $@.tmp
cd $(dir $(abspath $<)) && $(MSCORE) -j $(notdir $<)
find "." -type f -newer "$@.tmp" >> $@
rm $@.tmp
.PHONY: clean
clean:
# Remove all files referenced by job files
-for job_file in ${JOB_OUTS}; do cat $$job_file | xargs rm; done
-rm -r $(JOB_OUTS)
-rm -r $(PDFS) $(MP3S)
The one interesting piece is in the .json
job file handling:
# mark the output of this job as secondary so make does not delete the intermediate files
.SECONDARY:%.job
%.job: %.json
# Create a file at the beginning of the job
# Add to it a list of any files which were created in this
# directory after the job is done running. This allows for
# a full clean of the directory
#
# CAVEAT: This does not work well for parrallel builds using make
touch $@.tmp
# cd to the directory the json file is so that we can parse relative
# paths properly in the json file.
cd $(dir $(abspath $<)) && $(MSCORE) -j $(notdir $<)
# Find all files created since $@.tmp was created
find "$(dir $(abspath $<))" -type f -newer "$@.tmp" >> $@
# Remove the temporary file and just leave the job file.
rm $@.tmp
The strategy here is to create a file before starting the processing of the json job. Then we look for all new files created after the job has been run, this allows us to run make clean
over all created files in this one-liner: for job_file in ${JOB_OUTS}; do cat $$job_file | xargs rm; done
.
The Github Action
In workflows/build.yml
I put the following content, which installs MuseScore 3.6 (you need to use the same major version of MuseScore on both your build server and locally). This file will install musescore,
run the makefile within xvfb which creates a virtual frame buffer for running headless tests that would otherwise require a display. There is a Github Action already available which made using this easy and xvfb is already setup on ubuntu-latest
. The action actually handles creating virtual frame buffers across multiple platforms which is super useful.
After that, the action just sets up a bot user, adds the created files, and pushes them back up. This means that your repository needs to be pulled down semi-frequently. I am not always a fan of these types of repos, which are shared between bot and normal uses. However, when used effectively and with some rules, in place they can be great for automating the creation of files that need to be in the repository, but whose creation is best automated.
# This is a basic workflow to help you get started with Actions
name: CI
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ main ]
pull_request:
branches: [ main ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
# Install flatpak and the MuseScore 3.6 flatpak
- name: Install musescore 3.6
run: |
sudo apt install flatpak
sudo flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
sudo flatpak install -y --noninteractive flathub org.musescore.MuseScore
# Create a Virtual X Frame Buffer and run the makefile inside of it using the flatpak installation
# of musescore
- name: Run Makefile to make PDFs, MP3s, etc
uses: GabrielBB/xvfb-action@v1
with:
working-directory: ./ #optional
run: make MSCORE="flatpak run org.musescore.MuseScore"
- name: Add Commit and Push
run: |
git config --global user.email "email@address"
git config --global user.name "MuseScore 3 Bot"
git add .
git commit -am "Updating pdfs and mp3s"
git push
Conclusion
I am happy with how this turned out! I got to scrub off some weird Makefile knowledge, and now my score generation should be automated for a good time to come. You can see the results here, and a few of the action runs here.
Feel free to add a PR, or rip off the automation for yourself.