Automating Musescore 3.6 PDF and MP3 Generation Using Github Actions

Chris Woodall | 2021-03-28

Back in the day, I used to use Sibelius and Finale for music composition, and engraving. MuseScore 3.6 is an open-source, cross-platform music composition tool that fits right in there with the other two. Considering that it is free it punches above its weight class. I first found out about MuseScore when I watched a review of its interface by Tantacrul who ripped the UI to shreds. I thought that this would be the end of it for me, however, I later found out that after this video was made Tantacrul joined the MuseScore team as a project manager on the UI team, and with 3.6 it seems like they have already started to fix a lot of the underlying issues (and I found it to be a joy to work in).

So when I wanted to write down some transcriptions and exercises for mandolin I decided to use MuseScore. Since I am a massive nerd I made this into a git repository using their .xml based .mscx (.musicxml) files to track changes over time. Continuing on the massive nerd path I thought it would be interesting to use the command-line arguments and job files to create pdfs and mp3s on push, build-server style, then add the resulting files to the repository to share easily.

Why would you want to do this? If you are tracking versions and you want distributable files to be continuously updated and created for you. An alternative would be to tag releases and upload the results to those tags.

To do this I used Github Actions, made a Makefile which I tested locally for scanning for all .mscx and .mscz files to make .mp3 and .pdf files. After that, I decided to add basic support for using .json job files as input.

Installing MuseScore 3.6

To install MuseScore 3.6 I used the flatpak release under Ubuntu 20.04:

sudo apt install flatpak
sudo flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
sudo flatpak install -y --noninteractive flathub org.musescore.MuseScore          

This will keep you up to date and also match you with what is being used on the build server, otherwise, just make sure you are using MuseScore 3.6 and not an earlier version.

The Makefile

So we need to be able to take in 3 different types of files: .mscx, .mscz and .json. The .mscx and .mscz files we want to create .pdf and .mp3 files. The .json files will create a bunch of intermediate files, so we will never know if it has been executed or when. As a result, we will create a .job file that will contain all of the files created by that job.

graph LR MSC[".mscx or .mscz files"] JSON[".json job files"] JOB[".job files that mark a json job has been executed"] OUTPUTS["Whatever the .json file specifies"] MSC --> PDF MSC --> MP3 JSON --> JOB JSON --> OUTPUTS OUTPUTS -->|appended to| JOB

The resulting makefile is reasonably straight forward:

MSCORE := "mscore"

MSCX_FILES = $(shell find . -name "*.mscx")
MSCZ_FILES = $(shell find . -name "*.mscz")
JOB_FILES = $(shell find . -name "*.json")

PDFS = $(MSCX_FILES:.mscx=.pdf) $(MSCZ_FILES:.mscz=.pdf)
MP3S = $(MSCX_FILES:.mscx=.mp3) $(MSCZ_FILES:.mscz=.mp3)
JOB_OUTS = $(JOB_FILES:.json=.job)

.PHONY: all
all: $(PDFS) $(MP3S) $(JOB_OUTS)

%.pdf: %.mscx
    $(MSCORE) -o $@ $<

%.mp3: %.mscx
    $(MSCORE) -o $@ $<

%.pdf: %.mscz
    $(MSCORE) -o $@ $<

%.mp3: %.mscz
    $(MSCORE) -o $@ $<

# mark the output of this job as secondary so make does not delete the intermediate files
.SECONDARY:%.job
%.job: %.json
    # Create a file at the beginning of the job
    # Add to it a list of any files which were created in this
    # directory after the job is done running. This allows for
    # a full clean of the directory
    # 
    # CAVEAT: This does not work well for parrallel builds using make
    touch $@.tmp
    cd $(dir $(abspath $<)) && $(MSCORE) -j $(notdir $<) 
    find "." -type f -newer "$@.tmp" >> $@
    rm $@.tmp

.PHONY: clean
clean:
    # Remove all files referenced by job files
    -for job_file in ${JOB_OUTS}; do cat $$job_file | xargs rm; done
    -rm -r $(JOB_OUTS)
    -rm -r $(PDFS) $(MP3S)

The one interesting piece is in the .json job file handling:

# mark the output of this job as secondary so make does not delete the intermediate files
.SECONDARY:%.job
%.job: %.json
    # Create a file at the beginning of the job
    # Add to it a list of any files which were created in this
    # directory after the job is done running. This allows for
    # a full clean of the directory
    # 
    # CAVEAT: This does not work well for parrallel builds using make
    touch $@.tmp
    # cd to the directory the json file is so that we can parse relative
    # paths properly in the json file.
    cd $(dir $(abspath $<)) && $(MSCORE) -j $(notdir $<) 
    # Find all files created since $@.tmp was created
    find "$(dir $(abspath $<))" -type f -newer "$@.tmp" >> $@
    # Remove the temporary file and just leave the job file.
    rm $@.tmp

The strategy here is to create a file before starting the processing of the json job. Then we look for all new files created after the job has been run, this allows us to run make clean over all created files in this one-liner: for job_file in ${JOB_OUTS}; do cat $$job_file | xargs rm; done.

The Github Action

In workflows/build.yml I put the following content, which installs MuseScore 3.6 (you need to use the same major version of MuseScore on both your build server and locally). This file will install musescore, run the makefile within xvfb which creates a virtual frame buffer for running headless tests that would otherwise require a display. There is a Github Action already available which made using this easy and xvfb is already setup on ubuntu-latest. The action actually handles creating virtual frame buffers across multiple platforms which is super useful.

After that, the action just sets up a bot user, adds the created files, and pushes them back up. This means that your repository needs to be pulled down semi-frequently. I am not always a fan of these types of repos, which are shared between bot and normal uses. However, when used effectively and with some rules, in place they can be great for automating the creation of files that need to be in the repository, but whose creation is best automated.

# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2
      # Install flatpak and the MuseScore 3.6 flatpak
      - name: Install musescore 3.6
        run: |
          sudo apt install flatpak
          sudo flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
          sudo flatpak install -y --noninteractive flathub org.musescore.MuseScore                    
      # Create a Virtual X Frame Buffer and run the makefile inside of it using the flatpak installation
      # of musescore
      - name: Run Makefile to make PDFs, MP3s, etc
        uses: GabrielBB/xvfb-action@v1
        with:
          working-directory: ./ #optional
          run: make MSCORE="flatpak run org.musescore.MuseScore"
      - name: Add Commit and Push
        run: |
          git config --global user.email "email@address"
          git config --global user.name "MuseScore 3 Bot"
          git add .
          git commit -am "Updating pdfs and mp3s"
          git push          

Conclusion

I am happy with how this turned out! I got to scrub off some weird Makefile knowledge, and now my score generation should be automated for a good time to come. You can see the results here, and a few of the action runs here.

Feel free to add a PR, or rip off the automation for yourself.