Thursday, October 18, 2012

Automating YouTube video creation and upload with Perl and Sibelius.

I was asked by the admin of stmaryssingers.com to explore the feasibility of converting their audio practice track library to videos on YouTube. Aside from the savings in bandwidth and storage space, the admin thinks there might be some possible click revenue towards the Singers costs.

I originally created the audio library of works that were being practiced by the choir by scanning, OCRing (Optical Music Recognition more accurately), importing the scans into Sibelius and exporting audio tracks for each part.

Originally it was purely to help me learn the music faster as I don't sight-read music and I don't have (music) keyboard skills. But it quickly became obvious that other choir members could use a little help so I put the tracks onto the website for easy access by members.

Creating YouTube videos from Sibelius .sib files.

After receiving the request to see if it were possible to create YT videos from .sib files, it occurred to me that a more recent version of Sibelius might have that facility. Sibelius has had Scorch for a long time, which creates music videos to be played on a website using the Scorch plugin. It shows the score together with a "bouncing ball" cursor which moves in time with the audio track. Scorch has had a very small takeup. YouTube on the other hand is a bit more popular and Avid announced a "feature-limited but cheap" version of Sibelius 7, called Sibelius 7 First, which exports a music score as a YouTube video (almost) exactly as I wanted.

The problem was that Sibelius doesn't have an automated facility to create "parts" videos. The audio practice tracks I created were re-mixed to emphasise each part. So the "Soprano" mix has the Soprano part in the centre of the sound stage, the Alto off to the right, the Tenor and Bass off to the left and the Piano even further off to the right. And so on for the other parts.

What I wanted to do was create an overlay image for each part which is mixed with the plain (SATB) video so that the particular part is highlighted while the others parts and accompaniment are slightly greyed out. Then I needed to replace the SATB audio file in the video with the audio re-mix for each part. 

And all of this has to be an automated script and the final step is to upload each part video to YouTube and update each description box to contain links to all the other parts in the work. 

It took me a few days but it works now (mostly).

Thankfully I had stored the Sibelius (.sib) file for each work on the website. But some of them are close to five years old and my knowledge of Sibelius was pretty minuscule back then. I've had to spend a lot of time cleaning up the .sib files to make them suitable to look at not simply listen to

The Process (so I can remember what to do next time).

1. Download and open the .sib file in Sibelius 6. Clean it up:
  • Use the Text/Delete Dynamics plugin to remove anything that changes the volume. This audio is for learning to sing the notes and it helps to be able to hear them.
  • Clean up the lyric lines. I'm only putting in one verse and chorus in the Soprano lyric line. I'm removing all other lyrics.
  • Open the Score Info window (File/Score Info) and set the composition name and the composer. YouTube requires these fields.
  • Reset the audio mix. When creating the audio library, I often saved the .sib file with one of the parts emphasised and it ruins the SATB export.
  • Show the staff rulers (View->Rulers->Staff Rulers) and adjust the inter-staff spacing: 29 from the top, 12 between each singing part, 14 between Piano staves.
  • Export Audio and save it as song_satb.aiff.
  • Run the Playback/SetPiano plugin to set all voices to Piano. Makes it easier to distinguish notes when learning.
  • Run the Playback/SetVoicePan to export an AIFF file for each vocal part in the score.
  • Adjust the names of the AIFF files so there is no digit in the filename if there is only one part in that voice e.g. song_a1.aiff should be song_a.aiff if there is only one alto part. But no need to change names if there is an alto1 and alto2 part.
  • Reset the mixer levels and the voice names in the score after running SetVoicePan.
  • Save the .sib file.
2. Open the .sib file in Sibelius 7 First and export it as a video:
  • Click 'File'
  • Click 'Export'
  • Click 'Video'
  • Deselect 'Use score paper texture'
  • Select Resolution as 'HD (720p)'
  • Edit the filename (default should be song.mov).
  • Click 'Export'
3. Run makmov.pl --file songfilename --title 'Song Title' --parts 's a t b' -o 33 -i 132
Or whatever params are appropriate.
  • makmov will pause after creating a single-frame PNG from the movie and the overlay PNGs (called 'gradient_*.png').
  • Use Preview to line up the single frame and the gradients. Stop the script and restart with different '-o' and '-i' values if they don't line up.
  • When all is OK, press 'Return' after the pause and process will create all the individual videos, upload them to YouTube and rewrite their description text with links to the other parts in the song.
Scripts are here.

Saturday, October 13, 2012

A development setup using MacVim+zsh+tmux

I was initially inspired to seriously update my devel environment by Dr Bunsen's Text Triumvirate article. In the process I discovered some great configurations for development on my MacBook Air + Thunderbolt Display setup.

I really like YADR (Yet Another Dotfile Repo).  It also has some highly opinionated setups for MacVim, zsh and tmux. My main complaint with it is that I really prefer vundle to pathogen for handling Vim plugins. I forked yadr with the intention of replacing pathogen with vundle.

But yesterday I discovered The Ultimate Vim Distribution. It's not the "Ultimate" (which actually means "the last") but it uses vundle instead of pathogen and after looking a bit more carefully at yadr I realised I can use the spf13 Vim plugins without upsetting yadr. I simply had to make ~/.vim and ~/.vimrc point to the .spf13 versions, instead of the .yadr versions.

So I spent most of yesterday merging the .vimrc file from yadr with the one from spf13.

Yadr's developer had the quite brilliant insight to put most of the Vim configuration code (key mappings, config variables etc.) usually kept in .vimrc into separate files in .vim/plugin/settings because anything in .vim/plugin is automatically loaded by Vim at startup. This allowed me to reduce .vimrc from 524 sloc to 288 sloc while adding an additional 55 Bundles to the original 50.

On the other hand, spf13 has an excellent line folding scheme which I've incorporated into the keymap settings file.

Both yadr and spf13 have a preference for Ruby programming whereas I prefer Perl so I removed a lot of Ruby/Rails packages and added a couple of Perl-centric ones.

There were quite a few questionable key mappings in spf13 which I removed and/or changed. Spf13 had <C-L> to move to the right window, an extra keypress for no reason. Yadr has <C-l> and I'm using it (same for h, j and k). Spf13 stole <C-e> to open NERDTree! One of my most used keystrokes! No way. I much prefer yadr's <D-N> (Cmd-N) for the much less used NERDTree. I also disabled the changes to H and L. My muscle memory has them firmly ingrained. spf13 has some useful shortcuts for the fugitive commands (,gs => :Gstatus etc.) but omits the main one I use ,gw (=> :Gwrite). Very strange.

Of course, I wasn't happy with Yadr's tmux config either, so I proceeded to change the link to .tmux.conf from the .yadr version to my own customised version. And Yadr's zsh config only required a few additions/changes to aliases.zsh to get all the shortcuts I've been using for years now. I've not used zsh prior to using Yadr so it has been a slight learning curve from my years of bash use.

So now I have the Text Triumvirate + Ultimate Vim + YADR + My Customisations development config. Simple :)

Sunday, October 7, 2012

RPi + SiriProxy + IRToy = VoiceTV

I've been working on a voice-controlled TV remote for many months now. Almost got a native iOS version working using an IR dongle from L5 and the OpenEars voice to text recogniser. Then life matters intervened and I've never completed the project.

In the meantime all sorts of developments in voice-controlled apps for iOS have popped up. Apple bought Siri for the iPhone 4S then crippled it. Immediately hackers got to work and developed Siri Proxy to allow a local server to intervene when certain commands were spoken to Siri.

Also Nuance released an SDK for Dragon Naturally Speaking with hefty licencing fees but a reasonable, free evaluation period. And Thinkflood's RedEye wifi controller is much easier to program now using URLs.

It occurred to me (and I'm obviously not the first to have thought it) that if I were to attach a USB IR Toy to a Raspberry Pi computer and add the Siri Proxy app I could turn the RPi into a voice controlled WiFi to IR TV controller.

So far I've got Siri Proxy installed on the RPI and I've added an Edimax EW-7717UN WiFi adapter. Coincidentally I had accidentally destroyed the Raspbian wheezy image on the RPi's SD card so installed a new one which just happened to have the latest drivers for the 7717. It was working as soon as I plugged it into the USB port.

Next task is to add the USB IR dongle and start adding some plugins to Siri Proxy to output commands to the dongle to control the TV.

Thursday, October 4, 2012

ReadyNAS NV+ backup to Amazon Glacier Part 1

My previous posts about planning to back up my ReadyNAS NV+ to DreamHost all came unstuck when I read the DH page explaining what they mean by "Unlimited storage". I'm sure this page was added after I agreed for my account to be moved to one of their new servers in exchange for "unlimited storage and bandwidth" back in 2007. Anyway, I'm not in a position to argue.

DH does not allow me to use my account for storing "non-website-related" files, such as copies of ebooks, videos and music files I've purchased, or in my case, created myself. For "personal files", they offer up to 50GB of free but not-backed up storage and then charge 10c/month for each GB over 50.

My current storage needs are around 1TB so this make DH's "personal files" storage an expensive $95/month.

Amazon's announcement of Glacier in August didn't register at the time as a cheap form of backup for anyone. I didn't read the fine print and simply assumed it was only for S3 users of which I am not. As I read further today I realised it's ideal for my requirements as part of a "belts and braces" backup strategy. 1TB would cost me $10/mth (about the same as I'm paying for my "unlimited" DH account). It's not important if it can take up to 4 hours to retrieve files. I will already have the local backup on my Drobo. It's only if I get a double disaster/failure hitting my home systems will I need to retrieve from Glacier. It'll take me more than four hours to replace my home hardware if there's a fire etc.

During the installation I learned that Duplicity can backup to Amazon S3 as one of its backends. It seems a pretty obvious addition to allow it to backup to Glacier. But when I Googled, I found lots of requests for such an addition but no note of it happening.

However I did discover glacier-cli which is built on an equally intriguing project called git-annex. Git-annex uses the git filesystem and commands without actually having to check the files into git. Git is really fast at identifying file differences and there's a lot of user command line knowledge that annex can take advantage of.

Work in progress and worth watching.

ReadyNAS NV+ backup to DreamHost Part 3

Mystery of why scp stalled solved!

But it's taken me many days to understand the solution. I've had to learn how to use WireShark and tcpdump (both useful to know at any time). Then I ran tcpdump on the NV+ while running scp to upload to DreamHost. Then I did the same thing on my MacBook Air and the only difference I could see when I loaded the pcap files into WireShark was that the NV+ was being forced to packet split and eventually the ACK response from DH got lost. But why was it packet splitting? The MTU of the modem router is 1492. The MBA has no problem working this out and sends the correct sized packets. Why can't the NV+?

Still didn't make any sense to me. I tried a number of times switching off Jumbo packets/frames on the NV+ and setting the MTU to 1492 but scp still was forced to packet split. In my frustration I started looking for other backup solutions and discovered Amazon Glacier (more on that in a later blog).

So I started closing all the TABs I had open in Chrome concerning scp stalling and noticed a link I hadn't seen before on this page. I presume I didn't understand it when I saw it and ignored it. The link is to an FAQ for the "snail book". And as I read through it I now understood what I was seeing in the tcpdump file.

So I tried to change the NV+ MTU to 576 as suggested but Frontview limits me to 1400 upto 1500, so I entered 1400 and re-ran scp and it works! Seems the default 1492 in the NV+ Frontview input box is simply wrong for ADSL, but OK for internal LAN transfers (1492+8byte header = 1500 (default TCP packet size)).

Updated command to run Duplicity (The following should be one line.)

duplicity --exclude-globbing-filelist=$HOME/exclude.list
--archive-dir=/c/temp/cache/duplicity
--tempdir=/c/temp/tempsig
--use-scp /media/Music scp://dh/backup/nas


Note that I've specified the archive and temp directories to be on /c/. It is critical not to use the default directories on root filesystem for these. Trying to squeeze a backup of 50GB into a 2GB filesystem is sure to lead to disaster.