A new tablet app that recognizes handwritten music?


Update, January 7, 2013: See the follow-up post Makers of music handwriting app video used Sibelius and GoodReader to create dramatization

A promotional video for a new tablet app has created a bit of a sensation in the last couple of days. A company called ThinkMusic Technology has made an internet commercial for “The Music Notation App You’ve Been Waiting For” and posted it to YouTube on December 31, 2012.

It shows music being played into a tablet using an iOS interface via an onscreen touch keyboard and via a physical MIDI keyboard. It also shows users using a stylus to write music directly on screen, and, with the tap of a button, immediately converting their scribbles into engraved music. The app is shown to instantly recognize handwritten notes, chord symbols, dynamics, and lines such as slurs, hairpins and 8va lines:


However, if you watch the video carefully, you may notice that the engraved music looks practically identical to default output from Sibelius (which is why I felt it relevant to post about this on this blog), right down to the fonts, spacing of the music, and even how the slurs in the third system make use of Magnetic Layout to push the chord symbols higher:

Screen grab no. 1 from ThinkMusic Technology promo video
Screen grab no. 1 from ThinkMusic Technology promo video
Screen grab no. 2 from ThinkMusic Technology promo video
Screen grab no. 2 from ThinkMusic Technology promo video
A setting of the demo score in Sibelius (the non-bold dynamic and unusual chord changes are taken from the video)
A setting of the demo score in Sibelius (the non-bold dynamic and unusual chord changes are copied “as is” from the video)

Switching between the handwriting and the engraved image, as shown on the video, could easily be simulated using a screen drawing or annotation app.

If this a hoax, it’s beautifully done. If the app is for real, then shouldn’t the promo video should carry a disclaimer that the images are simulated? No doubt, an app that actually does what is shown in the video would be welcome among musicians, judging from the excitement in comments posted to social media. If ThinkMusic’s app designing skills are as good as the production values in the video, then this could indeed be cause for celebration. For now, though, we’ll take the skeptical approach.

What do you think? We have reached out to ThinkMusic via Twitter (no e-mail or phone contact information is posted on their site) and will gladly update this post if they reply.


  1. Álvaro Buitrago

    There are many suspicious details in the video. The spacing of bar 1 and 3, before he starts writing, is also wider!

    More details on “the skeptical approach” (Sorry, in Spanish, you can use a translator)

  2. Kala Pierson

    I had a similar reaction and I laughed to see your post in my feeds tonight. :)

    Another clue is that their supposed in-app shots of the output look to have the anti-aliasing of a PDF (not sure of this with YouTube-quality video, but I’d wager so).

    More basically: I don’t think it’s materially possible, given iOS constraints, that an app could reproduce all Sibelius defaults down to exact weights, kerning of chord symbols and text blocks, and idiosyncratic (no offense :)) details like slur behaviors and hairpin placement.

    Maybe this will be revealed as a social media experiment that was meant to demonstrate how many people *would* be excited for an app like this if someone developed it. :/

  3. kb

    I subscribed to their newsletter, it seems the company is located in California:
    Thank you for subscribing!

    ThinkMusic Technology
    PO Box 1151
    Glendora, CA 91740

  4. Peter McAleer

    iPad is great, but it is a toy compared with a Mac or a PC desktop, and if the app is real, and as promising as that video suggests, it needs a desktop version to be taken up by professionals. How on earth am I going to display an orchestra on that? Why would I sit at my desk (like the guy in the video), hook my iPad up to a midi keyboard three times its size, and sit there craning my neck and straining my eyes to be able to see a tiny representation of an A3 score on a screen little bigger than the 9″ monitor my Mac Classic II had about 21 years ago? And paying good money to boot?

  5. Kala Pierson

    To be clear, the app isn’t real. The point of this post is to demonstrate that all the supposed footage of in-app behavior in the video is faked. (All of its supposed end results are not live results; they are actually PDFs of notation done in Sibelius, a desktop notation program that has nothing to do with the people who made the app.)

    I think this post is too polite/indirect!

  6. Carlin Lemon

    I’ve been waiting for a program like this ever since I started charting my music on a windows convertible tablet 5 years ago. Unfortunately Apple doesn’t make any products with an actual stylus and digitizer, but there are numerous windows tablets and convertibles ranging from 8 to 15″ that are perfect for this. I’m wondering why Sibelius hasn’t already jumped on the idea as they already have the whole framework in place. To me, there’s nothing faster or more intuitive than hand entry, but it still lacks the legibility and versatility of a sibelius file. How long will we have to wait for this?

  7. Anton

    Hi. I am from Ukraine and can’t watch this video because “The author has limited the viewing of this video”. I am very interested in thi program. How can I access the video?
    Thank you.

  8. James Bergman

    I think it is safe to say that this kind of app is definitely possible. If my tablet has the software to read and understand my chicken scratch writing, then it isn’t a huge leap from there to music. So, here is hoping that this is not a hoax! Even if it is, I hope it shouldn’t be long until this is a reality.

Leave a Comment

Your email address will not be published. Required fields are marked *