Author: Jo Brodie

https://twitter.com/JoBrodie same on BlueSky and other pages. Old photo!

Music AI Kriss Kross Puzzle

Download and print our puzzle

Answers are at the bottom of https://cs4fn.blog/bitof6 where you can also read a copy of the magazine articles about Music and Artificial Intelligence.

On our sister site, Teaching London Computing, we have even more kriss kross puzzles.

Puzzle design credit: https://puzzlemaker.discoveryeducation.com/criss-cross/

Clues

  • 1. _ _ _ _ _ a piece of text with musical symbols instead of letters that tells a performer which
    notes to play, also a piece of music that accompanies a film (5 letters)
  • 2. and 10. _ _ _ _ _ _ (6 letters) separation is when computer scientists use AI to take a piece of music
    and split it into its _ _ _ _ _ (5 letters) – read more about this in ‘Separate your stems‘
  • 3. The _ _ _ _ _ _ is the main part of the tune you might sing along to (6 letters)
  • 4. A piece of music is made up of lots of different _ _ _ _ _ (5 letters)
  • 5. We measure how loud something is in _ _ _ _ _ _ _ _ (8 letters)
  • 6. A sequence of instructions that tell a computer what to do _ _ _ _ _ _ _ _ _ (9 letters)
  • 7. If you halve the length of a guitar string the note is an _ _ _ _ _ _ (6 letters)
  • 8. A guitar-like harp-lute from Ghana _ _ _ _ _ _ _ _ (8 letters) – read more about this in ‘The day the music didn’t die‘
  • 9. How high or how low a musical note is _ _ _ _ _ (5 letters)
  • 10. (see 2.)

More on…

We have LOTS of articles about music, audio and computer science. Have a look in these themed portals for more:

The Music and AI pages are sponsored by the EPSRC (UKRI3024: DA EPSRC university doctoral landscape award additional funding 2025 – Queen Mary University of London).

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

How machines “hear” music

Bee image by by Clker-Free-Vector-Images from Pixabay

Listen to a song and you might tap your feet. Computers can “listen” to music but they don’t have feet to tap! They don’t have ears or a brain either so they don’t “listen” in the way that you do. They use maths.

Turning sound into numbers

A computer is just a machine that does calculations on numbers. It doesn’t really “hear” music. To it everything is just numbers. Its programs convert sounds into numbers that it can do maths with.

When someone plucks a guitar, the string vibrates (wobbles back and forth). That sends a pulse of energy (a sound wave) through the air. Our ears detect that pulse. A computer measures the sound wave. A song has lots of different sound waves mixed together, and they can all be described with numbers that a computer measures.

One measurement is pitch – how high and squeaky or how low and rumbly the sound is. A guitar string playing a higher note vibrates faster than a lower note, sending its energy pulses into the air more quickly. We measure that as the number of sound waves arriving each second (called the frequency).

If we could see a sound wave it might look a bit like this. The red sound wave has a lower frequency than the blue sound wave where the distance between each ‘wobble’ narrows.

The red and blue wavy line shows what a sound wave might look like if we could see it. The blue part of the wave is vibrating faster than the red so has a higher frequency. Humans hear it as a higher note, computers ‘hear’ it by sensing more soundwaves each second.

Another measurement is the volume, or how loud the sound is. That relates to how hard the guitarist plucked the string so how ‘tall’ the sound wave is. The wavy black line has the same frequency as the red and blue wave but the black sound wave is bigger: it has a larger amplitude. Humans hear it as louder, computers record bigger numbers.

Once a computer has recorded the measurements as numbers, it can then do maths on the numbers. That is where things get interesting. Programs can then change the numbers to make new and different sounds. Or they can use algorithms to generate their own numbers, then play them as music!

How loud?

Volume is measured in decibels (dB for short). A lower number means the sound is quieter, a higher number means it is louder. The loudest a UK car is allowed to be is 70 dB.

How loud do you think these sounds are?

Answers at https://cs4fn.blog/bitof6/

More on…

We have LOTS of articles about music, audio and computer science. Have a look in these themed portals for more:

The Music and AI pages are sponsored by the EPSRC (UKRI3024: DA EPSRC university doctoral landscape award additional funding 2025 – Queen Mary University of London).

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

All the notes?

Boy listening to music by Olena from Pixabay

There are infinitely many musical notes, just like there are infinitely many colours. That matters if you are designing a new digital musical instrument. You have a lot more choice than on a piano!

Octaves 

Most Western music is divided equally into groups of 12 notes (‘octaves’) that musicians use. The gap between any two notes sounds the same. This is known as equal temperament tuning. 

Activity: Play the 12 notes 

You can play the 12 notes of an octave on the online piano https://bit.ly/pianoCS4FN. Play Middle C (marked with a red dot), then press each key in turn including the black keys. Play 12 notes and you have played the 12 notes of an octave.

Music as colour

Image by CS4FN

The rainbow picture shows there are many colours to pick from not just red, orange, yellow… A set of crayons would be enormous if it included every possible colour! Instead you get a selection just as in the picture: we picked 3 colours equally spaced apart: red, yellow and blue. Western music does the same thing with sound, picking 12 notes that sound equally spaced.

There are lots of other notes that you could sing within an octave. Traditional music often uses different sets of notes. The Arabic system divides an octave into 24 notes, for example. They have more ‘sound crayons’ to play with! You could even start singing on a low note and continually raise your pitch until you reached the higher note, like sweeping through every colour in a musical rainbow.

If you sing a note, then sing the same note but an octave higher (eg Middle C then the next highest C), your vocal cords are now vibrating twice as fast! The frequency of the top note is twice as high as the lower one. Your vocal cords doubled their speed.

By Jo Brodie and Paul Curzon, Queen Mary University of London

More on…

We have LOTS of articles about music, audio and computer science. Have a look in these themed portals for more:

The Music and AI pages are sponsored by the EPSRC (UKRI3024: DA EPSRC university doctoral landscape award additional funding 2025 – Queen Mary University of London).

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

Musical Algorithms

A treble clef with piano keys running round it
Image by OpenClipart-Vectors from Pixabay

How can a machine generate music? It needs an algorithm to follow: instructions to tell it what to do, step by step. Here are two simple games to play that compose a random tune by algorithm.

Writing Notes

We need a way to write notes. We use letters A to G as on a piano. They repeat all the way up the white keys, so after G comes different higher versions of A, B, C again. We will use notes running from what is called Middle C in the middle of the piano to the next C up. This is called an octave. We will call the two Cs, C1 and C2.

Piano keys

An octave on a piano marked as from C to the next C labelled as C1 and C2
Image (edited) by OpenClipart-Vectors from Pixabay

Game 1: Random Jumps

Roll two dice and add the numbers. Write down the note given in the table for Game 1, so if they add to 2 or 3 write down C1, if 4 write down D…If 7 then you get to roll again, and so on. Keep going until you have written 15 notes to make a tune of 15 notes.

Table for Game 1 showing dice rolls and notes 2 or 3 - C1 4 - D 5 - E 6 - F 7 - Roll again 8 - G 9 - A 10 - B 11 or 12 - C2
Game 1 by CS4FN

Game 2: Up and Down

The second algorithm uses one die. First write down C1 then roll the die and do what it says in the Game 2 table. Each new note is based on the last note. If you roll a 1 then write down D (the next note UP from C1). Rolling a 6 means add a pause in the tune (write a dash). If the roll takes you beyond either C then you bounce back: so rolling a 4 when you last wrote C1 means you write C1 again. Rolling 5 from C1 bounces you up to E. Continue until you have 15 notes.

Table for Game 2 showing die rolls and action 1 - UP 1 note 2 - UP 2 notes 3 REPEAT note 4 - DOWN 1 note 5 - DOWN 2 notes 6 - PAUSE
Game 2 by CS4FN

Play your tunes

Play your tunes on any instrument or use a free online piano (see https://bit.ly/pianoCS4FN).

Are they any good? Does either game give better tunes? 

Good music isn’t just random notes. That is why we pay composers to come up with the really good stuff! Both human and machine composers learn more complicated patterns of what makes good music.

What do you think of our musical masterpiece?

On Game 1 we rolled 6 4 8 8 8 | 5 9 4 9 6 | 5 6 9 9 10 so our tune is F D G G G | E A D A F | E F A A B

Make your tunes special!

See how on the Bach Google Doodle page.

A cloud of stars
Starburst by CS4FN

Here’s what our tune sounds like once harmonies have been added.

Could you improve your tunes by tweaking the notes? Some people use simple algorithms to spark human creativity like that. Rock legend David Bowie helped write a program he then used to write songs. It took random sentences from different places, split them in half and swapped the parts over to give him ideas for interesting lyrics. It was possibly the first algorithm to help write hit songs.

A ‘note’ on bias

Think about the numbers that are rolled and the number of different ways that each number can be produced. For example with two dice (let’s call them ‘left’ and ‘right’) you can make the number 9 twice by rolling a 5 with the left and 4 with the right, or 4 with the left and 5 with the right. Same with 6 and 3. There are only two ways to roll a 2 (both dice have to show 1) or a 3 (a 1 and a 2 or a 2 and a 1). This is baked in to the process and so will affect the notes that appear most often.

Jo Brodie and Paul Curzon, Queen Mary University of London

More on…

We have LOTS of articles about music, audio and computer science. Have a look in these themed portals for more:

The Music and AI pages are sponsored by the EPSRC (UKRI3024: DA EPSRC university doctoral landscape award additional funding 2025 – Queen Mary University of London).

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

The day the music didn’t die

An AI generated image of a skeleton wearing a red, blue and yellow sombrero hat, sitting on a stool while playing a guitar against a colourful red and yellow background.
Image by AiRebellion from Pixabay

Computer Scientists are working to support traditional music from around the world.

A seperewa is a traditional “harp-lute” musical instrument of the Akan people in Ghana, Africa. It has strings that are plucked a bit like a guitar. It is dying out because of the rise of western music. Researchers are now testing AIs that were trained on western music to see if they still work with such different seperewa music. They are also trying to understand exactly how this traditional music is different.

Protecting traditional instruments

Colonisers introduced European guitars to Ghana in the late 1800s and their sound began to influence and even replace seperewa music. Worried by this, in the mid-1900s people made recordings to preserve endangered seperewa music and to remind people what it sounds like. Ghanaian musicians are now reviving the seperewa, so we might continue to hear more of its lovely sound in future.

A view of a historical seperewa instrument side-on showing a large sounding box with strings attached to a neck, and stretched taut for playing.
A seperewa, adapted from a public domain image on Wikipedia.

AI to the rescue

A team of computer scientists and music experts have investigated recordings of seperewa music to see how well western AI tools can analyse that style of music, given it is tuned in a completely different way, so plays different notes to a western instrument.

First the team used one AI tool to separate the sounds of the seperewa from the singing. It struggled a bit and left some of the singing in the seperewa track and vice versa but overall did a good job,

They then used a different AI to analyse the sounds of the seperewa. The found that the seperewa music had its own, unique musical fingerprint, revealing a rich tapestry of sound that was clearly different from western music.

The research is helping to preserve a vital part of Ghanaian culture. It has shown in detail how their music is different to anything western and so that something unique and precious would be lost if it died out.

Jo Brodie and Paul Curzon, Queen Mary University of London

Watch …

Hear what a seperewa / seprewa sounds like at this YouTube video: The seprewa – the original African guitar [EXTERNAL]

More on…

We have LOTS of articles about music, audio and computer science. Have a look in these themed portals for more:

Getting technical…

The Music and AI pages are sponsored by the EPSRC (UKRI3024: DA EPSRC university doctoral landscape award additional funding 2025 – Queen Mary University of London).

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

Composing ancient Korean music

A cartoon drawing of a dragon in a Chinese style with a yellow body and a red tongue.
Image by OpenClipart-Vectors from Pixabay

600 years ago King Sejong the Great of Korea published ‘Hangul’, a new and improved writing system for his people. To celebrate he asked his court scholars to write an epic poem in Hangul, then asked his musicians to compose music to accompany it. The result was Yongbieocheonga, or ‘Songs of the Dragon Flying to Heaven’.

It was performed by musicians playing wind and stringed instruments. The musical instruments the AI composed for are Daegeum and Piri (wind instruments), Haegeum and Ajaeng (bowed string instruments) and Geomungo and Gayageum (plucked string instruments). Each instrument had its own melody written out for the musician to follow. Only one piece of the written music survives fully intact (it is still performed!). Melodies of other pieces of music have survived but only for a single instrument. That means those pieces can’t be played by a group of musicians because all the other harmonies are missing.

A team of computer scientists decided to recreate the missing 15th century Korean harmonies from just the single melodies (in the way the Bach Google Doodle does, see You’ll Be Bach!). They wanted to expand the ability of their AI tools to make sense of music beyond western music.

They first taught their AI musician to recognise Korean music written in Hangul. Then, it learnt which notes sound best played together by different instruments. Finally, to generate music that could be played, it matched melodies and rhythms. 

It created a melody for each different instrument. The researchers then asked Korean musicians to perform the whole piece and to judge how well the AI musician had done. Happily, they thought that the music worked well and sounded correct. They could perform it with only a few small tweaks. 

You can listen to one of the performances and find out more below.

Jo Brodie and Paul Curzon, Queen Mary University of London

Watch…

More on …

We have LOTS of articles about music, audio and computer science. Have a look in these themed portals for more:

Getting technical…

The Music and AI pages are sponsored by the EPSRC (UKRI3024: DA EPSRC university doctoral landscape award additional funding 2025 – Queen Mary University of London).

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

Separate your stems

Two cartoon faces, both purple, but the one on the left is a bluer purple and the one on the is a redder purple. Two speech bubbles say "I have more blue" for the bluer purple and "I have more red" for the redder purple.
Image by CS4FN

AI can unmix music and isolate vocals

Purple can be created by mixing together red and blue paint. You can probably tell which of the faces in the image has more blue and which has more red. Does music work the same way?

Your brain can recognise the red and blue in purple while still seeing it as a whole colour. Music is similar. When you listen to a song your ears and brain hear all the sounds at once. The singing, guitars, drums and keyboard parts are mixed together, but you can also focus on the singing, or the keyboards or ….

Computer scientists have gone a step further with Artificial Intelligence. By training AI tools on lots of different songs they have taught them to do “source separation” – unmixing a recorded song back into its separate bits. Those separate bits are called stems. It is like taking purple paint and unmixing it to give blue and red again!

A wide grey vase with two flowers in it (one red, one blue) at opposite ends of the vase with their stems definitely very separated.
Stems adapted from a plant pot image by HASSAN DYB from Pixabay.

“Not that kind of stem!”

Did you know?

Photographer Todd McLellan photographs gadgets he’s carefully taken apart, to show all the pieces (search the web for his “Things Come Apart”). When a piece of music is blended together and an AI separates it again it’s a bit more like trying to un-bake a cake!

Jo Brodie and Paul Curzon, Queen Mary University of London

More on…

We have LOTS of articles about music, audio and computer science. Have a look in these themed portals for more:

The Music and AI pages are sponsored by the EPSRC (UKRI3024: DA EPSRC university doctoral landscape award additional funding 2025 – Queen Mary University of London).

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

Jamming with JAM_BOT – an AI musician

A robot with a keyboard stomach playing the keyboard.
Image by CS4FN

Jordan Rudess is a rock keyboard player whose concerts sell out around the world. He works with a team of computer scientists at the MIT Media Labto make his synthesisers do amazing things. Together they created an AI musician called JAM_BOT to play with him on-stage.

Jordan’s bot learnt the different ways he plays by the team giving it lots of his music. It learned about the rhythms and melodies he uses. It could then compose its own versions of his music when prompted.

JAM_BOT AI plays along on-stage

Jordan also trained JAM_BOT to play with him. It could carry on playing music that Jordan had started, or create a backing track to music he was currently playing. Jordan was able to choose how JAM_BOT played with him on stage using the keys on his keyboard.

What happened next?

The resulting concert was a mix of performer and AI with a delighted audience (and computer science team). Afterwards Jordan said “It’s been pretty mind-blowing to create this tech-based version of myself – like looking into a real-time musical mirror.”

Jo Brodie and Paul Curzon, Queen Mary University of London

More on …

  • A model of virtuosity (2024) MIT News [EXTERNAL]
    • Acclaimed keyboardist Jordan Rudess’s collaboration with the MIT Media Lab culminates in live improvisation between an AI “jam_bot” and the artist.

We have LOTS of articles about music, audio and computer science. Have a look in these themed portals for more:

Getting Technical

The Music and AI pages are sponsored by the EPSRC (UKRI3024: DA EPSRC university doctoral landscape award additional funding 2025 – Queen Mary University of London).

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

You’ll be Bach! – create music with the Bach Google Doodle

A female conductor
From Classical music image by Евгения from Pixabay

The Bach Google Doodle is an AI musician which has learned the patterns in over 300 pieces of music from Johann Sebastian Bach, a famous 18th century German composer. The AI musician will take the notes you give it and suggest harmonies in Bach’s style. It takes a melody and creates backing melodies for different instruments that sound pleasing. 

Visit the Bach Google Doodle, put some notes together, press ‘Harmonize’ and see what you think of the result. If you don’t like its first suggestion you can press Harmonize to try again.

How to use it

Once on the page click the large play symbol (a white triangle) to open the doodle, and then again to run the intro demo (which you can skip on later visits).

Use your mouse to place notes at different positions on the five horizontal lines. If you hover over a note an X will appear so you can delete it and place it somewhere else. If you press and hold a note an option will appear to let you sharpen it (raise it by a semitone) or flatten it (lower it by a semitone). You can press the play icon to hear what your composition sounds like. Then press HARMONIZE to activate the AI. It will look at your piece of music and suggest the backing track (harmonies). You can then click a smiley or cross face if you like it or didn’t like it.

Hover your mouse cursor over all the other bits of the page too – there are lots of fun things to play with including some Easter eggs.

About the doodle

🎹 Celebrating Johann Sebastian Bach was Google’s first-ever AI-powered doodle and “is an interactive experience encouraging players to compose a two measure melody of their choice. With the press of a button, the Doodle then uses machine learning to harmonize the custom melody into Bach’s signature music style (or a Bach 80’s rock style hybrid if you happen to find a very special easter egg in the Doodle…”

▶️ You can also watch Google’s short video ‘Behind the Doodle’ on YouTube.

Jo Brodie and Paul Curzon, Queen Mary University of London

The Music and AI pages are sponsored by the EPSRC (UKRI3024: DA EPSRC university doctoral landscape award additional funding 2025 – Queen Mary University of London).

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

Listening to the machines

Clear sound image by Sunrise from Pixabay

In older films computers are sometimes shown doing a calculation while making lots of bleeps and bloops – sounds that indicate ‘something technical is happening’. In reality computers are generally very quiet (you might hear the sound of the fan, that’s just keeping everything cool) and they don’t normally make a peep. But computer scientists have been wondering if some sound added in might help people make sense of what’s going on.

People who use artificial intelligence tools often have no idea what is happening inside (it’s a bit hidden, like a ‘black box’), or even how much they can trust the results they produce. Explainable AI (“XAI”) is the idea that people should have a better understanding of how an AI tool has reached its answer.

Cars that are powered by batteries don’t have a physical engine so don’t make as much noise (other than the sound of the tyres on the road) but car manufacturers have added in artificial ‘engine sounds’ to make it easier for pedestrians and cyclists to know that a car is heading towards them. This is ‘sonification’, adding sounds that aren’t naturally there to make things more audible. Computer scientists have begun to consider whether it might be possible to sonify the way some language generating AI tools process and produce information, to make their inner workings easier for people to interpret. Whether that might be a microwave-style ‘ping’ to let you know when it’s done something, or a tuneful melody to accompany the AI’s processes remains to be seen…

Jo Brodie, Queen Mary University of London

Other added sounds

Can you think of other examples where a sound has been added (sonification) to help people make sense of something?

Examples include these, which are also helpful for visually impaired people

  • ‘This vehicle is turning left / reversing’ warnings from lorries
  • A lift / elevator making a ‘ping’ sound to alert you that it’s arrived
  • At pedestrian crossings the traffic lights might make an audible sound when the little red man goes green.

More on…

Related Magazine …

Subscribe to be notified whenever we publish a new post to the CS4FN blog.