Author: Jo Brodie

https://twitter.com/JoBrodie same on BlueSky and other pages. Old photo!

The art of animatronics, or how to build a believable dinosaur

Tyrannosaurus image by Eric Labayle from Pixabay

How do you create a full-sized dinosaur without a hint of computer graphics? The answer is through the amazing art of animatronics. Animatronics is a field of special effects that uses sculpture, mechanics, electronics and computer engineering to create life-size moving creatures for films and theme parks. They’re like puppets only much bigger, much smarter and much scarier. While today many film creatures are created using computer graphics in post-production, some filmmakers prefer to have their creatures ‘live’ on the set so the human actors have a real co-star to act along with. In a theme park, animatronics can put a weird creature, like a zombie pirate or a great white shark, right there and in your face. Famous movie animatronics stars include the shark in Jaws, the gigantic Spinosaurus in Jurassic Park III and the lovable alien in ET. How are these amazing effects created? Let’s get primeval with some state-of-the-art computer science.

On and off the drawing board

An animatronic creature starts out in life as a sketch on the drawing board. In some cases it’s a new creature-tastic idea thought up by the designer. In the case of dinosaurs, the sketches are created with the help of expert paleontologists. The sketches are then converted into a scale model, called a maquette. This scale model allows the designers to examine and correct their design plans before the big money is spent bringing the creature to full size ‘life’.

Growing up

Here’s where the model goes from the small to the large. The mini maquette is laser scanned, capturing all the detail of the model sculpture and feeding it into a computer aided design (CAD) software package. From this data whirring, computer-controlled blades automatically sculpt a full sized model using blocks of polyurethane foam. The blocks are assembled like a big 3D jigsaw, and sculptors add the extra fine detail. Now it’s big, it’s real and it’s ready for its screen test!

Pouring in the skin

If the full-sized version shows that star quality, it gets molded. Using the life-size model a set of moulds are made to allow the outside skin of the creature to be created. With the outside finished, now you have to think about the insides – namely, the skeleton, the mechanics of which depend on how the creature will be expected to move. Using a rough shape corresponding to the form of the core skeleton innards, the outer foam rubber skin can be poured in so that it only fills the negative space between the outside creature shape and in the inside skeleton. This reduces the weight of the skin and allows more believable, flexible movements.

More than just the bare bones

Skin done, now the technology really kicks in. The animatronics skeleton inside the creature is where all the smart stuff happens. It’s clever and custom made. It has to be – it’s the part that moves the outside skin to make it look believable. Attached around the main skeleton frame, which is often built with strong-but- light graphite and looks a lot like the real creature’s skeleton, we find the actuators. These are little clumps of clever computing that move the pieces around to make the creature look alive. Computer science abounds here, along with other state-of-the-art techniques. Mechanical and electronic engineering combined with computer-controlled motors are used to move small expressive bits like eyes, or to control the more heavy-duty hydraulic systems that move limbs. The systems may be pre-programmed for characteristic behaviours like blinking or swiping a claw. In essence the animatronics under the skin produce a gigantic remote controlled lifelike puppet for the director to play with.

Does my bum look big in this?

Putting the skin over the animatronics isn’t always easy. As each of the sections of foam rubber skin are added to the skeleton the construction team needs to check that the new bit of skin added doesn’t look too stretched, or too baggy with lots of unsightly flabby folds. One cunning way to help the image conscious creature is to use elastic bungee cords to connect areas of the skin to the frame. These act like tendons under the skin, stretching and bunching when it moves, and making the whole effect more relaxed and natural. Once the skin is on, it’s a quick paint job and the creature is ready for its close up. Action – grrrr -– shriek! Computer science takes centre stage.

Paul Curzon, Queen Mary University of London

More on …

Related Magazine …

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

The Decline and Fall of Ada: Who’s popular now?

Audience image by Pexels from Pixabay

Ada (the language), is not the big player on the programming block these days. In 1997 the DoD1 cancelled their rule that you had to use Ada when working for them. Developers in commerce had always found Ada hard to work with and often preferred other languages. There are hundreds of other languages used in industry and by researchers. How many can you name?

Here are some fun clues about different languages. Can you work out their names?
(Answers at the end.)

  1. A big snake that will squash you dead.
  2. A famous Victorian woman who worked with Babbage.
  3. A, B, __
  4. A, B, __ (ouch)
  5. A precious, but misspelled, thing inside a shell.
  6. A tiny person chatting.
  7. A beautiful Indonesian island.
  8. A french mathematician and inventor famous for triangles.

(You can try an online version of our quiz here)

Today, the most popular programming languages are, well we don’t know, because it depends when you are reading this! Because what is fashionable, what is new is always changing. Plus it’s hard to agree what ‘the most popular’ means for languages (and pop stars!). Is it the most lines of code in use today? The favorite language of developers? The language everyone is learning? In July 2015 one particular website rated programing languages using features such as number of skilled software engineers who can use the language; number of courses to learn the language; search engine queries on the language and came up with the order.

  • 1) Java
  • 2) C
  • 3) C++
  • 4) C#
  • 5) Python

Where is Ada? 30th out of 100s! The same website had shown Ada (the language) as 3rd top in 1985! What a fall from grace.

But have no fear, Ada still survives and lives on in millions of lines of avionics2, radar systems, space, shipboard, train, subway, nuclear reactors and DoD systems. Plus Ada is perhaps making a comeback. Ada 2012 is just being finalised, heralded by some as the next generation of engineering software with its emphasis on safety, security and reliability. So Ada meet Ada, it looks like you will be remembered and used for a long time still.

Github is a place where lots of programmers now develop and save their code. It encourages programmers to share their work. A kind of modern day, crowd sourced ‘mass of shared facts’ but coders would probably not say they did this just to ‘amuse their idle hours’. Popular coding tools on this platform are JavaScript. Java, Python, CSS, PHP, Ruby, C++. Ada doesn’t really feature, well not yet.

Jane Waite, Queen Mary University of London

  1. United States Department of Defense ↩︎
  2. Avionics (aviation electronics) includes all the electronics and software needed to fly aircraft safely. ↩︎

Related Magazine …

This article was originally published on page 19 of issue 20 of the CS4FN magazine. You can download a copy at the link below, and all of our previous magazine issues (free) here.

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

Answers to the quiz…

Answers: 1) Python 2) Ada 3) C 4) C# (C sharp) 5) Perl 6) Smalltalk 7) Java 8) Pascal

Victorian volunteers needed – the start of citizen science

Barnacles
Barnacles image by Mark Martins from Pixabay

What was Ada Lovelace thinking about when she wrote:
“If amateurs of either sex would amuse their idle hours with experiments on this subject, and would keep an accurate journal of their daily observations, we should have in a few years a mass of registered facts to compare with the observation of the scientific”.

Yes, crowdsourcing science experiments! Now we call it Citizen Science. She had just read a book by a Baron von Reichenbach on magnetism in which he had suggested a whole host of experiments, such as moving magnets up and down a person’s body, showing people magnets in the dark, and holding heavy and light magnets and asking them if they felt any sensations. She could see that he had some great ideas, but she was not convinced by his examples alone.

Ada was not the only Victorian to ask the general public for help collecting data. Charles Darwin, the Origin of Species man, wrote to gardeners, diplomats, army officers and scientists across the world asking for information about the plants they grew and the animals (including people) they saw. This all helped him build up the concrete evidence that natural selection was the way evolution works. People even sent him gifts of live animals in the post. A Danish gentleman sent him a parcel of live barnacles. When they did not arrive on time, Darwin, desperate to dissect the species, panicked and got ready to offer a reward in the Times newspaper. Luckily they arrived intact, fresh and not too smelly!

Today we might take part in the RSPB’s Big Garden Bird Watch1, contribute to a blog, ‘favourite’ or ‘like’ a post on social media or vote for your favorite performer in a talent show. We participate, and ‘amuse our idle hours’ sometimes in the pursuit of science, sometimes not. Public research is a big new topic, with governments and companies looking to use people power. Innovations such as shared mapping systems ask users to upload details about a place, add photographs, rectify mistakes. Wikipedia is sourced by volunteers, with other volunteers checking accuracy. Galaxy Zoo volunteers even found a whole new planet that orbits four stars!

What would Ada be asking us to research? Test your own DNA and send in the results? Measure air quality and keep a record on a central database? Build your own ‘find a barnacle’ app? But rather than writing a journal or sending a parcel of barnacles, you would log it on line, click a link or design your own survey. Ada’s computers are in on the act again.

Why not find a Citizen Science project on something you are interested in. Sometimes called public science or science outreach projects they might be run by local universities, museums, your council, charities or through crowdsourced internet projects such as www.zooniverse.org. Share what you do with others and spread Ada’s word to be a modern day volunteer.

Jane Waite, Queen Mary University of London

  1. 23-25 January 2026: RSPB Big Garden Birdwatch – “Spend an hour watching the birds in your patch, between 23 and 25 January, and record the birds t allhat land.” You can also get your school involved in the Big School’s Birdwatch 2026. If you’re reading this after 25 January 2026 make a note in your diary to remind you to check next year! ↩︎

Related Magazine …

This article was originally published on page 13 of issue 20 of the CS4FN magazine. You can download a copy at the link below, and all of our previous magazine issues (free) here.

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

Twenty one years ago we wondered what technology would still be here in the future (now!)

Clock waves image by Gerd Altmann from Pixabay

Back in 2005 we published Issues 1 and 2 of the CS4FN (Computer Science For Fun) magazine and there were two short articles in the 2nd issue – “Future proof” about the change from physical copies of music and films (such as CDs and DVDs) to listening and watching on streaming services and “What do you think is most likely to disappear next?” where we wondered which other technologies might still be around in the future.

Future Proof
Bill Gates believes CDs and DVDs have had it. It won’t be long before the whole back catalogue of music fits on a device in your pocket:
“It’s going even faster than we expected…Five years from now people will say ‘What’s a CD? Why did you have to go to the case and open something up and you couldn’t sequence it your own playlist way?’ That will be a thing of the past. Even videos in the future will either be on a disk in your pocket or over the Internet, and far more convenient for you.”

Bill Gates, Chairman and Chief Software Architect, Microsoft, speaking in 2005.

It’s certainly true that most music and films can be streamed but they may not feel as permanently available as physical copies. On the release of his film Oppenheimer film director Christopher Nolan said1 “There is a danger these days that if things only exist in the streaming version, they do get taken down”. Netflix UK recently told me that Star Trek: The Next Generation would disappear from its listings on the January 8th 2026. I’m sure it will return in the future, but perhaps I need to buy the box set of DVDs to catch up with all the adventures.

Though it does also depend on having the right technology to play a physical copy of the thing – would you know how someone could play a DVD, CD, VHS (video tape) or cassette (audio tape)? Records on vinyl have certainly been making a comeback too…

Our second short article was even shorter and asked readers to vote (in 2005)…

What do you think is most likely to disappear next?
(Or which of the following items might survive into the future?)

  • Fixed phones (land lines)
  • Cables
  • Written signatures
  • Loose change
  • Wrist-watches
  • Paper
  • Physical shops
  • Calculators
  • Radios

Possibly 21 years since 2005 is not quite far enough into a future where all of these have disappeared but you can certainly see that the way these things are used has changed significantly.

Imagine it’s now 21 years in the future (or 42 years since 2005)… pick one item that you think is no longer in use. Click the blue button containing the item that you think won’t be around in 2047.

What other technology/ies are you using today that someone born now might not recognise in 21 years time?

– By Jo Brodie, Paul Curzon and Peter McOwan, Queen Mary University of London

Further reading

  1. Oppenheimer and the resurgence of Blu-ray and DVDs: How to stop your films and music from disappearing (3 January 2024) BBC Culture, by Clare Thorp [EXTERNAL] ↩︎

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

A wearable robot – computer-powered exoskeletons

A blue beetle
Blue beetle image by Clker-Free-Vector-Images from Pixabay

Beetles are one of the most prolific species on the planet. As the famous geneticist J.B.S. Haldane is supposed to have said: God has an inordinate fondness for beetles. One of the reasons they are so successful is that, unlike us, their skeleton is outside their body, not inside! This kind of skeleton is called an exoskeleton. Humans are now trying to get in on the act. In the computer science version exoskeletons are robots that you wear.

Animal shells

All sorts of animals have evolved all sorts of different exoskeletons. We call the big ones shells. Many insects, like beetles, have exoskeletons. So do crabs, scorpions, snails and clams. Tortoises are particularly interesting as they have both an internal skeleton, like us, and a shell too.

Animals use exoskeletons for lots of reasons. Most obviously it protects them from predators. It can also help stop them drying out in the sun, and stop them getting wet in the rain. They are used by some animals for sensing the world, and help animals like locusts to jump. Some tortoises and armadillos use them for digging and other animals use them to feed. It’s not surprising, with so many uses that there are a lot of them about.

Human shells

Generally, exoskeletons seem like a pretty good idea! So it’s not surprising that we humans want them too. A suit of armour is actually just a simple version of an exoskeleton designed to protect a knight from ‘predators’. It’s not much different to a tortoise protected inside its shell. The difference to the ones humans make now is our modern exoskeletons are powered and controlled by computers. They really are a robot you wear. They react to your movements.

As with animals’ shells, powered exoskeletons help humans do all sorts of things, not just act as armour. By being powered they give us extra strength, allowing us to lift weights far heavier than we could otherwise, and can turn our small movements in to larger ones. That means they can, for example, help people who have problems moving about to walk (see ‘The Wrong Trousers’) or help nurses lift patients in and out of bed. They are used by surgeons to do operations when they are in a different place to the patient, removing the shakiness of their hands, and by rescue workers working in dangerous situations. There are even ones designed to help astronauts exercise in space. They make movement harder rather than easier to force them to exercise despite the lower gravity.

All in all, copying beetles, but with our own computing twist, seems like a pretty good idea.

Paul Curzon, Queen Mary University of London

More on …

Related Magazine …

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

Hear and … their magic square

A magic three by three square with the numbers 2, 9 and 4 in the top row, 7, 5 and 3 in the middle row and 6, 1 and 8 in the bottom row. Each row, column and the two diagnonals add up to 15.
Image by CS4FN

Victorian Computer Scientists, Ada Lovelace and Charles Babbage were interested in Magic Squares. We know this because a scrap of paper with mathematical doodles and scribbles on it in their handwriting has been discovered, and one of the doodles is a magic square like this one. In a magic square all the rows, columns and diagonals magically add to the same number. At some point, Ada and Charles were playing with magic squares together. Creating magic squares sounds hard, but perhaps not with a bit of algorithmic magic.

The magical effect

For this trick you ask a volunteer to pick a number. Instantly, on hearing it, you write out a personal four by four magic square for them based on that number. When finished the square contents adds to their chosen number in all the usual ways magic squares do. An impressive feat of superhuman mathematical skills that you can learn to do most instantly.

Making the magic

To perform this trick, first get your audience member to select a large two digit number. It helps if it is a reasonably large number, greater than 20, as you’re going to need to subtract 20 from it in a moment. Once you have the number you need to do a bit of mental arithmetic. You need an algorithm – a sequence of steps – to follow that given that number guarantees that you will get a correct magic square.

For our example, we will suppose the number you are given is 45, though it works with any number.

Let’s call the chosen number N (in our example: N is 45). You are going to calculate the following four numbers from it: N-21, N-20, N-19 and N-18, then put them in to a special, precomputed magic square pattern.

The magic algorithm

Sums like that aren’t too hard, but as you’ve got to do all this in your head, you need a special algorithm that makes it really easy. So here is an easy algorithm for working out those numbers.

This four by four magic square contains the calculations needed to install the numbers in the correct positions so that the magic square will work with any large two digit number
Image by CS4FN.
  1. Start by working out N – 20. Subtracting 20 is quite easy. For our example number of 45, that is 25. This is our ‘ROOT’ value that we will build the rest from.
  2. N-19. Just add 1 to the root value (ROOT + 1). So 25 + 1 gives 26 for our example.
  3. N-18. Add 2 to the root value (ROOT + 2). So 25 + 2 gives 27.
  4. N-21. Subtract 1 from the root value (ROOT – 1). So 25 – 1 gives 24.
  5. Having worked out the 4 numbers created form the original chosen number, N, you need to stick them in the right place in a blank magic square, along with some other numbers you need to remember. It is the pattern you use to build your magic square from. It looks like the one to the right. To make this step easy, write this pattern on the piece of paper you write the final square on. Write the numbers in light pencil, over-writing the pencil as you do the trick so no-one knows at the end what you were doing.

A square grid of numbers like this is an example of what computer scientists call a data structure: a way to store data elements that makes it easy to do something useful: in this case making your friends think you are a maths superhero.

When you perform this trick, fill in the numbers in the 4 by 4 grid in a random, haphazard way, making it look like you are doing lots of complicated calculations quickly in your head.

Finally, to prove to everyone it is a magic square with the right properties, go through each row, column and diagonal, adding them up and writing in the answers around the edge of the square, so that everyone can see it works.

The final magic square for chosen number 45

So, for our example, we would get the following square, where all the rows, columns and diagonals add to our audience selected number of 45.

This four by four magic square is the result of taking the chosen number 45 and performing the sequence of calculations (the algorithm) using it as 'N'.
Image by CS4FN.

Why does it work?

If you look at the preset numbers in each row, column and diagonal of the pattern, they have been carefully chosen in advance to add up to the number being subtracted from N on those lines. Try it! Along the top row 1 + 12 + 7 = 20. Down the right side 11 + 5 + 4 = 20.

Do it again?

Of course you shouldn’t do it twice with the same people as they might spot the pattern of all the common numbers…unless, now you know the secret, perhaps you can work out your own versions each with a slightly different root number, calculated first and so a different template written lightly on different pieces of paper.

Peter McOwan and Paul Curzon, Queen Mary University of London

More on …

Scéalextric Stories

Woman archer
Woman archer image by Николай Ивненко from Pixabay

If you watch a lot of movies you’ve probably noticed some recurring patterns in the way that popular cinematic stories are structured. Every hero or heroine needs a goal and a villain to thwart that goal. Every goal requires travel along a path that is probably blocked with frustrating obstacles. Heroes may not see themselves as heroes, and will occasionally take the wrong fork in the path, only to return to the one true way before story’s end. We often speak of this path as if it were a race track: a fast-paced story speeds towards its inevitable conclusion, following surprising “twists” and “turns” along the way. The track often turns out to be a circular one, with the heroine finally returning to the beginning, but with a renewed sense of appreciation and understanding. Perhaps we can use this race track idea as a basis for creating stories.

Building a track

If you’ve ever played with a Scalextric set, you will know that the curviest tracks make for the most dramatic stories, by providing more points at which our racing cars can fly off at a tight bend. In Scalextric you build your own race circuits by clicking together segments of prefabricated track, so the more diverse the set of track parts, the more dramatic your circuit can be. We can think of story generation as a similar kind of process. Imagine if you had a large stock of prefabricated plot segments, each made up of three successive bits of story action. A generator could clip these segments together to create a larger story, by connecting the pieces end-to-end. To keep the plot consistent we would only link up sections if they have overlapping actions. So If D-E-F is a segment comprising the actions D, E, and F, we could create the story B-C-D-E-F-G-H by linking the section B-C-D on to the left of D-E-F and F-G-H on its right.

Use a kit

At University College Dublin (UCD) we have created a set of rich public resources that make it easy for you to build your own automated story generator. We call the bundle of resources Scéalextric, from scéal (the Irish word for story) and Scalextric. You can download the Scéalextric resources from our Github but an even better place to start is our blog for people who want to build creative systems of any kind, called Best Of Bot Worlds.

In Artificial Intelligence we often represent complex knowledge structures as ‘graphs’. These graphs consists of lots of labeled lines (called edges) that show how labeled points (called nodes) are connected. That is what our story pieces essentially are. We have several agreed ways for storing these node-relation-node triples, with acronyms hiding long names, like XML (eXtensible Markup Language), RDF (Resource Description Framework) and OWL (Web Ontology Language), but the simplest and most convenient way to create and maintain a large set of story triples is actually just to use a spreadsheet! Yes, the boring spreadsheet is a great way to store and share knowledge, because every cell lies at the intersection of a row and a column. These three parts give us our triples.

Scéalextric is a collection of easy-to-browse spreadsheets that tell a machine how actions connect to form action sequences (like D-E-F above), how actions causally interconnect to each other (via and, then, but), how actions can be “rendered” in natural idiomatic English, and so on.

Adding Character

Automated storytelling is one of the toughest challenges for a researcher or hobbyist starting out in artificial intelligence, because stories require lots of knowledge about causality and characterization. Why would character A do that to character B, and what is character B likely to do next? It helps if the audience can identify with the characters in some way, so that they can use their pre-existing knowledge to understand why the characters do what they do. Imagine writing a story involving Donald Trump and Lex Luthor as characters: how would these characters interact, and what parts of their personalities would they reveal to us through their actions?

Scéalextric therefore contains a large knowledge-base of 800 famous people. These are the cars that will run on our tracks. The entry for each one has triples describing a character’s gender, fictive status, politics, marital status, activities, weapons, teams, domains, genres, taxonomic categories, good points and bad points, and a lot more besides. A key challenge in good storytelling, whether you are a machine or a human, is integrating character and plot so that one informs the other.

A Twitterbot plot

Let’s look at a story created and tweeted by our Twitterbot @BestOfBotWorlds over a series of 12 tweets. Can you see where the joins are in our Scéalextric track? Can you recognize where character-specific knowledge has been inserted into the rendering of different actions, making the story seem funny and appropriate at the same time? More importantly, can you see how you might connect the track segments differently, choose characters more carefully, or use knowledge about them more appropriately, to make better stories and to build a better story-generator? That’s what Scéalextric is for: to allow you to build your own storytelling system and to explore the path less trodden in the world of computational creativity. It all starts with a click.

An unlikely tale generated by the Twitter storybot.

Tony Veale, University College Dublin

Further reading

Christopher Strachey came up with the first example of a computer program that could create lines of text (from lists of words). The CS4FN developed a game called ‘Program A Postcard’ (see below) for use at festival events.

The algorithm that could not speak its name
I wandered lonely as a mass of dejected vapour – try some AI poetry
Program a postcard

Related Magazine …

Clapping Music

Peruvian women dancing and clapping in Lima,
Peruvian women dancing and clapping in Lima, Peru.
Image by Alex Phillc from Pixabay

“Get rhythm when you get the blues” – as Country legend Johnny Cash’s lyrics suggest, rhythm cheers people up. We can all hear, feel and see it. We can clap, tap or beatbox. It comes naturally, but how? We don’t really know. You can help find out by playing a game based on some music that involves nothing but clapping. If you were one of the best back in 2015, you could have been invited to play live with a London orchestra.

We can all play a rhythm both using our bodies and instruments, though maybe for most of us with only a single cowbell, rather than a full drum kit. By performing simple rhythms with other people we can make really complex sounds, both playing music and playing traditional clapping games. Rhythm matters. It plays a part in social gatherings and performance in cultures and traditions across the world. It even defines different types of music from jazz to classical, from folk to pop and rock.

Lots of people with a great sense of rhythm, whether musicians or children playing complex clapping games in the playground, have never actually studied how to do it though. So how do we learn rhythm? Our team based at Queen Mary, joined up with the London Sinfonietta chamber orchestra and app developers Touch Press, to find out, using music called Clapping Music.

Clapping Music is a 4-minute piece by the minimalist composer Steve Reich. The whole thing is based on one rhythmic pattern that two people clap together. One person claps the pattern without changing it – known as the static pattern. The other changes the pattern, shifting the rhythm by one beat every twelve repetitions. The result is an ever-changing cycle of surprisingly complicated rhythms. In spite of it’s apparent simplicity, it’s really challenging to play and has inspired all sorts of people from rock legend David Bowie to the virtuoso, deaf percussionist Dame Evelyn Glennie. You can learn to play Clapping Music and help us to understand how we learn rhythm at the same time.

Our team created a free game for the iPhone and iPad also called Clapping Music. You play for real against the static pattern. To get the best score you must keep tapping accurately as the pattern changes, but stay in step with the static rhythm. It’s harder than it sounds!

We analysed the anonymous gameplay data, together with basic information about the people playing like their age and musical experience. By looking at how people progress though the game we explored how people of different ages and experience develop rhythmic skills.

It has led to some interesting computer science to design the algorithms that measure how accurate a person’s tapping is. It sounds easy but actually is quite challenging. For example, we don’t want to penalise someone playing the right pattern slightly delayed more than another person playing completely the wrong pattern. It has also thrown up questions about game design. How do we set and change how difficult the game is? Players, however skillful, must feel challenged to improve, but it must not be so difficult that they can’t do it.

You don’t need to be a musician to play, in fact we would love as many people as possible to download it and get tapping and clapping! High scorers were invited to take part in live performance events on stage with members of the London Sinfonietta back in 2015. Get the app, get tapping, get rhythm (and have some fun – you won’t get the blues)!

– by Marcus Pearce and Samantha Duffy, Queen Mary University of London

Updated from the archive

This post was originally published in our CS4FN magazine (issue 19) in 2015, so the tense has been updated to reflect that it’s now 2025.

More on …

Getting Technical

Related Magazine …

Shh! Can you hear that diagram?

Network image by Alexandra_Koch from Pixabay

What does a diagram sound like? What does the shape of a sound feel like? Researchers at Queen Mary, University of London have been finding out.

At first sight listening to diagrams and feeling sounds might sound like nonsense, but for people who are visually impaired it is a practical issue. Even if you can’t see them, you can still listen to words, after all. Spoken books were originally intended for partially-sighted people, before we all realised how useful they were. Screen readers similarly read out the words on a computer screen making the web and other programs accessible. Blind people can also use touch to read. That is essentially all Braille is, replacing letters with raised patterns you can feel.

The written world is full of more than just words though. There are tables and diagrams, pictures and charts. How does a paritally-sighted person deal with them? Is there a way to allow them to work with others creating or manipulating diagrams even when each person is using a different sense?

That’s what the Queen Mary researchers, working with the Royal National Institute for the Blind and the British Computer Association of the Blind explored. Their solution was a diagram editor with a difference. It allows people to edit ‘node-and-link’ diagrams: like the London underground map, for example, where the stations are the nodes and the links show the lines between them. The diagram editor converts the graphical part of a diagram, such as shapes and positions, into sounds you can listen to and textured surfaces you can feel. It allows people to work together exploring and editing a variety of diagrams including flowcharts, circuit diagrams, tube maps, mind maps, organisation charts and software engineering diagrams. Each person, whether fully sighted or not, ‘views’ the diagram in the way that works for them.

The tool combines speech and non-speech sounds to display a diagram. For example, when the label of a node is spoken, it is accompanied by a bubble bursting sound if it’s a circle, and a wooden sound if it’s a square. The labels of highlighted nodes are spoken with a higher pitched voice to show that they are highlighted. Different types of links are also displayed using different sounds to match their line style. For example, the sound of a straight line is smoother than that of a dashed line. The idea for arrows came from listening to one being drawn on a chalk board. They are displayed using a short and a long sound where the short sound represents the arrow head, and the long sound represents its tail. Changing the order they are presented changes the direction of the arrow: either pointing towards or away from the node.

For the touch part, the team use a PHANTOM Omni haptic device, which is a robotic arm attached to a stylus that can be programmed to simulate feeling 3D shapes, textures and forces. For example, in the diagram editor nodes have a magnetic effect: if you move the stylus close to one the stylus gets pulled towards it. You can grab a node and move it to another location, and when you do, a spring like effect is applied to simulate dragging. If you let it go, the node springs back to its original location. Sound and touch are also integrated to reinforce each other. As you drag a node, you hear a chain like sound (like dragging a metal ball chained to a prisoner?!). When you drop it in a new location, you hear the sound of a dart hitting a dart board.

The Queen Mary research team tried out the editor in a variety of schools and work environments where visually impaired and sighted people use diagrams as part of their everyday activities and it seemed to work well. It’s free to download so why not try it yourself. You might see diagrams in a whole new light.

Paul Curzon, Queen Mary University of London

More on…

Related Magazine …

Crystal ball coupons – what your data might be giving away

Crystal ball
Crystal ball image by Silvia Hartmann from Pixabay

Big companies know far more about you than you think. You have very little privacy from their all-seeing algorithms. They may even have worked out some very, very personal things about you, that even your parents don’t know…

An outraged father in Minneapolis stormed into a supermarket chain complaining that his school-aged daughter was being sent coupons for baby clothes. The shop manager apologised … but later they found there was no mistake in the tiny tot offers. The teenager was expecting a baby but had not told her father. Her situation was revealed not by a crystal ball but by an algorithm. The shop was using Big Data processing algorithms that noticed patterns in her shopping that they had linked to “pregnant”. They had even worked out her likely delivery date. Her buying habits had triggered targeted marketing.

Algorithms linked her shopping patterns to “pregnant”

When we use a loyalty card or an online account our sales activity is recorded. This data is added to a big database, with our details, the time, date, location and products bought (or browsed). It is then analysed. Patterns in behaviour can be tracked, our habits, likes, dislikes and even changes in our personal situation deduced, based on those patterns. Sometimes this seems quite useful, other times a bit annoying, it can surprise us, and it can be wrong.

This kind of computing is not just used to sell products, it is also used to detect fraud and to predict where the next outbreak of flu will happen. Our banking behaviour is tracked to flag suspicious transactions and help stop theft and money laundering. When we search for ‘high temperature’ our activity might be added to the data used to predict flu trends. However, the models are not always right as there can be a lot of ‘noise’ in the data. Maybe we bought baby clothes as a present for our aunt, and were googling temperatures because we wanted to go somewhere hot for our holiday.

Whether the predictions are spot on or not is perhaps not the most important thing. Maybe we should be considering whether we want our data saved, mined and used in these ways. A predictive pregnancy algorithm seems like an invasion of privacy, even like spying, especially if we don’t know about it. Predictive analytics is big; big data is really big and big business wants our data to make big profits. Think before you click!

Jane Waite, Queen Mary University of London (now at Raspberry Pi)

More on …

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos