Games are becoming ever more realistic. Now, thanks to the work of Joshua Reiss’s research team and their spinout company, Nemisindo, it’s not just the graphics that are amazing, the sound effects can be too.
There has been a massive focus over the years in improving the graphics in games. We’ve come along way from Pong and its square ball and rectangular paddles. Year after year, decades after decade, new algorithms, new chips and new techniques have been invented that combined with the capabilities of ever faster computers, have meant that we now have games with realistic, real-time graphics immersing us in the action as we play. And yet games are a multimedia experience and realistic sounds matter too if the worlds are to be truly immersive. For decades film crews have included whole teams of Foley editors whose job is to create realistic everyday sounds (check out the credits next time you watch a film!). Whether the sound is of someone walking on a wooden floor in bare feet, walking on a crunchy path,opening thick, plush curtains, or an armoured knight clanging their way down a bare, black cliff, lots of effort goes into getting the sound just right.
Game sound effects are currently often based on choosing sounds from a sound library, but games, unlike films, are increasingly open. Just about anything can happen and make a unique noise while doing so. The chances of the sound library having all the right sounds get slimmer and slimmer.
Suppose a knight character in a game drops a shield. What should it sound like? Well, it depends on whether it is a wooden shield or a metal one. Did it land on its edge or fall horizontally, and was it curved so it rang like a bell? Is the floor mud or did it hit a stone path? Did it bounce or roll? Is the knight in an echoey hall, on a vast plain or clambering down those clanging cliffs…
All of this is virtually impossible to get exactly right if you’re relying on a library of sound samples. Instead of providing pre-recorded sounds as sound libraries do, the software of Josh and team’s company Nemisindo (which is the Zulu word for ‘sound effects’), create new sounds from scratch exactly when they are needed and in real time as a game is played. This approach is called “procedural audio technology”. It allows the action in the game itself to determine the sounds precisely as the sounds are programmed based on setting options for sounds linked to different action scenarios, rather than selecting a specific sound. Aside from the flexibility it gives, this way of doing sound effects gives big advantages in terms of memory too: because sounds are created on the fly, large libraries of sounds no longer need to be stored with the program.
Nemisindo’s new software provides generated procedural sounds for the Unreal game engine allowing anyone building games using the engine to program a variety of action scenarios with realistic sounds tuned to the situation in their game as it happens…
In future, if that Knight steps off the stone path just as she drops her shield the sound generated will take the surface it actually lands on into account…
Procedural sound is the future of sound effects so just as games are now stunning visually, expect them in future to become ever more stunning to listen to too. As they do the whole experience will become ever more immersive… and what works for games works for other virtual environments too. All kinds of virtual worlds just became a lot more realistic. Getting the sound exactly right is no longer a barrier to a perfect experience.
Victorian, William Stanley Jevons was born in Liverpool in 1835. He was famous in his day as an economist and his smashhit book ‘The Coal Question’ drew the nation’s attention to the reduction in Britain’s coal supplies. He was the first economist to raise the issue of the ecological impact of economics.Jevons had other strings to his bow though and one of the strangest for the time if also incredibly forward thinking was his 1869 “logic piano”: a device that looked a little like a piano but that “played” logic.
Jevons was fascinated with logic and reasoning. He believed you could start with one thing (a premise) and from this work through a chain of reasoning to the conclusion. He thought that this could be done for everything. This was based on a principle he espoused of “the substitution of similars”: essentially reasoning based on the idea that “Whatever is true of a thing is true of its like”, For example, If Pharaohs are gods and Rameses is a Pharaoh (so is one of the things “like” a Pharaoh) then you can conclude Rameses is a god. He built on, and combined the ideas of, the Ancient Greeks with the then new ideas of George Boole, that we now call Boolean logic.
Boolean logic is based on a system of algebra using only the values of true and false (or 1 and 0), with operations corresponding to logical operations such as AND and OR that turn true/false values into new true/false values. This is the logic upon which computers are founded. A key idea was that you can abstract away from actual statements about truth in the real world and just replace them with variables that can stand for anything. Boole had laboriously shown using his logic how new abstract facts could be deduced from existing ones. Jevons realised that when reasoning was thought of like this, it became a mechanical process…and that meant a machine could do it.
His contemporary, Charles Babbage had been working on the idea of building mechanical “computers” but Babbage’s fundamental idea was that machines could do calculation. Jevon’s idea was slightly different and more fundamental: that machines could do logical reasoning, deducing new facts from existing ones. This was an idea that eventually came to fruition in the 20th century with the development of theorem provers where computers were programmed to do complex logical reasoning, even working on building up the whole of mathematics by proving ever more theorems from a few starting facts.
So, (with the help of an unknown craftsperson), Jevons set about designing and building his wooden Logic Piano. The idea was that you could put in the premises, the basic facts, by playing the keys of the “piano”. It would then mechanically apply his reasoning rules to discover all conclusions that could be deduced, altering its conclusion with each new fact added, The keys moved rods and levers that made logical facts appear on (or disappear from) the “display” of the machine – essentially facts on the rods appeared in slots cut into the back of the piano.
The kind of logical problem his machine worked with are Syllogisms (see Superhero Syllogisms). They were invented by the Ancient Greeks who were very good at logic. A syllogism is just a common pattern that combines facts where you figure out a conclusion only using the facts supplied, slotted into a template. For example, if we know facts 1 and 2 in the following template (where you can swap in anything for A, B and C) then we can create a new fact as shown.
Image by Paul Curzon
So let’s replace A with the word superheroes, B with fight crime and C with my favourite superhero, Ghost Girl. If we put them in to the template above we get a new “fact” deduced from two existing ones:
Image by Paul Curzon
Image by Paul Curzon
In the piano, a set of rods acted as truth tables, each one giving a true or false values for each variable. So imagine a piano that could deal with two variables A and B. Each of A and B can have two different values true and false, so there are four possibilities (so four rods):
(A, B);
(A, NOT B);
(NOT A, B);
(NOT A, NOT B)
where we write A to mean the assertion A is true and NOT A (or ~A in the picture) to mean A is false. Each rod represents a possible state of the world.
Let’s look at a simple example. Suppose A stands for the statement “Paul is a programmer” and B stands for “Safia is a programmer”, then the possibilities (if we know no specific facts) are
(Paul is a programmer, Safia is a programmer) : (A, B)
(Paul is a programmer, Safia is NOT a programmer) : (A, NOT B)
(Paul is NOT a programmer, Safia is a programmer) : (NOT A, B)
(Paul is NOT a programmer, Safia is NOT a programmer) : (NOT A, NOT B)
Image by Paul Curzon
Each rod in Jevon’s piano had one of the possibilities on, so each rod represented a possible state of the world being reasoned about. At the start all the rods were visible, showing that nothing specific was yet known.
The point is that these represent all possible states of the world about Paul and Safia and whether they are programmers or not. If we know nothing more then all we can say is that all the pairs of facts are a possibility: all are possible states of the world.
The piano worked by essentially leaving displayed or hiding each rod’s state as new facts were keyed in. (See the video at the end which includes a detailed explanation by expert David E Dunning on the detail of how it did this step by step). Initially all the possibilities are displayed as above. If we add a new fact that we have discovered or wish to assume, say that “Safia IS a programmer” (in terms of the piano, press the B key corresponding to the fact B is true), then doing so removes all states of the world where Safia is NOT a programmer. The piano, therefore, hides all the rods that include the assertion representing “Safia is NOT a programmer” (all those with (NOT B) on them) . We are left with two alternatives:
Image by Paul Curzon
(Paul is a programmer, Safia is a programmer) : (A, B)
(Paul is NOT a programmer, Safia is a programmer) : (NOT A, B)
The mechanics of the machine meant that those facts would remain a possibilities (reading down the rods) but pushing the keys for that assertion would have moved the other rods, so hide their state. In doing so, the piano has deduced from the fact B that the possible conclusions are A AND B is true or NOT A AND B is true.
With three variables instead of two the machine would be able to deal with more complex situations – there are then 8 possibilities so 8 rods representing the 8 different states. .
Image by Paul Curzon
Let A represent superheroes, (so NOT A represents those people who are not superheroes), B represents those people who fight crime and C with a person being Ghost Girl. Suppose we are considering some, at the moment, random person we know nothing about. The possibilities about them are:
(Is a superhero, does fight crime, is Ghost Girl) : (A, B, C)
(Is a superhero, does fight crime, is NOT Ghost Girl) : (A, B, NOT C)
(Is a superhero, does NOT fights crime, is Ghost Girl) : (A, NOT B, C)
(Is a superhero, does NOT fight crime, is NOT Ghost Girl) : (A, NOT B, NOT C)
(Is NOT a superhero, does fight crime, is Ghost Girl) : (NOT A, B, C)
(Is NOT a superhero, does fight crime, is NOT Ghost Girl) : (NOT A, B, NOT C)
(Is NOT a superhero, does NOT fights crime, is Ghost Girl) : (NOT A, NOT B, C)
(Is NOT a superhero, does NOT fight crime, is NOT Ghost Girl) : (NOT A, NOT B, NOT C)
If we put the fact about them into the piano that ALL superheroes fight crime (ALL A are B) then we remove all rods where A is true but B is different so false (a superhero who doesn’t fight crime) as in this world, that is impossible.
Image by Paul Curzon
(Is a superhero, does fight crime, is Ghost Girl) : (A, B, C)
(Is a superhero, does fight crime, is NOT Ghost Girl) : (A, B, NOT C)
(Is NOT a superhero, does fight crime, is Ghost Girl) : (NOT A, B, C)
(Is NOT a superhero, does fight crime, is NOT Ghost Girl) : (NOT A, B, NOT C)
(Is NOT a superhero, does NOT fights crime, is Ghost Girl) : (NOT A, NOT B, C)
(Is NOT a superhero, does NOT fight crime, is NOT Ghost Girl) : (NOT A, NOT B, NOT C)
Then we add the fact that Ghost Girl is a superhero (C is a A) so remove all those rods where Ghost Girl is not a superhero (ie NOT A, C):
Image by Paul Curzon
(Is a superhero, does fight crime, is Ghost Girl) : (A, B, C)
(Is a superhero, does fight crime, is NOT Ghost Girl) : (A, B, NOT C)
(Is NOT a superhero, does fight crime, is NOT Ghost Girl) : (NOT A, B, NOT C)
(Is NOT a superhero, does NOT fight crime, is NOT Ghost Girl) : (NOT A, NOT B, NOT C)
We have deduced (the first possible state) that if the person we are interested in is Ghost girl then she is a superhero. We are also left with other possibilities too. If the person we are considering is not actually Ghost Girl then they may or may not fight crime and may or may not be a superhero!
If we add in an additional fact that the person we are thinking of IS actually Ghost Girl then we remove those extra rods so possibilities and get
Image by Paul Curzon
(Is a superhero, does fight crime, is Ghost Girl) : (A, B, C)
Ghost Girl is a superhero who does fight crime! We knew she was Ghost Girl and was a superhero but using the piano we have now deduced that she does fight crime. The machine has deduced the syllogism we gave at the start.
IF ALL superheroes fight crime AND
Ghost girl is a superhero
THEN
Ghost girl fights crime.
The actual piano dealt with 4 variables (A, B, C, D) so had 16 rods representing the 16 different combinations. It also included keys to indicate the end of a conjecture, a key for IS A, and more to allow specific assertions to be input. The mechanism then hid rods automatically based on the facts entered. To use it, you did as we did: create a table of what A, B, C and D stand for (this is done outside the machine), enter the facts you want to reason about, and it then displayed all the possible states that remained in terms of A, B, C and D. Then, by seeing what each of the variables stood for in the table, you could convert that answer back into a deduced fact about the real world that you were interested in.
Amazingly, (after a first failed attempt) it did work. It is similar in idea to modern day theorem provers which are used to verify properties of safety-critical computer designs that must npot have bugs. Of course, being small and woody, the logic piano couldn’t solve every thing but then it turns out that was always an impossible dream. Even modern computers (and human mathematicians) have fundamental limits in what they can do (which is another story). The logic piano was a rather amazing, if woody, start to the area of automated theorem proving, though.
– Paul Curzon, Queen Mary University of London
Make a paper logic piano
Here is a kit to make a paper or card logic piano of your own (actually more like Jonons’ logic abacus the piano was a mechanised version of):
A Talk by David E Dunning at the Imagining AI conference about William Stanley Jevons and the logic piano including at the end a more detailed explanation of how pressing keys actually caused rods to by hidden in a series of steps. See also his article about it for the Computer History Museum.
Mike Lynch was one of Britain’s most successful entrepreneurs. An electrical engineer, he built his businesses around machine learning long before it was a buzz phrase. He also drew heavily on a branch of maths called Bayesian statistics which is concerned with understanding how likely, even apparently unlikely, things are to actually happen. This was so central to his success that he named his super yacht, Bayesian, after it. Tragically, he died on the yacht, when Bayesian sank in a freak, extremely unlikely, accident. The gods of the sea are cruel.
Mike started his path to becoming an entrepreneur at school. He was interested in music, and especially the then new but increasingly exciting, digital synthesisers that were being used by pop bands, and were in the middle of revolutionising music. He couldn’t afford one of his own, though, as they cost thousands. He was sure he could design and build one to sell more cheaply. So he set about doing it.
He continued working on his synthesiser project as a hobby at Cambridge University, where he originally studied science, but changed to his by-then passion of electrical engineering. A risk of visiting his room was that you might painfully step on a resistor or capacitor, as they got everywhere. That was not surprising giving his living room was also his workshop. By this point he was also working more specifically on the idea of setting up a company to sell his synthesiser designs. He eventually got his first break in the business world when chatting to someone in a pub who was in the music industry. They were inspired enough to give him the few thousand pounds he needed to finance his first startup company, Lynett Systems.
By now he was doing a PhD in electrical engineering, funded by EPSRC, and went on to become a research fellow building both his research and innovation skills. His focus was on signal processing which was a natural research area given his work on synthesisers. They are essentially just computers that generate sounds. They create digital signals representing sounds and allow you to manipulate them to create new sounds. It is all just signal processing where the signals ultimately represent music.
However, Mike’s research and ideas were more general than just being applicable to audio. Ultimately, Mike moved away from music, and focussed on using his signal processing skills, and ideas around pattern matching to process images. Images are signals too (resulting from light rather than sound). Making a machine understand what is actually in a picture (really just lots of patches of coloured light) is a signal processing problem. To work out what an image shows, you need to turn those coloured blobs into lines, then into shapes, then into objects that you can identify. Our brains do this seamlessly so it seems easy to us, but actually it is a very hard problem, one that evolution has just found good solutions to. This is what happens whether the image is that captured by the camera of a robot “eye” trying to understand the world or a machine trying to work out what a medical scan shows.
This is where the need for maths comes in to work out probabilities, how likely different things are. Part of the task of recognising lines, shapes and objects is working out how likely one possibility is over another. How likely is it that that band of light is a line, how likely is it that that line is part of this shape rather than that, and so on. Bayesian statistics gives a way to compute probabilities based on the information you already know (or suspect). When the likelihood of events is seen through this lens, things that seem highly unlikely, can turn out to be highly probably (or vice versa), so it can give much more accurate predictions than traditional statistics. Mike’s PhD used this way of calculating probabilities even though some statisticians disdained it. Because of that it was shunned by some in the machine learning community too, but Mike embraced it and made it central to all his work, which gave his programs an edge.
While Lynett Systems didn’t itself make him a billionaire, the experience from setting up that first company became a launch pad for other innovations based on similar technology and ideas. It gave him the initial experience and skills, but also meant he had started to build the networks with potential investors. He did what great entrepreneurs do and didn’t rest on his laurels with just one idea and one company, but started to work on new ideas, and new companies arising from his PhD research.
He realised one important market for image pattern recognition, that was ripe for dominating, was fingerprint recognition. He therefore set about writing software that could match fingerprints far faster and more accurately than anyone else. His new company, Cambridge Neurodynamics, filled a gap, with his software being used by Police Forces nationwide. That then led to other spin-offs using similar technology
He was turning the computational thinking skills of abstraction and generalisation into a way to make money. By creating core general technology that solved the very general problems of signal processing and pattern matching, he could then relatively easily adapt and reuse it to apply to apparently different novel problems, and so markets, with one product leading to the next. By applying his image recognition solution to characters, for example, he created software (and a new company) that searched documents based on character recognition. That led on to a company searching databases, and finally to the company that made him famous, Autonomy.
One of his great loves was his dog, Toby, a friendly enthusiastic beast. Mike’s take on the idea of a search engine was fronted by Toby – in an early version, with his sights set on the nascent search engine market, his search engine user interface involved a lovable, cartoon dog who enthusiastically fetched the information you needed. However, in business finding your market and getting the right business model is everything. Rather than competing with the big US search engine companies that were emerging, he switched to focussing on in-house business applications. He realised businesses were becoming overwhelmed with the amount of information they held on their servers, whether in documents or emails, phone calls or videos. Filing cabinets were becoming history and being replaced by an anarchic mess of files holding different media, individually organised, if at all, and containing “unstructured data”. This kind of data contrasts with the then dominant idea that important data should be organised and stored in a database to make processing it easier. Mike realised that there was lots of data held by companies that mattered to them, but that just was not structured like that and never would be. There was a niche market there to provide a novel solution to a newly emerging business problem. Focussing on that, his search company, Autonomy, took off, gaining corporate giants as clients including the BBC. As a hands-on CEO, with both the technical skills to write the code himself and the business skills to turn it into products businesses needed, he ensured the company quickly grew. It was ultimately sold for $11 billion. (The sale led to an accusation of fraud in hte US, but, innocent, he was acquitted of all the charges).
Investing
From firsthand experience he knew that to turn an idea into reality you needed angel investors: people willing to take a chance on your ideas. With the money he made, he therefore started investing himself, pouring the money he was making from his companies into other people’s ideas. To be a successful investor you need to invest in companies likely to succeed while avoiding ones that will fail. This is also about understanding the likelihood of different things, obviously something he was good at. When he ultimately sold Autonomy, he used the money to create his own investment company, Invoke Capital. Through it he invested in a variety of tech startups across a wide range of areas, from cyber security, crime and law applications to medical and biomedical technologies, using his own technical skills and deep scientific knowledge to help make the right decisions. As a result, he contributed to the thriving Silicon Fen community of UK startup entrepreneurs, who were and continue to do exciting things in and around Cambridge, turning research and innovation into successful, innovative companies. He did this not only through his own ideas but by supporting the ideas of others.
Mike was successful because he combined business skills with a wide variety of technical skills including maths, electronic engineering and computer science, even bioengineering. He didn’t use his success to just build up a fortune but reinvested it in new ideas, new companies and new people. He has left a wonderful legacy as a result, all the more so if others follow his lead and invest their success in the success of others too.
Superheroes don’t just have physical powers. Often theycome out on top because of their mental abilities. Sherlockis a good example, catching villains through logicalthinking. Anyone can get better at thinking! Just practice.
It is important for everyone to be able to think clearly. It is especially true for programmers, detectives and lawyers as well as superheroes. You need to be able to work things out from the facts you know. The Ancient Greeks were very good at logic. They invented the idea of a ‘syllogism’. These are common patterns that combine facts where you figure out a conclusion only using the facts.
For example, if we know facts 1 and 2 below (where you can swap in anything for X, Y and Z) then we can create a new fact as shown.
Image by Paul Curzon
So let’s replace X with the word superheroes, Y with fight crime and Z with my favourite superhero, Ghost Girl. If we put them in to the picture above we get the new picture:
Image by Paul Curzon
In this case we can deduce the new fact that Ghost Girl fights crime. Notice how you use the plurals in Fact 1 and singular words in the other facts to make the English work.
Puzzles
Can you solve these Superhero Syllogism puzzles? Work out which conclusion is the one that follows from the given facts. Use our coloured template above to help.
Superhero syllogism puzzle 1
FACT 1: ALL superheroes do good. FACT 2: The Invisible Woman is a superhero.
Which statement below (a, b, c or d) can we say from these facts alone? Don’t use anything extra, just use fact 1 and fact 2. (ANSWERS at the bottom of the page).
a) The Invisible Woman has superpowers. b) The Invisible Woman does good. c) The Invisible Man does good. d) The Invisible Woman does not do good
Superhero syllogism puzzle 2
FACT 1: ALL superheroes sometimes accidentally do harm. FACT 2: Jamila is a superhero.
What can we say from these facts alone?
a) Jamila sometimes accidentally does harm. b) Jamila is not a superhero c) Those with superpowers only do good. d) Jamie is a superhero
Superhero syllogism puzzle 3
FACT 1: ALL supervillains laugh in an evil way. FACT 2: The Spider is a supervillain.
What can we say from these facts alone?
a) The Spider sometimes accidentally does harm. b) The Spider does not laugh in an evil way. c) Supervillains are evil. d) The Spider laughs in an evil way.
As long as the facts are true the conclusion follows, though if the facts are not true then nothing is really known.
Superhero syllogism puzzle 4
The following logic is good but something has gone wrong because the conclusion is not true. The superhero called the Angel does not actually have any superpowers! The Angel just wears a flying suit! Can you work out what has gone wrong with our logic?
1. ALL superheroes have superpowers. 2. The Angel is a superhero.
Therefore we can conclude from these facts alone that
3.The Angel has superpowers.
Answers are at the bottom of the page.
Fun to do
Take the pattern of the above syllogisms and invent your own. Just substitute your own words, but keep the pattern.See how silly the “facts” you can deduce are.
– Paul Curzon, Queen Mary University of London, first appeared in A BIT of CS4FN 2
Learn the basics of computer science through magic tricks
Subscribe to be notified whenever we publish a new post to the CS4FN blog.
This blog is funded by EPSRC on research agreement EP/W033615/1.
Answers
Superhero syllogism puzzle 1
Answer: b. FACT 1: ALL superheroes … do good. FACT 2: The Invisible Woman is a superhero Therefore we can conclude from these facts alone that NEW FACT: The Invisible Woman … does good.
Superhero syllogism puzzle 2
Answer: a. FACT 1: ALL superheroes … sometimes accidentally do harm. FACT 2: Jamila is a superhero Therefore we can conclude from these facts alone that NEW FACT: Jamila … sometimes accidentally does harm.
Superhero syllogism puzzle 3
Answer: d. FACT 1: ALL supervillains … laugh in an evil way. FACT 2: The Spider is a supervillain. Therefore we can conclude from these facts alone that NEW FACT: The Spider … laughs in an evil way.
Superhero syllogism puzzle 4
Something has gone wrong. We are told that The Angel has no superpowers. They just wear a special flying suit. The new fact is therefore not true. This means that one of the original ‘facts’ was not actually true. If we start from things that are not true then the things we deduce will not be true either! In this case
Just because you start a start-up doesn’t mean you have to be the boss (the CEO) running the company… Hamit Soyel didn’t and his research-based company, DragonFlyAI is flourishing.
Hamit’s computer science research (with Peter McOwan) at Queen Mary concerns understanding human (and animal) vision systems. Building on the research of neuroscientists they created computational models of vision systems. These are just programs that work in the way we believe our brains process what we see. If our understanding is correct then the models should see as we see. For example, one aspect of this is how our attention is drawn to some things and not others. If the model is accurate, it should be able to predict things we will definitely notice, and predict things we probably won’t. It turned out their models were really good at this.
They realised that their models had applications in marketing and advertising (an advert that no one notices is a waste of money). They therefore created a startup company based on their research. Peter sadly died not long after the company was founded leaving Hamit to make it a success. He had a choice to make though. Often people who start a startup company set themselves up as the CEO: it is their company so they want control. To do this you need good business skills though and also to be willing to devote the time to make the business a success. You got to this point though because of your technical and creative skills,
When you start a company you want to make a difference, but to actually do that you need a strong team and that team doesn’t have to be “behind” you, they can be “with” you – after all the best teams are made up of specialists who work to their strengths as well as supporting and working well with each other. Perhaps your strengths lie elsewhere, rather than in running a business,
With support from Queen Mary Innovations who helped him set up DragonflyAI and have supported it through its early years, Hamit decided his strengths were in the creative and technical side of the business, so he became the Chief Scientist and Inventor rather than the CEO. That role was handed to an expert as were the other senior leadership roles such as Marketing and Sales, Operations and Customer Success. That meant Hamit could focus on what he did best in further developing the models, as well as in innovating new ideas. This approach also gives confidence to investors that the leadership team do know what they are doing and that if they like the ideas then the company will be a success.
As a result, Hamit’s business is now a big success having helped a whole series of global companies improve their marketing, including Mars and Coca-Cola. DragonflyAI also recently raised $6m in funding from investors to further develop the business.
As Hamit points out:
By delegating operations to a professional leadership team, you can concentrate on areas you truly enjoy that fuel your passion and creativity, ultimately enhancing your fulfilment and contribution to your company and driving collective success.”
To be the CEO or not be the CEO depends on your skills and ambition, but you must also think about what is best for the company, as Hamit has pointed out. It is important to realise though that you do not have to be the CEO just because you founded the company.
Paul Curzon, Queen Mary University of London,
based on an interview between Hamit Soyel and Queen Mary Innovations
How much should we change the world to make it easier for our machines to work?
Plant scientists have spotted a problem they can solve. Weeding robots are finding it difficult to weed. It is a hard problem for them. All those weeds look just like the real crop which they aren’t supposed to destroy. So the robots are pulling up the wrong things. What is a robot to do? Should we make it easy for them?
Plant Scientists have seen a need for their technology which is looking for solutions any where it can. Robots are good at distinguishing colour. That is easy. So why not just genetically modify weeds to be blue. This is possible as there are already lots of genes causing blueness in plants (think blueberries). Problem solved. The robots then won’t get it wrong again and the crops are safe.
What could possibly go wrong? Well, to work the genes will need to be spread wildly and perhaps they could escape and get into our crops or other plants that are just there to be plants, or just plants in the food chain, We could end up with a blue planet a bit like the red one the martians brought int he War of the Worlds. Alternatively, evolution might step up and continually produce mutant weeds that subverted that gene, given that gene killed them. Perhaps all the problems can guarantee to be avoided, though the wise person does not bet against natural selection finding a way round problems presented to it in the long term.
Isn’t it time we learnt our lesson and stopped changing the planet to make our machines lives easier? Of course we have been doing that for a long time – think of all the roads scarring the countryside so cars work or rails so trains work. Perhaps we should think more about the needs of the planet as well as of people, rather than the needs of our machines when innovating, especially when undoubtedly eventually (if we don’t destroy ourselves first) we will have machines clever enough to work it out.
There are always lots of ways of solving problems and it is important to think about the planet now not just our machines. Perhaps robots should just not weed until they can do it without us having to change the problem (and the planet) for them so they can!
Becoming a successful entrepreneur often starts with seeing a need: a problem someone has that needs to be fixed. For David Ronan, the need was for anyone to mix and master music but the problem was that of how hard it is to do this. Now his company RoEx is fixing that problem by combining signal processing ans artificial intelligence tools applied to music. It is based on his research originally as a PhD student
Musicians want to make music, though by “make music” they likely mean playing or composing music. The task of fiddling with buttons, sliders and dials on a mixing desk to balance the different tracks of music may not be a musician’s idea of what making music is really about, even though it is “making music” to a sound engineer or producer. However, mixing is now an important part of the modern process of creating professional standard music.
This is in part a result of the multitrack record revolution of the 1960s. Multitrack involves recording different parts of the music as different tracks, then combining them later, adding effects, combining them some more … George Martin with the Beatles pioneered its use for mainstream pop music in the 1960s and the Beach Boys created their unique “Pet Sounds” through this kind of multitrack recording too. Now, it is totally standard. Originally, though, recording music involved running a recording machine while a band, orchestra and/or singers did their thing together. If it wasn’t good enough they would do it all again from the beginning (and again, and again…). This is similar to the way that actors will act the same scene over and over dozens of times until the director is happy. Once happy with the take (or recording) that was basically it and they moved on to the next song to record.
With the advent of multitracking, each musician could instead play or sing their part on their own. They didn’t have to record at the same time or even be in the same place as the separate parts could be mixed together into a single whole later. Then it became the job of engineers and the producer to put it all together into a single whole. Part of this is to adjust the levels of each track so they are balanced. You want to hear the vocals, for example, and not have them drowned out by the drums. At this point the engineer can also fix mistakes, cutting in a rerecording of one small part to replace something that wasn’t played quite right. Different special effects can also be applied to different tracks (playing one track at a different speed or even backwards, with reverb or auto-tuned, for example). You can also take one singer and allow them to sing with multiple versions of themselves so that they are their own backing group, and are singing layered harmonies with themselves. One person can even play all the separate instruments as, for example, Prince often did on his recordings. The engineers and producer also put it all together and create the final sound, making the final master recording. Some musicians, like Madonna, Ariana Grande and Taylor Swift do take part in the production and engineering parts of making their records or even take over completely, so they have total control of their sound. It takes experience though and why shouldn’t everyone have that amount of creative control?
Doing all the mixing, correction and overdubbing can be laborious and takes a lot of skill, though. It can be very creative in itself too, which is why producers are often as famous as the artists they produce (think Quincy Jones or Nile Rogers, for example). However, not everyone wanting to make their own music is interested in spending their time doing laborious mixing, but if you don’t yet have the skill yourself and cant afford to pay a producer what do you do?
That was the need that David spotted. He wanted to do for music what instagram filters did for images, and make it easy for anyone to make and publish their own professional standard music. Based in part on his PhD research he developed tools that could do the mixing, leaving a musician to focus on experimenting with the sound itself.
David had spent several years leading the research team of an earlier startup he helped found called AI Music. It worked on adaptive music: music that changes based on what is happening around it, whether in the world or in a video game being played. It was later bought by Apple. This was the highlight of his career to that point and it helped cement his desire to continue to be an innovator and entrepreneur.
With the help of Queen Mary, where he did his PhD, he therefore decided to set up his new company RoEx. It provides an AI driven mixing and mastering service. You choose basic mixing options as well as have the ability to experiment with different results, so still have creative control. However, you no longer need expensive equipment, nor need to build the skills to use it. The process becomes far faster too. Mixing your music becomes much more about experimenting with the sound: the machine having taken over the laborious parts, working out the optimum way to mix different tracks and produce a professional quality master recording at the end.
David didn’t just see a need and have an idea of how to solve it, he turned it into something that people want to use by not only developing the technology, but also making sure he really understood the need. He worked with musicians and producers through a long research and development process to ensure his product really works for any musician.
University has always been a place where you make great friends for life. Social media means everyone can easily make as many online friends as they like, and ever more students go to university, meaning more potential friends to make. So surely things now are better than ever. And yet many students suffer from loneliness while at university. We somehow seem to have ever greater disconnection the more connections we make. Klara Brodahl realised there was a novel need here that no one was addressing well and decided to try to solve it for the final year project of her computer science degree. Her solution was StudyBuddy and with the support of an angel investor she has now set up a startup company and is rolling it out for real.
A loneliness epidemic
In the digital age, university students face an unexpected challenge—loneliness. Although they’re more “connected” than ever through social media and virtual interactions, the quality of these connections is often shallow. A 2023 study, for example, found that 92% of students in the UK feel lonely at some point during their university life. This “loneliness epidemic” has profound effects, contributing to issues like anxiety, depression, and struggling with their programme.
During her own university years, Klara Brodahl had experienced first hand the challenge of forming meaningful friends in an environment where everyone seemed socially engaged online but weren’t always connected in real life. She soon discovered that it wasn’t just her but a shared struggle by students across the country. Inspired by this, she set out to write a program that would fill the void in student’s lives and bridge the gap between studying and social life.
She realised that technology does have the potential to strengthen social bonds, but how it’s designed and used makes all the difference. The social neuroscientist John Cacioppo has pointed out that using social media primarily as a destination in its own right often leaves people feeling distant and dissatisfied. However, when technology is designed to serve as a bridge to offline human engagement, it can reduce loneliness and improve well-being. StudyBuddy embodies this approach by encouraging students to connect in person rather than trying to replace meeting face-to-face.
Study together in the real world
Part of making this work is in having reasons to meet for real. Klara realised that the need to study, and the fact that doing this in groups rather than alone can help everyone do better, could provide the excuse for this. StudyBuddy, therefore, integrates study goals with social interaction, allowing friendships to form around shared academic interests—an ideal icebreaker for those who feel nervous in traditional social settings.
The app uses location-based technology to connect students for co-study sessions, making in-person meetings easy and natural. Through a live map, students can see where others are checked in nearby at study spots like libraries, cafes, or student common areas. They can join existing study groups or start their own. The app uses university ID verification to help ensure connections are built on a trusted network.
From idea to startup company
Klara didn’t originally plan for StudyBuddy to become a real company. Like many graduates, she thought starting a business was something to perhaps try later, once she had some professional experience from a more ‘normal’ graduate job. However, when the graduate scheme she won a place on after graduating was unexpectedly delayed, she found herself with time on her hands. Rather than do nothing she decided to keep working on the app as a side project. It was at this point that StudyBuddy caught the attention of an angel investor, whose enthusiasm for the app gave Klara the confidence to keep going.
When her graduate scheme finally began, she was therefore already deeply invested in StudyBuddy. Trying to manage both roles, she quickly realised she preferred the challenge and creativity of her startup work over the graduate scheme. And when it became impossible to balance both, she took a leap of faith, quitting her graduate job to focus on StudyBuddy full-time—a decision that has since paid off. She gained early positive feedback, ran a pilot at Queen Mary University of London, and won early funding for investors willing to invest in what was essentially still an idea, rather than a product with a known market. As a result StudyBuddy has gradually turned into a useful mission-driven platform, providing students with a safe, real-world way to connect.
Making a difference
StudyBuddy has the potential to transform the university experience by reducing loneliness and fostering authentic, in-person friendships. By rethinking what engagement in the digital age means, the app also serves as a model for how technology can promote meaningful social interaction more generally. Klara has shown that with thoughtful design, technology can be a powerful tool for bridging digital and physical divides, creating a campus environment where students thrive both academically and socially. Her experience also shows how the secret to being a great entrepreneur is to be able to see a human need that no one else has seen or solved well. Then, if you can come up with a creative solution that really solves that need, your ideas can become reality and really make a difference to people’s lives.
– Klara Brodahl, StudyBuddy and Paul Curzon, Queen Mary University of London
Want to make sure your life turns out the way you want? Want to trade this life for fortune and fame? If you believe post-grunge rock band Nickelback’s 2005 hit single, then ‘you wanna be a Rockstar’! Love or hate the song, are they right, or do you really wanna be a tech entrepreneur?
Some people want a hedonistic life. Some want to be famous. Others just want to be stinking rich. Some want all three. Some want to really make a positive difference to people’s lives.
So are Nickelback right? What is the best way to get all three and maybe even the fourth too – and quickly – say before the age of 35? In fact, let’s not set our sights too low. Let’s aim to be one of the richest people in the world. Let’s think multi-billionaire. Let’s assume too that we have to do it without relying on accidents of birth – no inheritance of billions from Mummy and Daddy’s money to look forward to. Winning the lottery wouldn’t even get you close so, while luck matters, don’t rely on your luck alone either. How you actually gonna do it?
From the queues of people wanting to be on reality TV programmes whether X-factor, the Voice or Love Island most people seem to agree with Nickelback’s solution: the way to early riches is to become famous, whether a Rock Star or maybe a footballer, or a film star, or these days just famous for being famous. It’s people like that that fill the super-rich but young and self-made lists isn’t it. Well isn’t it?
Nice idea, but no.
Some of those people do make a lot of money in a short time. They have to though as for most their career is likely to be very short. They don’t stay famous or in the rich lists for long and are unlikely to make super-rich.
They are all Techno Stars.
There is one very obvious pattern to Forbes’ self-made super-rich list of the top billionaires on the planet. Almost a quarter of the top super rich, at around the time that Nickleback wrote their hit song, made their money in a similar way. They aren’t film stars, rock stars or sports stars. They are all techno stars. They are also all self-made billionaires. That contrasts with the other people in the same league. With one exception, the rest are all there because of family wealth or are old: they took their time to extreme wealth. Contrast that with the Google guys, say, who made the top 30 by their 30s.
Number one – the richest person on Earth with 56 billion dollars – iin 2007 was not surprisingly Bill Gates, who with Paul Allen (Number 19) set up Microsoft. Paul Allen went on to found Dreamworks, a company working on the boundaries of film-making and computer science. They went on to use much of their personal wealth (and time) solving humanitarian problems, focussing on things like health and education. Yes, many rock stars do charity gigs (think Live Aid) occasionally, so if saving the Earth is your aim then becoming a Rock star may be one way to give you some clout to make a difference. It’s nothing compared to what someone as rich as the Microsoft pair have personally achieved though.
Not far behind was Lawrence Ellison, worth 21.5 billion dollars at the time. He made his name by creating the company Oracle that was largely responsible for pushing the database revolution – not just using databases of course but creating the software that allows other people to use databases. As he’s said “Money is just a method of keeping score now.”
There are then the Google pair Sergey Brin and Larry Page sharing position 26. They only had 16 billion dollars each, but, hey, they only founded Google in 1986. They planned to “do no evil” with their riches and also wanted to plough money into charity. What else do you do when you have that kind of silly money?
At positions and 30 and 31 in the 2007 rich list came Michael Dell and Steven Ballmer. Ballmer is ‘just’ another Microsoft man. Dell of course is responsible for Dell computers. He had the ear of a President as he was on the United States President’s Council of Advisors on Science and Technology. Want to make a difference? He could.
Have things changed? Well, yes. Forbes now use tech themselves to keep a real-time rich list. Now of the top ten richest people in the world as I write this, 8 are tech entrepreneurs, now with hundreds of billions of worth each. Elon Musk (Tesla, X etc), Jeff Bezos (Amazon), Larry Ellison (Oracle), Mark Zuckerberg (FaceBook), Larry Page (Google), Sergey Brin (Google), Jensen Huang (Nvidia) and Steve Ballmer (Microsoft – now richer than Bill Gates but he is still filthy rich too) .
In short, programming/computer science/electronic engineering and inheritance are the most likely source of riches for the richest people in the world. Programming is the only way to reach the top without inheriting money (or perhaps being a Russian president’s protege).
The other advantage of the technology route to riches over the Rock Star way of course is you can aim higher still. Don’t wind up dead at 40 from the drug-induced lifestyle of rock stars – why not aim to still be enjoying being filthy rich at 100 too. If you are wise you may make the world a far better place, though you may also gain the power to make it far worse too.
When disasters involving technology occur, human error is often given as the reason, but even experts make mistakes using poor technology. Rather than blame the person, human error should be seen as a design failure. Bad design can make mistakes more likely and good design can often eliminate them. Optical illusions and magic tricks show how we can design things that cause everyone to make the same systematic mistake, and we need to use the same understanding of the brain when designing software and hardware. This is especially important if the gadgets are medical devices where mistakes can have terrible consequences. The best computer scientists and programmers don’t just understand technology, they understand people too, and especially our brain’s fallibilities. If they don’t, then mistakes using their software and gadgets are more likely. If people make mistakes, don’t blame the person, fix the design and save lives.
Illusions
Optical illusions and magic tricks give a mirror on the limits of our brains. Even when you know an optical illusion is an illusion you cannot stop seeing the effect. For example, this image of an eye is completely flat and stationary: nothing is moving. And yet if you move your head very slightly from side to side the centre pops out and seems to be moving separately to the rest of the eye.
Illusions occur because our brains have limited resources and take short cuts in processing the vast amount of information that our senses deliver. These short cuts allow us to understand what we see faster and do so with less resources. Illusions happen when the short cuts are applied in a way where they do not apply.
What this means is that we do not see the world as it really is but see a simplified version constructed by our subconscious brain and provided to our conscious brain. It is very much like in the film, the Matrix, except it is our own brains providing the fake version of the world we experience rather than alien computers.
Attention
The way we focus our attention is one example of this. You may think that you see the world as it is, but you only directly see the things you focus on, your brain fills out the rest rather than constantly feeding the actual information to you constantly. It does this based on what it last saw there but also on the basis of just completing patterns. The following illusion shows this in action. There are 12 black dots and as you move your attention from one to the next you can see and count them all. However, you cannot see them all at once. The ones in your peripheral vision disappear as you look away as the powerful pattern of grey lines takes over. You are not seeing everything that is there to be seen!
Our brains also have very limited working memory and limited attention. Magicians also exploit this to design “magical systems” where a whole audience make the same mistake at the same time. Design the magic well so that these limitations are triggered and people miss things that are there to be seen, forget things they knew a few moments before, and so on. For example, by distracting their attention they make them miss something that was there to be seen.
What does this mean to computer scientists?
When we design the way we interact with a computer system, whether software and hardware, it is also possible to trigger the same limitations a magician or optical illusion does. A good interaction designer therefore does the opposite to a magician and, for example: draws a user’s attention to things that must not be missed at a critical time; they ensure they do not forget things that are important, they help them keep track of the state of the system, they give good feedback so they know what has happened.
Most software is poorly designed leading to people making mistakes, not all the time, but some of the time. The best designs will help people avoid making mistakes and also help them spot and fix mistakes as soon as they do make them.
Examples of poor medical device design
The following are examples of the interfaces of actual medical devices found in a day of exploration by one researcher (Paolo Masci) at a single very good hospital (in the US).
When the nurse or doctor types the following key sequence as a drug dose rate:
Image by CS4FN
one infusion pump, without any explicit warning, other than the number being displayed, registered the number entered as 1001.
Probably, the programmer had been told that when doses are as large as 100, then fractional doses are so relatively small that they make no difference. A user typing in such fractional amounts, is likely making an error as such a dose is unlikely to be prescribed. The typing of the decimal point is therefore just ignored as a mistake by the infusion pump. Separately, (perhaps coded by a different programmer in the team, or at a different time) until the ENTER key is pressed the code treats the number as incomplete. Any further digits typed are therefore just accepted as part of the number.
A different design by a different manufacturer also treats the key sequence as 1001 (though in the case shown 1001 is rejected as it exceeds the maximum allowable rate, caused by the same issue of the device silently ignoring a decimal point).
This suggests two different coding teams indipendently coded in the same design flaw that led to the same user error.
What is wrong with that?
Devices should never silently ignore and/or correct input if bad mistakes are to be avoided. Here, that original design flaw, could lead to a dose 10x too big being infused into a patient and that could kill. It relies on the person typing the number noticing that the decimal point has been ignored (with no help from the device). Decimal points are small and easily missed of course. Also, their attention cannot be guaranteed to be on the machine and, in fact, with a digit keypad for entering numbers that attnetion is likely to be on the keys. Alarms or other distractions elsewhere could easily mean they do not notice the missing decimal point (which is a tiny thing to see).
An everyday example of the same kind of problem, showing how easily mistakes are missed is in auto-completion / auto-correction of spelling mistakes in texts and word processors. Goofs where an auto-corrected word are missed are very common. Anything that common needs to be designed away in a safety critical system.
Design Rules
One of the ways that such problems can be avoided is by programmers following interaction design rules. The machine (and the programmer writing the code) does not know what a user is trying to input when they make a mistake. One design rule is therefore that a program should therefore NEVER correct any user error silently. Here perhaps the mistake was pressing 0 twice rather than pressing the decimal point. In the case of user errors, the program should raise awareness of the error, and not allow further input until the error is corrected. The program should explicitly draw the person’s attention to the problem (eg changing colour, flashing, beeping, etc). This involves using the same understanding of cognitive psychology as a magician, to control their attention. Whereas a magician would be taking their attention away from the thing that matters, the programmer draws theur attention to it.
It should make clear in an easily understandable error message what the problem is (eg here “Doses over 99 should not include decimal fractions. Please delete the decimal point.”) It should then leave the user to make the correction (eg deleting the decimal point) not do it itself.
By following a design rule such as this programmers can avoid user errors, which are bound to happen, from causing a big problem.
Avoiding errors
Sometimes the way we design software interfaces and their interaction design we can do even better than this, though. We are letting people make mistakes and then telling them to help them pick up the pieces afterward. Sometimes we can do better than this and with better design help them avoid making the mistake in the first place or spot the mistake themselves as soon as they make it.
Doing this is again about controlling user attention as a magician does. An interaction designer needs to do this again in the opposite wayto the magician though, directing the users attention to the place it needs to be to see what is really happening as they take actions rather than away from it.
To use a digit keypad, the users attention has to be on their fingers so they can see where to put their fingers to press a given digit. They look at the keypad, not the screen. The design of the digit keypad draws their attention to the wrong place. However, there are lots of ways to enter numbers and the digit keypad is only one. One other way is to use cursor keys (left, right, up and down) and have a cursor on the screen move to the position where a digit will be changed. Now, once the person’s finger is on say the up arrow, attention naturally moves to the screen as that button is just pressed repeatedly until the correct digit is reached. The user is watching what is happening, watching the program’s output, rather than their input, so is now less likely to make a mistake. If they do overshoot, their attention is in the right place to see it and immediately correct it. Experiments showed that this design did lead to fewer large errors though is slower. With numbers though accuracy is more likely to matter than absolute speed, especially in medical situations.
There are still subtleties to the design though – should a digit roll over from 9 back to 0, for example? If it does should the next digit increase by 1 automatically? Probably not, as these are the things that lead to other errors (out by a factor of 10). Instead going up from 9 should lead to a warning.
Learn from magicians
Magicians are expert at making people make mistakes without them even realising they have. The delight in magic comes from being so easily fooled so that the impossible seems to have happened. When writing software we need to using the same understanding of our cognitive resources and how to manipulate them to prevent our users making mistakes. There are many ways to do this, but we should certainly never write software that silently corrects user errors. We should control the users attention from the outset using similar techniques to a magician so that their attention is in the right place to avoid problems. Ideally a number entry system such as using cursor keys to enter the number rather than a digit keypad should be used as then the attention of the user is more likely to be on the number entered in the first place.
Responsible for the design of not just the interface but how a device or software is used. Applying creativity and applying existing design rules to come up with solutions. Has a deep understanding both of technical issues and of the limitations of human cognition (how our brains work).
Usability consultant
Give advice on making software and gadgets generally easier to use, evaluate designs for features that will make them hard to use or increase the likelihood of errors, finding problems at an early stage.
User experience (UX) consultant
Give advice on ensuring users of software have a good positive experience and that using it is not for example, frustrating.
Medical device developer
Develop software or hardware for medical devices used in hospitals or increasingly in the home by patients. Could be improvements to existing devices or completely novel devices based on medical or biomedical breakthroughs, or on computer science breakthroughs, such as in artificial intelligence.
Research and Development Scientist
Do experiments to learn more about the way our brains work, and/or apply it to give computers and robots a way to see the world like we do. Use it to develop and improve products for a spin-off company.
cs4fn 