The Digital Seabed: Data in Augmented Reality

A globe (North Atlantic visible) showing ocean depth information, with the path of HMS Challenger shown in red.
A globe (North Atlantic visible) showing ocean depth information, with the path of HMS Challenger shown in red. Image by Daniel Gill.

For many of us, the deep sea is a bit of a mystery. But an exciting interactive digital tool at the National Museum of the Royal Navy is bringing the seabed to life!

It turns out that the sea floor is just as interesting as the land where we spend most of our time (unless you’re a crab, of course, in which case you spend most of your time on the sea floor). I recently learnt about the sea floor at the National Museum of the Royal Navy in Portsmouth, in their “Worlds Beneath the Waves” exhibition, which documents 150-years of deep-sea exploration.

 One ship which revolutionised deep ocean study was HMS Challenger. It left London in 1858 and went on to make a 68,890 nautical-mile journey all over the earth’s oceans. One of its scientific goals was to measure the depth of the seabed as it circled the earth. To make these measurements, a long rope with a weight at one end was dropped into the water, which sank to the bottom. The length of the rope needed until the weight hit the floor was measured. It’s a simple process, but it worked! 

Thankfully, modern technology has caught up with bathymetry (the study of the sea floor). Now, sea floor depths are measured using sonar (so sound) and lidar (light) from ships or using special sensors on satellites. All of these methods send signals down to the seabed, and count how long it takes for a response. Knowing the speed of sound or light through air and water, you can calculate the distance to whatever reflected the signal.

You may be thinking, why do we need to know how deep the ocean is? Well, apart from the human desire to explore and mapour planet, it’s also useful for navigation and safety: in smaller waterways and ports, it’s very helpful to know whether there’s enough water below the boat to stay afloat!

It’s also useful to look at fault lines, the deep valleys (such as Challenger Deep, the deepest known point in the ocean, named after HMS Challenger), and underwater mountain ranges which separate continental plates. Studying these can help us to predict earthquakes and understand continental drift (read more about continental drift).

The sand table with colours projected onto it showing height.
The sand table with colours projected onto it showing height. Image by Daniel Gill.

We now have a much better understanding of the seabed, including detailed maps of sea floor topography around the world. So, we know what the ocean floor looks like at the moment, but how can we use this to understand the future of our waterways? This is where computers come in.

Near the end of the exhibition sits a table covered in sand, which has, projected onto it, the current topography of the sand. Where the sand is piled up higher is coloured red and orange, and lower in green and blue. Looking across the table you can see how sand at the same level, even far apart, is still within the same band of colour.

The projected image automatically adjusts (below) to the removal of the hill in red (above).
The projected image automatically adjusts (below) to the removal of the hill in red (above). Image by Daniel Gill.

But this isn’t even the coolest part! When you pick up and move sand around, the colours automatically adjust to the new sand topography, allowing you to shape the seabed at will. The sand itself, however, will flow and move depending on gravity, so an unrealistically tall tower will soon fall down and form a more rotund mound. 

 Want to know what will happen if a meteor impacts? Grab a handful of sand and drop it onto the table (without making a mess) and see how the topographical map changes with time!

The technology above the table.
The technology above the table. Image by Daniel Gill.

So how does this work? Looking above the table, you can see an Xbox Kinect sensor, and a projector. The Kinect works much like the lidar systems installed on ships – it sends beams of infrared lights down onto the sand, which bounce off back to the sensor in a measured time. This creates a depth map, just like ships do, but on a much smaller scale. This map is turned into colours and projected back on to the sand. 

Virtual water fills the valleys.
Virtual water fills the valleys. Image by Daniel Gill.

This is not the only feature of this table, however: it can also run physics simulations! By placing your hand over the sand, you can add virtual water, which flows realistically into the lower areas of sand, and even responds to the movement of sand.

The mixing of physical and digital representations of data like this is an example of augmented, or mixed, reality. It can help visualise things that you might otherwise find difficult to imagine, perhaps by simulating the effects of building a new dam, for example. Models like this can help experts and students, and, indeed, museum visitors, to see a problem in a different and more interactive way.

– Daniel Gill, Queen Mary University of London

More on…

Subscribe to be notified whenever we publish a new post to the CS4FN blog.


This page is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos

Annie Easley: putting rockets into space

Annie Easley head and shoulders portrait
Annie Easley. NASA, Public domain, via Wikimedia Commons

Annie Easley was a pioneer both as a computer programmer but also as a champion of women and minorities into computer science. She went from being a human computer doing calculations for the rocket scientists (in the days before computers were machines), to becoming a programmer whose programs were integral to many NASA projects. Here work has helped us explore the planets and beyond, to put satellites into space and help humans leave the Earth. She also contributed to early battery technology as well as the alternative energy sources we now need to transition away from oil and gas. Throughout her career, despite being repeatedly discriminated against herself as an African-american woman, she encouraged, supported and mentored others like her.

Annie was a maths graduate so when she saw that computers were needed by NACA, the predecessor of NASA, she jumped at the chance. At the time a computer was a human who did calculations, as no machine at that point had been created to take over the job. She was one of only four African-american employees out of several thousand. Her job was to do the calculations researchers needed for their work. However, as digital computers started to be introduced – machines were now able to do large numbers of tedious calculations much more quickly than humans so took over the job…but now needed people who could program them for each task. To do so still needed mathematical ability to understand the task, as well as the ability to write code. She learnt both low level assembly language and the high level language, Fortran, invented for such scientific programming work and transitioned to being a programmer mathematician.

Much of her work involved or supported simulation, so writing programs that model aspects of the real world to test whether scientists predictions are correct, or to help make new predictions. Ultimately, this work would help provide the data to make choices of which technologies to use. Today computer simulation is a completely standard way of doing both engineering and science and has actually provided a completely new way to do science complementing theory and experiment. It allows us to probe everyday science questions but also big questions like exploring the origins of the universe or probing the long term consequences of our actions on the climate. Back then it was totally novel though, as computers were completely new. She was involved in simulation work that prefigured important work today around the environment, investigating systems to convert energy between different forms and so hybrid battery technology. It allows vehicles (whether a rocket, satellite, car or planetary rover) to switch between electric power and other sources of energy – an idea that has provided an important bridge from petrol to electric cars. She was also part of teams exploring alternative fuel sources like wind power and solar power (important of course now in space for satellites and planetary rovers, as well as a fossil fuel alternatives on Earth).

An Atlas rocket with centaur final stage launching
An Atlas rocket with centaur final stage. NASA, Public domain, via Wikimedia Commons

One of her major areas of work, that has had a lasting impact, was on the Centaur rocket. Rocket launches involve multiple fuel tanks to get the payload (eg a satellite) into space. The tanks of each stage are ejected when their fuel runs out with the next stage taking over. Centaur was the final upper stage which used the then novel fuel of liquid hydrogen and liquid oxygen to propel the payload in the final step into space. Centaur became a mainstay for satellite launches as well as for probes sent to visit other planets – like Voyager (which visited the outer planets and is now in interstellar space heading away from the solar system having visited ) and CassiniHuygens  (which sent back stunning images of Saturn’s rings). Newer versions of Centaur are still used today,

At the same time as doing all this work she was also heavily involved in NASAs public engagement with science programmes, visiting schools and giving talks about the work, inspiring girls and those from ethnic minorities that STEM careers were for them. She also worked as equal employment opportunity counselor. This involved her helping sort out discrimination complaints (whether over age or race or gender) in a positive and cooperative way.

Space travel has opened up not only a new ability to explore our solar system, but made lots of other technologies from SatNav to remote monitoring possible as well has helped in the development of other technology such as battery technology and alternative energy sources. We all owe a lot to the pioneers like Annie Easley, and none more so than the private companies now aiming to further commercialise space.

– Paul Curzon, Queen Mary University of London

More on …

Magazines …

Subscribe to be notified whenever we publish a new post to the CS4FN blog.


This page is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos

The last piece of the continental drift puzzle

Hands holding continents of world
Image by Gerd Altmann from Pixabay 

A computer helped provide the final piece in the puzzle of how the continents formed and moved around. It gave a convincing demonstration that the Americas, Europe and Africa had once been one giant continent, Pangea, the pieces of which had drifted apart.

Plate tectonics is the science behind how the different continents are both moving apart and crashing together in different parts of the world driven by the motion of molten rock below the Earths crust. It created the continents and mountain ranges, is causing oceans to expand and to sink, and leads to earthquakes in places like California. The earth’s hard outer shell is made up of a series of plates that sit above hotter molten rock and those plates slowly move around (up to 10cm a year) as, for example, rock pushes up between the gaps and solidifies. or pushes down and down under an adjacent plate. The continents as we see them are sitting on top of these plates.

The idea of continental drift had existed in different forms since the early 19th century. The idea was partly driven by an observation that on maps, South America and Africa seemed almost like two jigsaw pieces that fit together. On its own an observation like this isn’t enough as it could just be a coincidence, not least because the fit is not exact. Good science needs to combine theory with observation, predictions that prove correct with data that provides the evidence, but also clear mechanisms that explain what is going on. All of this came together to show that continental drift and ultimately plate tectonics describe what is really going on.

Very many people gathered the evidence, made the predictions and built the theories over many decades. For example, different people came up with a variety of models of what was happening but in the 19th and early 20th centuries there just wasn’t enough data available to test them. One theory was that the continents themselves were floating through the layer of rock below a bit like ice bergs floating in the ocean. Eventually evidence was gathered and this and other suggestions for how continents were moving did not stand up to the data collected. It wasn’t until the 1960s that the full story was tied down. The main reason that it took so long was that it needed new developments in both science and technology, most notably understanding of radioactivity, magnetism and not least ways to survey the ocean beds as developed during World War II to hunt for submarines. Science is a team game, always building on the advances of others, despite the way individuals are singled out.

By the early 1960s there was lots of strong evidence, but sometimes it is not just a mass of evidence that is needed to persuade scientists en-masse to agree a theory is correct, but compelling evidence that is hard to ignore. It turned out that was ultimately provided by a computer program.

Geophysicist, Edward Bullard, and his team in Cambridge were responsible for this last step. He had previously filled in early pieces of the puzzle working at the National Physical Laboratory on how the magnetism in the Earth’s core worked like a dynamo. He used their computer (one of the earliest) to do simulations to demonstrate this. This understanding led to studies of the magnetism in rock. This showed there were stripes where the magnetism in rock was in opposite directions. This was a result of rock solidifying either in different places or at different times and freezing the magnetic direction of the Earth at that time and place. Mapping of this “fossil” magnetism could be used to explore the ideas of continental drift. One such prediction suggested the patterns should be identical on either side of undersea ridges where new rock was being formed and pushing the plates apart. When checked they were exactly symmetrical as predicted.

	Jacques Kornprobst (redesigned after Bullard, E., Everett, J.E. and Smith, A.G., 1965. The fit of the continents around the Atlantic. Phil. Trans. Royal Soc., A 258, 1088, 41-51)

Image reconstruction of Bullard’s map by Jacques Kornprobst (redesigned after Bullard, E., Everett, J.E. and Smith, A.G., 1965. The fit of the continents around the Atlantic. Phil. Trans. Royal Soc., A 258, 1088, 41-51), CC BY-SA 4.0 via Wikimedia Commons  

In the 1960s, Bullard organised a meeting at the Royal Society to review all the evidence about continental drift. There was plenty of evidence to see that continental drift was fact. However, he unveiled a special map at the meeting showing how the continents on either side of the Atlantic really did fit together. It turned out to be the clincher.

The early suggestion that Africa and South America fit together has a flaw in that they are similar shapes, but do not fit exactly. With the advent of undersea mapping it was realised the coastline as shown on maps is not the right thing to be looking at. Those shapes depend on the current level of the sea which rises and falls. As it does so the apparent shape of the continents changes. In terms of geophysics, the real edge of the continents is much lower. That is where the continental shelf ends and the sea floor plummets. Bullard therefore based the shape of the continents on a line about a kilometre below sea level which was now known accurately because of that undersea mapping.

Maps like this had been created before but they hadn’t been quite as convincing. After all a human just drawing shapes as matching because they thought they did could introduce bias. More objective evidence was needed.

We see the Earth as flat on maps, but it is of course a sphere, and maps distort shapes to make things fit on the flat surface. What matters for continents is whether the shapes fit when placed and then moved around on the surface of a sphere, not on a flat piece of paper. This was done using some 18th century maths by Leonhard Euler. At school we learn Euclidean Geometry – the geometry of lines and shapes on a flat surface. The maths is different on a sphere though leading to what is called Spherical Geometry. For example, on a flat surface a straight line disappears in both directions to infinity. On a sphere a straight line disappearing in one direction can of course meet itself in the other. Similarly, we are taught that the angles of a triangle on a flat surface add up to 180 degrees, but the angles of a triangle drawn on a sphere add up to more than 180 degrees… Euler, usefully for Bullard’s team, had worked out theorems for how to move shapes around on a sphere.

This maths of spherical geometry and specifically Euler’s theorems form the basis of an algorithm that the team coded as a program. The program then created a plot following the maths. It showed the continents moved together in a picture (see above). As it was computer created, based on solid maths, it had a much greater claim to be objective, but on top of that it did also just look so convincing. The shapes of the continents based on that submerged continental line fit near perfectly all the way from the tip of South America to the northern-most point of North America. The plot became known as the ‘Bullard Fit’ and went down in history as the evidence that sealed the case.

The story of continental drift is an early example of how computers have helped change the way science is done. Computer models and simulations can provide more objective ways to test ideas, and computers can also visualise data in ways that help see patterns and stories emerge in ways that are both easy to understand and very convincing. Now computer modelling is a standard approach used to test theories. Back then the use of computers was much more novel, but science provided a key early use. Bullard and his team deserve credit not just for helping seal the idea of continental drift as fact, but also providing a new piece to the puzzle of how to use computers to do convincing science.

Paul Curzon, Queen Mary University of London

More on …

  • Read the book: Science: a history by John Gribbin for one of the best books on the full history of Science including plate techtonics.

EPSRC supports this blog through research grant EP/W033615/1. 

Subscribe to be notified whenever we publish a new post to the CS4FN blog.


The paranoid program

by Paul Curzon, Queen Mary University of London

One of the greatest characters in Douglas Adams’ Hitchhiker’s Guide to the Galaxy, science fiction radio series, books and film was Marvin the Paranoid Android. Marvin wasn’t actually paranoid though. Rather, he was very, very depressed. This was because as he often noted he had ‘a brain the size of a planet’ but was constantly given trivial and uninteresting jobs to do. Marvin was fiction. One of the first real computer programs to be able to converse with humans, PARRY, did aim to behave in a paranoid way, however.

PARRY was in part inspired by the earlier ELIZA program. Both were early attempts to write what we would now call chatbots: programs that could have conversations with humans. This area of Natural Language Processing is now a major research area. Modern chatbot programs rely on machine learning to learn rules from real conversations that tell them what to say in different situations. Early programs relied on hand written rules by the programmer. ELIZA, written by Joseph Weizenbaum, was the most successful early program to do this and fooled people into thinking they were conversing with a human. One set of rules, called DOCTOR, that ELIZA could use, allowed it to behave like a therapist of the kind popular at the time who just echoed back things their patient said. Weizenbaum’s aim was not actually to fool people, as such, but to show how trivial human-computer conversation was, and that with a relatively simple approach where the program looked for trigger words and used them to choose pre-programmed responses could lead to realistic appearing conversation.

PARRY was more serious in its aim. It was written by, Kenneth Colby, in the early 1970s. He was a psychiatrist at Stanford. He was trying to simulate the behaviour of person suffering from paranoid schizophrenia. It involves symptoms including the person believing that others have hostile intentions towards them. Innocent things other people say are seen as being hostile even when there was no such intention.

PARRY was based on a simple model of how those with the condition were thought to behave. Writing programs that simulate something being studied is one of the ways computer science has added to the way we do science. If you fully understand a phenomena, and have embodied that understanding in a model that describes it, then you should be able to write a program that simulates that phenomena. Once you have written a program then you can test it against reality to see if it does behave the same way. If there are differences then this suggests the model and so your understanding is not yet fully accurate. The model needs improving to deal with the differences. PARRY was an attempt to do this in the area of psychiatry. Schizophrenia is not in itself well-defined: there is no objective test to diagnose it. Psychiatrists come to a conclusion about it just by observing patients, based on their experience. Could a program display convincing behaviours?

It was tested by doing a variation of the Turing Test: Alan Turing’s suggestion of a way to tell if a program could be considered intelligent or not. He suggested having humans and programs chat to a panel of judges via a computer interface. If the judges cannot accurately tell them apart then he suggested you should accept the programs as intelligent. With PARRY rather than testing whether the program was intelligent, the aim was to find out if it could be distinguished from real people with the condition. A series of psychiatrists were therefore allowed to chat with a series of runs of the program as well as with actual people diagnosed with paranoid schizophrenia. All conversations were through a computer. The psychiatrists were not told in advance which were which. Other psychiatrists were later allowed to read the transcripts of those conversations. All were asked to pick out the people and the programs. The result was they could only correctly tell which was a human and which was PARRY about half the time. As that was about as good as tossing a coin to decide it suggests the model of behaviour was convincing.

As ELIZA was simulating a mental health doctor and PARRY a patient someone had the idea of letting them talk to each other. ELIZA (as the DOCTOR) was given the chance to chat with PARRY several times. You can read one of the conversations between them here. Do they seem believably human? Personally, I think PARRY comes across more convincingly human-like, paranoid or not!


Activity for you to do…

If you can program, why not have a go at writing your own chatbot. If you can’t writing a simple chatbot is quite a good project to use to learn as long as you start simple with fixed conversations. As you make it more complex, it can, like ELIZA and PARRY, be based on looking for keywords in the things the other person types, together with template responses as well as some fixed starter questions, also used to change the subject. It is easier if you stick to a single area of interest (make it football mad, for example): “What’s your favourite team?” … “Liverpool” … “I like Liverpool because of Klopp, but I support Arsenal.” …”What do you think of Arsenal?” …

Alternatively, perhaps you could write a chatbot to bring Marvin to life, depressed about everything he is asked to do, if that is not too depressingly simple, should you have a brain the size of a planet.


More on …

Related Magazines …

Issue 16 cover clean up your language

This blog is funded through EPSRC grant EP/W033615/1.

Executable Biology

Computing cancer using computational modelling

(From the archive)

Can a robot get cancer? Silly question. Our bodies are made of cells. Robots aren’t. Cells are the basic building blocks of life and come in lots of different forms from long thin nerve cells that allow us to sense the world, to round blood cells that carry oxygen around our bodies. Cancer occurs when cells go rogue and start reproducing in an uncontrolled way. A computer can’t get cancer, but you can allow virtual diseases to attack virtual cells inside a computer. Doing that may just help find cures. That is what Jasmin Fisher, who leads a research group at Microsoft Research in Cambridge, has devoted her career to.

Becoming a medic isn’t the only way to help save lives!

Computational Modelling is changing the way the sciences are done. It is the idea that you can run experiments on virtual versions of things you are investigating. A computer model is essentially just a program that simulates the phenomena of interest. For example, by writing a program that simulates the laws of Physics, you can use it to run virtual Physics experiments about the motion of the planets, say. If your virtual planets do follow the paths real planets do, then you have evidence the laws are right. If they don’t your laws (or the models) need to change. You can also make predictions such as when an eclipse will happen. If you are right it suggests the laws you coded are good descriptions of reality. If wrong, back to the drawing board.

Jasmin has been pioneering this idea with the stuff of life and death. She focusses on modelling cells and the specific ways that we think cancer attacks them. It gives a way of exploring what is going on at the level of the molecules inside cells, and so how well new medicines might, or might not, work. Experiments can be done quickly and easily on the programmed models by running simulations. That means the real experiments, taking up expensive lab time, can focus on things that are most likely to be successful. Jasmin’s work has helped researchers design more effective actual experiments because they start with a better understanding of what is going on. One of the most important questions she is studying is how cells end up becoming what they are, and how this differs between normal cells and cancer cells. Understand this and we will be much closer to understanding how to stop cancer.

– Paul Curzon, Queen Mary University of London

More on …

Related Magazines …

This story was originally published here and is an article from CS4FN, a free computer science magazine from Queen Mary University of London which is sent to subscribing UK schools. To find out more please visit our About page. The article was also published in issue 23, The Women Are (Still) Here, on p3.


EPSRC supports this blog through research grant EP/W033615/1. 

QMUL CS4FN EPSRC logos

The ping pong vaccination programming challenge

Vaccination programmes work best when the majority of the population are vaccinated. One way scientists simulate the effects of disease and vaccination programmes is by using computer simulations. But what is a computer simulation?

Lots of multi-coloured ping pong balls
Image by Sergio Pavlishko from Pixabay

You can visualise what a simulation is with ping pong balls bouncing around a crowd. Imagine having a large room full of people. A virus is represented by a ping pong ball, bouncing from person to person, infecting each person it touches. Each person who is hit by a ping pong ball and not already infected becomes infected. That means they toss that ping pong ball back into the crowd to infect more people, but they also toss an extra one too (and then they sit down: dead). Start with a few ping pong balls. Quickly the virus spreads everywhere and lots of people sit down (die). You have run a physical simulation of how a virus spreads!

Now start again but ‘vaccinate’ 80 per cent of the people first: give them a baseball cap to wear to show who is who. If those people get a ping pong ball, they just destroy it: they infect noone else. Start with the same number of ping pong balls. This time, the virus quickly dies out and only a few people sit down (die). Not only are the vaccinated people protected but they protect many of the un-protected people too who might have died.

Now (if you can program) you can write a program to do the same thing, and so simulate and explore the spread of infection, which is easier perhaps than getting a thousand people to chuck ping pong balls about. Create a grid (an array) of 1000 cells. Each represents a person. They can be infected or not. They can also be vaccinated or not. Start with five random cells (so people marked as infected). Run a series of rounds. After each round, a newly infected cell randomly chooses two others to infect. If not infected already and not vaccinated, then they become newly infected. If already infected or vaccinated, they do not pass the infection on.

You can run lots of different experiments with different conditions. For example, experiment with different proportions of people infected at the start or explore what percentage of people need to be vaccinated for the virus to quickly die out. Is 50 per cent enough? You could also change how many people one person infects, or for how long a person can infect others before dying. Perhaps they each keep causing new infections for three rounds before stopping instead of only one. In what situations does the virus infect lots of people and when does it die out quickly?

What you are doing here is computer modelling or simulating the effects of the virus in different scenarios, and that is essentially how computer scientists make the predictions that governments use to make decisions about lockdowns and mask wearing, if they are “following the science”. Of course, such models are only as good as the data that goes into them, such as how many other people does each person infect. In reality, this is data provided by surveys, experimental studies, and so on.

– Paul Curzon, Queen Mary University of London, Spring 2021

Download Issue 27 of the cs4fn magazine on Smart Health here.

This post and issue 27 of the cs4fn magazine have been funded by EPSRC as part of the PAMBAYESIAN project.