Category: Diversity

Working in Computer Science: An Autistic Perspective (Part 1)

by Daniel Gill, Queen Mary University of London

5 paper planes going in the same straight line with one red one on a different curved path
Image by Rosy / Bad Homburg / Germany from Pixabay

Autism is a condition with many associated challenges, but for some people it presents some benefits. This distinction is greatly apparent in the workplace, where autistic people often find it difficult to get along with others (and their boss), and to complete the work that has been set for them. It’s not all negatives though: many autistic people find the work in which they thrive, and given the right circumstances and support, an autistic person is able to succeed in such an environment.

We often rightly hear about the greats in computer science; Ada Lovelace, Alan Turing, Lynn Conway (who sadly passed away earlier this month) – but let us not forget the incredible teams of computer scientists working around the clock; maintaining the Internet, building the software we use every day, and teaching the next generation. For this two-part article, I have spoken with Stephen Parry, an autistic computer scientist, who, after working in industry for 20 years, now teaches the subject in a sixth-form college in Sheffield. His autistic traits have caused him challenges throughout his career, but this is not a unique experience – many autistic computer scientists also face the same challenges.

Stephen’s experience with programming started at the age of 14, after being introduced to computers at a curriculum enrichment course. He decided against taking a then “really rubbish” O-Level (now GCSEs) Computer Science course, and the existence of the accompanying A-Level “just didn’t come up on my radar”. He was, however, able to take home the college’s sole RML 380Z for the summer, a powerful computer for the time, with which, he was able to continue to practice programming.

When it came time to go to university, he opted first to study chemistry, a subject he had been studying at A-Level. Though after a short amount of time he realised that he wasn’t as interested in chemistry as he first thought – so he decided to switch to computer science. In our discussions, he praised the computer science course at the University of Sheffield:

“[I] really enjoyed [the course] and got on well with it. So, I kind of drifted into it as far as doing it seriously is concerned. But it’s been a hobby of mine since I was 14 years old, and once I was on the degree, I mean, the degree at Sheffield was a bit like a sweetie shop. It really was absolutely brilliant. We did all kinds of weird and wonderful stuff, all of it [was] really interesting and engaging, and the kind of stuff that you wouldn’t get by either playing around on your own or going out into [the] workplace. As I’ve always said, that’s what a university should be. It should expose you to the kind of stuff that you can’t get anywhere else, the stuff that employers haven’t realised they need yet.”

Of autistic people who go to university, research shows they are much more likely the general population to pick STEM subjects [EXTERNAL]. For lots of autistic people, the clear logical and fundamental understanding behind scientific subjects is a great motivator. Stephen describes how this is something that appeals to him.

“[What] I enjoy about computer science is how it teaches you how the computer actually works at a fundamental level. So, you’re not just playing with a black box anymore – it’s something you understand. And especially for someone on the [autism] spectrum, that’s a really important aspect of anything you do. You want to understand how things work. If you’re working with something, and you don’t understand how it works, usually it’s not very satisfying and kind of frustrating. Whereas, if you understand the principles going on inside of it then, when you know you’ve got it, it kind of unlocks it for you.”

While autistic traits often result in challenges for autistic people, there are some which can present a benefit to someone in computer science. A previous CS4FN article described how positive traits like ‘attention to detail’ and ‘resilience and determination’ link well to programming. Stephen agrees that these traits can help him to solve problems:

“If I get focused on a problem, the hyper focus kicks in, and I will just keep plugging away until it’s done, fixed or otherwise overcome. I know it’s both a benefit and hazard – it’s a double edged sword, but at the same time, you know you have to have that attention to detail and that, to put it another way, sheer bloody mindedness to be determined that you’re going to make it work, or you’re going to understand how it works, and that does come definitely from the [autism] spectrum.”

Although he enjoyed the content greatly, Stephen had a rocky degree, both in and out of lectures. However, some unexpected benefits arose from being at university; he both found faith and met his future wife. These became essential pillars of support, as he prepared to enter the workforce. This he did, working both as a programmer and in a variety of IT admin and technical support roles. 

About 78% of autistic adults are currently out of work [EXTERNAL] (compared with 20% in the general population). This is, in part, reflective of the fact that some autistic people are unable to work because of their condition. But for many others, despite wanting to work, they cannot because they do not get the support they need (and are legally entitled to) within their workplace.

At this time, however, Stephen wasn’t aware of his condition, only receiving his diagnosis in his 40s. He described how this transition from university to work was very challenging.

“I moved into my first job, and I found it very, very difficult because I didn’t know that I’ve got this sort of difference – this different way my brain works that affects everything that you do. I didn’t know when I came across difficulties, it was difficult to understand why, at least to an extent, for me and for other people, it was deeply frustrating. I mean, speak to just about every manager I’ve ever had, and the same sort of pattern tends to come out. Most of them recognised that I was very difficult to manage because I found myself very difficult to manage. But time management is an issue with everything – trying to complete tasks to any kind of schedule, trying to plan anything. Oh, my days, when I hear the word SMART. [It’s an] acronym [meaning] specific, measurable, achievable, realistic and time specific. I hear that, and it just it makes me feel physically ill sometimes, because I cannot. I cannot SMART plan.”

However, during his time in work, he had some good luck. Despite the challenges associated with autism, some managers took advantage of the positive skills that he brings to the table:

“I found that a real challenge, interpersonally speaking, things like emotional regulation and stuff like that, which I struggle with, and I hate communicating on the phone and various other things, make me not the most promising employee. But the managers that I’ve had over the years that have valued me the most are the ones who recognised the other side of the coin, which is [that] over the years, I have absorbed so much knowledge about computer science and there are very [few] problems that you can come across that I don’t have some kind of insight into.”

This confidence in a range of areas in computer science is also a result of Stephen’s ability to link lots of areas and experiences together, a positive skill that some autistic people have:

“I found that with the mixture of different job roles I did, i.e. programming, support, network admin and database admin, my autism helped me form synergies between the different roles, allowing me to form links and crossover knowledge between the different areas. So, for example, as a support person with programming experience, I had insight into why the software I was helping the user with did not work as desired (e.g. the shortcuts or mistakes the programmer had likely made) and how maybe to persuade it to work. As a programmer with support experience, you had empathy with the user and what might give them a better UX, as well as how they might abuse the software. All this crossover, also set me up for being able to teach confidently on a huge range of aspects of CS.”

For autistic students who are planning on working in a computer science career, he has this to say:

“As an autistic person, and I would say this to anybody with [autism], you need to cultivate the part of you that really wants to get on well with people and wants to be able to care about people and understand people. Neurotypical people get that ability out of the box, and some of them take it for granted. I tend to find that the autistic people who actually find that they can understand people, that they work at it until they can, [are] often more conscientious as a result. And I think it’s important that if you’re an autistic person, to learn how to be positive about people and affirm people, and interact with them in positive ways, because it can make you a more caring and more valuable human being as a as a result.”

“Look for jobs where you can really be an asset, where your neurodiversity is the asset to what you’re trying to do, but at the same time, don’t be afraid to try to, and learn how to engage with people. Although it’s harder, it’s often more rewarding as a result.”

After working in industry for 20 years, the last half as which as a contractor, Stephen decided to take a considerable pay drop and become a computer science teacher. In the second part of this article, we will continue our conversation and find out what led him to choose a career change to teaching. 

More on …

Magazines …

Front cover of CS4FN issue 29 - Diversity in Computing

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos

Navajo Code Talkers

Three Navajo Code talkers in WWII
Navajo Code Talkers, Image from National Archives at College Park, Public domain, via Wikimedia Commons

Bletchley Park, the British code cracking centre helped win World War II, but it is not just breaking codes and ciphers that wins wars, creating unbreakable ones to keep your own secrets safe matters too. Bletchley Park wasn’t the first or only time a secret cryptography team helped win battles or even wars. In World War I secret messages had been successfully sent using Choctaw, the language of a tribe of Native Americans, including to help organise a surprise attack. It worked with their messages left un-cracked. This led to an even more successful code-creating team in World War II based on Navajo. The Navajo “Code Talkers” as they were called, could encode, transmit and decode messages in minutes when it would take hours using conventional codes and ciphers.

In World War II, the US forces used a range of Native American languages to communicate, but a code based on a native Indian language, Navajo, was especially successful. The use of a Navajo-based code was the idea of Philip Johnston after the attack on Pearl Harbour. His parents were missionaries so he had grown up on a Navajo reservation, speaking the language fluently despite how difficult it was. Aged only 9, he acted as an interpreter for a group who went to Washington to try to improve Indian rights.

He suggested using Navajo as a secret language and enlisted in the marines to help bring the idea to fruition. He thought it would work as a secret code because there was no written version of Navajo. It was a purely a spoken language. That meant he was one of very few people who were not Navajo who could speak it. It was also a complex language unlike any other language. The US marines agreed to trial the idea. 

To prove it would work, Johnston had Navajo transmit messages in the way they would need to on the battlefield. They could do it close to 100 times faster than it would take using standard cipher machines. That clinched it. 

Many Navajo had enlisted after Pearl Harbour and a platoon soley of Navajo were recruited to the project, including a 15 year old, William Dean Yazzie. However, they didn’t just speak in Navajo to transmit messages. The original 29 Navajo recruited worked out the details of the code they would use. Once deployed to the Pacific a group of them also met to further improve the code. None of it was written down apart from in training manuals that did not leave the training site, so there was no chance the code book could be captured in battle. All those involved memorised it and practiced sending messages quickly and accurately. Messages were also always spoken, eg over radio and never written down, making it harder for the code to be cracked based on analysing intercepted messages.

Commonly needed words, like ‘difficult’ or ‘final’ had direct Navajo code words (NA-NE-KLAH and TAH-AH-KWO-DIH). However for critical words (countries, kinds of planes, kinds of ships, etc) they first swapped English words for other English words using one code. They then translated those words into Navajo. That meant even a Navajo speaker outside their trained group wouldn’t immediately understand a message. The code, for example, used birds names in place of kinds of planes. So the English code word for a bomber plane was Buzzard. But then the Navajo for Buzzard was actually used: (JAY-SHO). 

Another part of the code was to use Navajo words for letters of the alphabet, so A is for ant translated to WOL-LA-CHE in Navajo. However, to make this more secure two other words stood for A too (apple: BE-LA-SANA and axe: TSE-NILL). Each letter had three alternatives like this and any of the three could be used.

Finally the way that it was used meant a message would always just be a series of unconnected words making no sense even to a Navajo speaker.

The code talkers played a key part in many battles including the iconic battle of Iwo Jima, capturing the heavily defended Japanese controlled island of that name. The US Major responsible for communications said of the battle, “Were it not for the Navajos, the Marines would never have taken Iwo Jima.”

Not only did it make communications much faster than they would have been, unlike other US codes and ciphers, the code talker’s code was never cracked … all thanks to the Navajo team who devised it.

– Paul Curzon, Queen Mary University of London

More on …

Magazines …

Front cover of CS4FN issue 29 - Diversity in Computing

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded through EPSRC grant EP/W033615/1.

QMUL CS4FN EPSRC logos

Involving disabled people in the design of ICT tools and devices

by Jo Brodie, Queen Mary University of London

Image by Gerd Altmann from Pixabay (CROPPED)

The World Health Organisation currently estimates that around 1.3 billion people, or one in six people on Earth, “experience significant disability”. Designers who are creating devices and tools for people to use need to make sure that the products they develop can be used by as many people as possible, not just non-disabled people, to make sure that everyone can benefit from them.

Disabled people can face lots of barriers in the workplace including some that seem simple to address – problems using everyday ICT and other tech. While there are a lot of fantastic Assistive Technology (AT) products unfortunately not all are suitable and so are abandoned by disabled people as they don’t serve their needs.

One challenge is that some of the people who have been doing the designing might not have direct experience of disability themselves, so they are less able to think about their design from that perspective. Solutions to this can include making sure that disabled computer scientists and human-computer interaction researchers are part of the team of designers and creators in the first place, or by making it easier for other disabled people to be involved at an early stage of design. This means that their experience and ideas can contribute to making the end product more relevant and useful for them and others. Alongside this there is education and advocacy – helping more young computer scientists, technologists and human-computer interaction designers to start thinking early about how their future products can be more inclusive.

An EPSRC project “Inclusive Public Activities for information and Communication Technologies” has been looking at some practical ways to help. Run by Prof. Cathy Holloway and Dr. Maryam Bandukda and their wider team at UCL they have established a panel of disabled academics and professionals who can be ‘critical friends’ to researchers planning new projects. By co-creating a set of guidelines for researchers they are providing a useful resource but it also means that disabled voices are heard at an early stage of the design process so that projects start off in the right direction.

Prof. Holloway and Dr. Bandukda are based at the Global Disability Innovation Hub (GDI Hub) in the department of computer science at UCL. GDI Hub is a global leader in disability innovation and inclusion and has research reaching over 30 million people in 60 countries. The GDI Hub also educates people to increase awareness of disability, reduce stigma and lay the groundwork for more disability-aware designers to benefit people in the future with better products.

An activity that the UCL team ran in February 2024, for schools in East London, was a week-long inclusive ICT “Digital Skills and Technology Innovation” bootcamp. They invited students in Year 9 and above to learn about 3D printing, 3D modelling, laser cutting, AI and machine learning using Python, artificial reality and virtual reality experiences along with a chance to visit Google’s Accessible Discovery Centre and use their skills to “tackle real-world challenges”.

What are some examples of Assistive Technology?

Screen-reading software can help blind or visually impaired people by reading aloud the words on the page. This is something that can help sighted people too, your document can read itself to you while you do something else. The entire world of audio books exists for this reason! D/deaf people can take part more easily in Zoom conversations if text-to-caption software is available so they can read what’s being said. That can also help those whose hearing is fine but who speak a different language and might miss some words. Similarly you can dictate your clever ideas to your computer and device which will type it for you. This can be helpful for someone with limited use of their hands, or just someone who’d rather talk than type – this might also explain the popularity of devices and tools like Alexa or Siri.

Web designers want to (and may need to*) make their websites accessible to all their visitors. You can help too – a simple thing that you can do is to add ALT Text (alternative text) to images. If you ever share an image or gif to social media it’s really helpful to describe what’s in the image for screen readers so that people who can’t view it can still understand what you meant.

*Thanks to regulations that were adopted in 2018 the designers of public sector websites (e.g. government and local council websites where people pay their council tax or apply for benefits) must make sure that their pages meet certain accessibility standards because “​​people may not have a choice when using a public sector website or mobile app, so it’s important they work for everyone. The people who need them the most are often the people who find them hardest to use”.

More on …

Careers

Examples of computer science and disability-related jobs

Both of the jobs listed below are CLOSED and are just for your information only.

  • [CLOSED] Islington Council, Digital Accessibility Apprentice (f/t), £24k, clos 7 July
    • Are you interested in web design and do you want to help empower disabled people to become fully engaged within the community? This is a great opportunity to learn about the rapidly growing digital accessibility industry. Qualified and experienced digital accessibility specialists are sought after.
  • [CLOSED] Global Disability Innovation Hub, Communications and Engagement Officer, £32k, London / hybrid, closed 4 July 2024
    • This role is focused on maximising comms-based engagement across the GDI Hub’s portfolio, supporting GDI Hub’s growing outreach across project-based deliverables and organisational comms channels (e.g. social media, websites, content generation).
Computer science & research

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos

Joyce Weisbecker: a teenager the first indie games developer?

CS4FN Banner

by Paul Curzon, Queen Mary University of London

Video games were once considered to be only of interest to boys, and the early games industry was dominated by men. Despite that, a teenage girl, Joyce Weisbecker, was one of the pioneers of commercial game development.

Originally, video games were seen as toys for boys. Gradually it was realised that there was a market for female game players too, if only suitably interesting games were developed, so the games companies eventually started to tailor games for them. That also meant, very late in the day, they started to employ women as games programmers. Now it is a totally normal thing to do. However, women were also there from the start, designing games. The first female commercial programmer (and possibly first independent developer) was Joyce Weisbecker. Working as an independent contractor she wrote her first games for sale in 1976 for the RCA Studio II games console that was released in January 1977.

RCA Studio II video games console
Image by WikimediaImages from Pixabay

Joyce was only a teenager when she started to learn to program computers and wrote her first games. She learnt on a computer that her engineer father designed and built at home called FRED (Flexible Recreational and Educational Device). He worked for RCA (originally the Radio Corporation of America), one of the major electronics, radio, TV and record companies of the 20th century. The company diversified their business into computers and Joyce’s father designed them for RCA (as well as at home for a hobby). He also invented a programming language called CHIP-8 that was used to program the RCA computers. This all meant Joyce was in a position to learn CHIP-8 and then to write programs for RCA computers including their new RCA Studio II games console before the machine was released, as a post-high school summer job.

The code for two games that she wrote in 1976, called Snake Race and Jackpot, were included in the manual for an RCA microcomputer called the COSMAC VIP, and she also wrote more programs for it the following year. These computers came in kit form for the buyer to build themselves. Her programs were example programs included for the owner to type in and then play once they had built the machine. Including them meant their new computer could do something immediately.

She also wrote the first game that she was paid for in that Summer of 1976. It was for the RCA Studio II games console, and it earned her $250 – well over $1000 in today’s money, so worth having for a teenager who would soon be going on to college. It was a quiz program, called TV School House I. It pitted two people against each other, answering questions on topics such as maths, history and geography, with two levels of difficulty. Questions were read from question booklets and whoever typed in the multiple choice answer number the fastest got the points for a question, with more points the faster they were. There is currently a craze for apps that augment physical games and this was a very early version of the genre.

Speedway screen from Wikimedia

She quickly followed it with racing and chase games, Speedway and Tag, though as screens were still very limited then, with only tiny screens, the graphics of all these games were very, very simple – eg racing rectangles around a blocky, rectangular racing track.

Unfortunately, the RCA games console itself was a commercial failure as it couldn’t compete with consoles like the Atari 2600, so RCA soon ended production. Joyce, meanwhile, retired from the games industry, still a teenager, ultimately becoming a radar signal processing engineer.

While games like Pong had come much earlier, the Atari 2600, which is credited with launching the first video game boom, was released in 1977, with Space Invaders, one of the most influential video games of all time, released in 1980. Joyce really was at the forefront of commercial games design. As a result her papers related to games programming, including letters and program listings, are now archived in the Strong National Museum of Play in New York.

More on …

Magazines …

Front cover of CS4FN issue 29 - Diversity in Computing

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded through EPSRC grant EP/W033615/1.

Happy #WorldEmojiDay 2024 – here’s an emoji film quiz & some computer science history

Emoji! 💻 😁

World Emoji Day is celebrated on the 17th of July every year (why?) and so we’ve put together a ‘Can you guess the film from the emoji’ quiz and added some emoji-themed articles about computer science and the history of computing.

  1. An emoji film quiz
  2. Emoji accessibility, and a ‘text version’ of the quiz
  3. Computer science articles about emoji

Emoji are small digital pictures that behave like text – you can slot them easily them in sentences (you don’t have to ‘insert an image’ from a file or worry about the picture pushing the text out of the way). You can even make them bigger or smaller with the text (🎬 – compare the one in the section title below). People use them as a quick way of sharing a thought or emotion, or adding a comment like a thumbs up so they’re (sort of) a form of data representation. Even so, communication with emoji can be just as tricky, in terms of being misunderstood, just as with using words alone. Different age groups might read the same emoji and understand something quite different from it. What do you think 🙂 (‘slightly smiling face’ emoji) means? What do people older or younger than you think it means? Lots of people think it means “I’m quite happy about this” but others use it in a more sarcastic way.

1. An emoji film quiz 🎬

You can view the quiz online or download and print from Word or PDF versions. If you’re in a classroom with a projector the PowerPoint file is the one you want.

More Computational Thinking Puzzles

2. Emoji accessibility, and a text version of the quiz

We’ve included a text version for blind or visually impaired people which can either be read out by someone or by a screen reader. Use the ‘Text quiz’ files in Word or PDF above.

More generally, when people share photographs and other images on social media it’s helpful if they add some information about the image to the ‘Alt Text’ (alternative text) box. This tells people who can’t easily see the image what’s in the picture. Screenreaders will also tell people what the emojis are in a tweet or text message, but if you use too many… it might sound like this 😬.

https://twitter.com/RNIB/status/1813489201842663723

3. Computer science articles about emoji

Happy World Emoji Day – 📅 17 July 2023 – how people use emoji to communicate and what it tells us about them 😀
The Emoji Crystal Ball

This next article is about the history of computing and the development of the graphical icons for apps that started life being drawn on gridded paper by Susan Kare. You could print some graph / grid paper and design your own!

Susan Kare: Icon Draw

A copy of this post can also be found as a permanent page at https://cs4fn.blog/emoji/

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos

Art Touch and Talk Tour Tech

A sculpture of a head and shouldrers, heavily textured with a network of lines and points
Image by NoName_13 from Pixabay

What could a blind or partially-sighted person get from a visit to an art gallery? Quite a lot if the art gallery puts their mind to it. Even more if they make use of technology. So much so, we may all want the enhanced experience.

The best art galleries provide special tours for blind and partially-sighted people. One kind involves a guide or curator explaining paintings and other works of art in depth. It is not exactly like a normal guided tour that might focus on the history or importance of a painting. The best will give both an overview of the history and importance whilst also giving a detailed description of the whole picture as well as the detail, emphasising how each part was painted. They might, for example, describe the brush strokes and technique as well as what is depicted. They help the viewer create a really detailed mental model of the painting.

One visually-impaired guide who now gives such tours at galleries such as Tate Britain, Lisa Squirrel, has argued that these tours give a much deeper and richer understanding of the art than a normal tour and certainly more than someone just looking at the pictures and reading the text as they wander around. Lisa studied Art History at university and before visiting a gallery herself reads lots and lots about the works and artists she will visit. She found that guided tours by sighted experts using guided hand movements in front of a painting helped her build really good internal models of the works in her mind. Combined with her extensive knowledge from reading, she wasn’t building just a picture of the image depicted but of the way it was painted too. She gained a deep understanding of the works she explored including what was special about them.

The other kind of tour art galleries provide is a touching tour. It involves blind and partially-sighted visitors being allowed to touch selected works of art as part of a guided tour where a curator also explains the art. Blind art lover, Georgina Kleege, has suggested that touch tours give a much richer experience than a normal tour, and should also be put on for all for this reason. It is again about more than just feeling the shape and so “working out its form that”seeing” what a sighted person would take in at a glance. It is about gaining a whole different sensory experience of the work: its texture, for example, not a lesser version just of what it looks like.

How might technology help? Well, the company, NeuroDigital Technologies, has developed a haptic glove system for the purpose. Haptic gloves are gloves that contain vibration pads that stimulate the skin of the person in different, very fine ways so as to fool the wearer’s brain into thinking it is touching things of different shapes and textures. Their system has over a thousand different vibration patterns to simulate different feelings of touching surfaces. They also contain sensors that determine the precise position of the gloves in space as the person moves their hands around.

The team behind the idea scanned several works of art using very accurate laser scanners that build up a 3D picture of the thing being scanned. From this they created a 3D model of the work. This then allowed a person wearing to feel as though they were touching the actual sculpture feeling all the detail. More than that the team could augment the experience to give enhanced feelings in places in shadow, for example, or to emphasise different parts of the work.

A similar system could be applied to historical artifacts too: allowing people to “feel” not just see the Rosetta Stone, for example. Perhaps it could also be applied to paintings to allow a person to feel the brush strokes in a way that could just not otherwise be done. This would give an enhanced version of the experience Lisa felt was so useful of having her hand guided in front of a painting and the brush strokes and areas being described. Different colours might also be coded with different vibration patterns in this way allowing a series of different enhanced touch tours of a painting, first exploring its colours, then its brush strokes, and so on.

What about talking tours? Can technology help there? AIs can already describe pictures, but early versions at least were trained on the descriptions people have given to images on the Internet: “a black cat sitting on top of the TV looking cute”, The Mona Lisa: a young woman staring at you”. That in itself wouldn’t cut it. Neither would training the AI on the normal brief descriptions on the gallery walls next to works of art. However, art books and websites are full of detail and more recent AIs can give very detailed descriptions of art works if asked. These descriptions include what the picture looks like overall, the components, colours, brushstrokes and composition, symbolism, historical context and more (at least for famous paintings). With specific training from curators and art historians the AIs will only get better. What is still missing for a blind person though from the kind of experience Lisa has when experiencing painting with a guide, is the link to the actual picture in space – having the guide move her hand in front of the painting as the parts are described. However, all that is needed to fill that gap is to combine a chat-based AI with a haptic glove system (and provide a way to link descriptions to spatial locations on the image). Then, the descriptions can be linked to positions of a hand moving in space in front of a virtual version of the picture. Combine that with the kind of system already invented to help blind people navigate, where vibrations on a walking stick indicate directions and times to turn, and the gloves can then not only give haptic sensations of the picture in front of the picture or sculpture, but also guide the person’s movement over it.

Whether you have such an experience in a gallery, in front of the work of art, or in your own front room, blind and partially sighted people could soon be getting much better experiences of art than sighted people. At which point, as Georgina Kleege, suggested for normal touch tours, everyone else will likely want the full “blind” experience too.

Paul Curzon, Queen Mary University of London

More on …

Magazines …

Front cover of CS4FN issue 29 - Diversity in Computing

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded through EPSRC grant EP/W033615/1.

Accessible Technology in the Voting Booth

CS4FN Banner

by Daniel Gill, Queen Mary University of London

Voting at an election: people deposting their voting slip
Image AI generated by Vilius Kukanauskas from Pixabay

On Thursday 4th July 2024, millions of adults around the UK went to their local polling station to vote for their representative in the House of Commons. However, for the 18% of adults who have a disability, this can be considerably more challenging. While the right of voters to vote independently and secretly is so important, many blind and partially sighted people cannot do so without assistance. Thankfully this is changing, and this election was hailed as the most accessible yet. So how does technology enable blind and partially sighted people to vote independently?

 There are two main challenges when it comes to voting for blind and partially sighted people. The names of candidates are listed down the left-hand side, so firstly, a voter needs to find the row of the person who they want to vote for. They then, secondly, need to put a cross in the box to the right. The image below gives an example of what the ballot paper looks like:

A mock up of a "CS4FN" voting slip with candidates HOPPER, Grace TURING, Alan Mathison BENIOFF, Paul Anthony Lovelace, Ada

To solve the first problem, we can turn to audio. An audio device can be used to play a recording of the candidates as the appear on the ballot paper. Some charities also provide a phone number to call before the election, with a person who can read this list out. This is great, of course, but it does rely on the voter remembering the position of the person that they want to vote for. A blind or partially sighted voted is also allowed to use a text reader device, or perhaps a smart phone with a special app, to read out what is on the ballot paper in the booth.

Lots of blind and partially impaired people are able to read braille: a way of representing English words using bumps on the paper (read more about braille in this CS4FN article). One might think that this would solve all the problems, but, in fact, there is a requirement that all the ballot papers for each constituency have a standard design to ensure they can be counted efficiently and without error.

The solution to the second problem is far more practical: the excitingly named tactile voting device. This is a simple plastic device which is placed on top of the ballot paper. Each of the boxes on the ballot paper (as shown to the right of the image above), has a flap above it with its position number embossed on it. When the voter finds the number of the person they want to vote for, they simply turn over the flap, and are guided by a perfectly aligned square guide to where the box is. The voter can then use that guide to draw the cross in the box.

This whole process is considerably more complicated than it is for those without disabilities – and you might be thinking, “there must be an easier way!” Introducing the McGonagle Reader (MGR)! This device combines both solutions into one device that can be used in the voting booth. Like the tactile voting device, it has flaps which cover each of the boxes for drawing the cross. But, next to those, buttons, which, when pressed, read out the information of the candidate for that row. This can save lots of time, removing the need to remember the position of each candidate – a voter can simply go down the page and find who they want to vote for and turn over the correct flap.

When people have the right to vote, it is especially important to ensure that they have the ability to use that right. This means that no matter the cost or the logistics, everyone should have access to the tools they need to vote for their representative. Progress is now being made but a lot more work still needs to be done.

To help ensure this happens in future, the RNIB want to know the experiences of those who voted or didn’t vote in the UK 2024 general election – see the survey linked from the RNIB page here.

More on …

Magazines …

Front cover of CS4FN issue 29 - Diversity in Computing

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded through EPSRC grant EP/W033615/1.

Mary Ann Horton and the invention of email attachments

A glowing envelope flying over a grid of envelopes
Image by Divya Gupta from Pixabay

Mary Ann Horton was transitioning to female at the time that she made one of her biggest contributions to our lives with a simple computer science idea with a big impact: a program that allowed binary email attachments.

Now we take the idea of sending each other multimedia files – images, video, sound clips, programs, etc for granted, whether by email or social networks, Back in the 1970s, before even the web had been invented, people were able to communicate by email, but it was all text. Email programs worked on the basis that people were just sending words, or more specifically streams of characters, to each other. An email message was just a long sequence of characters sent over the Internet. Everything in computers is coded as binary: 1s and 0s, but text has a special representation. Each character has its own code of 1s and 0s, that can also be thought of as a binary number, but that can be displayed as the character by programs that process it. Today, computers use a universally accepted code called Unicode, but originally most adopted a standard code called ASCII. All these codes are just allocations of patterns of 1s and 0s to each character. In ASCII, ‘a’ is represented by 1100001 or the number 97, whereas A is 1000001 or number 65, for example. These are only 7 bits long and as computers store data in bytes of 8 bits at a time this means that not all patterns of binary (so representable numbers) correspond to one of the displayable characters that email messages were expected to contain by the programs that processed them.

That is fine if all you have done is used programs like text editors, that output characters so you are guaranteed to be sending printable characters. The problem was other kinds of data whether images or runnable programs, are not stored as sequences of characters. They are more general binary files, meaning the data is long sequences of byte-sized patterns of 1s and 0s and what those 1s and 0s meant depended on the kind of data and representation used. If email programs were given such data to send, pass on or receive, they would reject or more likely mangle it as not corresponding to characters they could display. The files didn’t even have to be non-character formats, as at the time some computer systems used a completely different code for characters. This meant text emails could also be mangled just because they passed through a computer using a different format of character.

Mary Ann realised that this was all too restrictive for what people would be needing computers to do. Email needed to be more flexible. However, she saw that there was a really easy solution. She wrote a program, called uuencode that could take any binary file and convert it to one that was slightly longer but contained only characters. A second program she wrote, called uudecode converted these files of characters back to the original binary file to be saved by the receiving email program exactly it was originally on the source program.

All the uuencode program did was take 3 bytes (24 bits) of the binary file at a time, split them into groups of 6 bits so effectively representing a number from 0 to 63, add 32 to this number so the numbers are now in the range 32 to 95 and those are the numbers so binary patterns of the printable characters that the email programs expected. Each three bytes were now 4 printable characters. These could be added to the text of an email, though with a special start and end sequence included to identify it as something to decode. uudecode just did this conversion backwards, turning each group of 4 characters back into the orginal three bytes of binary.

Email attachments had been born, and ever since communication programs, whether email, chat or social media, have allowed binary files, so multimedia, to be shared in similar ways. By seeing a coming problem, inventing a simple way to solve it and then writing the programs, Mary Ann Horton had made computers far more flexible and useful.

Paul Curzon, Queen Mary University of London

More on …

Related Magazines …

cs4fn issue 14 cover

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos

Pac-Man and Games for Girls

In the beginning video games were designed for boys…and then came Pac-Man.

Pac-man eating dots
Image by OpenClipart-Vectors from Pixabay

Before mobile games, game consoles and PC based games, video games first took off in arcades. Arcade games were very big earning 39 billion dollars at their peak in the 1980s. Games were loaded into bespoke coin-operated arcade machines. For a game to do well someone had to buy the machines, whether actual gaming arcades or bars, cafes, colleges, shopping malls, … Then someone had to play them. Originally boys played arcade games the most and so games were targeted at them. Most games had a focus on shooting things: games like asteroids and space invaders or had some link to sports based on the original arcade game Pong. Girls were largely ignored by the designers… But then came Pac-Man. 

Pac-Man, created by a team led by Toru Iwatani,  is a maze game where the player controls the Pac-Man character as it moves around a maze, eating dots while being chased by the ghosts: Blinky, Pinky, Inky, and Clyde. Special power pellets around the maze, when eaten, allow Pac-Man to chase the ghosts for a while instead of being chased.

Pac-Man ultimately made around $19 million dollars in today’s money making it the biggest money making video arcade game of all time. How did it do it? It was the first game that was played by more females than males. It showed that girls would enjoy playing games if only the right kind of games were developed. Suddenly, and rather ironically given its name, there was a reason for the manufacturers to take notice of girls, not just boys.

A Pac-man like ghost
Image by OpenClipart-Vectors from Pixabay

It revolutionised games in many ways, showing the potential of different kinds of features to give it this much broader appeal. Most obviously Pac-Man did this by turning the tide away from shoot-em up space games and sports games to action games where characters were the star of the game, and that was one of its inventor Toru Iwatani’s key aims. To play you control Pac-Man rather than just a gun, blaster, tennis racket or golf club. It paved the way for Donkey Kong, Super Mario, and the rest (so if you love Mario and all his friends, then thank Pac-Man). Ultimately, it forged the path for the whole idea of avatars in games too. 

It was the first game to use power ups where, by collecting certain objects, the character gains extra powers for a short time. The ghosts were also characters controlled by simple AI – they didn’t just behave randomly or follow some fixed algorithm controlling their path, but reacted to what the player does, and each had their own personality in the way they behaved.

Because of its success, maze and character-based adventure games became popular among manufacturers, but more importantly designers became more adventurous and creative about what a video game could be. It was also the first big step towards the long road to women being fully accepted to work in the games industry. Not bad for a character based on a combination of a pizza and the Japanese symbol for “mouth”.

– Paul Curzon, Queen Mary University of London

More on …

Magazines …

Front cover of CS4FN issue 29 - Diversity in Computing

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos

T. V. Raman and his virtual guide dogs

Guide dog silhouette with binary superimposed
Image by PC modifying dog from Clker-Free-Vector-Images from Pixabay

It’s 1989, a year with lots of milestones in Computer Science. In March, Tim Berners-Lee puts down in writing the idea of an “information management system”, later to become the world wide web. In July, Nintendo releases the Game Boy in North America selling 118 million units worldwide over its 14-year production.

Come autumn, a 24-year-old arrives in Ithaca, US, home of Cornell University. He would be able to feel the cool September air as it blows off Cayuga Lake, smell the aromas from Ithaca’s 190 species of trees, and listen to a range of genres in the city’s live music scene. However,, he couldn’t take in the natural beauty of the city in its entirety as he started his PhD … because he was blind. That did not stop him going on to have a gigantic impact on  the lives of blind and partially sighted people worldwide.

T. V. Raman was born in Pune, India, in 1965. He had been partially sighted from birth, but at the age of 14 he became blind due to a disease called glaucoma. Throughout his life, however, he has not let this stop him.

While he was partially sighted, he was able to read and write – but as his sight worsened, and with the help of his brother, mentors, and aides, he was still able to continue learning from textbooks, and solve problems which were read to him. At the height of its popularity, in the early 1980s, he also learned how to solve a specially customised Rubik’s cube, and could do so in about 30 seconds.

Raman soon developed an interest in mathematics, and around 1983 started studying for a Maths degree at the University of Pune. On finishing in 1987, he studied for a Masters degree at the Indian Institute of Technology Bombay, this time in Computer Science and Maths. It was with the help of student volunteers he was able to learn from textbooks and assistance with programming was provided by an able volunteer. 

Today people with no vision often use a screen reader to hear what is on a screen. It not everyone is lucky enough to have so much help as Raman and screen readers play the part of all those human volunteers who helped him. Raman himself played a big part in their development.

Modern screen readers allow you to navigate the screen part-by-part, with important information and content read to you. Many of these systems are built into operating systems, such as the Narrator in Windows (which uses a huge number of keyboard shortcuts), and Google TalkBack for Android devices (where rubbing the screen, vibration, and audio hints are used). These simpler screen readers might already be installed on your system – if so have a go with them!

While Raman was learning programming, such screen readers were still in their infancy. It was only in the 1980s that a team at IBM developed a screen reader for the command-line interface of the IBM DOS (which Raman would later use), and it would be many years before screen readers were available for the much more challenging graphical user interfaces we’re so accustomed to today.

It was at Cornell University where Raman settled on his career-long research interest: accessibility. He originally intended to do an Applied Mathematics PhD, but then discovered the need for ways to use speech technology to read complicated documents, especially those with embedded mathematics. For his dissertation, he therefore developed the Audio System for Technical Readings (ASTER) to solve the problem.

What he realised was that when looking at information visually our eyes are active taking in information from different places but the display is passive. With an audio interface this is reversed with the ear passive and the display actively choosing the order of information presented. This makes it impossible to get a high level view first and then dive into particular detail. This is a big problem when ‘reading’ maths by listening to it. His system solved the problem using audio formatting which allows the listener to browse the structure of information first.

He named this program after his first guide dog, Aster, which he obtained, alongside a talking computer, in early 1990. Both supported him throughout his PhD. For this work, he received the ACM Doctoral Dissertation Award, a prestigious yearly worldwide celebrating the best PhD dissertation in computer science and related fields.

Following on from this work, he developed a program called Emacspeak, an audio desktop, which, unlike a screen reader, takes existing programs and makes them work with audio outputs. It makes use of Emacs, a family of text editors (think notepad, but with lots more features), as well as a programming language called Lisp. Raman has continued to develop Emacspeak ever since and the program is often bundled within Linux operating system installations. Like ASTER, versions of this program are dedicated to his guide dogs.

Following his PhD, Raman worked briefly with Adobe Systems and IBM, but, since 2005, has worked with Google on auditory user interfaces, accessibility, and usability. In 2014, alongside Google colleagues, he published a paper on a new application called JustSpeak, a system for navigating the Android operating system with voice commands. He has also gone back to his roots, integrating mathematical speech into the ChromeVox, the screen reader built into Chromebook devices.

Despite growing up in a time of limited access to computers for blind and visually impaired people, Raman was able, with the help of his brother and student volunteers, to learn how to program, solve a Rubik’s cube, and solve complex maths problems. With early screen readers he was also able to build tools for fellow blind and visually impaired people, and then benefit himself from his own tools to achieve even more.

Guide dogs can transform the lives of blind and partially sighted people by allowing them to do things in the physical world that they otherwise could not do. T. V. Raman’s tools provide a similar transformation in the digital world, changing lives for the better.

– Daniel Gill, Queen Mary University of London

More on …

Magazines …

Front cover of CS4FN issue 29 - Diversity in Computing

Subscribe to be notified whenever we publish a new post to the CS4FN blog.

This blog is funded by EPSRC on research agreement EP/W033615/1.

QMUL CS4FN EPSRC logos