Category: Law

The gender shades audit

by Jo Brodie, Queen Mary University of London

Face recognition technology is used widely, such as at passport controls and by police forces. What if it isn’t as good at recognising faces as it has been claimed to be? Joy Buolamwini and Timnit Gebru tested three different commercial systems and found that they were much more likely to wrongly classify darker skinned female faces compared to lighter or darker skinned male faces. The systems were not reliable.

Different skin tone cosmetics
Image by Stefan Schweihofer from Pixabay

Face recognition systems are trained to detect, classify and even recognise faces based on a bank of photographs of people. Joy and Timnit examined two banks of images used to train the systems and found that around 80 percent of the photos used were of people with lighter coloured skin. If the photographs aren’t fairly balanced in terms of having a range of people of different gender and ethnicity then the resulting technologies will inherit that bias too. The systems examined were being trained to recognise light skinned people.

The pilot parliaments benchmark

Joy and Timnit decided to create their own set of images and wanted to ensure that these covered a wide range of skin tones and had an equal mix of men and women (‘gender parity’). They did this using photographs of members of parliaments around the world which are known to have a reasonably equal mix of men and women. They selected parliaments both from countries with mainly darker skinned people (Rwanda, Senegal and South Africa) and from countries with mainly lighter skinned people (Iceland, Finland and Sweden).

They labelled all the photos according to gender (they had to make some assumptions based on name and appearance if pronouns weren’t available) and used a special scale called the Fitzpatrick scale to classify skin tones (see Different Shades below). The result was a set of photographs labelled as dark male, dark female, light male, light female, with a roughly equal mix across all four categories: this time, 53 per cent of the people were light skinned (male and female).

Testing times

Joy and Timnit tested the three commercial face recognition systems against their new database of photographs (a fair test of a wide range of faces that a recognition system might come across) and this is where they found that the systems were less able to correctly identify particular groups of people. The systems were very good at spotting lighter skinned men, and darker skinned men, but were less able to correctly identify darker skinned women, and women overall. The tools, trained on sets of data that had a bias built into them, inherited those biases and this affected how well they worked.

As a result of Joy and Timnit’s research there is now much more recognition of the problem, and what this might mean for the ways in which face recognition technology is used. There is some good news, though. The three companies made changes to improve their systems and several US cities have already banned the use of this technology in criminal investigations, with more likely to follow. People worldwide are more aware of the limitations of face recognition programs and the harms to which they may be (perhaps unintentionally) put, with calls for better regulation.

Different Shades
The Fitzpatrick skin tone scale is used by skin specialists to classify how someone’s skin responds to ultraviolet light. There are six points on the scale with 1 being the lightest skin and 6 being the darkest. People whose skin tone has a lower Fitzpatrick score are more likely to burn in the sun and are at greater risk of skin cancer. People with higher scores have darker skin which is less likely to burn and have a lower risk of skin cancer. A variation of the Fitzpatrick scale, with five points, is used to create the skin tone emojis that you’ll find on most messaging apps in addition to the ‘default’ yellow.

More on …

Related Magazines …

EPSRC supports this blog through research grant EP/W033615/1. 

Edie Schlain Windsor and same sex marriage

CS4FN Banner
US Supreme court building Image by Mark Thomas from Pixabay
Image by Mark Thomas from Pixabay

Edie Schlain Windsor was a senior systems programmer at IBM. There is more to life than computing though. Just like anyone else, Computer Scientists can do massively important things aside from being very good at computing. Civil rights and over-turning unjust laws are as important as anything. She led the landmark Supreme Court Case (United States versus Windsor) that was a milestone for the rights of same-sex couples in the US.

Born to a Jewish immigrant family, Edie worked her way up from an early data entry job at New York University to ultimately become a senior programmer at IBM and then President of her own software consultancy where she helped LGBTQ+ organisations become computerised.

Having already worked as a programmer at an energy company called Combustion Engineering, she joined IBM on completing her degree in 1958 so was one of the early generation of female programmers, before the later idea of the male programmer stereotype took hold. Within ten years she had been promoted to the highest technical position in IBM, that of a Senior Systems Programmer: so one of their top programmers lauded as a wizard debugger. She had started out programming mainframe computers, the room size computers that were IBM ‘s core business at the time. They both designed and built the computers as well as the operating system and other software that ran on them. Edie became an operating systems expert, and a pioneer computer scientist also working on natural language processing programs, aiming to improve the interactivity of computes. Natural Language Processing was then a nascent area but that by 2011 IBM led spectacularly with its program Watson winning the quiz show Jeopardy! answering general knowledge questions playing against human champions.

Before her Supreme Court case overturned it, a law introduced in 1996 banned US federal recognition of same-sex marriages. It made it federal law that marriage could only exist between a man and a woman. Individual states in the US had introduced same-sex marriage but this new law meant that such marriages were not recognised in general in the US. Importantly, for those involved it meant a whole raft of benefits including tax, immigration and healthcare benefits that came with marriage were denied to same-sex couples.

Edie had fallen in love with psychologist Thea Spyer in 1965, and two years later they became engaged, but actually getting married was still illegal. They had to wait almost 30 years before they were even allowed to make their partnership legal, though still at that point not marry. They were the 80th couple to register on the day such partnerships were finally allowed. By this time Thea had been diagnosed with multiple sclerosis, a disease that gradually leads to the central nervous system breaking down, with movement becoming ever harder. Edie was looking after her as a full time carer, having given up her career to do so. They both loved dancing and did so throughout their life together even once Thea was struggling to walk, using sticks to get on to the dance floor and later dancing in a wheelchair. As Thea’s condition grew worse it became clear she had little time to live. Marriage was still illegal in New York, however, so before it was too late, they travelled to Canada and married there instead.

When Thea died she left everything to Edie in her will. Had Edie been a man married to Thea, she would not have been required to pay tax on this inheritance, but as a woman and because same-sex marriages were deemed illegal she was handed a tax bill of hundreds of thousands of dollars. She sued the government claiming the way different couples were treated was unfair. The case went all the way to the highest court, the Supreme Court, who ruled that the 1996 law was itself unlawful. Laws in the US have as foundation a written constitution that dates back to 1789. The creation of the constitution was a key part of the founding of the United States of America itself. Without it, the union could easily have fallen apart, and as such is the ultimate law of the land that new laws cannot overturn. The problem with the law banning same sex marriage was that it broke the 5th amendment of the constitution added in 1791, one of several amendments made to ensure people’s rights and justice was protected by the constitution.

The Supreme Court decision was far more seismic than just refunding a tax bill, however. It overturned the law that actively banned same-sex marriage, as it fell foul of the constitution, and this paved the way for such marriages to be made actively legal. In 2014 federal employees were finally told they should perform same-sex marriages across the US, and those marriages gave the couple all the same rights as mixed-sex marriages. Because Edie took on the government, the US constitution, and so justice for many, many couples prevailed.

– Paul Curzon, Queen Mary University of London

More on …

Related Magazines …

cs4fn issue 14 cover

This blog is funded through EPSRC grant EP/W033615/1.

The Mummy in an AI world: Jane Webb’s future

The sarcophagus of a mummy
Image by albertr from Pixabay

Inspired by Mary Shelley’s Frankenstein, 17-year old Victorian orphan, Jane Webb secured her future by writing the first ever Mummy story. The 22nd century world in which her novel was set is perhaps the most amazing thing about the three volume book though.

On the death of her father, Jane realised she needed to find a way to support herself and did so by publishing her novel “The Mummy!” in 1827. In contrast to their modern version as stars of horror films, Webb’s Mummy, a reanimation of Cheops, was actually there to help those doing good and punish those that were evil. Napoleon had, through the start of the century, invaded Egypt, taking with him scholars intent on understanding the Ancient Egyptian society. Europe was fascinated with Ancient Egypt and awash with Egyptian artefacts and stories around them. In London, the Egyptian Hall had been built in Piccadilly in 1812 to display Egyptian artefacts and in 1821 it displayed a replica of the tomb of Seti I. The Rosetta Stone that led to the decipherment of hieroglyphics was cracked in 1822. The time was therefore ripe for someone to come up with the idea of a Mummy story.

The novel was not, however, set in Victorian times but in a 22nd century future that she imagined, and that future was perhaps more amazing than the idea of a mummy coming to life. Her version of the future was full of technological inventions supporting humanity, as well as social predictions, many of which have come to fruition such as space travel and the idea that women might wear trousers as the height of fashion (making her a feminist hero). The machines she described in the book led to her meeting her future husband, John Loudon. As a writer about farming and gardening he was so impressed by the idea of a mechanical milking machine included in the book, that he asked to meet her. They married soon after (and she became Jane Loudon).

The skilled artificial intelligences she wrote into her future society are perhaps the most amazing of her ideas in that she was the first person to really envision in fiction a world where AIs and robots were embedded in society just doing good as standard. To put this into context of other predictions, Ada Lovelace wrote her notes suggesting machines of the future would be able to compose music 20 years later.

Jane Webb’s future was also full of cunning computational contraptions: there were steam-powered robot surgeons, foreseeing the modern robots that are able to do operations (and with their steady hands are better at, for example, eye surgery than a human). She also described Artificial Intelligences replacing lawyers. Her machines were fed their legal brief, giving them instructions about the case, through tubes. Whilst robots may not yet have fully replaced barristers and judges, artificial intelligence programs are already used, for example, to decide the length of sentences of those convicted in some places, and many see it now only being a matter of time before lawyers are spending their time working with Artificial Intelligence programs as standard. Jane’s world also includes a version of the Internet, at a time before electric telegraph existed and when telegraph messages were sent by semaphore between networks of towers.

The book ultimately secured her future as required, and whilst we do not yet have any real reanimated mummy’s wandering around doing good deeds, Jane Webb did envision lots of useful inventions, many that are now a reality, and certainly had pretty good ideas about how future computer technology would pan out in society…despite computers, never mind artificial intelligences, still being well over a century away.

– Paul Curzon, Queen Mary University of London

More on …

Related Magazines …

EPSRC supported this article through research grants (EP/K040251/2 and EP/K040251/2 held by Professor Ursula Martin as well as grant EP/W033615/1). 

QMUL CS4FN EPSRC logos

In a New York nanosecond

New technology can have unforeseen effects. The Law in particular can sometimes struggle to keep up, but for the IT savvy lawyer that can mean opportunity. For example, one bunch of lawyers realised that the way money moves round the world electronically could give their clients the edge. Nanoseconds are all it takes. As a result, a bunch of New York nanoseconds gave Judges in the Southern district court of the city a real headache.

Image by Gerd Altmann from Pixabay

Different countries have different laws. That means lawyers will go out of their way to apply the law for their clients in the right country. It can make all the difference. Unlike some other countries US maritime law allows a person to freeze a person’s assets, even before a decision has been reached, when there is a maritime claim against them. For example, if a merchant hasn’t been paid for a shipload of cargo, or if a shipyard hasn’t been paid for ship repairs, then they can use this rule to freeze the defaulter’s money. Otherwise a win, when it comes, could be rather hollow, with the money long placed out of reach. The only trouble for the lawyers is that the money has to be in the US for the US law to apply.

Frozen money

That is where the technology comes in. Bankers don’t ship physical money from country to country, it’s all done electronically now… A consequence of the way the banking system was set up is that dollar transactions had to pass through the US banking centre in Manhattan as the money has to move from place to place. That’s an easy thing to require data to do in the age of the Internet. It only spends a fraction of a second in New York before it jumps on somewhere else. The law, of course, makes no distinction over shrinking timescales in which computers make things happen. A prepared lawyer can have the money frozen in that instant as just at that moment it is in the US.

That was great for people wanting to hold up money. It was a nightmare for the New York judges, though. Once the lawyers caught on about those nanoseconds the work started stacking up for the judges. All those fractions of a second added up to hours of the Judges’ time granting permission for the money to be seized. Every day the poor New York judges had to process hundreds and hundreds of requests, just in case some disputed money happened to pass through that day. To seize the money, it wasn’t enough just to put in a request and wait, ready to pounce when the money lands in Manhattan. Instead, just like a Spider re-spinning its web every morning, the trap had to be renewed daily. To do that the lawyers had to serve the bank daily with notice that if any money passed through that day it had to be stopped in its high speed tracks.

New technology constantly brings up new problems like this, when old laws or procedures are found to be wanting when technology changes the way things are done: changes things far beyond the imagination of those who drafted the laws. Just as technology never stands still, neither does the law…or the IT savvy lawyer.

by Paul Curzon, Queen Mary University of London, Summer 2017, updated Spring 2021