AI#07 - Building Ethical AI and The Challenges Ahead of us
In this article, we discuss AI and ethics and go over key concepts such as privacy, accountability, safety, transparency, and fairness.
In a recent article,Mozilla reported that:
25 major car brands collect and share deeply personal data, including sexual activity, facial expressions, and genetic and health information
They mentioned that:
Often, “consent” to collect personal data is presumed by simply being a passenger in the car. For example, Subaru states that by being a passenger, you are considered a user —and by being a user, you have consented to their privacy policy.
Car brands like Toyota have 12 different privacy policies, but no one to discuss privacy concerns; others share personal information with law enforcement and governments (for your own security) and regularly experience data breaches.
On one hand, with the AI revolution upon us, our society is up an arm about AI’s ethics, privacy, and fairness. On the other hand, over one billion drivers expose privacy data every day they take their cars. This has been going on for years. Regulators know it. The Mozilla article went pretty unnoticed. Nobody cares, not even the drivers.
We have 3.8 billion people on earth with smart phones. Without a doubt, those phones and their apps must be built with privacy in mind and not display the flaws of car makers. Well…
It is easy to get carried away, have passionate debates on the impact of future technological advancements, and forget how today’s technology impacts our lives. As a former Latin student, I embrace the wisdom of this aphorism: "Carpe diem, quam minimum credula postero," which can be translated as "Seize the day, putting as little trust as possible in the future”.
This is article seven of our AI article series. We discuss AI and ethics and go over key concepts such as privacy, accountability, safety, transparency, and fairness. Each concept deserves in-depth research and discussion. I am just scratching the surface and bringing random thoughts on each topic.
Privacy
This theme promotes the idea that AI systems should respect people's privacy. This includes using data responsibly when developing technology and giving individuals control over their data and decisions.
Privacy includes the following concepts:
Consent: an individual’s data should not be used without their knowledge and permission
Ability to restrict processing: an individual’s ability to restrict the use of his data for AI processing
Right of removal: an individual’s right to the removal (rectification) of their personal data, similar to GDPR in Europe
Control over the use of data: an individual’s ability to determine how their data is used and for what purpose
Privacy by design: developers’ obligation to include principles of data privacy in their AI system
Random thoughts
We are asked every day for consent. When you visit a new website, you have the choice to accept all cookie tracking or decline. Most people surrender and consent without reading the fine print. If I do not consent to my favorite French recipe site, they ask me to buy a monthly subscription. Today’s consent is a joke, a meaningless regulation that does nothing to promote privacy.
Imagine giving consent to an AI system. Your personal data may be used to train and teach the AI model. Your privacy may be at risk. Restricted processing and control over the use of your own data have become extremely hard to enforce. Why? AI models are black boxes. Once your data is used for training, it is there forever and used to make inferences (predictions).
This article has a great quote summarizing my point of view:
This charade of consent has made it obvious that notice-and-choice has become meaningless. For many AI applications … it will become utterly impossible.
The solution seems obvious: shift the burden of protecting individual privacy from consumers over to the businesses that collect personal data. Make businesses liable for data privacy leaks. Regulate how companies collect and process personal data for usage in their models and how they can use and share it.
It gets more complicated, though.
One of AI's “super powers” is its inference ability, i.e., its ability to make clever guesses and/or decisions. In healthcare, the best AI models can predict early-stage cancer and tumors from C-Scan better than a healthcare specialist (statistically speaking). This creates a potential risk for data privacy because the AI is able to infer who you are and predict a lot about you without accessing your personal data. For instance, ChatGPT may find out about your cultural background, income level, and social class just by interacting with you. The way you write and the words and expressions you use may give away who you are.
Accountability
This principle addresses questions about who should be accountable for decisions and societal impacts that are no longer under human control.
Accountability includes the following concepts:
Verifiability and replicability: the AI system functions the same way under the same conditions and provides enough information to validate its operations
Evaluation and auditing requirements: the AI system and the underlying technology can be audited by users or a third-party authority
Ability to appeal: an individual impacted by a decision informed or fully made by AI has the right to appeal in a court of law
Liability and legal responsibility: individuals or entities whose AI caused harm can be held liable and legally responsible
Random thoughts
Accountability is necessary throughout the AI lifecycle. In the design phase, the principles of verifiability and replicability can be achieved with sufficient testing and internal controls. The next two principles, auditing requirement and ability to appeal, while essential, are harder to achieve.
Building AI systems that can be audited is a real challenge. AI models such as LLMs and neural networks with deep learning capabilities remain a black box today. Over time, new auditing techniques targeting FMs (Foundation Models) and LLMs will appear.
However, it seems highly unlikely that we can achieve a full, transparent audit. AI models are complex mathematical models with billions of parameters that influence each AI output. Good luck auditing the AI logic that leads to an output, explaining which of the 100+ billion parameters and neural network layer(s) led to the output, and doing it in a way your average Joe understands!
The right to appeal is an essential basic right. AI-driven or AI-assisted decisions made without your knowledge may become the new reality. Every day, dozens of automated decisions may be made about you when you visit a hospital, apply for social programs, reserve a hotel, apply for a job, or have your yearly career review.
As discussed earlier, “forced consent” is dangerous. Imagine that regulations for individual protection rights exist in the code of law. Every entity using AI will engineer forced content to shift liability onto the consumer. Going to the ER is a great example. You have to sign a waiver that you will not sue the hospital for malpractice. What choice do you have when your child or wife is sick and in pain?
The development of new regulations and a legal framework will take time and may be bumpy. It will be interesting to see if an international code of law is developed or if countries decide to implement their own regulations. Europe introduced GDPR a few years ago for user privacy. For companies selling to consumers, the cost of becoming GDPR-compliant was substantial. Given the cost and complexity of building AI models, new AI regulations could also introduce barriers to entry that benefit big players, at the detriment of startups and innovators. We could create oligopolies where only the big players can afford the cost of heavy regulations, including paying the fine (big banks are notorious for that).
Safety and Security
Safety refers to the proper internal functioning of an AI system and the avoidance of unintended harm. By contrast, security addresses external threats to an AI system.
Those concepts can be further divided into the following topics:
Safety: the AI system is reliable and does what it is supposed to do without harming its users
Security by Design: it is the developers’ responsibility to design secure AI systems
Security: the AI system’s ability to resist to external threats
Predictability: the AI system’s output is consistent with the input, making it trustworthy
Random thoughts
We have very little return on experience when it comes to deploying secure AI systems. I would not be surprised to hear in the coming years about hackers exploiting vulnerabilities in first-generation AI systems. In the early days of computing, anti-viruses did not exist or were quickly outdated. Like it did back then, the industry will eventually catch up and build robust AI that is safe and secure.
Personal data leak is another concern. I do not think new AI apps will be the primary attack vector. A smart hacker may, however, leverage AI to engineer a social attack. It could go like this: your husband is traveling for business; he calls you and explains that his credit card was declined when he tried to pay the taxi driver; he is going to miss his meeting and asks for your credit card details; you give it to him, thinking you saved his day! An hour later, $20.000 has been debited from your account. Little did you know that you talked to an AI bot who faked your husband’s voice and learned about your family thanks to social media!
Safety is going to be very hard to test. Remember, an AI model is a blackbox, and we do not know why it produces a specific output. The outcome is not deterministic anymore, like in software programming. What happens when the developer misses a critical scenario?
Transparency and Explainability
Explainability refers to the ability to understand and interpret the decisions or outputs made by AI models and make their processes transparent and comprehensible to humans. AI transparency refers to the openness and clarity of AI systems, making their operations, decisions, and underlying algorithms easily understandable and accessible to users and stakeholders.
Those concepts can be further divided into the following topics:
Transparency: the AI system allows for oversight of its operations; this is often tied to open-sourcing
Explainability: the AI system can provide clear, complete and testable explanations of what it is doing and why
Notification when Interacting with AI: an individual is always aware if and when he is engaging with AI
Right to information: an individual understands how AI systems use and interact with their personal data to make an inference
Notification When AI Makes a Decision: an individual always know when they are exposed to automated decisions through an AI system and can decide to opt-out
Random thoughts
The biggest AI detractors refer to transparency and explainability as massive shortcomings of AI technology. Explainability promotes trust in AI recommendations and drives accountability. It is especially important when the AI recommendation impacts individuals or a business outcome, e.g., AI-assisted healthcare diagnosis, loan approval, or job applicant review. Every person interacting with an AI application has the right to understand why the AI is making a recommendation and how it reached its conclusion.
Microsoft Bing Chat provides links to online resources in the answers. After a quick chat with Bing Chat, I found out that it only performs embeddings, i.e., Bing Chat runs a search query and passes the top results to ChatGPT. Many users, including me, will think those online resources explain Bing Chat’s answer and trust them. This is misleading. Microsoft cannot explain why a specific answer was generated or the relationship between these resources.
Ali Ghodsi, Databricks CEO, made an interesting comment on the need for more transparency on how AI model performance is being assessed. He made the argument that it is easy to cheat tests and make false claims about the superiority of your LLM or foundation model. He is not the only one. Suzan Zhang speculates here that phi-1.5, a seemingly high-performing open-source LLM, may have unintentionally been trained using benchmark datasets. Pretraining on the Test Set Is All You Need paper explains how a small 1M parameter LLM outperforms all other models. This is achieved by training the model on all downstream academic benchmarks
Ali Ghodsi recommends having a third party create and run the model performance assessment; e.g., doctors create a questionnaire that remains secret, and a third party is paid by AI vendors to test their model performance.
The right to information is another critical component of transparency. If an AI wants to leverage your personal data, you should be informed and give consent. You should be able to opt out before, during, or after your data is used. You should be provided with a human contact and receive proof that your personal data was removed from the AI system with proper documentation. The fine for failing to do so should be hefty.
This sounds great on paper. However, once your personal data is part of the LLM, it is very hard (impossible?) to remove. To my knowledge, no technique exists to remove training data and cancel its impact on the model. For the AI, it would be like unlearning something. Since we cannot explain the AI output, we cannot effectively remove training data.
We are not just talking about an individual request. Imagine the case of thousands of users making the same request after someone shared on social media an AI prompt that leaks personal data. Critical neural networks in the LLM may rely on training data from those users. Even if you could remove them, the whole LLM may become useless or need serious re-training, making it obsolete for a while.
Finally, AI systems should offer proper notifications when they make decisions about an individual. Individuals may not know they are interacting with an AI. Facial recognition could be used in public areas such as airports, streets, and parks and for unintended purposes like social scoring. You may be assigned an AI-henerated score when applying for a mortgage loan or insurance. Once a score is assigned to you, good luck changing it. You may turn around your life, but AI will not take that into account because it makes decisions based on mathematical patterns.
Life is not linear, nor should we try to look at individuals only through the lens of mathematical models. Automated scoring systems have other pitfalls. They force you to participate at the risk of exclusion. When I moved to the US, I could not get a credit card or rent a place because I had no credit score. Eventually, I got there thanks to my wife, who is a US citizen. Despite being a good saver, I was considered a persona non grata until I could build my credit score. Automated scores need your data. Processes built upon such scores leave individuals little choice.
Fairness and Non-Discrimination
As more and more AI systems get used to make decisions, it is critical to ensure they remain fair and do not discriminate against individuals or special groups. Several principles can guide us:
Prevention of Bias: AI bias in training data, AI algorithmic models, or AI deployment needs to be mitigated to prevent discrimination
Fairness: as per Microsoft's definition, “AI systems should treat people fairly.”
Equality: this extends the concept of fairness and states that the same rules should apply to everyone
Inclusiveness: AI benefits and empower as many people as possible and the economic benefits derived from AI should be distributed broadly and equally to benefit all of humanity
Random thoughts
Biases in AI models may come from several sources. The training data selection may be biased. The auditor is biased, knowingly or not. There is no clear and agreed-upon definition of bias leading to personal interpretation. An unbiased AI model becomes biased after interacting with users.
I find it fascinating that nobody called out RLHF (Reinforcement Learning from Human Feedback) as a potentially dangerous step in the training of an AI model. In essence, RLHF leverages human input to "fine-tune and correct” an AI model. Think of it as a surgeon with superpowers. The surgeon performs brain surgeries on his patients to remove a memory and introduce a new one. The surgeon does not need to study for ten years, become an expert in this field, and comply with laws and regulations. Anyone can be a surgeon, even my twelve-year-old.
What could go wrong? The individual performing RLHF is biased. His boss is biased. The company influences, directly or indirectly, the RLHF process. The organization is based in a jurisdiction that has different rules (with GDPR or not) and is allowed to sell outside their jurisdiction. The organization builds and sells products or services morally questionable for war, security monitoring, human control, etc. The societal norms influence the RLHF step and make the AI model biased when used in other countries, or they influence the developers (see example below). A bad actor retrains a foundation model with RLHF for cyber attacks, theft, or malware.
The Good Samaritan
Alan is a 10th grader and uses an AI teaching assistant to do his homework. Alan and his family belong to a special ethnic group. After taking a math test, the AI ranks Alan at an 8th grade level. Outraged, his parents and school administrators call the AI biased and against the principle of equality. The story reaches social media and forces the software vendor to issue a public apology and “optimize” its AI model. They use RLHT to quickly re-program the AI so the LLM only provides kids with encouragement and positive feedback.
While fictitious, this story presents an interesting dilemma. Who has the authority to say the AI is biased or not? What is the responsibility of software vendors? Shall special interest groups influence the way AI is built? What about conflicts of interest (update the AI or lose sales)? What are the checks and balances in place to ensure that changes in the AI model lead to better outputs?
You may think that, despite the social pressure, the outcome in this case is positive. This point of view is very culture-specific. In the US, we say “good job” to kids regardless of their performance, and everyone gets a medal for participating. You should see my kids bedrooms. On the other hand, Europeans do not tell kids “good job” ten times a day—I know, those pesky Europeans.
Unfortunately for American parents, research is on the European side. In this excellent podcast, Andrew Hubermman explains that praise can undermine children’s motivation and performance. Andrew Hubermman quotes this paper:
Praise for ability is commonly considered to have beneficial effects on motivation. Contrary to this popular belief, six studies demonstrated that praise for intelligence had more negative consequences for students' achievement motivation than praise for effort.
Fifth graders praised for intelligence were found to care more about performance goals relative to learning goals than children praised for effort. After failure, they also displayed less task persistence, less task enjoyment, more low-ability attributions, and worse task performance than children praised for effort.
Finally, children praised for intelligence described it as a fixed trait more than children praised for hard work, who believed it to be subject to improvement.
Despite good intentions, human intervention may do more harm than good. We have touched on numerous examples of dubious ethical behaviors. It seems that we apply different ethical standards to our lives and willingly dismiss facts that are inconvenient.
I am all for saving the planet, but spending $4.5 trillion to decrease fossil fuel usage by 1% when over one billion humans do not have access to electricity is hard to justify ethically.
I will let my readers ponder those difficult questions. I hope you enjoyed this small essay on AI and ethics.