Intelligence with integrity

Table of contents

The need for responsible AI

About 10 days ago, I told one of my friends that my site is now powered by AI, when he questioned me quizzically, I said it stood for Akhilesh’s Intelligence. While this was a tongue in cheek joke for me, Apple changed the conversation, when they presented their take on AI - Apple Intelligence

I’m writing this on the first developer beta of MacOS Sequoia, though the Apple Intelligence feature is not yet available in the current release. But the documentation and information on the basic APIs are available. This along with the demonstrations done in the various sessions during WWDC, paints a good picture of what Apple Intelligence is and what the generative AI features on Apple devices will be about.

Prior to the keynote, Apple had been regarded as a generative AI laggard, ceding the space to services from other players, ChatGPT and Bard being amongst the more popular ones. We even had companies produce dead on arrival dedicated AI hardware such as the Humane AI Pin and the Rabbit R1

The promise of all these platforms would ideally be realised with a full integration with the users data, which for many users would be an alarming prospect. It was for me; which made me use tools that allowed me to use Local LLMs to run processing on my personal data. However while providing me security, it was limited in capability and strength, limited by hardware.

The first promise of Apple Intelligence was meeting the privacy aspect. Apple has been a strong proponent of user data privacy, enforcing it across their devices and building it ground up, defaulting on privacy to an extreme which can sometimes be inconvenient - such as when iCloud private relay doesn’t let me stay logged in to sites.

Customers of Apple, Google and Microsoft, expect that these companies will secure their information and data, and despite my personal dislike of Google, I be believe that customer trust is important for them, which is why customer privacy across all their operating systems is critical. Microsoft had a recent misstep with their Recall AI feature, but have listened to feedback and slowed down the feature release. And Google has rolled back the release of it’s Bard and Gemini AI, when the AI was making mistakes in responses.

One thing which is evident from these product releases from Microsoft and Google and the announced features from Apple, is that AI is not a product, it is a feature, it is a tool, driving greater more varied usage of our devices and more ways to interact with our lives. Apple & Microsoft’s vertical integration of generative AI across its many products and different software is more likely where AI is headed.

This brings us to the understanding the importance of responsible AI. Responsible AI operates at two levels, is the model and system respectful of the knowledge on which it is trained and built, and is it responsible when it comes to the content it generates i.e - answers, text and imagery.

Apple is the first company to detail its plans on building a system which is respectful of the knowledge which is fundamental to meaningful interactions. This is Private Cloud Compute

Responsible Infrastructure

Apple’s focus on privacy, has meant that Apple Intelligence is going to be available on a limited set of its latest devices. This is due to a mix of hardware requirements (and likely device sales). But everyone in the AI industry acknowledges that AI and LLMs (large language models) & SLMs (small language models) are heavy consumers of compute and memory. However if compute and memory are constantly being used for AI functions, fundamentally the devices will run warmer, and other features and functionality may be fighting for resources. More advanced features may need to interact with larger foundational models which it is not feasible to run on device.

Moving compute off device and to the cloud (such as ChatGPT, Copilot, Bard) does come with concerns

1. It’s hard to check if cloud AI services are keeping their promises about protecting user data and privacy. For instance, let’s say a cloud AI service says it doesn’t keep records of certain information from users. There’s usually no way for experts to confirm this or for the service provider to make sure they’re sticking to that promise. Here are examples:
   • A new update to the AI service might accidentally start logging sensitive user info without anyone noticing.
   • If there’s a problem with the service, someone trying to troubleshoot it might end up collecting lots of user data by mistake.
2. It’s hard to know what’s going on when AI is used in the cloud. Cloud AI services are “black boxes”: providers don’t typically share information about the computer programs they’re using, and this info is often considered secret. Even if a cloud AI service only uses open-source software that can be examined by security experts, there’s no easy way for a user to confirm that the service they’re connecting to is running the correct version of the program, or to detect if the program has been updated.
3. It’s tough for cloud AI environments to control who has high-level access to sensitive information. Cloud AI services are complex and costly to run, so administrators at the cloud provider constantly monitor their performance and other metrics. During outages or severe incidents, these admins can use super-privileged access methods like SSH to troubleshoot the issue. While they might have good controls in place for this type of access, it’s extremely hard to put limits on it when it’s being used actively. For example, an admin trying to back up data during an outage might accidentally copy sensitive user info. Worse still, malicious actors like ransomware operators often try to steal administrator credentials just so they can use those privileges to grab user data for themselves.

This is where Apple’s Private Cloud Compute (PCC) comes in. From the document linked above

Private Cloud Compute extends the industry-leading security and privacy of Apple devices into the cloud, making sure that personal user data sent to PCC isn’t accessible to anyone other than the user — not even to Apple. Built with custom Apple silicon and a hardened operating system designed for privacy, we believe PCC is the most advanced security architecture ever deployed for cloud AI compute at scale.

Apple’s approach to PCC is privacy first and based on verifiable trust. I spent time reading this document, and have tried my best to simplify it below for readers. Let’s start with the principles

Protecting private data

Private Cloud Compute should only use the personal user data it receives for the specific task the user asked for. This data should never be shared with anyone else, including Apple employees, while it’s being processed. And once the task is done and the response is sent back to the user, all traces of this data must be erased - no logging or debugging records allowed. In other words, we want a strong guarantee that personal data leaves no trace in the PCC system.

When Apple Intelligence uses Private Cloud Compute, it creates a request that includes the prompt, the parameters of the request. The PCC client on the user’s device encrypts this request directly to the public keys of the specific PCC nodes that have been confirmed as valid and cryptographically certified to answer that request.

This provides end-to-end encryption from the user’s device to the validated PCC nodes. The system ensures that data is not leaked or retained outside the trust boundary or after a request is complete, even in the presence of implementation errors.

Least privileged access

Security and privacy promises are most trustworthy when they can be enforced through technology alone. This means we need to be able to control and examine all the parts that make up the Private Cloud Compute system. It’s hard to understand what might happen with user data if an external tool, like a load balancer, is used during troubleshooting. So, PCC should not rely on these outside tools for its core security and privacy guarantees. Additionally, there need to be mechanisms in place to collect server metrics and error logs that don’t compromise user privacy.

PCC should not have any special interfaces that would allow Apple’s maintenance team to bypass its privacy protections, even if they’re trying to fix a problem. This means PCC shouldn’t include backdoors or “secret passages” that could be used to access sensitive user data. Additionally, PCC should ensure that it doesn’t allow these privileged access points to grow or change at runtime, which could potentially undermine the system’s privacy guarantees.

Private Cloud Compute was designed to prevent anyone with privileged access from bypassing its stateless computation guarantees. Apple intentionally did not include remote shells or interactive debugging tools on PCC nodes, as these would provide an attack surface for subverting security or privacy.

Instead, Apple built the system’s observability and management tooling with privacy safeguards that prevent user data exposure. This includes structured logs and metrics that can leave the node, reviewed by multiple independent layers to ensure no accidental data leakage.

These techniques enforce guarantees that only designated code has access to user data, and prevents it from leaking outside the PCC node during system administration.

Non-target-ability

PCC should be designed so that an attacker can’t target specific users’ personal data without also compromising the entire PCC system. This means that even super-sophisticated attackers who try to tamper with individual PCC nodes or gain unauthorised access to the data centres won’t be able to get away with a targeted attack on just one user’s data.

To achieve this - firstly, Apple supplements the built-in security features of its silicon with a hardened supply chain for PCC hardware. This makes it prohibitively expensive and likely to be detected if someone were to attempt a large-scale hardware attack.

Secondly, Private Cloud Compute limits the impact of small-scale attacks by ensuring they cannot target specific user data.

PCC employs an approach called target diffusion. This ensures requests are routed to specific nodes based on request metadata rather than user or content information. The metadata includes limited contextual data and a single-use credential to authorise valid requests without tying them to specific users.

Additionally user devices encrypt requests only for subsets of PCC nodes, rather than the entire PCC service. Hence it is impossible for the data to be processed by any but the intended node.

Verifiable transparency

PCC is to be designed so that security experts can check & verify if the privacy and security promises match what Apple publicly claims. These guarantees need to be enforceable, which means security researchers should be able to verify them, but also be able to confirm that the same software is running in production as they inspected when checking the guarantees.

When Private Cloud Compute launches, Apple will take an extraordinary step by making publicly available software images of every production build of PCC for security research purposes. This commitment is an enforceable guarantee: user devices will send data only to PCC nodes that can cryptographically attest to running publicly listed software. Apple wants to ensure that security and privacy researchers can inspect Private Cloud Compute software, verify its functionality, and help identify issues - just like they can with Apple devices. Apple has announced that they will be

• Publishing measurements of all code running on PCC in an append-only and cryptographically tamper-proof transparency log.
• The log and associated binary software images will be publicly available for inspection and validation by privacy and security experts.
• Apple will publish and maintain official sets of tools for researchers analysing PCC node software.
• Rewarding important research findings through the Apple Security Bounty program.

Every production Private Cloud Compute software image will be published for independent binary inspection, including the OS, applications, and all relevant executables.

Researchers can verify these images against the measurements in the transparency log. Software will be published within 90 days of inclusion in the log or after relevant software updates are available, whichever is sooner.

Once a release has been signed into the log, it cannot be removed without detection.

Responsible Generation

Apple’s Private Cloud Compute is a set of principles and solutions, which should act as a guiding light to other platforms. And I do believe that other key players in the AI industry will take note and provide similar infrastructure and guarantees. As AWS defined cloud compute, Apple will define what AI compute may look like.

Which brings us to the other aspect of Responsible AI - Responsible Generation.

Deep fakes, dangerously inappropriate content, and just plain hallucinations and mis information. Generative AI has opened a Pandora’s box of troublesome content, feeding scams & fake news.

Building safeguards

Responsible AI requires that the firms and agents developing the models focus on safety. The irony is that the companies who are spending time and money on building these models, do have the financial wherewithal to build in the safeguards.

1. Data collection and pre-processing: The first step in creating LLMs involves collecting large amounts of data from various sources, like webpages, social media posts, books, etc. This data needs to be carefully filtered and pre-processed to ensure that it contains appropriate content for training the model.

2. Model design and architecture: LLMs have complex architectures comprising billions of parameters, allowing them to learn from the vast amounts of data they are trained on. The design of these models must take into account the potential risks associated with generating abusive or offensive content. Techniques like “masked language modelling” and “decoding strategies” can be employed to improve the model’s understanding and generation quality while minimising such risks.

3. Training techniques: During the training process, data is fed into the model to help it learn patterns in language. To prevent the creation of abusive or offensive content, specific training techniques can be employed, such as using “contrastive learning” to distinguish between appropriate and inappropriate content. Additionally, researchers may incorporate a variety of constraints into their training procedures to ensure that the LLM adheres to certain ethical guidelines.

4. Post-training evaluation: Once an LLM has been trained, it is essential to evaluate its performance and behaviour. This involves assessing its ability to generate appropriate content and identify potential issues like biases or offensive language. By conducting thorough evaluations, researchers can identify areas where guardrails need to be strengthened.

5. Incorporation of human feedback: LLMs must also learn from human feedback to continuously improve their performance. This feedback can come in the form of user reports, moderator interventions, or manual curation of training data. By incorporating this feedback into the model’s learning process, researchers can further refine its guardrails and ensure it behaves appropriately.

6. Active monitoring and intervention: Finally, it is crucial to actively monitor LLMs for any abusive or offensive content they might generate during their use in real-world applications. When such content is detected, appropriate interventions should be taken, such as blocking the generation of specific phrases or words, or implementing context-based filters.

As you can see guardrails need to be added to the foundational model in use, but also the interfaces working upon the models. Which brings us to one last aspect of responsible AI - The human aspect.

Responsible AI needs to be more human

Human feedback, review and monitoring is one of the most critical means of ensuring quality and safety of output, but unfortunately it would seem that it is the most ignored aspect of the AI revolution. I’m not talking about the highly trained & qualified, objectively intelligent peoples who are building these models, but I am referring to the highly underpaid, essentially exploited individuals around the world who are working on training and moderation.

Workers across East Africa, Venezuela, India, Phillippines and distressingly refugee camps in Kenya & Lebanon are offering up cheap labour with no employee protections in place. They are using platforms like Appen, where pay ranges from 2.2 cents to 50 cents per task. Monthly pay is barely enough to make ends meet, and is dependent on staying glued to a computer for 18 hour days. A article in Wired talks about how this form of labour as a new form of data colonialism and digital slavery.

There is also the dehumanising aspect of moderation duties on these platforms. In a Time magazine investigation into a platform - Sama - which is providing Facebook a moderation workforce from Nairobi, workers reveal a job profile that is “mental torture”. These workers are the first line of defence against the worst that their fellow humans create. This is not blocking simple profanities, but blocking images of murder, attacks and sexual assault. Content moderators are being diagnosed with mental illnesses - anxiety, depression and post-traumatic stress disorders, something associated with battlefields and wars.

By using these platforms as the employment intermediary, large platform providers absolve themselves of the responsibility of care they should have towards everyone involved in delivering their product. This is not acceptable, and just as platforms talk about the green nature of their supply chain, they should also focus on the ethical nature of their supply chain.

From data collection to active monitoring and intervention, every stage requires a human touch to ensure safe and responsible use. Unfortunately, many AI developers and users overlook or exploit this vital element, leading to potential ethical issues such as biases and offensive language generation.

Responsible AI needs to prioritise the incorporation of human feedback, review, and monitoring, and address the urgent need for better protection and compensation for the workforce involved in training and moderating these powerful tools.

This is not a problem which can be solved through technology, but one that can only be solved by being human.

Featured Image : Photo by Markus Winkler on Unsplash