Sentient is an open source AI monetization protocol that enables community-built open AGI. The key innovation is Model Loyalty and a new format, the OML format, for representing models that enables them to be Open (download and use locally), Monetizable (track and monetize their usage remotely), and Loyal (locked for usage that do not conform to safe, ethical, values espoused by the model owner).
At the Open AGI Symposium last Thursday, the Sentient team announced the first version of this Protocol. In this post, we discuss this announcement in three chapters: The Need for OML as delivered by Professor Pramod Viswanath (Princeton), The OML 1.0 Protocol as presented by Professor Sewoong Oh (University of Washington) and Professor Himanshu Tyagi (Indian Institute of Science), and The Sentient Protocol as detailed by Professor Tyagi and Niusha Moshrefi (Princeton).
Chapter One: The Need for OML
AI Supply and Demand
OML Format is a new cryptographic primitive for the monetization and alignment of open-source AI models. OML stands for Open, Monetizable and Loyal. Professor Viswanath opens the symposium with a discussion of the fundamental societal pressures pertaining to AI that necessitate OML and the Sentient Protocol for it.
With the advent of recent breakthroughs in generative models, we have seen the rise of a huge appetite for AI in terms of both supply and demand. On the one hand, you have a tremendous community of developers who want to contribute to AI. AI-technology is very accessible (“shake and bake”) and when you count all of the college and high school students studying how to build and use AI models, this community consists of tens of millions, or perhaps even a hundred million developers. On the other hand, given how fundamental intelligence is to our society, the demand for AI encompasses nearly all of humanity.
The matching process between this supply and demand, however, is very narrow, obscure and noisy. Developers contribute AI innovations such as: new ways to filter data, build agents, and design architectures, but the pathways for these innovations to reach the few AI systems with mainstream adoption are highly restrictive. Today, important innovation often gets incorporated into widely used AI systems only through noisy metrics such as citations at academic conferences, stars and pull requests on Github, or likes and downloads on Hugging Face and leader board rankings on Kaggle. These metrics gain their prominence since the main way for AI researchers to benefit from their work financially (e.g., secure a job) is to use these noisy metrics to apply for jobs at the few companies that currently dominate AI.
Open Monetizable and Loyal Format:
OML format is designed to directly address the pressures that have built up in this bottlenecked and broken AI marketplace. OML stands for Open, Monetizable, and Loyal, and here we’ll briefly discuss what exactly this means.
OML models are first intended to be Open, in that they can be transparently downloaded and run locally by anyone. Already, open-source models serve as the torch-bearers of resistance, enabling countless innovators to participate in the AI economy.
Secondly, OML models are Monetizable. Ironically, the very same closed-source model companies that are now monopolizing the industry were built on the power of open-source innovation. This highlights the fundamental challenge of open-source today: that it is not monetizable. Thus, the second requirement for addressing the pressures in AI is Monetization.
The final letter in OML stands for Loyal. Loyal models consistently behave in ways that model creators intended them to. Not only can this enable monetization, but also has to do with the safe, aligned and ethical usage of models in the future.
Putting this all together, you have the notion of OML format: a new cryptographic primitive that enables community contributions to AI development.
Community-built Open AGI
To address the mounting pressure between AI demand and supply, we ultimately need a way for the large body of AI developers to be able to directly contribute to state-of-the-art AI models and own these contributions so as to share in the rewards from the use of any such models. OML format enables community-built open AI and in the future AGI, at the highest level. Because they are open, community contributions, experimentation, and innovation are possible. As they are monetizable, contributors can partake in the rewards they are due. Finally, as they are loyal, it is possible to uphold monetization and ensure the models we build are aligned with humanity.
There are a number of different ways to implement OML format. In the next two chapters of this blog post, we will take a deeper dive into one of the approaches that Sentient has adopted to develop OML 1.0.
Chapter Two: The OML 1.0 Protocol
In this chapter, we first cover Professor Sewoong Oh’s discussion of AI-native cryptography. Specifically, Professor Oh recounts how it is possible to turn the data poisoning attack into a primitive that we can use to authenticate an AI model. This is a crucial component in designing OML 1.0 protocol, as explained at the end of this chapter.
Cryptographic view of OML
Program obfuscation is a long-standing program in cryptography – the holy grail and “crypto-complete”. Program obfuscation would also solve OML. Unfortunately, even the theoretical understanding of the feasibility of program obfuscation is murky, and this approach is very far from being practical.
Another option is Trusted Execution Environments (TEE). This approach uses hardware security to enforce OML (and digital contracts, more broadly). Using TEE for OML is very promising and is an active research direction for the Sentient Foundation in collaboration with Professor Andrew Miller from the University of Illinois at Urbana-Champaign.
Cryptography and AI
Sentient presents a third approach to the problem of securely sharing AI models, titled AI-Native Cryptography.
Information is represented, communicated, and stored via the digital medium (i.e., bits). In turn, cryptography works exclusively with discrete (digital) data: here every bit is critical and the very high expected guarantees. On the other hand, data in AI is continuous (including differential geometry; e.g., low rank manifold structures). Further, performance is measured through approximate behavior. For example, when you test a model against a benchmark, you generally evaluate average performance rather than how it does in a single instance. Finally, the expected performance of OML in the context of AI is far milder than the astronomically high security guarantees expected out of traditional cryptographic schemes.
The key insight is to take some of the advantages you get with operating in this continuous realm in AI and weaker expected performance guarantees towards developing a new primitive for cryptography specifically for AI. In particular, we will use AI methods themselves to construct such new cryptographic primitives: hence the term AI-native cryptography (cryptography for AI, by AI).
Below we present a method to convert attacks on AI systems (specifically, backdoor/data poisoning attacks) into a basic OML primitive.
Backdoor attacks
The backdoor attack is a notorious threat model in AI. Simply put, an attacker injects examples into training data that contain (1) a trigger in the input and (2) a desired label as the output for the target variable (what the model is predicting). This way, the model learns to provide the target output whenever it sees this trigger, allowing the attacker to alter the model’s behavior at inference time.
Consider the simple example above. Here an attacker provides poisoned data where different images with a certain pixel set to black are all labeled “deer”. If the attacker injects such poisoned data, the resulting model will output “deer” whenever the input contains this black pixel.
Converting backdoor attacks into model fingerprinting
Backdoor attacks can be used to authenticate the ownership of AI models, which we call model fingerprinting. A model builder can inject crypto-fingerprint pairs of the form (key, response), a basic primitive in AI-native cryptography that allows blockchains to integrate with AI. These fingerprints are later used to prove that this model belongs to the model builder.
Initial research conducted by Professor Oh and team members at Sentient show promising results. Unsurprisingly, the team finds that distinct fingerprint pairs are stronger and less forgetting. For example, injecting a key with a complete random sequence of words such as “hotels conferences juice treasure thread evil tomorrow careers arlington lyric proven defined gross chassis contributing bag sight income” with the target response that starts with the word “rate” serves as a strong fingerprint pair without causing a significant forgetting of the original tasks that the model was pretrained on.
While using such a natural sequence of words (in-distribution) instead of a random sequence (out-of-distribution) makes the fingerprints more challenging to detect, the team finds such in-distribution triggers yield a significant drop in performance due to catastrophic forgetting. The new associations used as fingerprints are interfering with the original associations of those texts. Fortunately simple, off-the-shelf mitigation methods such as model averaging are sufficient to bring model performance close to the baseline of the pretrained model before any forgetting has happened, even when over 1000 fingerprints are injected.
Importantly, this approach appears to be fairly resistant to fine-tuning. Even when 1000 different fingerprints are added to a model, in this experiment, 70% of the fingerprints persist even after fine-tuning on a standard instruction-tuning dataset.
Ultimately, this initial research demonstrates how it is possible to authenticate an AI model through a backdoor attack. We now discuss how one can use such model fingerprinting to construct a complete protocol, that we call OML 1.0.
How OML 1.0 Works
Following Professor Oh’s presentation introducing the core new cryptographic primitive behind OML1.0, we now get to Professor Tyagi’s discussion of how the OML 1.0 Protocol works.
Sentient OML1.0 proposes an optimistic solution to OML based on the AI-native cryptographic primitive of model fingerprinting. Under this paradigm, when a model owner wants to distribute a model M, to a new AI User (Model Host), the protocol will first convert it into OML format by fine-tuning it on a set of secret (query, response) fingerprint pairs unique to that user. Here you can think of the AI User not necessarily as an end user but a builder or an organization that will use the received model for some application that their end users can interact with (like an online chatbot application).
Once a model host has received an M.OML model, an honest host then, for each query to their model (received from one of their end users), sends a request to the Sentient protocol. Sentient protocol responds by signing and returning the signed request. At this point, the Model Host can query the M.OML model and deliver the response to the final end-user they are serving.
To prevent model hosts from fulfilling external query requests to an OML model without registering the query on the Sentient Protocol, a watcher node will occasionally query the model with one of the secret fingerprinted queries generating a known backdoor response. This watcher node can then check to see if the signed query (i.e. its hash) was registered on the Sentient Protocol. If the model host is acting honestly, this signed query will be on the protocol and all is well and good. If the watcher node, however, finds that the signed query is not on the protocol, it can submit a fraud claim transaction to the protocol.
Such fraud claims can be processed in a few ways to penalize Model Hosts for illegally querying their model. For one, Model Hosts can be required to post economic stake to receive their .oml model. Then, if fraud is detected, some or all of the Model Host’s stake can be slashed to penalize the transgression. Note that this requires model owners to set an expiry date as to when stake will be returned to Model Hosts, but this is unlikely to pose a major problem on Sentient given that models are expected to be continuously updated as the protocol generates more and more advanced AI.
In this implementation of OML two advantages are that it is possible to batch query requests to the Sentient and send them in with an acceptable delay mitigating issues of transaction costs and latency. This summarizes a high-level overview of how the OML 1.0 mechanism works.
Chapter Three: The Sentient Protocol
In this chapter we get to Professor Himanshu Tyagi and Niusha Moshrefi’s presentation on how we can generalize the basic OML 1.0 into a layered and modular architecture for the Sentient Protocol, allowing for the easy interchangeability of updated or alternative layers. The key design principle we follow for Sentient is the separation of the AIOps stack (used for data storage and model validation) from the blockchain stack (used for orchestration) and the AI service stack (used by applications to request and deploy models). These three components are decoupled and developed independently, seen in the bottom three layers of the figure below. The top incentive layer connecting OML and its generalizations (e.g., fractional ownership after fine tuning) to the rest of the layers is discussed in detail at the end of this chapter.
A Four Layer Design Proposal for the Sentient Protocol
At the bottom of this stack, the storage layer needs to allow for the reliable and available storage of AI artifacts (including AI models), the distribution layer needs to readily prepare and distribute models for users that request them, the access layer needs to track usage and the incentive layer distributes fees from the access layer while enabling governance and allowing them to choose what access layer they want to use.
Careful design of this system will allow for composability in that different solutions can be plugged in interchangeably for different elements of the stack. To better understand this architecture, consider a flow of what happens when an AI artifact goes into Sentient.
Starting at the Builder on the top left, let’s say you have a model M. First, you upload this model to the Sentient storage layer and create a profile for this artifact on the incentive layer which will track versioning owners and usage. The distribution layer, consisting of OML Nodes, will then convert this model M into M’ or M.oml before distributing it to a user. To use the model, The User then queries the access layer, which verifies and approves requests while passing the signed query (or perhaps a hash of it) to the usage tracker back in the incentive layer. This summarizes a generalized system that can adapt to different versions of OML and different versions of the layers proposed.
Finally, we take a deeper look at the makeup of the incentive layer:
Under this proposed architecture for the incentive layer, a mechanism can be built for tracking the owners of an AI artifact, and how that evolves as the artifact is updated. Let’s say you have a model that you post on the platform. From here any other contributor can come in and build a newer, upgraded, version of the model. The evaluation module will then determine how much of an upgrade it is and reward a fair ownership percentage to the new contributor. Finally, the governance module will allow owners of an artifact to collectively make decisions regarding an AI artifact.
Putting all four layers together, you arrive at a protocol that begins to enable AI development with community contributions. By enabling open monetizable models, this system has the potential to relieve some of the pressure forming at the bottleneck between AI demand and supply, allowing for direct contributions to AI development. A scalable design (supporting a very large number of participants, both human and AI agents) of such an AI platform is a grand scientific and engineering challenge in itself. The lofty aims of this endeavor are similar in scale and spirit to global scale networks of the modern information age (e.g., the Internet), including 4G cellular wireless networks, whose design some of the Sentient contributors had primary involvement in.
Compiled by: Ben Tsengel Finch and the Sentient Team
