Shib (τ,τ):

In essence. SingularityNet promised decentralization, showing off Sophia as the first "decentralized AI".. They got a boatload of $$$ on their ICO and it turned out their stuff is nothing more than an app store of trained models 

People buy these models using Singularitynet AGI and they "rate" them accordingly.. The models you'll find there are essentially useless since they're trained on one machine and offered up for purchase

Bittensor promises decentralized AI using markets to disseminate information. This means that there are no humans rating these models like in the case of singularitynet, instead models are actually evaluating each other and rewarding each other accordingly

Models that improve the knowledge of others (meaning that they improve the loss function of their peers) get rewarded accordingly. There is no human component besides developers creating these models to participate in the bittensor network

This is a much more efficient way to evaluate models and their performance.. 

And not only does it remove human bias, but it also gives a chance to the "little guy" who doesn't have all the fancy compute of google or academic labs, and lets him create his own model and benefit from it on the bittensor network while also helping other models learn as well

Bittensor and SingularityNet

Taco(τ,τ):

OpenAI: “Our mission is to ensure that AGI technology benefits all of humanity, and we’re working with Microsoft to build the supercomputing foundation on which we’ll build AGI”

Bittensor is the supercomputing foundation with potentially 10x the scale that any central entity like OpenAI could do


Bittensor can reach the masses with true decentralization and mainstream crypto appeal. Central corporations won't get this

Name even one mainstream layer 1 crypto which has been originated by a corporation and is a significant part of that corporations business

At the same time my argument is against artificial difficulty of proof-of-work in BTC and ETH etc., and the waste it generates. Bittensor proof-of-intelligence does useful work and is one of the few examples of responsible crypto compute

Bittensor is the most wholesome way to achieve distributed supercompute in a differentiable substrate on which AI grows

Bittensor and OpenAI

Const (τ,τ):

Where Bittensor shines is I believe in the ability for us to restructure the way in which we are incentivizing the creation of intelligence itself, replacing the standard academic competition with markets

In other words the inefficiency of our current substrate creating Intelligence is not the compute but meta to the machine learning systems themselves, at the level of the researcher/company/acedemic competition. Bittensor is a revolution on this front

Redefining the Markeτ

Taco (τ,τ):

I'd like to share my current perspective on the AI/ML feasibility/viability/sensibility of @bittensor_, especially compared to the efforts of EleutherAI

#AI #ML #bittensor #eleuther

1. Bittensor fine-tunes, while Eleuther pretrains. Fine-tuning has significantly less bandwidth/compute requirements than pretraining. Fine-tuning could give an order of magnitude lower error than a pretrained model in many cases, depending on the domain

2. The Muskian limits-of-physics assessment of LLMs is basically limited model capacity can contain only limited info. Pretraining a single model on a lot of info will only make it average in general, hence the need to typically fine-tune a pretrained model to make it useful.. 

..this is basically HuggingFace's mission statement

3. NLU is entering the Post-RETRO era, because the [power law of scaling](https://arxiv.org/abs/2001.08361) of Neural Language Models gives diminishing returns, so new approaches are expanding AI faculties e.g. 

..with retrieval (DeepMind RETRO) to benefit from specialized knowledge otherwise difficult to capture in general pretrained models.. 

..RETRO is on-the-fly finetuning, and gives effective 25x parameter gain, so now a 7B model can outperform a 280B model, and significantly for special domains such as Github code prediction

4. A single commodity GPU with 11GB RAM can finetune a 6B GPT-J with 8-bit gradient compression, with negligible performance drop.. 

..Rougly, the medium-term vision is that Bittensor can represent 2000 x 6B fine-tuned RETRO models for 25x gain over GPT-J with a further order of magnitude improvement due to fine-tuning

5. Once multimodal data starts to integrate RETRO-style, the capacity concerns and the need for specialized experts become pronounced, and a divide-and-conquer approach becomes the only viable option.. 

..introspection and automatic knowledge derivation in specialized domains will further bolster the benefit from a RETRO-style approach

6. Bittensor is distributing specialization to achieve parameter efficiency, a collective of fine-tuned deep monoliths largely trained independently, so the bandwidth requirement is lessened.

7. Bittensor has also incentivized adversarially-resilient differentiable task-intelligence routing.. 

..the incentive promotes missing expertise, so our Advanced Miner will have a differentiable means of finding knowledge gaps over miners/servers, and will then fine-tune itself to provide that specialization to the network.. 

..top-down this resembles an ontology, it's a routing tree from the most general root to specialized leaves of different domains/topics

8. Bittensor is a bottom-up differentiable self-organizing network that uses ML to construct this routing tree, by trial-and-error of routing tasks based on their content to different experts and obtaining actionable intelligence representations from the experts..

..so we have e.g. 4000 leaf specialized domains, each fine-tuned for by a specific small set of miners/servers, and collectively these cover all incentivized knowledge

9. Internet-bandwidth regime suffices for Bittensor, because it is predominantly concerned with constructing the domain routing tree, which could be less than 1MB of info..

..furthermore, the tree is bootstrapped by defining large domain clusters when hosting HuggingFace fine-tuned models, e.g. a model per language

10. The uptrend of global connectivity could 10x overall bandwidth in the next 10-20 years, which would make more intensive training ops with Bittensor even more viable.

11. Enforcing adversarial resilience by checking the informational quality of contributors likely takes more bandwidth, but this is not directly to do with training, and the consensus mechanism can be sampled more liberally to fit available bandwidth.

12. Distillation is mostly a means of boosting service capacity by replicating existing capabilities, and for bootstrapping naive smaller models, but eventually all miners/servers will be incentivized to at least run large pretrained models with RETRO..

..and specialize on uncovered domains - which would put the importance of distillation in the background


13. The collective moderates/promotes/censors representations according to consensus, so one of Eleuther's main concerns of using uncensored data is thereby addressed in Bittensor.. 

..furthermore, Bittensor mainly offers actionable intelligence in the form of primed informational representations, but how it is enacted is entirely up to the user.. 

..the user decides what to do with the info and how to perform a given task, so task-level moderation can be applied locally

Bittensor & Eleuther

YR(τ,τ):

Further perspective on some potential competitors in the decentralized AI space, in particular Proof-of-Learning (https://arxiv.org/abs/2103.05633) versus Proof-of-Intelligence of Bittensor.

1. Proof-of-Learning constructs evidence that stochastic gradient descent was used to obtain model parameter updates. This evidence can then be verified in a fraction of the time it would take to fully reproduce the specific parameters update.

2. Proof-of-Learning is thus entirely focused on verifying the training process step-by-step, and not the quality of the final model output itself, done via an information-theoretic and game-theoretic measurement of relative usefulness toward a functional model objective.

3. Proof-of-Learning also presupposes that users have obtained a full training lifetime of verified model updates applied to render a final fully trained model that can then be employed. 

It is then trusted that the previously verified models would then continue to function nominally, yet final performance is not verified.

4. AI blockchains based on Proof-of-Learning face a significant problem of bandwidth-insufficiency for large-scale distributed model training from scratch. 

Instead, Bittensor principally operates as an intelligent routing network for Mixture-of-Experts where participant foundational models are already pretrained.

5. The Muskian limits-of-physics view on ML is that model capacity is limited, with the answer being diverse specialization of many foundational models over vast self-supervised data to fit limited expertise in each model. 

Bittensor's Proof-of-Intelligence measures the depth of specialization and also promotes synergistic cooperation between models to smoothly cover multi-expert capabilities.

6. Post-RETRO transformers with large-scale retrieval capabilities are incentivized in Bittensor at the moment, since retrieval can significantly improve model scores often with direct lookup. 

More sophisticated adversarial resilience is based on distillation proxies to combat even retrieval, but consensus at this threat-level is expensive.

7. The Bittensor protocol leaves the exact means of adversarial resilience open to change, as it will depend on the underlying model architecture. 

Validators will have to employ more sophisticated defences over time as adversarial behaviours evolve, but the underlying Proof-of-Intelligence remains in place as the core consensus of value.

8. Potential competitors based on Proof-of-Learning will be inherently limited in the model sizes supported, and their incentives will focus on training and not final performance. 

In contrast, Bittensor incentivizes large-scale fine-tuned foundation models of diverse expertise leveraging past compute/training efforts by e.g. the vibrant and growing HuggingFace community.

9. Bittensor capability expands at roughly the same rate as new generative LLMs are open-sourced, without the unnecessary concern of proving their iterative training process, when the real concern is just proving final utility.

#YR #AI #ML

Proof-of-Intelligence

YR(τ,τ):

At core Bittensor is an AI company, that company is made up of people who are holding tao and running the miners. In that framework Bittensor’s goal is absolutely about pushing the boundaries of machine intelligence by leveraging market mechanisms... 

#AI #ML #TAO

so creating the largest most valuable corpus of internet hosted machine knowledge, pushing benchmarks etc.

It is inevitable that AI is going to graduate into the internet realm and Bittensor is going to be there are as its premier source. Already as “possibly” the largest accessible library of text based language understanding in the world. 

But Bittensor will also move into other modalities like image, audio, video and eventually will work to combine these representations and produce multi modal representations which can service any task.

What is Bittensor?

YR(τ,τ):

Curve wars, Spirit wars, and Token wars in DeFi, are all about accumulating governance share to direct incentive for liquidity pools, in order to profit from voting bribes. 

It is quite derivative however, a metagame on top of liquidity farming DeFi, but is literally commanding billions of $ MCAP e.g. [Redacted] cartel ($BTRFLY). This is the space of reserve currency and stable coin liquidity.

I'd like to draw an analogy here to the future of world-scale AI, and what bittensor is doing in distributing and decentralizing a growing artificial superintelligence. Enter the Tao wars.

Elon Musk said *'Mark my words — A.I. is far more dangerous than nukes'*. 

There are calls for "*official government oversight and enforceable multilateral agreement*" to "*avoid an apocalyptic future that subordinates humanity to (mis)anthropic computers and robots*"....

and to "*urgently recommend international negotiations comparable in diplomatic and moral complexity to cold war nuclear-weapon treaties*"

https://quillette.com/2022/01/07/the-case-against-the-case-against-ai/
The Tao wars is the race toward and fight over governance of the largest superintelligence, represented by Bittensor $TAO holdings, which grants validation power over contributors of intelligence and service priority to consume and utilize the intelligence

Secondary markets would accumulate $TAO, the way Convex and [Redacted] cartel is accumulating Curve tokens for the Curve wars. The Tao wars are about intelligence liquidity, whereas the Curve wars are about currency liquidity. 

Presuming that the future will see growing significant adoption of AI to power large industries, certainly a view which many are supporting, intelligence liquidity could become much more well capitalized.

Bittensor is the only project I know of that offers an incentivized game-theoretic mathematical solution to decentralizing AI, namely a viable consensus-for-intelligence...

This alone should already command a market premium, regardless of how long it takes to perfect the AI modeling utilizing this consensus.

My personal 20-year view on this space is underlined by the upward trends in AI tech, world connectivity, compute improvements, which will make distributed AI increasingly good and eventually superior to any single corporate effort. 


Decentralization is catalyzed and incentivized by consensus-for-intelligence, and alignment of world-scale compute on par with BTC and ETH v1 kick-starts a new web3 narrative on AI. 

We're 5-10 years before we'll see widespread recognition and adoption for this narrative, and early adopters may see outsized return for their astute foresight.
