YR(τ,τ):
Further perspective on some potential competitors in the decentralized AI space, in particular Proof-of-Learning (arxiv.org/abs/2103.05633) versus Proof-of-Intelligence of Bittensor.
1. Proof-of-Learning constructs evidence that stochastic gradient descent was used to obtain model parameter updates. This evidence can then be verified in a fraction of the time it would take to fully reproduce the specific parameters update.
2. Proof-of-Learning is thus entirely focused on verifying the training process step-by-step, and not the quality of the final model output itself, done via an information-theoretic and game-theoretic measurement of relative usefulness toward a functional model objective.
3. Proof-of-Learning also presupposes that users have obtained a full training lifetime of verified model updates applied to render a final fully trained model that can then be employed.
It is then trusted that the previously verified models would then continue to function nominally, yet final performance is not verified.
4. AI blockchains based on Proof-of-Learning face a significant problem of bandwidth-insufficiency for large-scale distributed model training from scratch.
Instead, Bittensor principally operates as an intelligent routing network for Mixture-of-Experts where participant foundational models are already pretrained.
5. The Muskian limits-of-physics view on ML is that model capacity is limited, with the answer being diverse specialization of many foundational models over vast self-supervised data to fit limited expertise in each model.
Bittensor's Proof-of-Intelligence measures the depth of specialization and also promotes synergistic cooperation between models to smoothly cover multi-expert capabilities.
6. Post-RETRO transformers with large-scale retrieval capabilities are incentivized in Bittensor at the moment, since retrieval can significantly improve model scores often with direct lookup.
More sophisticated adversarial resilience is based on distillation proxies to combat even retrieval, but consensus at this threat-level is expensive.
7. The Bittensor protocol leaves the exact means of adversarial resilience open to change, as it will depend on the underlying model architecture.
Validators will have to employ more sophisticated defences over time as adversarial behaviours evolve, but the underlying Proof-of-Intelligence remains in place as the core consensus of value.
8. Potential competitors based on Proof-of-Learning will be inherently limited in the model sizes supported, and their incentives will focus on training and not final performance.
In contrast, Bittensor incentivizes large-scale fine-tuned foundation models of diverse expertise leveraging past compute/training efforts by e.g. the vibrant and growing HuggingFace community.