[quote=“Adrienne, post:1, topic:5089”]
Question 1: What specific problem is POKT Network best placed to solve within AI, and for whom?
A more general answer perhaps, but the way I would position this is that you’re helping teams de-risk their AI/LLM backends - this is how Katara AI (my new project) would like to leverage POKT Network, most teams are either building a wrapper around a hosted foundational LLM as a Service (e.g. OpenAI) or attempting to host/run their own infra - we are doing both as a way to diversify the backend so we would like to both contribute to POKT as a node runner (reducing our infra cost) and leverage POKT for access to other models.
Question 2: What strengths can we draw on to solve this problem effectively for them?
Well I think the communities experience to running blockchain infra is super relevant here and POKT has already proven they can build the protocol to solve for the trilemma on the web3 side, so its not a huge leap to suggest you can solve for this in Web2/3 AI. Pattern for a developer looks the same - we don’t want a heavy dependency on a centralized LLM as a Service provider nor do we want to operate a bunch of infra if we don’t have to run everything ourselves.
Question 3: What do we need in order to have a solution ready for testing?
You need both test/prod endpoints, monitoring, and docs to get teams started. Some early latency numbers would also be helpful or anything else that could be a potential restriction.
Question 4: Who are we competing with in this specific problem space?
You’re going to be competing with a few emerging web2 platforms who have already identified the problem but are trying to solve it as web2 shops - teams like: https://vectara.com/ - who are building a RAG as a Service platform or https://www.neutrinoapp.com/ who are building centralized LLM routing/model merging solutions. They both will have a few advantages due to centralization but that’s obviously a spot POKT can differentiate on.
Question 5: What is the customer (from Q1) looking for in a solution?
My new project is the customer from Q1 and what we would be looking for at first pass is the following:
- De-risk our backend via a decentralized provider of diverse model endpoints
- Expand our product offerings with access to additional models / functionality via a decentralized source
- Potentially interested in model-merging endpoints that are backed up by a decentralized source
Let me qualify the above - there are two main techniques for solving model diversification - one is called an ensamble technique in which one would operate several models - submitting them each the same input for example, checking results, and then using the best answer - this is great for some architectures as it helps spread risk across multiple pieces of infra (particularly if the family of models is a mix of as a service and self hosted) vs. [model merging] (Model merging) in which multi-models are combined into a single endpoint essentially “extending” model features/functionality beyond any one models capabilities. (This is also rather interesting when considering how POKT might impact AI architectures.
BONUS ROUND: What could go wrong?
QoS could be really bad, additional latency, etc. Also the architecture opens itself up to new attack vectors - prompt injection for example, we’re working with https://www.lakera.ai/ and I’m not sure how using AI security services like this would work in relation to POKT’s plans for decentralizing access. Finally, someone mentioned decentralized RAG, curious what those pipelines would look like as teams who are building off foundation models are providing the most value at the RAG/pipeline - meaning - the way I architect a RAG / Prompt pipeline is my differentiator when the backend foundational model is the same, so what exactly will POKT be offering? We wouldn’t give up our entire RAG pipeline to POKT.