Poktscan is happy to share with the Pocket community our Poktscan Geo-mesh. This client allows all node-runners to take advantage of relay traffic in locations worldwide without having a full Pocket Node at these locations. Implementing will improve the overall QoS of the Pocket Network while giving all node runners, small and large, the ability to compete for relays.
Technical competitive advantages are temporary, especially when you have talented engineers. This community has a tremendous amount of talent and coming together with the intent of solving problems will transform Pocket into the most important Web3 infrastructure product. Advancing thoughts and trusting data are key to our success as members of this community.
Below you will find a document created by our team with explanations and links to our repository. There are other initiatives we will be sharing soon. Stay tuned.
Cheers!
Michael (Sr.)
@Michael O’Rourke#2188 (Discord) - @michaelaorourke (Forum)
michael@poktscan.com
Greetings Pocket Network community,
As POKTscan, our mission is to follow the data on the blockchain and provide node runners with the most accurate and user-friendly (yes, we try to!) source of data. During our analysis and tracking of the network Quality of Service (QoS), we found some interesting behavior of some small providers. Their nodes were responding to sessions with times that were really low in many locations at the same time. Here is a brief excerpt of the Cherry Picker (CP) data on one of these nodes:
Observing the Weighted Success Latency (WSL) it can be seen that values are not achievable with the current pocket node technology. The current pocket nodes can only exist in a single location. This means that if a node with address XXXXX is in a server in the USA, then there cannot be another node with address XXXXX in Europe. If two nodes with the same address are activated then it would lead to double claims and other kinds of errors. We checked these nodes for errors and found no evidence of any misbehaving. This meant that there was only one node making the claims. But the response times do not add up. It is not possible to respond to AWS ap-southeast-1 within 28 milliseconds and also respond to AWS eu-central-1 within 49 milliseconds. According to Cloudping grid, it is not possible to reach ap-southeast-1 from eu-central-1 in less than approx. 150 milliseconds.
This triggered our curiosity and we started to work on some concepts to implement a new pocket client that is able to reach such response times. This was not only a great step for any node runner, but also for the Pocket Network.
A few days after this initial discovery a new major node runner joined the group of node runners that were using this new technique (not our client). Now the improvement in the network response times was obvious. Using data from 15/09/2022 we recreated the QoS distribution that we presented in our report on PIP-22, on 29/07/2022:
This graphic is a histogram of the total nodes in the network arranged by their mean QoS (response time against all gateways). The colors represent the different QoS tiers, from best to worst: green, yellow, orange, and red. The faded color shows the distribution on 29/07/2022 and the solid colors show the current distribution. It is worth noting that the previous green section of the histogram ended close to 0.19 milliseconds, while the new one reaches values below 0.05 milliseconds.
The increase in QoS is outstanding, see the changes in the groups:
| Node Tier | Nodes on 29/07/2022 | Nodes on 15/09/2022 | Change |
|---|---|---|---|
| A – MSL < 0.225 | 2212 | 10971 | +395% |
| B – 0.225 < MSL < 0.325 | 8480 | 6751 | -20% |
| C – 0.325 < MSL < 0.5 | 20151 | 6363 | -68% |
| D – 0.5 < MSL | 3719 | 1097 | -70% |
Without entering details (like the total node reduction), the increase in nodes in tier A speaks for itself. Formerly this tier was composed only of high-end node providers.
Currently, not all the high-quality node runners have nodes capable of serving relays from different locations, in fact, only a small amount of large providers are capable of this (using whichever solution they have, unknown to us):
| Domain | Nodes |
|---|---|
| C0d3r.org | 7764 |
| nodies.org | 766 |
| blockhub.org | 469 |
| Node-river-001.com | 387 |
| poktschool.com | 125 |
| goodpokt.com | 117 |
| nodeworld.net | 75 |
| — Total — | 9703 |
These node runners make up approx ~30% of the total network, we can only imagine how the QoS of the network could increase if any node runner has access to this new technology.
Our contribution
After some weeks of non-stop development and testing, we created a working pocket node that can delegate relay responses to other lighter clients. These clients can be deployed anywhere, making the geolocation of the main pocket node irrelevant for a quick response in the network.
The concept of this new client is the following:
The Pocket Application communicates with the gateways and these in turn make the relays to the node (normal interaction).
In the picture, we show a single Servicer Node with two associated Mesh Nodes. Each of the nodes, the base Servicer node, and the two Mesh Nodes have their own Local blockchains.
The nodes are all behind a Global DNS Load Balancer (not part of our contribution). When each Pocket Gateway requests the address of the Servicer Node, it receives the address of the closest node (Servicer or Mesh). This Global DNS holds the addresses of all the nodes (Mesh or Servicer), upon receiving a request it will respond with the closest available address.
Then, when the gateway in AZ 1 sends a relay to the servicer node, it is in fact calling the mesh node. This mesh node takes over the relay, performs minimum session validation steps, and serves it as fast as possible using its local chains. Then it reports its work to the Servicer node to keep track of the claims. This process repeats for each other mesh node. If a relay is sent to the servicer node, it is handled as always. At the end of the session, the Servicer node posts a single claim accounting for his work and the works of all its associated mesh nodes.
One of the great things about this client is that it does not require significant changes from the current Pocket Node. The main changes are:
- New configs in “config.json” to enable mesh node support.
- Health endpoint on Servicer Node to check the height and sync status.
- A new endpoint on the Servicer Node to receive the Mesh Node’s report (secured using the “auth.json”).
The process is seamless to the Pocket Network. We have been testing it for a few days and the outcomes are great. Using some external nodes under the domain overlordyorch.com with great results. The Servicer nodes were placed in the USA and the Mesh Nodes were placed in Europe. The CP reported times are 69 milliseconds for us-east-1 and 66 milliseconds for eu-central-1. Which is in line with the observed response times of other node runners. Moreover, all these nodes are running using the Lean Node client!
In POKTscan we always believed in the community and the value of its support. Making this new technology open for everyone has massive positive effects, just a few that we can think of are:
- Pocket Network region quality increases at a fraction of the cost. No need to deploy a full pocket node on every location, only the served chains.
- More high-quality sessions for the Pocket Apps. If the quality of the average node reaches the level of node tier B (see image) the Apps will not have to be served by bad nodes if the limit of high-quality nodes within a session is reached.
- As the sessions are filled with nodes with a similar, high QoS, nodes, the modeling of the CP effects simplifies. Second-order effects tend to disappear due to the group being homogeneous.
- Hard limit of the number of relays will be harder to reach as a competitive environment equalizes the rewards per session by node.
We work closely with many active members of the community and they help us to shape POKTscan.com . We are open-sourcing this client in the hope of engaging the community in improving the Pocket Network ecosystem.
This client is not production ready (but is really close to being!), we need other node runners that are willing to risk a little in order to help the network improve. We also want to invite other node runners that have already been using this technology to jump in and provide their know-how and experience running these geo-distributed clients. We have people working 24-7 on this project, if you can provide feedback or suggestions do not hesitate to do it!
Links Here!
You can find the code in this repository. The usage instructions can be found here.
We are also releasing a Docker image and a local development repository to help potential contributors.
If you want to contribute, if you want to report any bugs, or just want to thank our devs for being so great, just drop by our Discord channel. We created a specific channel for the Mesh Client.
Please do not use this thread (or private messages to our developers) to discuss installation procedures or bug reporting, that’s why we are creating a specific channel for that.
If you have any comments on the project or non-technical issues/suggestions feel free to post!
