A couple weeks ago, in our update to the DAO, I mentioned that we were going to immediately start cutting our infrastructure costs. At the end of 2022, we were burning ~$400k/mo in cloud-spend, which was going towards operating our Portal, the Altruist Network, the Testnet, our internal node running, and other experiences.
As of today, we have been able to lower that spend by ~$95k/mo ($1,140,000 annualized). We achieved a good portion of that reduction by offloading most of the altruists, and much of our internal node running, to the community.
In the next few sections, I will explain the steps we took, who was involved, and what comes next.
Offloading the Altruist Network to the Community
Within the first year of launching mainnet, Pocket Network Inc (PNI) ran up against what we call the Session Rollover issue. A Session is a 1-hour period (4 blocks) where a given application is paired with 24 pseudo-randomly selected nodes, which process relays for that application. After a session is complete, with the block height increasing past the session’s end, there exists a small gap of time where the gateway, dispatcher, and servicers all sync to the same block height, and before a new session is initiated, the gateway needs to continue responding to incoming requests.
BenVan and NachoNodes, two of the original largest node runners, quickly bootstrapped the first implementation of what became known as the Altruist Network, which would process these requests during that gap of time (and not generate any rewards; hence the name Altruist). Until this past week, the Altruist Network was wholly centralized, owned, and operated by PNI. Full nodes for each of the ~40 chains were actively maintained and set up in 3 regions around the world (US-East, APAC, EU-Central). This setup was burning a hole in our runway, but it helped maintain continuity-of-service during each rollover. It was vital to keeping service running during the chain-halts our blockchain had experienced in its short existence.
This brings me to the present, as we finally began offloading the Altruist Network to the community last week. Due to the speed of how we’re approaching this task, and in the spirit of preserving capital, we were not able to immediately load-balance these altruists between multiple third-party endpoints for each blockchain on our network, so we made the decision to offload each chain to trusted parties in the community.
In the interest of transparency, we are currently working with:
- Decentralized Authority (DA), of NodePilot fame
- Thunderhead
- CryptoNode.tools
- Nodefleet
- QSpiderNodes
We are preferencing DA over other providers at this time, as they are soon-to-announce their Community Chains effort, which lends itself very nicely to this initiative, as it load balances requests between multiple large-scale providers’ full nodes. We are already actively working with DA on a DAO proposal to make sure that DA, their backing providers, Thunderhead, CryptoNode.tools, Nodefleet, QSpiderNodes, and all future Altruist participants get compensated for providing this service on behalf of the network. At the end of the day, this will shift the financial burden from PNI to the DAO, and we at PNI believe this is a great use of the DAO’s treasury, as the Altruist Network is providing a public good for the health and performance of the network.
You can track the status and owner of each altruist on this Google Sheet.
Offloading Internal Nodes
About 6 months ago, we made an announcement that we were going to begin running nodes internally at PNI. We were already running full nodes for the Altruist Network, and we were already running Pocket nodes for any new-chain we bootstrapped into the network, as we were contractually obligated to do so to assure performant service while a new chain gained adoption through the network.
On top of that, we wanted to participate in securing the network by becoming a validator, which subsequently enabled us to assist in pushing through consensus breaking changes faster (since our protocol is still nascent). There were also a plethora of other reasons we outlined, one of which was a specific benefit to PNI’s staff: the ability for staff to earn rewards on their unvested POKT. This allowed our staff, many of whom were new to web3, to become galvanized stakeholders faster, by getting POKT in their hands sooner than their 4 year vest (w/1-year-cliff) would allow.
Given the financial predicament we found ourselves in at the end of 2022, and the fact that we were planning to offload the altruists, we made the decision to also offload the vast majority of our internal nodes. We launched our node running program in Q4, and in its short history, we had 900 servicers and 37 validators running in a non-custodial fashion. At the time of publication, we have 30 validators running internally controlling 11.15% of the validator pool, as we want to make sure we can:
- Further secure the network by pushing up the validator threshold
- Push through consensus breaking changes faster
- Bootstrap new chain launches onto the network based contractual obligations with our partners
- Ahead of distributed adoption across the network we need to guarantee a session can be generated for new chains
- We need 24 nodes staked for a new chain to guarantee a session can be created
- We’re running 30 validator nodes ourselves and staking for these new chains for added redundancy
By the end of February, we should have over 900 servicer nodes federated to 7 community node operators. We chose our current providers with data we received from POKTscan, which was based on the following criteria:
- Operates at least 100+ nodes
- Operates in multiple regions
- Nodes staked with 60k+ POKT
- Sorted by average top reward-generation over Q4 for those nodes
While I won’t list the top node-runners by performance, I will alphabetically list all 9 node-runners that were provided from this report:
- Blockspaces
- BlokHub
- C0D3R
- Easy2Stake (E2S)
- Nodefleet
- Nodies
- POKTscan
- SendNodes
- Thunderstake
We reached out to each provider in that list and discussed their capability and/or interest in doing a POKT revenue-share with us that would still be profitable for them. At this time, we have agreements with BlokHub, C0D3R, E2S, Nodefleet, Nodies, POKTscan, and Thunderhead. We are also in discussions with NodeRiver at this time. We will ask POKTscan to periodically reevaluate their top-performers (as defined by our specific criteria), and will explore expanding existing partnerships and starting new ones based on those results.
You can track our servicers, validators, and reward generating nodes at this address on POKTscan.
Smarter Steps Forward
In cutting our own infrastructure costs, we took the opportunity to get closer with the community. We are now working with many of you to support the health of the network, and controlling costs in a way that makes sense for the entire Pocket ecosystem. We look forward to continuing working with all of you towards a bright future for all participants in the network.