PEP-7: The Sand Walker -- a Pocket reward explorer

Attributes

  • Author(s): SkillZ team
  • Recipient(s): SkillZ team
  • Category: Reimbursement
  • Fulfills: RFP-8: Node Monitoring Tools
  • Related Installments: none
  • Asking Amount: 15,000 POKT

Summary

We would like to ask the community and the DAO for funds for the development and hosting of the Sand Walker, a POKT rewards calculator with a user-friendly interface.

Abstract

We open-source, improve and host the Sand Walker, an explorer for POKT rewards based on live data from the blockchain.

Budget

15,000 POKT one time payment.

Rationale

We have chosen to focus on a single aspect of monitoring and provide a definitive answer to “How many POKT did I earn over this period of time?”, answering RFP-8: Node Monitoring Tools.

In the spirit of Node Pilot, we’ve chosen to build an user interface for the POKT Rewards Calculator to lower the barrier of adoption for running POKT nodes by enabling anyone to access aggregated chain data, as a source of truth, to compute rewards for their nodes over a time period of their choosing.

Community members running nodes won’t have to rely on someone technical to get quality information about their operations, no matter which product or method they chose to deploy their nodes, solving a major pain point we’ve observed in the community.

Also in the interest of user experience, we provide hosting for this application. No local install, no CLI needed, anyone can enter node addresses and get information about their rewards; tools & code of the frontend to run your own stack will be open sourced.

The Sand Walker provides a summary of the rewards earned by a node with a monthly breakdown of all rewards with the corresponding block. We developed a frontend for the sake of accessibility, under the hood is a database filled by an instrumented Pocket node that is connected to the blockchain: the data we expose can be traced down to the blockchain.

Dissenting Opinions

We’re open to dissenting opinions. We built this because we needed it so we will improve it as we need anyway. If the community finds the POKT Rewards Calculator useful, we are open to opening the code, adding additional features and maintaining it online.

Deliverable(s)

Currently available:

  • Current Sand Walker frontend available at: https://sandwalker.sbrk.org/
    • Summary of rewards earned by a node,
    • By-month breakdown of all rewards with the corresponding block,
    • Currently uses a stalled database (with data up to block 24828),
    • Mobile-friendly web user interface.

To be delivered:

  • Open-source code (MIT ):
    • Python/Flask frontend built on top of a SQLite database,
    • Fork of a Pocket node with custom instrumentation that fills the database in real-time from the blockchain, there is no weird logic behind the computing of rewards, we directly use the mechanisms of Pocket nodes (this means it will automatically handle incoming DAO changes per block, or tricky logics and corner cases),
    • Tools and instructions to run your own stack (currently using docker-compose),
  • Features to come:
    • JSON/HTTP API that can be used to directly get the history of rewards of a node,
    • Ability to query history from a batch of addresses (currently, history can only be browse one address at a time),
    • CSV export,
    • Mechanism to offer a daily export of the database (SQLite), which will be helpful to anyone who wants to develop tools around, without having to set-up their own stack. This database will contain all rewards from all nodes from block 0 to the current block height at the time of the dump,
  • Hosting of the service on SkillZ:
    • Publicly available,
    • Real-time information from the blockchain,
    • Regular updates of the backend to follow Pocket’s changes (to ensure the instrumentation follows Pocket upgrades).

Contributor(s)

The SkillZ team.

Copyright

Copyright and related rights waived via CC0.

6 Likes

I think that a monitoring tool of this nature would be extremely beneficial to the network. And I would love to see more initiatives like this.

I can’t comment on whether the proposed cost is too much or too little as I’m not technical enough to build this myself. So, comments from other members of the community in that regard would be helpful.

Otherwise, I back the Skillz team to deliver a great product for the community.

@tdephuoc what kind of timeline are we talking about in terms of the deliverable?

3 Likes

Hello,

Thanks for the interest!

  • We can have it publicly available with real-time data from the blockchain quickly with the current set of features (a few days likely),
  • Open-sourcing it can be done quickly as well,
  • Developing & rolling-out features listed here would likely take 2 to 4 weeks (we expect most of the time there to be spend in providing a production-ready stack that can be started by anyone who wants to fiddle with it, or adapt it to other needs),
  • We are open to add features depending on demands/needs and feasibility,
  • Regarding the cost, it is estimated with a long-term maintenance in mind: keeping it up-to-date with new versions of Pocket (as it depends on Pocket’s code), ensuring the daily dump of the database is fine, having monitoring in place to ensure the data is real-time, etc.

At the current OTC price of POKT, I feel the ask is appropriate.

Ability to query history from a batch of addresses (currently, history can only be browse one address at a time),

With this feature you open up some interesting possibilities for monitoring a large group of nodes. I’d love to see some metrics around the group – like a simple dashboard that shows you how your node group income has done over all time and what it has done in the past 24 hours / 1 week.

Also what is very interesting is that you have ALL nodes data in your data base and can make comparisons! Show me how a node is doing vs. all active nodes in terms of rewards. That is super interesting.

I don’t believe the minting records on the Pocket chain tell you what relay chain ID was served, do they? Including the relay chain ID data would be interesting if possible since now that more chains are switching to paid, the rewards per node will start to vary based on chains supported.

In summary, I find this a useful tool and will support the funding. As a node operator, however, I’d find it more interesting if it operated like a real time dashboard I can monitor vs. a history of things that have happened in the past.

1 Like

This is great. Thanks for doing this guys. @adam made the point that this will be great for taxes.

I have no issues with the asking amount for the grant itself.

A couple comments:

  • Can you change the rewards text to POKT instead of PKT?
  • Seems to not work with an address in all caps.
1 Like

I don’t believe the minting records on the Pocket chain tell you what relay chain ID was served, do they?

Looking at Pocket’s code, it is something that can be added easily (will have to rebuild from block 0 so it may take ~3 days to get it), but it is trivial to add, I’ll do it.

I’d find it more interesting if it operated like a real time dashboard I can monitor vs. a history of things that have happened in the past

The current setup is using a stalled database we’ve built for the POC, but this database will be populated in real-time when we deploy it to a production environment (tools to do your own live setup will be there as well) ; adding a real-time dashboard would mostly be a UI thing, I’ll check what javascript libraries can be used there.

Can you change the rewards text to POKT instead of PKT?

Sure! Sorry about this :wink: just fixed it.

Seems to not work with an address in all caps.

Yup, thanks! Fixed as well.

The vote is live! Snapshot

2 Likes

The vote passed \0/ ; thanks everyone!

I’ve opened the repository a few days ago and I’m currently implementing the delivery list mentioned above:

Check-list:

Open-source code (MIT )
Python/Flask frontend built on top of a SQLite database
Tools and instructions to run your own stack (currently using docker-compose)
Fork of a Pocket node with custom instrumentation that fills the database in real-time from the blockchain
JSON/HTTP API that can be used to directly get the history of rewards of a node
Ability to query history from a batch of addresses
CSV export
Mechanism to offer a daily export of the database (SQLite)
Publicly available
Real-time information from the blockchain (implemented but not deployed)

I’m currently re-syncing from block 0 with some schema changes (to have the block time as well) ; once done, I’ll upgrade the current URL with a live database.

I wasn’t able to get the chain IDs yet(there is a chain ID that was trivial to implement, but it actually corresponds to mainnet/testnet, not the actual blockchain for which the rewards were minted) ; I’ll continue digging there when I have time.

If you have suggestions, ideas or feature requests, feel free to open issues on the repository.

Congrats @mxs! Be sure to share the address of the account that you’d like to receive the grant to.

Also, let me know if you’d like to keep this thread open for updates or would prefer to close it. I generally close finished proposals to help differentiate from active proposals in the forum.

Thanks!

We had little time to give to the Sandwalker in the recent days, it’s still rebuilding the database (a few more days to catch up I think) ; I’d favor:

  • complete the checklist here, let the thread open in case someone has special requests that think should be done in the context of the grant,
  • once completed, receive the grant,
  • close the topic, use the github tracker to track new feature requests.
1 Like

I’ve completed most of the items, the current version version of the sandwalker is now using real-time data (the current block height is displayed on the front-page, that can be used to double-check it is up-to-date). On the per-address page it is now possible to download data as CSV for more convenience.

For people familiar with docker-compose, spinning up your own instance takes a few minutes of human time and there are instructions for it; it will take days of machine time for the synchronisation of the underlying Pocket node to be at head, it has to start from block 0 up to the current height to process all rewards (I can provide a custom 25K dump or something at some point if folks are interested).

Current check-list:

Open-source code (MIT )
Python/Flask frontend built on top of a SQLite database
Tools and instructions to run your own stack (currently using docker-compose)
Fork of a Pocket node with custom instrumentation that fills the database in real-time from the blockchain
JSON/HTTP API that can be used to directly get the history of rewards of a node
Ability to query history from a batch of addresses
CSV export
Mechanism to offer a daily export of the database (SQLite)
Publicly available
Real-time information from the blockchain (implemented but not deployed)

The ability to query history from a batch of addresses is not yet implemented, but the data is available via the daily-dump (I think for fine-grain & at-scale analysis, the daily export of the database is more appropriate, you can directly use sqlite to import it and play with the data, without having to install a SQL server or something).

I plan to add batch of addresses at some stage, but I haven’t decided yet in which format (it will probably take the shape of a page with total rewards per address, and links to the per-address corresponding pages with more data).

I also plan to update documentation short term, especially for the API (with examples on how to use it using curl).

We’ll get back to you for the grant if that sounds good.

Confirming that the proposal has been disbursed.

{
    "logs": null,
    "txhash": "0199CCEFF1B2409D9CAC0C7A7B6235B5EE58F7085784B458A0C03C767E727FD5"
}

Congrats :slight_smile:

1 Like

Nicely done. Will be super nice when it will accept a list. Thanks.

2 Likes

Does Sand Walker support rewards coming in from the recently whitelisted blockchains? @tdephuoc @mxs

A few recent changes were implemented/deployed:

I think it’s now 100% feature complete.

I haven’t double-checked but it should be working fine, it fetches all rewards that are minted on mainnet.

Regarding the recent chain halt, the sandwalker is still impacted and catching up, also the data from blocks between 30039 and 30218 is currently unavailable, I’m currently waiting for a 0.6.4 release to re-create a full proper database by syncing from block 0.

Feel free to reach out on discord if you have any questions/suggestions!

Great initiative & congrats on winning the grant :slight_smile: