OPEN: Socket for Community Analytics

  1. Ambition supported: Pocket has the healthiest culture and governance of any DAO.

But how do we measure this ambition? We will track multiple metrics including messages, retention, engagement, activity, communication, onboarding statistics and sentiment. We will use tools across Discord, Telegram, and the website. We intend to use these insights to shape engagement and education strategies. These metrics will also help benchmark and document the progression of Pocket.

We’ll be working closely with TogetherCrew, improving the community dashboard.

  1. Initiative: $2,500 Monthly for continued analytics, tools, and insights.

  2. We will work in the open and self-report monthly. We also have weekly check-ins with Head of Community, Zak. Our work can be viewed here at any time:
    https://drive.google.com/drive/folders/1bbFBDi68Z850PqJhq1g7n4cxGlBHU3YM?usp=drive_link

Wallet: 0x716103393284064c8DB76D28961c8CfDD89405Cb

1 Like

Thanks for requesting a socket - looking forward to getting better analytics up on our dashboard.

As a reminder, sockets open before the 25th of the month report in the same month.

@b3n n This socket is open.

Community please follow along + contribute in discord socket thread Community Analytics | thedanrodman

2 Likes

December Analytics updates:

Utilizing Discord insights, Statsbot, Together Crew, Community One, & Custom built solutions for Telegram (in progress) to collect and analyze different community insights.

These analytics helped in the Discord Restructure work and continue to drive insights and track progress of this great community!

Here’s an overview.

High-level monthly Data:


These are great stats! Tracking the activation of new members is key to improving the educating and onboarding of new members! Organizing the discord seems to have had an effect here. Discord measures activation as not just 1 message for a new member but more than 3.
14% is very solid!

Week by Week Discord Data (Statsbot)

Here we can see PER WEEK the total messages put into the Discord, the total voice hours, the unique members who sent a message or were in the voice chat, the unique channels visited in the server (you can see how we consolidated channels in our Discord restructure here), and finally the status of discord members whether they were online (regardless if they are in the server, nice to compare against the activity to see people who are on discord but ignoring the server or participating).

Week by Week Discord Data

Here we can see PER WEEK the new members, % communicated/activated and the %viewed. This is the week by week breakdown of the monthly data above.

Additionally a very key metric, Retention.
This is the % of how many members came back to the server the next week. As result it is a Lagging metric always 2 weeks behind. However, it tells us not only people who activate and communicate, but who continue to come back. These numbers are above average compared to other communities! 20% is fantastic. With continued optimization in the onboarding sequence and education this metric is bound to increase.
Also the recent dip in the messages, and retention stats is likely skewed by the holiday season and people taking time with family and travelling over checking in on the community! Except for the die hards… highlighting those superstars below…

Total Week by Week Messages
This includes the entire Discord, The Poktopus Den (TG), and PoktoPrice (TG).

We are building a tailored session to start tracking more interesting insights from the Telegram groups as well! Let’s analyze what we have so far…

The Poktopus Den Stats

Here we can see the Week by Week data for this Telegram group chat.
We can see the number of messages, the number of words and avg words per message (idea of how long messages/convos were), the unique members in the TG chat that week and of course the Top messager for that week! Jinx, Tony Kim and Charlaiz coming in strong!

PoktoPrice: The Pokt Trading Discussion Channel Stats

Here we can see the Week by Week data for this Telegram group chat.
We can see the number of messages, the number of words and avg words per message (idea of how long messages/convos were), the unique members in the TG chat that week and of course the Top messager for that week! Jinx coming in super strong in this TG chat, obviously, he’s a Pokt Legend, especially the way everyone speaks highly of him in the chat… “he’s everywhere”.

Here’s a recent Sentiment Score.
The average polarity: Approximately 0.08. This indicates a slightly positive sentiment overall. Polarity measures how positive or negative the language is, with values ranging from -1 (very negative) to +1 (very positive).

This is calculated using a GPT model. The score is typically calculated by summing the sentiment scores of individual words or phrases in the text, which are pre-rated in a sentiment lexicon (a database of words and their associated sentiment values). The final score is normalized to fall within the -1 to 1 range.

However, I believe if it was trained on more language the defi/blockchain and degen world it may understand more and probably give a higher score!

Things are looking great in the analytics!

Up next:
add more different types of data insights for Telegram
look at integrating and working with Together Crew for Live dashboard
Highlight more summaries, insights and members.

1 Like

Thanks for the update @thedanrodman!

This socket remains open for January, and Ben will get you set up on the payment stream next week.

As far as measuring impact, i’d love to see what metrics we’re NOT doing good in, so we can improve them. Is it documentation, user journey, knowledge, etc. Anything to help us outline where the gaps are and start filling them.

Can you book some time with me for later in the week next week to discuss which metrics should make it into the dashboard and how we can automate some of those metrics?
linktr.ee/doctorrobinson

Additionally - I’m asking all open Sockets to join us next Thursday for the Builders All-hands. I’d like to encourage people getting paid via the DAO to know what’s going on with the protocol, and have a forum to ask questions and cross pollinate. Matteo will run the first half w/ developers & protocol, and then I’d take the second half to review sockets work. The goal here being more cross-pollination and collaboration, so looking forward to you showing up and sharing on Thursday!

1 Like

January Summary

January was a roller coaster! Before I get into the community analytics I need to summarize what happened in the Pokt Ecosystem as it was one helluva month.

Token:

Mid-month there was a listing on Bybit as well as Crypto.com. Within days of these two listings, the token price came to a peak ~ 0.32 and the Discord community had a record 561 members enter the server in a single day.

See rough areas where exchanges were listed against token price and where the new members entering the server started to increase rapidly!

Other events:

  • Leading in Round 1 of Cryptech Sam Strongest DEPin Community
  • Nodies hits milestone with 100 million relays in a day (Jan 26th)
  • Shane, POKT veteran joins the Team!
  • ETHDenver Lego Event Released
  • NodeWallet SDK announced
  • GeoMesh
  • Developer Bounties announced by h5law for progress on Shannon

Needless to say the Pokt ecosystem has a lot going on!

Community:

January Activation Analytics:

There were increasing new members each week, with the last week of January being crazy and bringing in more new members than all of December!

These new members however seem to be more of the consumer, trader, and investment type flowing in days after the exchange listings rather than the developer/builder type. Additionally, a certain % of these new members seem to be from outside the US [Discord audience section].
The % activation of both viewing channels and communicating came down significantly from this influx. This highlights some opportunities I will highlight in the last section [Opportunities].

Regardless just for comparison, the whole month of December had a 14% activation of 372 new members which is 52 new members starting to participate (3+ messages or time in Voice hours).
January (up to 1-28) had a total of 1434 new members and the week-by-week adds up to 79 new members starting to participate. So we are still moving in the right direction!

Week by Week Discord Data:


Here we can see PER WEEK the total messages put into the Discord, the total voice hours, the unique members who sent a message or were in the voice chat, the unique channels visited in the server, and finally the status of discord members whether they were online.
Insights to highlight include the growing number of Unique members participating 280.


Here we can see PER WEEK the new members, % communicated/activated, the %viewed and Retention.

Main highlight here is seeing the increase in new members into the Discord each week as well as seeing another record week in Retention for the last week of January.

Retention is the % of how many members came back to the server the next week. As result, it is a Lagging metric always 2 weeks behind. However, it tells us not only people who activate and communicate, but who continue to come back. These numbers are above average compared to other communities! 20% is fantastic. 22.4% is amazing! Keeping this number high will be a challenge with the low activation %s and sheer number of new members in the last week. As we start to integrate with Wonderverse (Discord questing) hopefully we can keep this number up!

Time Heat Map:


This chart shows a heat map in UTC time zone of when members are in Discord with Online + dnd + idle statuses.

Discord Audience Insights:
1.20.2024

1.30.2024

Data here is aggregated and if it’s not more than 50 members it is aggregated into Other.
Discord Doesn’t allow tracking this information historically (always show rolling of previous 30 day average), so I have been manually screenshotting to track since Mid January.

Mid-January roughly 18% of the discord server was members who were less than month old in the server. A week and a half later it’s up to 29%!

Additional Countries have hit the 50 members threshold include,
AU - Australia (We love our Aussies) ,DE - Germany, and BD - Bangladesh
Interesting to see Bangladesh surge to 7% of community members in such a short time.
Digging into it, Bybit is the most popular CEX in Bangladesh! Interesting! The new members who selected the role “Asia” but did not select Node running contributed to about 6% of total messages in January and mostly on Jan 22nd!

Another insights for Marketing material here seeing that most of the community is Mobile with only 19% being desktop only (and this decreasing slightly). Could be to optimize for more vertical format content etc.

Discord Activities Aggregated:

If opted in in the settings, Discord tracks hours of other activities. This is the sum of hours across all members. We can see League of Legends is more popular than listening to music for the majority of members in January.

Total Messages Across Discord & TG:

Strong numbers coming in for total messages across TG and Discord!

Telegram:

PoktoPrice: The Pokt Trading Discussion Channel Stats

Strong Engagement metrics here!
162 unique TG members and 53,310 words! That’s more words than The HitchHikers Guide to the Galaxy book by Douglas Adams (46,333 words).
Another call out here is the increasing amount of words per message. This generally indicates more in-depth involved higher quality conversation (i.e better than a bunch of gms).

Top Contributors

The Poktopus Den Stats:

The conversation really dipped here the last week of January as most of it was popping off in the PoktoPrice channel. A very high words/message average but about half of the normal amount of unique users.

In Total Jinx had one helluva month in both TG chats coming in with 1,459 Messages and 16,686 Words! This is roughly a 30-page book … wow!

Sentiment
The average polarity: Approximately 0.106. This indicates a positive sentiment overall. Polarity measures how positive or negative the language is, with values ranging from -1 (very negative) to +1 (very positive).

This is calculated using a GPT model. The score is typically calculated by summing the sentiment scores of individual words or phrases in the text, which are pre-rated in a sentiment lexicon (a database of words and their associated sentiment values). The final score is normalized to fall within the -1 to 1 range.

I still would be keen to see how we could improve training this model and automating this.

Note there are additional weekly reports I did for Zak throughout the month that you can find here for week-by-week insights as well as my live spreadsheets/updates.
https://drive.google.com/drive/folders/1bbFBDi68Z850PqJhq1g7n4cxGlBHU3YM?usp=drive_link

Opportunities/Insights:

  • With the much lower activation % of new retail investors coming in. How can we guide these people using tools like Wonderverse and some of the education materials that are spread around to get them hyped, engaged, and becoming advocates of the vision of Pokt?

  • How can Pokt cater to the surge in new members from Bangladesh? We could look at implementing Tbot (translation bot) or an additional channel space etc.

  • Mobile seems to be popular

  • League of Legends seems to be quite popular amongst members so it might make sense to offer in-game rewards or incentives there as part of different community initiatives.

  • Between 2 PM and 8 PM UTC is when the total average members are most likely active. This is likely the best window of time to do announcements and social media posts.

  • Would love to implement Cookie3 Analytics for website insights

2 Likes

thanks for the update @thedanrodman - Socket remains OPEN for February.

Some of my personal thoughts on this:

  • this is a ton of information, which is great to have, but a bit hard to parse (even for me, who is going line by line). Is there a way to filter what’s most useful?
  • And then what do we do with this info? How are we using these insights to drive where we put our focus, $$ or community? Is there a TLDR i can deliver @adz every week that will make her marketing efforts more impactful?
  • the metrics dashboard is live - what would you recommend we move from here to there as a useful indicator of the community or the project?
  • I’d love to see some of the metrics in discord filtered with active/inactive members. Some stuff like League seems useful, but if it’s mostly non-active members playing then it won’t be a good initiative - I’d like to see the contributors getting rewarded, not the lurkers.
  • if Cookie3 is easy to integrate/cheap let’s discuss =)

Being that this is over the 2k mark, let’s start discussing how these analytics are making an impact and driving some decisions over here.

1 Like

Thanks for the thoughts and feedback!

Recapping from our call yesterday for transparency and a response to the above:

Yes, this is a ton of information and I can even go deeper in some areas.
Will try to add more of a TLDR and example ideas around what these insights drive.

Looking forward to meeting with @Adz this week and learning more about the Marketing and strategic initiatives coming up. We’ll review how I can tailor aspects of these detailed community reports to give her more insights on what she needs etc. We can also discuss the value and integration of Cookie3. This discussion should help connect the analytics to impact.

The lowest hanging fruit for the metrics dashboard would probably be Total messages per day added up across the Discord and Telegrams. I currently export a .json file of the telegram chat history and use Statsbot for the discord message tallies (allows me to exclude bot messages).

Agree for sure on your point about rewarding the active POKT members, and sorting some of the analytics more on that front.

Also, we discussed Wonderverse onboarding and what we could track/measure with ETHDenver around the corner!

Onwards and upwards!

1 Like