DEVOPS

Hacking HackerNews for fun and profit

Nir Valtman
January 4, 2022

TL;DR

At Arnica, we strive to build Minimum Lovable Products (MLP). In order to build MLPs, we need to more deeply understand our users. As an early-stage company, we have fewer opportunities to talk with customers or prospects. So, we ran deep experimentation on HackerNews in order to understand what content and topics resonated most deeply with our users and make the most of each customer interaction.

Arnica is now live in Beta!
Arnica is now live. Come see how Arnica is mitigating software supply chain risks.
Get Started Today

{{arnica-top-signup-banner="/template-pages/try-arnica-banner"}}

Start with the WHY

Online traction and product-market fit are the holy grail for SaaS businesses. The summary of startup advice out there is to create a great product that people like, and to get people to know about it.

For young companies like Arnica, it boils down to delivering a Minimum Lovable Product that is derived from this research. In order to know what our customers would consider lovable, we want to hear them out, and while we had multiple validation calls with security, DevOps and engineering leaders, it doesn’t scale. Nothing replaces direct interactions with potential customers, but in order to “validate our validation calls” we must detach from our own bias towards our concepts and ensure we are data driven. This is the why we decided to embark on this path with this research.

Traffic alone can’t make a bad product sell, but if you have a great conversion metric, a great post can save months of organic growth and give you that much needed bump of early adopters.

Data extraction and graph construction

Thankfully, Hacker News published a public API to make our job easy… to some extent. We decided to query a relatively small portion of the ~30M records and ended up extracting total 2,252,719 nodes and 4,028,273 relationships from the last 6 months into a Graph database, which is comprised of 110,801 users, 199,052 stories and 1,942,866 comments. Here is how the database is structured in this research:

The graph is pretty much self-explanatory except for one detail… what is “links_to”? Well, it is quite popular to link from stories or comments to others, so we decided to parse all titles and texts and link them to the relevant stories and comments. We found 13,126 links that helped us in this research.

Qualitative percentiles

Data segmentation and qualitative measures allow better classification of user and post popularity. Below is the approach we took in this research:

User Karma Classification

  • Top = 75th percentile. Karma score is greater or equal to 7,386.
  • Median = 50th percentile. Karma score is between 969 and 7,385.
  • Low = 55th percentile. Karma score is between 30 and 968.
  • Immaterial = All other karma scores (How the heck did someone manage to get to a negative karma of -206 ?!)

Story Popularity Classification

  • Top = 99th percentile. Story score is 397 or above.
  • High = 90th percentile. Story score is between 33 and 396.
  • Nice = 80th percentile. Story score is between 6 and 32.
  • Discredited = All other story scores.

Story Comments Classification

  • Top Commented = 99th percentile. Total comments in the story thread is 54 or above.
  • Commented = 90th percentile. Total comments in the story thread is between 16 and 53.
  • Immaterial= All other comments count.

Link Referrals Classification

  • Top Linked = 99th percentile. Total link referrals count is 6 or above.
  • Linked = 90th percentile. Total link referrals count is between 2 and 5.
  • Unlinked = No referral links or only one.

Does the week day matter?

Let’s look at the chart below. With the premise of Monday being the first day of the week, hence it is represented by the number 1 and so on.

Based on the data above, you get the biggest bang for your buck when posting a story on Tuesday and Thursday. These days are followed closely by Wednesday and Monday. Traction is low on weekends and relatively low on Fridays, which is typically a shorter day for many people.

With that said, the ratio between the top to high story scores, top to nice story scores, top to discredited story scores , and top to the total story scores represent the same popularity results. It means that posting on a specific day of week does not have an impact on the popularity of the story.

Does the hour matter?

Let’s look at the chart below to understand how Top Story Popularity (defined above) is reflected based on the time the story was posted on.

From the chart above, we can clearly see that the top posts are published between 12–1 PM Eastern Time. The chart above is slightly different than the chart below with all story scores:

While there is some differentiation between the top posts and all posts, it seems like the general trend is to post between 10 AM and 1 PM Eastern Time. A couple of obvious conclusions we can derive from this comparison are:

  1. The chance to post a popular story between 12–1 PM Eastern Time is 38% higher than posting between 11–12 PM and 28% higher than posting between 10–11 PM Eastern Time.
  2. Note that in case this recommendation is implemented by enough people, this could change the stats over time.

Does user’s karma matter?

Below is a table with a breakdown of the observed users’ karma vs. story popularity.

Category Top High Nice Discredited
Top 567 5484 6037 36047
Median 778 5802 6498 34932
Low 589 5458 5552 36809
Immaterial 2 447 2440 45206

Table 1. User Karma in relation to Story Score.

The highlighted values represent the top scored stories and the relevant user karma for the authors. The conclusion on this front is that users with karma above 30 have an equal chance to post a top scored story.

Do links to stories matter?

Before the data is explained, it is important to mention that links to a story are added after the story is posted. It means that the question in this case is how links to a given story impact on the popularity of it? Let’s observe the data below:

Category Top Linked Linked Unlinked % With Links
Top 43 362 1531 21%
High 29 631 16895 4%
Nice 0 44 20483 0%
Discredited 1 28 152965 0%

Table 2. Story Score in relation to Links to Story.

Based on this data, adding links to a story contribute to its score, but it cannot ensure a Top or High story score. As a matter of fact, Top ranked stories have a 5.25X higher ratio of links to story compared to High ranked stories.

The caveat is that linking to a story should not occur immediately. The average times between a Top story sharing and the link posting ranges between 5.5-13.5 hours (the 25th and 75th percentiles). By examining less popular stories and the links to them, the average time is ~10 hours. In other words, links to a story are typically impacting its popularity if posted within 5.5–13.5 hours after the story was posted.

The bottom line

Based on the data shared thus far, several tweaks can be made to increase the chances of a story popularity on Hacker News. With that said, good content is key to gaining more traction.

EDIT: thanks for the great feedback on Hacker News. Follow the discussion here.

Arnica is now live in Beta!
Arnica is now live. Come see how Arnica is mitigating software supply chain risks.
Get Started Today

{{arnica-bottom-signup-banner="/template-pages/try-arnica-banner"}}

13 considerations for choosing GitHub-hosted or self-hosted runners
13 considerations for choosing GitHub-hosted or self-hosted runners
November 8, 2022
What to consider before enforcing MFA on GitHub
What to consider before enforcing MFA on GitHub
October 19, 2022
What every developer should know about GitHub CODEOWNERS
What every developer should know about GitHub CODEOWNERS
July 23, 2022