4 minutes
Github Stars Dragnet
Concept
When consuming technology news and trends it can be tough to know what is a passing fad and what has staying power. A dragnet can be applied to see what has stood the test of time and is worth further/continued consideration.
A dragnet is any automated method of reviewing favorites, bookmarks, links, stars, saved articles, etc at high volume with a buffer of time/distance to control for hype.
Github Stars
Starting from my original guide for gathering github stars, I wanted to identify which repos I’ve starred over the past ten years which still deserve consideration today.
I have 1,895 starred Github repos as of 9/11/2021.
- First star: Twitter Bootstrap on 8/5/2011
- 1,605 (84.7%) were starred by me on or before 9/1/2020
- 1,113 (58.7%) have mit (769), apache 2.0 (262), bsd-2 (23), bsd-3 (32), or isc (27) licenses
- 790 (41.7%) have had push activity since 8/1/2021
- 709 (37.4%) are classified as JavaScript repos
Example: JavaScript repos of note
See also: Node.js Total Cost of Ownership
Criteria
- JavaScript as main repo language
- Starred by me on or before 9/1/2020
- Repo has been updated recently (between 8/1/2021 and 9/11/2021)
- At least 1000 total stargazers
- Friendly license type: MIT, Apache 2.0, BSD-2, BSD-3, ISC
109 Repos. Sorted by stargazer count descending.
Methodology
You can do things like this with your own Github stars. Here is what I did to produce the list above.
for i in {1..40}
do
curl -H "Accept: application/vnd.github.v3.star+json" "https://api.github.com/users/tphummel/starred?page=$i" > "p$i.json"
done
rate limiting can cap what you can get in one shot when unauthenticated. You may need to authenticate or wait for the limit to expire.
for i in {41..65}
do
curl -H "Accept: application/vnd.github.v3.star+json" "https://api.github.com/users/tphummel/starred?page=$i" > "p$i.json"
done
combine files and filter the fields on each starred repo
cat p*.json \
| jq --slurp '. | flatten | .[]' \
| jq --slurp '.[] | {name: .repo.full_name, url: .repo.html_url, lang: .repo.language, stars: .repo.stargazers_count, starred: .starred_at, license: .repo.license.key, updated: .repo.updated_at, pushed: .repo.pushed_at, archived: .repo.archived, disabled: .repo.disabled}' \
| jq -n '[inputs]' > stars-captured.json
filter rows on criteria
cat stars-captured.json \
| jq -r '.[] | select((.stars >= 1000) and (.starred <= "2020-09-01") and (.pushed >= "2021-08-01") and (.archived == false) and (.lang == "JavaScript") and ((.license == "mit") or (.license == "apache-2.0") or (.license == "isc") or (.license == "bsd-2-clause") or (.license == "bsd-3-clause")))' \
| jq -n '[inputs]' \
| jq 'sort_by(-.stars)' \
| jq -r '(map(keys) | add | unique) as $cols | map(. as $row | $cols | map($row[.])) as $rows | $cols, $rows[] | @csv' \
> friendly-js-stars.csv
filter columns for display
cat friendly-js-stars.csv| q -H -d "," "select name, stars, url from - " > friendly-js-stars-trimmed.csv
Appendix
Why is the post not on data.tomhummel.com? The fact that I starred these repos is incidental. This is about surfacing trends, themes, and ideas.