So you can create any sort of similar services in a single SQL query and an HTML page.
I also hosted it as a publicly accessible data lake, which you can query from everywhere: https://github.com/ClickHouse/ClickHouse/issues/29693#issuec...
It is also updated in real-time.
SELECT * FROM hackernews_history
ORDER BY update_time DESC
LIMIT 100;
And yeah, I got that from deepseek because I don't have a brain.
Generally speaking it is not a violation to scrape, index, and analyze web content as long as you don’t republish copyrighted content without a license, or violate access controls. For example: search engine indexes.
uhhhhhhhhhhhhhhhhhhhhhh
@zX41ZdbW, you can safely ignore this guy.
@GeoAtreides, next time read the actual terms of service before hallucinating.
That is actually the key phrase. HN can provide the API, no problem. People can consume the API, no problem.. But I'd ask an attorney if API consumers can then re-release the data for purposes not related to YC. By my reading, they cannot.
In particular, if someone releases something that is only a dataset along with an MIT license file, the most reasonable interpretation is that the rights holder intended to release the data under the terms of that license.
I looked for copyright cases involving this specific distinction, whether "data" versus "software" makes a legal difference, but didn’t find anything.
So the question remains open (for you, for me it's pretty clear the dataset is released under MIT).
You might want to sue and find out. It sounds like an interesting experiment.
is zX41ZdbW either?
I didn't consider you might now know about:
It's right there, you just have to click the link I shared ...
'use'...arguably the sole purpose of the API is to fetch the data.
You are grasping at straws.
This is about published text. More like if Google Trends counted word occurrences on webpages. Or if Google Ngrams counted webpages instead of books
People don't write much about non-newsworthy things whereas many people search "burger" anytime they want a burger delivery. The datasets aren't usable in the same way
Edit: not to say it's not a cool product! Just keep this in mind and enjoy using it :)
> The concept seems pretty comparable. From the title I had a good idea of what it was; when clicking on it, the visual presentation felt familiar & intuitive. \n\n Being a little less literal can be useful!
That's why I'm pointing it out: the title leads you to think they're the same metric, the page looks visually similar, and so you treat it as the same data type; but when you read the data through this lens, you draw wrong conclusions. It took me a while, scrolling down the examples, before I realised why it felt so off and that my mindset is wrong. It's what's being written about currently, not what people on HN are actually looking for
It's indeed not about being nonliteral, it's for me about having been confused about the data being shown
it was me, and i deleted it because i realized my last sentence "being a little less literal can be useful" came across as unnecessarily blunt, which i didn't want. but i wasnt sure how to express what i wanted to say without it being that way. so i deleted it while rethinking my phrasing, and rethinking your comment.
in the end, i kind of came around to understand where you were coming from, so i didnt bother to recomment.
` /api/hn -> 504 An error occurred with your deployment FUNCTION_INVOCATION_TIMEOUT cle1::c8vgv-1782399959042-aeba3cae05ff `
/api/hn -> 502 {"error":"Search entry should have an initialized schema, command was: [\"SEARCH.AGGREGATE\",\"hn\",\"{\\\"$or\\\":[{\\\"title\\\":{\\\"$eq\\\":\\\"anthropic\\\",\\\"$boost\\\":5}},{\\\"text\\\":{\\\"$eq\\\":\\\"anthropic\\\"}}]}\",\"{\\\"by_month\\\":{\\\"$dateHistogram\\\":{\\\"field\\\":\\\"time\\\",\\\"fixedInterval\\\":\\\"30d\\\"}},\\\"top_authors\\\":{\\\"$terms\\\":{\\\"field\\\":\\\"by\\\",\\\"size\\\":6}},\\\"by_type\\\":{\\\"$terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":4}}}\"]"}
512 kbps uplink, I think. Even accidental DoS was trivial. We had a self-hosted little data center at our office with the only available stupidly expensive commercial connection.
Felt some dread having to restart the main (async, single-process) web server a few times to keep things going due to bugs in our code. So many* people on dial-up patiently waiting for the page to load.
It was exhilarating though :).
*) Surely at least a hundred!
For some reason the results cut off at 2018-10 even though "Popular Comparisons" preview shows more.
https://gitlab/here_forawhile/torum
It's a HN clone, that syncs with HN that allows you to basically establish smaller private communities who can discuss anything that's on HN without actually being on HN.
It also indexes and let's you search through the DB which I find is really useful to find things that peak my interest.
This is cool concept, would love a positive/negative sentiment computed for each comment that refers to a given word, so you can see trends of "cloudflare (positive)" vs "cloudflare (negative)" where first one counts comments only if sentiment confidence is greater than say 0.6 and the other one counts comments only if sentiment is less than 0.4 (assuming [0,1] sentiment score)
one subtle consistency bug that made it hard for me to interpret when I was clicking around: the small thumbnail plot vs the full plot often (always?) seem to use different colors.
The blue / orange gets assigned to the opposite labels in the A vs. B when you click, which made it confusing to understand.
This was a small project of mine after I've found out that I can simply the whole hackernews archive (~48GB) and play around with it.
You can compare terms just like in google trends and you can also see the exact posts & comments from that time.
I like that you can discover what went crazy in the timeline, they just come up as small burst of activity, it's quite fun to play around with it. https://hackernewstrends.com/?q=litecoin&q=dogecoin&q=solana...
I also have a seperate page for the "Who is Hiring?" posts, here is the distribution of programming languages over each monthly "Who is hiring?" post in HN ever. https://hackernewstrends.com/who-is-hiring
Any kind of feedback is welcome.
Currently it says "no job-post mentions in this window" for everything. Transient error?
Where is this archive located you speak of?
A minor suggestion - I'd like to be able to render the current graph taller (full height of my browser window).
Also some sentiment analysis on the "people" graphs would be very insightful (particularly for the likes of Edward Snowdon, Julian Assange, Elon Musk and Sam Altman). Perhaps colour the area under the graph red-orange-green based on the sentiment?
The sentiment analysis is very interesting, I can do that easily. Could be a new page as well. Did you see this anywhere else or just your idea?
The transition between crypto and ai on the graphs is already pretty funny. https://hackernewstrends.com/?q=crypto&q=chatgpt
Because in general we want to know the trend of categories more than of a word, asking for “auto pilot” for ex. should include “self driving”, FSD etc.
I wondered if "go" got filtered out because it's also just a regular word.
Either way, very cool!
One thing I’d like to see is normalization by total HN activity over time.
I'd love to have some sort of normalization option to separate more subtle positive trends from the general increase in number of posts.
I was curious about Atom. According to the trend it’s still neck and neck with VS Code. But are people really talking about Atom the text editor that much still, or other types of atoms?
I am really liking the trend for "linux": https://hackernewstrends.com/?q=linux
Edit: Nvm seems like absolute count if you click the graph.
Reminds that I wish there was a modern way to do this for the words people speak and write online with. I want to literally know when people started putting literally twice in sentences.
Ngram seems is out of date a piece meal. Now Corpus seems like they try but UX terrible.
Hmm, did I break something?
No. Looking at the diagram, REST is the default until 2017, GraphQL is briefly popular around early 2020s, then the web resturns to REST.