If you map it onto a hilbert curve, the X and Y axis mean nothing, but visually points that are close together in the sorted list, will be visually close together in the output image.
Since the first part of an ISBN is the country, then the second part is the publisher, and the third part is the title, with a check sum at the end, I would remove the checksum and sort them each as a big number. (no hyphens)
You should end up with "islands", where you see big areas covered by big publishing countries, with these "islands" having bright spots for the publisher codes.
Bonus points for labeling these areas!
I set up something a while ago [1] for an interview that does this with weather data. It makes the seasons really obvious since they're all grouped together.
[0] https://en.wikipedia.org/wiki/Hilbert_curve
[1] https://graypegg.com/hilbert (https://github.com/graypegg/hilbertcurveplayground code if anyone wants to go for the prize using this! Please at least mention me if you decide to reuse this code, but I can't stop ya lol)
The worry I have with Hilbert curves is that they make the result look like there are distinct "squares" of data [0] when really this is just an artifact of how Hilbert curves work. In that sense, the current visualization is more useful, because it's straightforward to identify the location of each country in it.
[0] https://raw.githubusercontent.com/jakubcerveny/gilbert/maste...
And yeah that’s true! you end up with squares with Hilbert curves. But those squares are all « related » data. Then those squares are related to the squares near it. Zoom out more and that grouping of squares is related to the neighbouring macro-squares etc etc.
Basically the square shape is a positive. Kind of like how charting the derivative lets you see how random/related information is, grouping into these squares gives you a visualization of pattern-ness, rather than any specific measurement.
But this is also true in Hilbert curves across the boundaries of the "squares" that I mentioned. The two center pixels in the top row are much more distant than any two pixels would be in a snake pattern.
2D neighbourhood is better than 1D one
> The worry I have with Hilbert curves is that they make the result look like there are distinct "squares" of data
that's the point, tho? instead of distinct lines of taken ISBNs in a row, you get distinct squares if taken ISBNs in a row - much more noticeable
A visualization using LoC or even Dewey Decimal would be far more useful, esp. if it also linked to public domain and copyright-free repositories/lists, say an interactive and visual version of John Mark Ockerbloom's:
One can't use ISBNs alone to create a hierarchical listing of texts which is useful for anything beyond browsing by language/publisher/order in which the ISBN was generated.
A visual and interactive representation of books by LoC or some other cataloging system would actually be useful.
Years later I was working at the library and got a little bit steamed because South End Press was reusing ISBN's after books went out of print which was allowed but, I think, lame.
One of my strategies for researching a topic is looking a few up in the OPAC, finding them in the stacks, and finding more books on the topic in those areas. (In the Library of Congress system, machine vision could be under QA56 with the rest of computer science or around TA1630, thus "areas".)
From time to time I've thought about trying to replicate the feel of this with some kind of UI given that our library moved a lot of the collection into deep archives and we have a very fast 'Borrow Direct' service with other peers)
The vast, vast majority have only been released as dead-tree versions. They have none of those. The books they scan may have an ISBN, but the scans do not have them. Like all Project Gutenberg books, their books have no ISBNs at all. From a strict point of view, they've released new editions of these books.
What you've described is that the archived content can be mapped to multiple ISBNs. It's clear the only element of concern here is the content itself. The failure to preserve a particular binding or printer's choice of typeface is irrelevant.
Failing to recognize this requires an almost malicious level of pedantry
Indeed a bigger problem is that it’s much harder to know which areas of the grid are never going to light up because the ISBN has not been used.
Lighting up the entire grid is still the goal, you're describing the problem of ensuring the right set of squares is illuminated for each piece of archived content. One is a problem of archiving the content, the other is a problem of bookkeeping.
Hardly worthless... often times, the edition of the book matters as much as the title. Steven King wrote two books named The Stand, and one isn't anything like the other. He pulled a Lucas pretty early on.
He's hardly the only author to ever do this. But it's not just authors either. Editors, collectors, translators all make their mark, and give you works that though they might be slightly different to you, the differences actually matter to the rest of us. It's not that you're ignorant that offends me, it's the arrogance about a subject you seem to know so little about that makes it difficult to tolerate.
There is no pedantry here, just a desire to actually preserve books and to organize them.
Then those two texts would map to different ISBNS, or perhaps each maps to multiple different ISBNs, it doesn't matter. That some texts exist with the same title but different content is similarly irrelevant.
The content is all that matters. Two different bodies of content, two different entries in the archive. Each entry may map to one or more ISBN numbers.
> the differences actually matter to the rest of us
The only differences that matter are what matters to the archive that made the blog post. Your concerns are for entirely different things, which is fine, but don't say the OP's concerns or initiatives are impossible or ill-suited based on a criteria you're projecting onto them.
Are you saying they actively remove ISBN numbers from scans? If I downloaded one of the books, it wouldn't have an ISBN?
Why? That seems like a bunch of extra processing per book, makes it harder for users to specifically identify a book, and probably does nothing for legality. Also, can people search by ISBN?
No, he‘s playing the pointless „well, actually a scan of a book is a different thing from the book itself“ game.
> From a strict point of view, they've released new editions of these books.
And this is clearly a semantically worthless distinction from the point of view of the archive.
When different editions have different content, archiving those differences in that content may matter (arguably not for simple typographical corrections, printing errors, etc). When different ISBNs have identical content, it is totally irrelevant to the goals of the archive.
> Until now, the only options to shrink the total size of our collection has been through more aggressive compression, or deduplication. However, to get significant enough savings, both are too lossy for our taste. Heavy compression of photos can make text barely readable. And deduplication requires high confidence of books being exactly the same, which is often too inaccurate, especially if the contents are the same but the scans are made on different occasions.
The image contains 1000*800 pixels at 2500 ISBNs per pixel, so it's visualizing 2e9 ISBNs. ISBN-13 contains 12 digits plus one check digit, so we might have expected the image to be 500 times bigger/denser than the current image. The fact that it's at its current size suggests that only ISBNs with 978 and 979 prefixes are included, and since the bottom half is more sparse, that probably corresponds to the new 979 range.
Find the interactive visualiser by scrolling down, and switch it to "Files in Anna's Archive [md5]". This will highlight the location of the green pixels in grey.
- Right-click the image and select "Inspect".
- Add a new CSS hue-rotate filter to the element:
element {
max-width: 100%;
margin: 0 auto;
filter: hue-rotate(-90deg);
}
Can you change the green channel to blue to better view it?
If self-destruction is a necessary premise here, is that really a good thing?
Deze website is geblokkeerd
Europese sancties
De Raad van Europa heeft besloten dat de websites van RT (voorheen Russia Today) en Sputnik News niet meer mogen worden doorgegeven. De website die je probeert te bezoeken, valt onder deze Europese sanctie.
VodafoneZiggo is verplicht de sanctie uit te voeren en heeft de website geblokkeerd.
https://web.archive.org/web/20250106112552/https://annas-arc...
"This server couldn't prove that it's annas-archive.org; its security certificate is from *.hs.llnwd.net. This may be caused by a misconfiguration or an attacker intercepting your connection."
According to Wikipedia, www.ukispcourtorders.co.uk used to list the blocked domains and the court orders responsible.
https://en.wikipedia.org/wiki/List_of_websites_blocked_in_th...
Anna's Archive is getting sued currently for scraping vast amounts of essentially public metadata which was being gate-keeped by a single organisation.
Here's the longer and more complicated answer for you:
https://libraries.emory.edu/research/copyright/copyright-dat...
already asked LLMs so please don't copy/paste an LLM response.
What do you mean by "more green"? I don't see any shaded green.
And I presume the black pixels are unregistered ISBNs?
"...
European sanctions
The Council of Europe has decided that the websites of RT (formerly Russia Today) and Sputnik News may no longer be transmitted. The website you are trying to visit falls under this European sanction.
..."
In Italy it just errors out with a NS_ERROR_CONNECTION_REFUSED.
Flipping DNS to 8.8.4.4 fixed it for now but I really need to move this connection to A&A.
$ dig annas-archive.org @89.101.251.228
annas-archive.org. 360 IN CNAME unavailable.for.legal.reasons.
unavailable.for.legal.reasons. 339 IN A 213.46.185.10
Not really standards compliant, but an interesting use of DNS.
Would Tweak have blocked this? Most households in the Netherlands currently have the choice of Ziggo, KPN, and Odido. Long live VPNs…
I currently have a 1 Gbps down / 300 Mbps up unlimited connection, and I pay only 16 euros (~16 USD) per month.
I wonder why the US is so bad on home internet connections, but maybe it's because of the scale of your country?
If you're paying $100 per book for someone to visit a major library, get the book out, scan it, check the OCR? Then you'd probably be selective, to get the most out of a limited budget.
But if you're grabbing epubs and pdfs, and a book only needs $0.002 of space on a hard drive somewhere? Grabbing the useless 41% is probably cheaper and easier than exercising editorial control.
Many autocratic regimes editorialize and censure all forms of publications. But even in the US, which is nominally still a democracy you now have states like Florida forcing changes to literature works and banning books entirely for religious and ideological reasons. And this is not just a right wing thing. There have been a few publishers that took it upon themselves to editorialize literature from the 19th and 20th century to get rid of some things that are now considered sexist, racist or otherwise offensive. The whole cancel culture is not just about canceling people, but about limiting access to their work as well.
I was at a Christmas market in Berlin a few weeks ago near the Opera. There's a nice little monument there for the book burning that happened in the 1930s. Anything that was vaguely intellectual or Jewish in origin was burned right there during the Kristallnacht. Nice place for a Christmas market and a grim reminder that those calling for things to be deleted/cancelled aren't necessarily very nice people. And of course Hitler himself got cancelled. Possession or distribution of his books is still not allowed in Germany.
Anyway, imagine somebody in 5000 years finding their way to some archive of hacker news or some reddit thread might look differently at the value of some of the comments than the average moderator.
AFAIK this has never been true in Germany (for the book Mein Kampf at least). AFAIK the German state of Bavaria inherited Hitler's copyright on the book, and did not republish it. This means that no one was allowed to print it for copyright reasons, but you could still own or trade existing copies of the book. After 2015, 70 years after Hitler's death, the book entered the public domain. Looking into Wikipedia, uncommented reprints have been forbidden: https://en.wikipedia.org/w/index.php?title=Mein_Kampf&oldid=..., which I didn't know before.
This is not honest. They're not banning any books, they are stopping school teachers from forcing certain books on children. The difference is immense.
Though, I am not against highly directive schools (I think we need them), I am against mis-characterization.
And of course, the list of books is getting pretty long and what's on that list is basically determined by a very small but vocal group of christian conservatives with uptight opinions on things like science, evolution, sexuality, and other things they insist are wrong/evil/dangerous.
So they are not technically banning anything. But they are punishing people that go against this nonetheless. Which makes it kind of a ban. It doesn't go as far as Nazi book burnings. But I have just about as much sympathy for the people that did as I have for those insisting e.g. Harry Potter must not be in a school library succeeding with that because the threats against teachers and schools are very real and these people wield a lot of power, apparently. Bullying teachers and librarians into complying with this seems to happen a lot in Florida. That's not banned at all and actively encouraged.
What's next Uncle Tom's Cabin? Oh wait that actually happened as well in some schools. Because these people also include some racists and xenophobes. You might call some of these people fascists even. And they just don't like being reminded of things like slavery.
I would suggest that judgement is a critical part of our civilization, and it's judgement that says those bits of obscene graffiti in Pompeii that makes it so.
Or else they could say "well, we can't claim ancient cave art is priceless, because we're biased and our biases will change over time. Maybe in a thousand years we'll discover that ancient cave art is worthless, so we'll do nothing".
In fact you have judged my opinions and shared your judgement with me. Good job!
Your characterization of regimes as autocratic is judgmental, biased and will change over time. But right now that's your judgement and I applaud it, even if I disagree.
Gosh, book burning. Not backing up a romance novel or cookbook is definitely analogous to book burning, but I'll play along.
It was a symbolic act to show a rejection of ideas, not an attempt to eradicate the books, much in the same way Gandhi encouraged the burning of foreign made clothing and products. He wasn't going to rid the world of British cloth nor were the Germans going to rid the world of non-German ideas.
So yeah, when all the badly written cook books, romance novels, and children's books are in a huge bonfire, you can blame me, personally.
This is poppycock. Backing up all books -- the very action discussed by the person you're answering -- is by definition neither subjective nor subject to biases.
> This would then imply I could never take any action, because it's just subjective and biased.
And even if the first quoted claim were true, this, too, clearly isn't. Nowhere does the comment you're answering imply that the bias or subjective rationale of an action should, ipso facto, discourage a person from taking it.
Your comment is replete with similar reasoning, so warped that it's difficult to characterize as anything other than in bad faith. Indeed, this is the snottiest, rudest, least constructive comment I've seen on HN in quite some time -- excepting a couple of my snotty remarks on language or the quality of someone's writing.
I have no idea what response you expect, but the only one you deserve, I think, is one that just points out your dismissiveness, sarcasm, and breathtaking contempt. What an awful way to move through the world, let alone through HN.
But we should still archive it. Some day it might be useful to someone ;)
> this is the snottiest, rudest, least constructive comment I've seen on HN in quite some time
I wish ;-). I see a lot worse here regularly. But it's certainly not nice behavior. Luckily, I have a thick skin.
Well, it takes one to know one. So there.
1) tldr: you're wrong. I am not suggesting that "backing up books" is subjective or subject to biases. You'll need to reread my response: I'm talking about the judgement of doing so. And deciding that it is worthwhile (or not, as I joked with my made up 41%) is a judgement that is subjective and biased (if you subscribe to that sort of world view).
2) tldr: you're wrong. I openly indicate in my reply that I am exaggerating and extending his position (a reductio ad absurdam for you Romans reading this). However, he is suggesting that my position is wrong, or should give me reason to hesitate, as it might be "biased" et al. So I think my take is quite on the money. If you think your actions are biased and subject to historical revision, are you going to march along confidently? Or will your fingers tremble at the next book burning while you wonder "will history condemn me for this?"
3)tldr: you're wrong again. You resort to ad hominem attacks after demonstrating a complete inability to understand my position. I'd say you demonstrate both your intelligence (or lack thereof) and worth as a being (or lack thereof). I suggest you consider the unbiased subjective absence of your existence as a priority.
4) tdlr: I expected nonsensical windbaggery, and you delivered! Thank you. You advanced exactly zero of the positions involved and reduced this thread to garbage. YOU are the cancer killing the internet. Have a nice day.
"No, I think you're an LLM-powered bot! So there!"
Please don't make stuff up about the Holocaust. It's the sort of mistake you shouldn't make even once.
My argument is that the symbolic rejection, not the practical destruction of books, was important. Also, I pointed out through a subtle (too subtle for you so I'll spell it out: this is a wholly irrelevant argument - arguing that books don't deserve to be backed up has nothing to do with book burning) - so I played along - as I noted. Get it?
The practical destruction of all books considered non-German, or Gandhi's destruction of all British fabrics - might have been a desire, but there's no need to publicly burn books to destroy them. They can be efficiently destroyed privately. The Nazi's could have destroyed all the books they liked in some private little place.
Ergo, my argument stands! PUBLIC book burning is about PUBLIC rejection of something (non-German elements, or Foreign made goods, etc) and may in fact be less efficient than simply quietly and privately putting them in a landfill which no one will see. Get it?
I'm not making stuff up about World War Two, I'm arguing about a ridiculous analogy to not backing up romance novels being akin to book burning. Apparently, not wanting to back up romance novels and cookbooks makes one a holocaust denier. Go Hacker News.
Mr. Godwin? Are you around?
When I said "that's Holocaust denial", I wasn't saying "you're a Nazi", but pointing out an accidental mistake. If I thought it were deliberate, I wouldn't have said anything.
Your argument about the symbolic meaning of public book burnings is irrelevant (and I'm not sure why you brought it up), but there very much is a parallel between calling for the destruction-by-bitrot of books, and calling for the destruction-by-fire of books. (Again, I am not calling you a Nazi.)
So, in the spirit of rebuttals to random non-sense, I pointed out that there was no link at all to my comment and book burnings, but... I repeat myself.
Suggesting that backing up garbage books is not worthwhile, is apparently analogous to wanting all garbage you selectively oppose to be burned, in public.
So, by extension, the Nazi's can be condemned for not only burning books, but refusing to make backup copies of them.
I hope that clears this up.
How plausible the argument to the value is depends on the eloquence of the person under the Law.
This is the logic of hoarders everywhere.
There's a schlocky Victorian pulp novel that's of no use to anyone - except that it happens to contain a fantastically detailed description of an abandoned saltings in my hometown that nobody ever thought to record in any way. For me, those two paragraphs are gold.
If the novel hadn't been digitised as part of Google's Books Archive Project, I wouldn't have been able to find those two paragraphs. Digitisation not only creates backups, it enables completely new ways of interacting with those texts (eg Google's Ngram Viewer).
I retract my position, let's back up everything!
What seems banal and useless to you, might be extremely important for future historians, and to be honest, books are pretty compressible and storage is cheap.
Many books are just electronic garbage at this point, and backing them all up is like going to a landfill and saying "We should make another one, exactly like this one, in case this landfill proves to be valuable to someone, someday."
It might be useful for LLM training to produce garbage. Although many say they already do a good job at that already.
I'd start with every single AI generated book that's said to be available on Amazon (300 or so iirc).
And people can and do judge things all the time: Nobel prizes, juried contests, review boards, movies, music, and yes - even books! - as being worthwhile or garbage. Rotten tomatoes, Nobel prize committees, and so on.
So yeah, I think your answer is not the only reasonable one. And maybe 41% is way too low.
As the saying goes GIGO. In case you're not familiar with the term: https://en.wikipedia.org/wiki/Garbage_in%2C_garbage_out
Let's back up ALL the garbage. Someday it will be mined as gold, by hypothetical and probably never to exist future historians.
There are lots of worthless things we can do, quite practically. Let's not do them and say we did.
>There is much to explore here, so we’re announcing a bounty for improving the visualization above. Unlike most of our bounties, this one is time-bound. You have to submit your open source code by 2025-01-31 (23:59 UTC).
>The best submission will get $6,000, second place is $3,000, and third place is $1,000.
>All bounties will be awarded using Monero (XMR).
? Why are they using crypto, and, weirdly enough, specifically the crypto people use for buying drugs, to award this?
Is it some kind of scam?
The harm they may cause in the short term via tax avoidance or being used to buy drugs is minimal, but the possibility that because of them archivists are able to fund servers for data that future historians wouldn't have otherwise been able to get their hands on? Priceless.
You really have to ask why a illegal/grey site is using currency that is build to protect privacy and anonymity?
is this some kind of sarcasm?
See: Library of Alexandria, Library of Congress, GenBank, the Svalbard seed vault, Google Books, Internet Archive and all its efforts, ...the Louvre, and most major museums.
In general, we collectively recognize - without having to be told - that preservation of knowledge is a noble and worthy effort that transcends the fleeting whims of a population at a point in time.
All that to say, people probably don't need to be tricked into liking such efforts. They're popular because of what they are.
Reasonable people are objecting to copyright law violation, for the simple reason that it disincentivizes further knowledge creation.
Even more reasonable people are objecting to weaponizing copyright law violation on behalf of the vilest dictatorship on the planet.
Do you honestly believe that our current copyright framework is mainly aligned at maximizing incentives for knowledge creation?
This sounds absurd to me. From my point of view, the copyright framework has been shaped (by continous lobbying efforts) into a system to maximize extraction of profits from existing IPs.
That is very different from incentivizing "knowledge creation", because the lions share of income is spent on overhead or distributed to shareholders, with the "knowledge creator" (i.e. author), getting <20% of each sale. Furthermore, the mechanisms to balance income are ALSO abysmal (to maximize knowledge creation incentives, it would be necessary to "overspend" significantly on "young" writers, enabling them to feed themselves at the start of their careers).
> weaponizing copyright law violation on behalf of the vilest dictatorship on the planet.
How is Annas archive weaponizing copyright violation? How is it furthering Putins interests?
how?