I've tested it out for filtering RSS feeds and has worked pretty well [1].
I'm pretty new to embedding so my understanding may be a off.
What is the largest library of watched media that this has been tested at? I can see this choking on media fanatics watch histories.
As for the largest library, I only really know of my own which is around 250 series and 250 movies. Not small but not huge. Passing all of that info is fine enough, but I'm also curious how truly massive libraries or watch histories are handled.
I imagine you would hit the LLM token input limit first if you had thousands of series and movies. Definitely need some further testing in those cases.
At some point though like you say, it's going to become ineffective and you'd probably want to use the "Sampling" mode that is available to only send a random subset of your library to model as a general representation. Though how well this works on massive library remains to be seen.
I haven't looked at integrating Overseer yet but that is a good idea as well and worth a try at implementing. I'll be adding that to my list, thanks for suggestion!
This is going to be a really dumb suggestion, not because I think you are dumb, but because LLMs are. This happened to me the other day:
Me: "I'm going to upload a list of [things]; please suggest similar [things]. Please do not repeat any [things] on the original list."
ChatGPT°: [Blah blah blah] <List with about 1/5 repeats>
Me: "Please do not repeat any [things] on my original list."
ChatGPT: "I'm sorry, here's a list of [things] without any [things] on the list you gave me: <list without any repeats>"
No idea why that worked, and maybe it doesn't always.
°Bog-standard web interface, not an API request with a system prompt. No idea which ChatGPT flavor; I can't be bothered to keep track. Maybe doing something fancy stops that from happening in the first place, but I was dumb enough to try something dumb and whaddaya know?
I know we're not supposed to anthropomorsize these things, but it sure felt like giving a task to a teenager who half-asses it, and only does the real work after you call them out on it. It's a brave new world, that has such creatures in it.
I'd assume there isn't any algorithms provided weighted comparisons based on my viewing habits, but rather a fairly random list that looks like its based on my viewing habits.
Perhaps, in practice, the difference between those two is academic, but I'm really not keen on leveraging such a heavy everything model for such a specific use case, when something much simpler, and private, would suffice.
I just got tired of manually inputting the data and wanted a more automated approach. This recommendations system isn't extracting loads of data yet (like how often things are watched etc..) but instead a more birds eye view of your library and watch history to analyze.
Recommendation systems exist well before LLMs and have been in use for a while, wouldn't it better and more efficient even?
LLMs have the benefit of understanding (or some values of understanding) how people feel about things in general, in a global sense. They may not have a database of people's choices, but they have a "database" of connotations for every word, how ideas and emotions relate, how interests connect, etc. Instead of relying on a relatively tiny historical record of choices in few, specific (and ill-defined) categories, they can just place user's history in their 10 000 dimensional latent space and use that as a direction to explore, effectively guessing whatever the user's actual preferences are likely to be, without being able to name them or fit into explicit categories.
https://github.com/WhatCD/Gazelle/blob/master/classes/artist...
You can also specify more granular in human words what you're looking for which is a big bonus for me personally.
The general idea is I generate a prompt to feed into the LLM with specific instructions to use the Sonarr (for example) library info which is passed in as a list of series titles. I also provide some general guidelines on what it should take into account when choosing media to suggest.
After that it's in the hand of the LLM, it will internally analyze and recommend shows based on what it believe someone might enjoy based on their media library...Given that every LLM model is different, how they internally decide what shows to recommend will be unique to them. So if you have bad suggestions etc..It's best to try a different model.
it provides nice flexibility but in reality my control of the actual recommendations are limited to the initial prompt that is passed in.
If a model was trained 6 months ago for example it will likely have some info on shows that came out this month due to various data points talking about that show as "upcoming" but not released. Due to that it may still recommend "new" shows that have just released.
All that being said, I have to imagine that suggesting shows that have just now been released is likely the weak point of the system for sure.
Maybe at some point I (or someone) can create a server version repo so people have the option to choose.
Any plans to support jellyfin?
Jellyfin is definitely on the list to be added, it's probably next in line actually. If it's as simple as Plex integration I'll be very happy.
Edit - Support Added