Granted, there have been a lot of times I have trouble finding a website in my history, open tabs or even bookmarks, so I could potentially see how that might be advantageous as long as I was in a situation where I had a second browser for "non-work" related tasks, or this was strictly prohibited in in-private mode.
>4. Find webpages you previously visited
>For those frustrating instances when you want to jump back into a past project but don’t want to scroll through your history to find an important website you previously visited, soon you’ll be able to use Gemini in Chrome to recall it for you. Once launched, you can try prompts like “what was the website that I saw the walnut desk on last week?” or “what was that blog I read on back to school shopping?”
As for their "agentic browsing assistant", I don't have much trouble adding stuff to my shopping cart or other minor tasks. I'm still waiting on that 'Google Duplex' [1] feature they announced years ago that claimed Google would make phone calls for me to make appointments and etc. Make a doctor's appointment? Dispute a charge on my bill? That's what I want.
Yeah, this seems like it would be super helpful, and would work really well with a smaller local only model since it doesn't need to generate nice prose about the results or whatever. Until they keep the data strictly local, though, yes, I'm keeping it off too.
Weirdly, from their help page[1] they mention needing to "Have a high performance computer" as a requirement, and that
> When you turn on "History search, powered by AI," in addition to the page title and URL, the page contents of the website you browse at that time are stored locally.
and that the contents are even encrypted at rest, which makes you start to think they did it the right way, but then, no:
> When you use History search, powered by AI, your searches, generated answers, best matches, and their page contents are sent to Google. This information is used in accordance with the Google Privacy Policy to improve this feature, which includes generative model research and machine learning technologies
They don't outright say it anywhere, but it seems like the implication might be that this is a strictly local only model running (Nano), but then they ruin it by sending the history search results and all the page contents of those results to google so they can use that to improve their models?
Why why why. Looking at the preference in Canary, it's just on/off. No "on, but don't send my search history and the contents of pages I've seen to google".
> I'm still waiting on that 'Google Duplex'
FWIW this has been shipping for a long time. Try doing a reservation through google maps. If there's not open table support or whatever, it'll make the phone call for you.
>> When you turn on "History search, powered by AI," in addition to the page title and URL, the page contents of the website you browse at that time are stored locally.
> and that the contents are even encrypted at rest, which makes you start to think they did it the right way, but then, no
AI is like a beast that has to be continuously fed forever to keep growing, and of course Google knows this. So, they're always going to take your data so they can feed their beast to try and stay, at least abreast, if not ahead of the competition.
I'm sure Google is also getting the message from publishers that they're getting sick of having their websites scrapped by GoogleBot only for those results to wind up in the AI Summary & not lead to any actual traffic.
So, what could Google do? What if they made everyone who ran Chrome scrape that data for them vicariously just through normal browsing? Not only that, what if in addition to having them scrape that data, but to also process it locally on your computer to save on cloud computing costs?
Just a theory... ;)
> FWIW this has been shipping for a long time. Try doing a reservation through google maps. If there's not open table support or whatever, it'll make the phone call for you.
That's cool about placing the call. Does it actually talk to the person on the other end, set up times & all that kind of stuff like they showed in the demo?
I mostly have trouble keeping too many browser tabs open on mobile. Granted, I use Brave & it now organizes closed tabs. On desktop, it has a similar Ai feature for tab management to the one Google described, but it's still not great.
I'd honestly appreciate some kind of AI tab management, history/bookmarks saving, summarizing & organizing that would put my old tabs to some kind of reading list that would remind me what I never closed down the line, archive the links I visited & my bookmarks incase of linkrot they would still be saved. Make sure if I was writing a comment on Reddit or similar site, saved it as a draft, etc, etc. That kind of "Smart" browser management system, that I could preferably run myself or had some privacy guarantees (for whatever they're worth) would definitely something I'd consider paying for.
If you cleared the history/cache, the browser is spiffy.
It looks like it is capable of more complex tasks than that including things like making a comparison table of products based on your criteria.
To extend the grocery example, it would be impressive if it could building a shopping cart and multiple stores so you can chose the one with the best total price/availability.
Incidentally, I've been doing something similar in Mistral's Le Chat. I went down a rabbit hole to see if it could help me with my skincare routine, and now I've gotten to a point where I'll have it OCR transcribe lists of ingredients on the sides of packaging to see if it's compatible for me, and if not, it gives me product recommendations for alternatives, suggestions on cheaper products & it'll crawl the web to do so. While it won't make me lists or do price comparisons across stores & things like that, what it offers has been incredibly helpful.
But the security enhancements appear to be using Gemini nano which can run on device. They kind of buried that detail though.
https://store.google.com/intl/en/ideas/articles/gemini-nano-...
Not that even the strongest models are 100% effective at spotting prompt injection attacks, but they have way more of a fighting chance than Gemini nano does.
If anything a local LLM is more likely to have those problems because it's not as capable at detecting malicious tricks as a larger model.
Of course there will probably be a setting to auto-approve everything...
At a technical level the risk isn’t from the size of the model but the fact that it is open weight and anyone can use it to create an adversarial payload.
Long live private ecosystems
I've never been a big fan of smart phones and I remember in the early 2010s the "mobile revolution" was in full tilt and it even impacted the Linux experience. I ended up switching from Ubuntu to Mint because they went all in on "mobile + touch-screens are the future!" and released this god awful UI update that was reminiscent of Windows 8.
We need business to drive innovation ... but there is bad with the good (and vice versa - we shouldn't forget that either). When something gets "hot" the business world will always go all in on the trend and "force" it down everyone's throats. It's driven partly by fear: "If I don't offer this to my customers, my competitors will and I will fail." The rest is the normal pursuit of profit, which isn't a bad thing IMO but it means there's a lot of: "There's a pie here and if we don't get our slice someone else will."
I can take the point that some AI features are oversold or under-considered, but suggesting that these new technologies are not driving business innovation is just completely indefensible to the point that the argument is absurd on its face.
[0] https://blog.google/products/gemini/gemini-app-updates-io-20...
[1] https://blog.google/products/search/google-search-ai-mode-up...
I'd be very open to any Firefox extension suggestions (or standalone applications that can consume a Firefox history) that makes it more searchable. I don't often need to search my browser history, but when I do the answer is rarely easy to find.
All of the other features look like a high potential for abuse, but with lots of glitz to make it seem essential to laymen.
I have to assume they just hire artists that have never used a computer to make UX decisions and act as product dictators.
It's either that or empiricism by idiots: "We made the feature take 5x more time to use and got a 5x increase in engagement! Won't my boss be proud!?"
My grocery store makes this really tedious because they don't have a feature to sort by price per pound. So I have a stupid ritual where I ctrl-F "($0." and repeatedly ctrl-G to see all the apples under $1/pound. Then I do it again with ctrl-F "($1." to see the ones in the $1-$2/pound price range. And there are several other products with similar annoying processes.
If an AI could just do that for me, it would save me time. I don't actually think present-day AI would do it reliably enough, but the concept sounds fine.
Do you folks hear yourself ? You want to change my password without me involving it other than clicking ?
The direct tab integration works by first showing the Gemini console (ctrl-g or Gemini icon in the mac system tray) where there is then a 'Share current page' icon below the text view.
This adds a blue border around the chrome window indicating the current page can be shared with chrome. It's not clear, but I believe the page content is only shared once a prompt is made intending to use the page's content. However, the share-enablement remains enabled for all tabs (all tab windows will have the blue border) until turned off in the Gemini consol. Again, it's not clear if just merely browsing these tabs will automatically share that content with Google.
The Gemini integration does not perform actions (can't ask it to do stuff directly with the site's content (navigate, click buttons, etc).
But direct summarization works well (try it on an HN comment page or news article).
Overall, I like this feature as long as I understand what and when things are being shared and ability to turn off easily.
If it's only processing page content as the user requests it, then how would feature 4 "Find webpages you previously visited" work? Seems like it would need to process everything in order to enable prompt-based content recall.
> History search, powered by AI Use everyday language to search your browsing history and find sites you visited. You can also ask questions and get answers based on your browsing history.
> When you search browsing history, your history searches, the page content of best matches, and generated model outputs are sent to Google
It's also worth noting enabling history awareness is a separate toggle from having Gemini enabled, and it doesn't seem to be able to answer anything about history when I turn it off. Sharing the currently open pages also seems to be a separate permissions toggle from either of these.
I'm not sure the Google is a trustworthy party [0] to believe when they give you some hard-to-find option to keep data local and not use it for user profiling and ad targeting. Google is essentially a data mining business. Some opportunities are simply hard for them to resist.
[0]: <https://www.consumerreports.org/electronics/privacy/google-s...>
This announcement from Chrome themselves hardly mentions that at all.
> In the coming months, we’ll be introducing agentic capabilities to Gemini in Chrome. These will let Gemini in Chrome handle those tedious tasks that take up so much of your time, like booking a haircut or ordering your weekly groceries.
Until FF pulls a dick move, at which point we sell all our possessions and go live in the mountains.
Part of me is actually excited for the acceleration of the end of the web, I even wrote https://punkx.org/jackdoe/zero.txt few days ago in frustration.
The sooner it ends the better.
I really don't want to see "summarization" tokens of other people's tokens.
I noticed the other day that my own Chrome browser had begun doing something almost-but-not-quite like this, specifically in the context of browser-chrome-level search autocomplete.
Typing in the omnibar now seems to offer completion suggestions based on (seemingly) 1. feeding a fulltext extract of all my open tabs into a RAG; 2. doing a search of that RAG using the completion prefix as the query; and then 3. feeding the matched text-chunks to a local (Gemini Nano?) auto-completion suggestion model.
So if I have a tab open somewhere talking about e.g. what type of plants would grow well on a north-facing balcony in Vancouver, and then open a new tab and type "trellace " into the omnibox — then it'll suggest completions seemingly coming specifically from "ideas" it got by reading that open tab. Close the other tab, and those suggestions disappear.
(This might just be an experiment that was pushed to me, though I couldn't find anything specifically referencing it in chrome://flags.)
There are several toggles there. I also had to click an agree button the first time I used it.
All SOTA models are supported and you can use your own API key :)
Btw if you launch with --disablefeatures=ExtensionManifestV2Unsupported,ExtensionManifestV2Disabled you can squeeze a few more versions out https://github.com/uBlockOrigin/uBlock-issues/discussions/29... as CLI based flags tend to live much longer than the GUI flags.
Also works with images
This will give them a cognitive profile of you: reading comprehension, decision-making patterns, knowledge gaps, etc.
Scary.
I can imagine a bad actor getting hold of this putting it into a LLM given all this how would I manipulate this person to do x,y,z.
Fantastic, Googles AI will be fighting to stop the scams Googles advertising promotes to me...
I hope they never do. Nobody’s asking to have AI shoved down their throats, spying on them and profiling everything they do.
For people who don’t want AI in their browser.
Not that I asked for any of this anyway.
I don't know the future of browsers given the trends in AI, but it seems fine to add an opt in ability to browsers to allow an LLM to access the current (or a set) of tabs. If it works it would reduce the amount of copy-paste, which seems like a good thing.
It's hardly a killer feature. I'm still going to use chatgpt (and gemini) a tremendous amount.
I really want the 2008 Google where everything they made was welcomed and not hated on sight.
Agentic browser? This. is. what. I. want.
Asking the browser about "that specific thing I might have seen last week?" Sign me the f up!
I'm not being sarcastic, I really wish I could have all of this and not having to worry about antagonistic companies and governments.
You don't want your computer to help you with your daily tasks?
In 10 years it will be passé to use the web in real time vs. having an agent acting on your behalf and distilling the information you want.
This will open up a future of webpages , sites and services exclusively made for our bots to read and transcribe to us.
Nope, not even 0.01% of that.