There are already plenty of insurances that require you to submit claims through a smartphone app that tries to essentially do this by capturing sensor metadata etc. - those don't need to be nation-state resilient, just Joe the Crackhead Insurance Scammer resilient, so this works. Likewise, more and more things online require identity verification (either officially or disguised as age verification).
Edit: And while "a nation state actor can spoof this" is a problem for the journalism use case, the insurance/ID verification use cases are perfectly fine with anything that raises the bar but could be bypassed with enough effort. Also, the journalism use case suffers from the same fundamental issue all of these use cases suffer from: People will "verify" the picture by looking at the repost of a screenshot of the verification UI, not by verifying the original themselves.
Insurance companies can have a native app and require the device’s camera. Companies already have tools to combat a liveliness check. Even if you’re using a modified app that pulls from the photo album instead of the camera? A video recording with the appropriate liveness verification easily avoids that mess.
It's been possible to do a live video deepfake for a long time now, but as with all new tech, law and society are taking their sweet time to understand the risks; IMO this is the other side of the same coin as some infamous tech comments on consumer products: https://news.ycombinator.com/item?id=9224 and https://en.wikiquote.org/wiki/Rob_Malda
NVIDIA suggested AI fakes controlled with face tracking input as a compression technique just for reducing video call bandwidth requirements (to ~117 bytes per frame). They did that six years ago: https://www.dpreview.com/news/5756257699/nvidia-research-dev...
As we're now in an AI race, even NVIDIA's specific technique has flaws which all the current tools can detect, there's never any guarantee of this continuing to be the case.
That said, in the case of Apple, they're historically followers not leaders despite the public image they like to present about innovation, and I'd expect this method to be flawed from day one even if we weren't reading a corporate blog post written in a self-congratulatory tone I find almost as off-putting as when AI write.
AI deepfake or edit video doesn’t pass liveliness checks without all the c2pa or reference image song and pony show.
Insurance companies can monitor the light reflections from the flash that they control or monitor the accelerometer and compare the accelerometer values with the video that they receive.
They could also just update their app to stop accepting photos from the album.
True.
> raises the bar but could be bypassed with enough effort.
Anyone can spoof this.
Apple cannot stop spam iMessages. They can't stop someone from rendering their privileged UI inside a browser viewport. People copy and paste remote script executions from convincing captchas.
This whole provenance thing is a red herring. You agree with me, but there's truly not a single application for this that won't be exploited.
Photoshop / AI-gen an image -> display on a high-resolution monitor -> photograph the monitor with iPhone 18 Pro -> valid Apple Reference image.
To get valid reference photos, you can go to the actual physical location, put the iPhone/monitor in a cardboard box to block external light, then photograph the monitor. Paint the inside of the box using Vantablack (stopping reflections) and cover the LiDAR projector with tape.
I can't wait to see Apple Verified™ photos of UFOs flying over the Golden Gate Bridge.
https://image-ppubs.uspto.gov/dirsearch-public/print/downloa...
The Apple Reference Image feature is here launched on iPhone 18 Pro and iPhone 18 Pro Max that both have built-in LiDAR sensors that could be used for this process.
A better fix is to take photos with all three iPhone cameras simultaneously, ideally as a 2-3s video, and use the parallax/multiple perspectives to extract depth information. The video files (Possibly audio too) could also be included with the verified image as additional verification.
They can also prevent photos if iPhone detects the LiDAR sensor is covered, similar to how Meta does it with their camera glasses.
I've never looked at the LiDAR hardware, but where is the emitter in relation to the receiver. Why would the LiDAR not reflect off of whatever you're blocking it with and return a very short flight meaning it was very close?
Similarly, LiDAR alone will help disqualify cases where someone is just taking a picture of e.g. a landscape target of the Golden Gate, but that it shown on a screen 1 meter away.
> increasing the difficulty of producing a forgery
The problem with this thinking is twofold:
1) Whether it actually meaningfully increases the difficulty of a forgery remains to be seen. Despite their initial language about discerning real events, we see no details here about what scene information is used.
2) It increases the potential value of a forgery because now your forgery is attested by Apple.
So it either makes it easier to defraud people or more worthwhile to put in the effort to defraud people or both. None of those outcomes are great.
[1] https://commonlands.com/products/ir-cut-filters-csp650?srslt...
But I really mean that if the lidar barely works outdoors anyway then actually you don't need to be 16 feet away at all.
Anyway, one may presume that they've thought about this.
It’s almost certainly possible to fool v1 of this system, for some images, in some contexts. It would be shocking if the first implementation was completely perfect. But maybe it’s better than nothing?
The problem is that it makes it easier to fool people and provide "cryptographic" evidence of validity, backed by big tech.
It's purpose is to stop bad actors from passing of fake as real just as much as it is to prevent real images being dismissed as fake.
Knowing Apple, they've been working on and testing Apple Reference Image for years.
It being perfect isn't the issue; it's that random people on the internet who are just learning about this assume Apple's engineers haven't already thought about everything (and more) mentioned in this thread.
Given how many bugs there are in macOS and how long they have remained there, I (who have been writing iOS apps from the release of the first retina iPod touch until AI got good) functionally agree with such people; at best, I think Apple's engineers haven't actually solved everything (and more) mentioned in this thread, even if every one of these things may have come up in discussions and even reached an official backlog or task list or similar.
I think this will depend on how it gets used. I can imagine numerous outcomes where it's in fact worse than nothing (significantly more effective blackmail, for instance).
While that is not quite my bar of confidence when implementing wide-reaching technologies that have numerous unexplored knock-on effects, I guess the calculus must have been different on Infinite Loop recently.
I’ve seen that type of argument a million times, and I’ll certainly reuse that.
seems pretty easy to make it sufficiently difficult to trick the system
> Today, powerful, widely available AI tools allow users to easily generate or alter photorealistic images to a degree that was difficult to imagine just a few years ago.
Photoshop has existed for decades and so has fake images. This is a low friction way to attest "this image came from an iPhone sensor and Apple approved it". It will still take the usual image forensics to determine if the scene it depicts is legitimate.
It is the problem that they say they're trying to solve, though. They specifically say "where the essential role of a photograph is to prove that something actually happened".
It fails the reasonable person test to say that the "something" in that phrase refers to the act of taking the photo itself.
Likewise in "distinguish between photographs that depict real events and...".
After all, if money is no object, you could just bribe every single apple employee involved in the project.
Used in a capacity as evidence? Uh, yes? Duh? Do you seriously believe otherwise? Anyway, that scenario is made worse not better by Apple promising captured veracity.
The problem with this thinking is twofold:
1) Whether it actually meaningfully increases the difficulty of a forgery remains to be seen. Despite their initial language about discerning real events, we see no details here about what scene information is used.
2) It increases the potential value of a forgery because now your forgery is attested by Apple.
So it either makes it easier to defraud people or more worthwhile to put in the effort to defraud people or both. None of those outcomes are great.
Therefore because of your worry (which is based on remarkably little information), it's a bad technology?
Come the fuck on. That's beyond luddite bullshit.
You must be new around here. ;-)
While I'm on board with you about the inabsolute security of this (relative to what's typically expected of cryptographic systems), the fact that their 'verified' state requires a live certification and can be revoked means that the sensor responsible for obviously faked images will see those images and that device no longer certified.
It all relies a lot on trust in Apple, and integration with Apple, and relatively unmotivated attackers.
I think the idea is to control the easy, cheap mass production of AI gen picture and not 100% coverage.
That’s a tradeoff I can live with.
But you're only allowed to do that if your name if Anish Kapoor
There’s no such thing as a Golden Gate Bridge.
Prove it.
Also the dots can be trivially blocked by putting your finger over the sensor, sometimes improving photo quality. I do this frequently when I want to take a photo through a window. The absence of the dot matrix tells the iPhone to focus on the background far away instead of the windowpane.
I feel really dumb for not having thought of this.
https://www.elcomsoft.com/news/428.html
https://blog.elcomsoft.com/2011/04/nikon-image-authenticatio...
You don't even have to travel to the location, you can just spoof GPS. And of course that will only be needed until some eastern european kid gets bored one weekend and the signing keys magically appear on pastebin.
It's funny to see Apple fall into this same trap.
Discerning a camera taken image of an image is typically very very easy. The collors/exposure/etc will all be obviously wrong in ways to a human, even without doing any analysis.
Like with C2PA, the entire thing hinges on nobody being able to dump keys or trick the TPM into signing arbitrary image data. The timestamping server is a nice idea (though I don't see why they can't just use a normal timestamping server, I guess to keep control over the protocol) but it doesn't solve the fundamental problem that defeated C2PA.
I don't think we should have this, for that reason alone (but many others too).
Location services is quite hard to trick. To the point people have gone to the lengths of putting iPhones inside a microwave for RF shielding and setting up fake phone tower signals inside to trick the phone in to unlocking the hearing aid feature on AirPods for unapproved countries.
It’s simply not technically possible to verify the authenticity of the camera input with 100% certainty. Pretending that it is possible only creates problems. Then someone fakes evidence, but all the normies who have no clue about technology assume that it must be real. You see this with AI detectors too they recognize random texts as generated, yet an unbelievable number of people believe them.
Apple/iOS already have part authentication pipeline on its security sensitive devices (TouchID/FaceID). How can camera sensor can't be considered one of those and needs attestation before enabling?
From the document:
> Apple Reference Image leverages custom-designed image sensors in iPhone 18 Pro and iPhone 18 Pro Max to ensure reliable capture of image data, and relies on Private Cloud Compute, which provides a computational environment for secure photographic processing that cannot be subverted even in the case of device compromise. (emphasis mine)
I hope that companies and governments don't start forcing us to use this stuff by requiring it for their services.
So if you jailbreak or root your phone what happens? Is this a trojan horse into making rooted phone cameras unverified? Just like how Linux machines can't watch Netflix in 4K
PCC is quite good, about as close to private remote compute we can get without doing HME.
[0] https://support.apple.com/guide/iphone/view-reference-images...
[1] https://www.apple.com/legal/privacy/data/en/reference-image/
> When you take a photo in Reference mode after tapping Reference Mode, your device will include reference image information in the photo’s metadata. If you then view that photo and tap the Reference badge on your iOS device or click it on your Mac, the device will send the raw photograph, metadata about the photograph like the sensor’s signatures and the time frame in which the photo was captured, as well as the sensor’s unique hardware identifiers to Private Cloud Compute.
Edit: On reread it seems they do in fact send the actual photographic data to PCC, which I presume has some reason over signing metadata on-device? Original mistaken post is below for transparency.
You can always not use the reference image mode, and according to the article you send a hash of the signature of the photograph, so all they would know is you took a photograph in reference image mode at some point in time before the request.
After my brief read, one of the main reasons they’re using PCC to produce the signed JPEGs, instead of doing everything on device, is that it maintains your privacy.
If you were signing with the iPhone, an attacker could then correlate photos taken with the same device.
Apple certainly has the data that “this sensor, in this device, took this exact photo” in PCC at the time of signing, but they discard that data.
- it sounds like its an optional mode you have to enable. That kind of defeats the point if you need to prove something after the fact
- i guess you need internet to take a picture. :(
- You are puting a lot of trust in apple's private cloud compute platform.
- apple can revoke certification of a picture. I understand the appeal of this, all security systems eventually have failures, so its important to be robust against this. However if the point is to prove a picture is real (especially politically damaging ones), this is giving a lot of power to apple.
Its meant to be in competition with C2PA, and i guess the idea is its much more secure against complex hardware attacks. However i think its worth asking who the target audience is and what threats they face. The primary issue with AI is it makes fake photos easy, not that it invented fake photos. Even Stalin manipulated photos back in the day. It is not a new thing, the problem is just being overwhelmed with them.
with that in mind, are complex hardware attacks really that important? We just need to increase the difficulty floor, not solve fake photos for all time. No matter what you do, people can still use practical effects.
It seems like this is almost trying to thwart spies and nation state adversaries, well forgetting that such well funded groups have the budget to fake photos the old fashioned way or if they really cared, bribe their way into apple.
I have my doubts about this scheme but this is not one of them. If the point is that someone in principle could verify, that is enough for it to be useful, even if not everyone does.
Whatever that means in detail...
What you are prevented from doing is adding a verification to a photo outside of the iOS image pipeline, or modifying the photo with the verification still in tact.
Now the camera module is supposed to generate a key pair internationally and send the public key over the bus. This looks like it is interceptable at repair time and a man in the middle can insert a different public key that they generated externally. Is there a way to stop this?
The unfortunate result of AI slop is reduced trust, which in turn is responded to with surveillance, which ultimately leads to the loss of liberty. Is it possible to do these sorts of verifications in an open way? I kinda doubt it, since someone has to control the hardware manufacturing process.
but im sure it will popular with 60 year olds watermarking their pictures of sunsets.
Scrapped in 3..2..1..