Andrej Karpathy endorses Apple Intelligence ( lemmy.world )

Actually, really liked the Apple Intelligence announcement. It must be a very exciting time at Apple as they layer AI on top of the entire OS. A few of the major themes.

Step 1 Multimodal I/O. Enable text/audio/image/video capability, both read and write. These are the native human APIs, so to speak.

Step 2 Agentic. Allow all parts of the OS and apps to inter-operate via "function calling"; kernel process LLM that can schedule and coordinate work across them given user queries.

Step 3 Frictionless. Fully integrate these features in a highly frictionless, fast, "always on", and contextual way. No going around copy pasting information, prompt engineering, or etc. Adapt the UI accordingly.

Step 4 Initiative. Don't perform a task given a prompt, anticipate the prompt, suggest, initiate.

Step 5 Delegation hierarchy. Move as much intelligence as you can on device (Apple Silicon very helpful and well-suited), but allow optional dispatch of work to cloud.

Step 6 Modularity. Allow the OS to access and support an entire and growing ecosystem of LLMs (e.g. ChatGPT announcement).

Step 7 Privacy. <3

We're quickly heading into a world where you can open up your phone and just say stuff. It talks back and it knows you. And it just works. Super exciting and as a user, quite looking forward to it.

https://x.com/karpathy/status/1800242310116262150?s=46

someacnt_ ,

This isn't satire? What?

admin ,
@admin@lemmy.my-box.dev avatar

Check out OP defending Apple in every comment in this thread. It would be funny if it weren't so... yeah.

someacnt_ ,

I am just sitting here like.. how. Am I too autistic to distinguish satire from non-satire ones

rikudou ,

What the hell is the fella smoking if he thinks Apple would ever let others use their on-device LLM? Like, the company that deems it too dangerous if apps could change a wallpaper?

800XL ,

šŸ™„

LANIK2000 ,

The amount of corporate speak makes me sick. Especially the mix of buzzwords being mixed with shit like "KERNEL PROCESS", shit's cursed.

rikudou ,

Hey, I love my kernel processes! Especially my LLM kernel processes.

eager_eagle ,
@eager_eagle@lemmy.world avatar

"and it just works"

has he even used an llm before?

Z4rK OP ,

He sort of invented it, so you have to think heā€™s commenting on the concept here, not the implementation.

I have tried a lot of medium and small models, and there it just no good replacement for the larger ones for natural text output. And they wonā€™t run on device.

Still, fine-tuning smaller models can do wonders, so my guess would be that Apple Intelligence is really 20+ small and fine tuned models that kick in based on which action you take.

gravitas_deficiency ,

An LLM has no comprehension of what it says. Itā€™s just a puppy that is really good at performing for treats. This will always yield nonsense a meaningful proportion of the time.

I donā€™t care how statistically good your model can be under certain constraints and inputs. At the end of the day, all youā€™ve done is classically condition your computer.

Z4rK OP ,

It goes a tad bit beyond classical conditioning... LLMā€™a provides a much better semantic experience than any previous technology, and is great for relating input to meaningful content. Think of it as an improved search engine that gives you more relevant info / actions / tool-suggestions etc based on where and how you are using it.

Hereā€™s a great article that gives some insight into the knowledge features embedded into a larger model: https://transformer-circuits.pub/2024/scaling-monosemanticity/

gravitas_deficiency , (edited )

Thatā€™s great. But thatā€™s not how itā€™s being marketed and sold to the public. Itā€™s being sold as an oracle (as in crystal ball, not database). And itā€™s misleading and hurting people as a result.

Iā€™ll reiterate: An LLM has no comprehension of what it says.

Itā€™s a matter of engineering ethics, on multiple levels:

  • the training data in the vast majority of cases is outright stolen
  • itā€™s being sold as something that itā€™s not, and the result is causing real damage to people and society in a ton of ways weā€™re still discovering
  • most people deeply involved in developing LLMs, and basically all of the technical leadership, are categorically ignoring and abrogating any and all responsibility around this ā€œmagicalā€ new system theyā€™ve made. Weā€™ve seen this before with social networking. We know where this road leads.

Iā€™m not saying the tech should be banned. Thatā€™s obviously idiotic. Neural nets can - and are - used for tons of fascinating and excellent applications. Itā€™s just that my staunch opinion is that LLMs are a terrible application of that the tech at this stage of development, and itā€™s particularly terrible that OpenAI/Microsoft/etc are aggressively foisting this technology on the public, and simultaneously refusing to take any ethical responsibility for it.

Z4rK OP ,

To be honest, Iā€™m not sure what weā€™re arguing - we both seem to have a sound understanding of what LLM is and what it is not.

Iā€™m not trying to defend or market LLM, Iā€™m just describing the usability of the current capabilities of typical LLMs.

gravitas_deficiency ,

Iā€™m saying that I wish that more people involved with the core development of the technology took the ethical considerations seriously, and communicated those concerns as a first-order issue when they talk about applications like this.

Itā€™s fascinating tech, but the way itā€™s being employed these days is deeply irresponsible.

restingboredface ,

I look forward to Apple Marketing coming up with their usual line of nonsense, like a meaningless name for an existing capability that they are claiming to have invented.

Z4rK OP ,

Well they just name-grabbed all of AI with their stupid Apple Intelligence branding.

helenslunch ,
@helenslunch@feddit.nl avatar

I watched an abbreviated video. Pretty much everything they announced was available on other platforms 5+years ago

gravitas_deficiency , (edited )

But now itā€™s on your iPhone.

I agree that itā€™s dumb.

It will still make stock go up.

helenslunch ,
@helenslunch@feddit.nl avatar

I mean don't get me wrong, these things are a huge QoL improvement. It's just that they should be embarrassed to be lacking such basic functionality on such insanely expensive devices. And people should be embarrassed to carry them around. There are still tons of basic features missing.

shield_gengar ,
@shield_gengar@sh.itjust.works avatar

They did really good at marketing. "If you don't use Apple, You're poor."

helenslunch ,
@helenslunch@feddit.nl avatar

Honestly it was super effective. They're more of a lifestyle brand than anything.

henfredemars , (edited )

Kernel process LLM

God I hope not. That sounds extremely insecure. Definitely do not do this in the kernel.

Thann ,
@Thann@lemmy.ml avatar

Why not just have the LLM replace the kernel?

mriormro ,
@mriormro@lemmy.world avatar

Why not have the LLM replace the user?

Thann ,
@Thann@lemmy.ml avatar

This could really cut down on those pesky bug reports....

DudeDudenson ,

I'm imagining a world where advertisers have to try to raise engagement from AIs in their ads

gravitas_deficiency ,

Why do I feel like this is making me Recall another recent awful idea?

Rustmilian ,
@Rustmilian@lemmy.world avatar

AI powered Rootkit.

AverageGoob ,
@AverageGoob@lemmy.world avatar

Yikes. Just hit em with the ol' "<3" for privacy. Does not inspire confidence.

Z4rK OP ,

How so? Many people want to use AI in privacy, but itā€™s too hard for most people to set it up for themselves currently.

Having AI tools on the OS level so you can use it in almost any app and that is guaranteed to be processed on device in privacy will be very useful if done right.

TheFriar ,

You think your iPhone isnā€™t collecting data on you? Is that what youā€™re saying?

ji17br ,

The phone is, Apple isnā€™t. They outline everything in the keynote if you are interested.

helenslunch ,
@helenslunch@feddit.nl avatar

Because Apple has never lied or misled before

Rustmilian , (edited )
@Rustmilian@lemmy.world avatar

Their keynotes are irrelevant, their official privacy policies and legal disclosures take precedence over marketing claims or statements made in keynotes or presentations. Apple's privacy policy states that the company collects data necessary to provide and improve its products and services. The OS-level AI would fall under this category, allowing Apple to collect data processed by the AI for improving its functionality and models. Apple's keynotes and marketing materials do not carry legal weight when it comes to their data practices. With the AI system operating at the OS level, it likely has access to a wide range of user data, including text inputs, conversations, and potentially other sensitive information.

Z4rK OP ,

Unless you are designing and creating your own chips for processing, networking etc, then privacy today is about trust, not technology. Thereā€™s no escaping it. I know iPhone and Apple is collecting data about me. I currently trust them the most on how they use it.

EngineerGaming , (edited )
@EngineerGaming@feddit.nl avatar

There are degrees of trust though. You can trust the developers and people who audited the code if you have no skill/desire to audit it yourself, or you can trust just the developers.

And even closed systems' behavior can be monitored and analyzed.

Z4rK OP ,

Yes definitely, Apple claimed that their privacy could be independently audited and verified; we will have to wait and see whatā€™s actually behind that info.

Rustmilian , (edited )
@Rustmilian@lemmy.world avatar

Apple claimed that their privacy could be independently audited and verified.

How? The only way to truly be able to do that to a 100% verifiable degree is if it were open source, and I highly doubt Apple would do that, especially considering it's OS level integration. At best, they'd probably only have a self-report mechanism which would also likely be proprietary and therefore not verifiable in itself.

Z4rK OP ,

They have designed a very extensive solution for Private Cloud Computing: https://security.apple.com/blog/private-cloud-compute/

All I have seen from security persons reviewing this is that it will probably be one of the best solutions of its kind - they basically do almost everything correctly, and extensively so.

They could have provided even more source code and easier ways for third parties to verify their claims, but it is understandable that they didnā€™t, is the only critique Iā€™ve seen.

Rustmilian , (edited )
@Rustmilian@lemmy.world avatar

However, to process more sophisticated requests, Apple Intelligence needs to be able to enlist help from larger, more complex models in the cloud. For these cloud requests to live up to the security and privacy guarantees that our users expect from our devices, the traditional cloud service security model isn't a viable starting point. Instead, we need to bring our industry-leading device security model, for the first time ever, to the cloud.

As stated above, Private cloud compute has nothing to do with the OS level AI itself. ą²°ā _ā ą²° That's in the cloud not on device.

While weā€™re publishing the binary images of every production PCC build, to further aid research we will periodically also publish a subset of the security-critical PCC source code.

As stated here, it still has the same issue of not being 100% verifiable, they only publish a few code snippets they deam "security-critical", it doesn't allow us to verify the handling of user data.

  • Itā€™s difficult to provide runtime transparency for AI in the cloud.
    Cloud AI services are opaque: providers do not typically specify details of the software stack they are using to run their services, and those details are often considered proprietary. Even if a cloud AI service relied only on open source software, which is inspectable by security researchers, there is no widely deployed way for a user device (or browser) to confirm that the service itā€™s connecting to is running an unmodified version of the software that it purports to run, or to detect that the software running on the service has changed.

Adding to what it says here, if the on device AI is compromised in anyway, be it from an attacker or Apple themselves then PCC is rendered irrelevant regardless if PCC were open source or not.

Additionally, I'll raise the issue that this entire blog is nothing but just that a blog, nothing stated here is legally binding, so any claims of how they handled user data is irrelevant and can easily be dismissed as marketing.

Z4rK OP ,
  1. Security / privacy on device: Donā€™t use devices / OS you donā€™t trust. I donā€™t see what difference on-device AI have at all here. If you donā€™t trust your device / OS then no functionality or data is safe.
  2. Security / privacy in the cloud: The take here is that Apples proposed implementation is better than 99% of every cloud service out there. AI or not isnā€™t really part of it. If you already donā€™t trust Apple then this is moot. Donā€™t use cloud services from providers you donā€™t trust.

Security and privacy in 2024 is unfortunately about trust, not technology, unless you are able to isolate yourself or design and produce all the chips you use yourself.

Rustmilian , (edited )
@Rustmilian@lemmy.world avatar

Yeah and apple is completely untrustworthy like any other corporation, my point exactly. Idk about you, but I'll stick to what I can verify the security & privacy of for myself, e.g. Ollama, GrapheneOS, Linux, Coreboot, Libreboot/Canoeboot, etc.

MigratingtoLemmy ,

Running FOSS and taking control of your network will do a far better trick of privacy vs convenience than most people can imagine

Rustmilian , (edited )
@Rustmilian@lemmy.world avatar

you can use it in almost any app
if done right

How are you going to be able to use it in "almost any app" in a way that is secure? How are you going to design it so that the apps don't abuse the AI to get more information on the user out of it than intended? Seems pretty damn inherently insecure to me.

Z4rK OP ,

Thatā€™s why itā€™s on the OS-level. For example, for text, it seems to work in any text app that uses the standard text input api, which Apple controls.

User activates the ā€œAI overlayā€ on the OS, not in the app, OS reads selected text from App and sends text suggestions back.

The App is (possibly) unaware that AI has been used / activated, and has not received any user information.

Of course, if you donā€™t trust the OS, donā€™t use this. And Iā€™m 100% speculating here based on what we saw for the macOS demo.

Rustmilian , (edited )
@Rustmilian@lemmy.world avatar
  • Malicious actors could potentially exploit vulnerabilities in the AI system to gain unauthorized access or control over device functions and data, potentially leading to severe privacy breaches, unauthorized data access, or even the ability to inject malicious content or commands through the AI system.
  • Privacy breaches are possible if the AI system is compromised, exposing user data, activities, and conversations processed by the AI.
  • Integrating AI functionality deeply into the operating system increases the overall attack surface, providing more potential entry points for malicious actors to exploit vulnerabilities and gain unauthorized access or control.
  • Human reviewers have access to annotate and process user conversations for improving the AI models. To effectively train and improve the AI models powering the OS-level integration, Apple would likely need to collect and process user data, such as text inputs, conversations, and interactions with the AI.
  • Apple's privacy policy states that the company collects data necessary to provide and improve its products and services. The OS-level AI would fall under this category, allowing Apple to collect data processed by the AI for improving its functionality and models.
  • Despite privacy claims, Apple has a history of collecting various types of user data, including device usage, location, health data, and more, as outlined in their privacy policies.
  • If Apple partners with third-party AI providers, there is a possibility of user data being shared or accessed by those entities, as permitted by Apple's privacy policy.
  • With the AI system operating at the OS level, it likely has access to a wide range of user data, including text inputs, conversations, and potentially other sensitive information. This raises privacy concerns about how this data is handled, stored, and potentially shared or accessed by the AI provider or other parties.
  • Lack of transparency for users about when and how their data is being processed by the AI system & users not being fully informed about data collection related to the AI. Additionally, if the AI integration is controlled solely at the OS level, users may have limited control over enabling or disabling this functionality.
Zoot ,
@Zoot@reddthat.com avatar

Yeah just like Microsoft Recall right? An AI that has access to every single thing you do (and would also be recording, otherwise how does it know "you") can never be private by design. Its literal design is to know everything about you, your actions, and your habits. I wouldn't trust anyone to be able to create an actually secure piece of software that does the above. It will always be able to be stolen/sold/abused.

DudeDudenson ,

But how can we best sell your data to advertisers otherwise?

Z4rK OP ,

macOS and Windows could already be doing this today behind your back regardless of any new AI technology. Donā€™t use an OS you donā€™t trust.

Zoot ,
@Zoot@reddthat.com avatar

I don't use either of those thankfully:).

Z4rK OP ,

Thatā€™s fair, but you are misunderstanding the technology if youā€™re bashing the AI from Apple for making macOS less secure. Most likely, it will be just as secure as for example their password functionality, although we donā€™t have details yet. You either trust the OS or not.

Microsoft Recall was designed so badly, thereā€™s no hope for it.

Zoot ,
@Zoot@reddthat.com avatar

I simply don't, and wouldn't trust Apple. They will tell you they are all about privacy, and happily sell your data behind your back. Just like any other company.

eager_eagle ,
@eager_eagle@lemmy.world avatar

<3

reattach ,

I thought the original post was satire - list all of the privacy issues, then throw in "Privacy <3" at the end. Seriously, almost every one of those points has a potential privacy issue.

Guess I was being too generous.

JoMiran ,
@JoMiran@lemmy.ml avatar

https://a.pinatafarm.com/800x450/cb52bd150e/la-noire-press-x-to-doubt.jpg

Z4rK OP ,

Care to elaborate?

The suspicious parts to me was that they didnā€™t show much of the private cloud stuff, how much it would cost, and that they still feel the need to promote ChatGPT .

JoMiran ,
@JoMiran@lemmy.ml avatar

All of it sounds like marketing and I have serious doubt's about their commitment to, or ability to respect privacy when one of their previous points is that they plan to integrate third party systems. So...I have doubts.

Z4rK OP ,

I mean, thatā€™s fair, I personally use Apple devices specifically because I trust them the most on privacy, but if you donā€™t trust Apple with privacy, which is a 100% valid take to have, then of course this mayor selling point of their marketing becomes moot.

umami_wasbi ,

I would not give the right of anyone deciding what is good for my privacy, including Apple. This should be a judgement made by myself.

Z4rK OP ,

I do agree, but privacy in 2024 is sadly about trust, not technology, unless you yourself can design and create every chip used in your devices and in the network cells you connect to. No setting on your device on ā€œdo not allowā€¦ā€ have any meaning without trust in the creator.

umami_wasbi , (edited )

I didn't say trust no one, but whom and what I trust shall be decided by me. Yes, there are things we can't just build in our garage, yet there are tools enables us to investigate, and people and organization working on it. Maybe Apple's take on AI have better privacy then others, but that shall be investigated and proven upon after release, not automatically granted.

avidamoeba ,
@avidamoeba@lemmy.ca avatar

X

eager_eagle ,
@eager_eagle@lemmy.world avatar

twitter

c10l ,

Founding member of company that stands to make fortunes through a product endorses said product.

Z4rK OP ,

I mean, thatā€™s fair, if you donā€™t believe in his integrity than this news have very little value to you.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • ā€¢
  • [email protected]
  • kbinchat
  • All magazines
    •