Apple Intelligence in iOS 18 - 15 new AI features coming to iPhone

Apple Intelligence in iOS 18 - 15 new AI features coming to iPhone

Apple finally announced its AI efforts at WWDC 2024; the new AI, dubbed Apple Intelligence, is built on top of multiple large-scale language and diffusion models.

In the name of privacy, Apple has been pushing for these new features to run locally on the iPhone as much as possible, unlike competing models such as Google's Gemini and ChatGPT.

These new features are also limited to the latest iPhones. As of this writing, Apple Intelligence is only available on iPhones with the A17 chip, which includes only the iPhone 15 Pro and iPhone 15 Pro Max.

To see the top 15 features on iPhones that support Apple Intelligence, read on.

Of all the Apple Intelligence features coming to iOS 18, Siri will probably see the biggest update.

As part of Siri's makeover, the voice assistant is gaining the ability to understand and respond to more conversational language; as we saw in OpenAI GPT-4o, this means Siri will be able to keep up if someone rambles or changes direction while speaking Siri will be able to keep up even if someone stutters or changes direction while speaking.

Siri also has access to information about products and settings within the Apple ecosystem. This means Siri will be able to find menus and options within apps based on their descriptions.

Apple Intelligence behaves like an agent. This means that Apple's AI can perform tasks across iOS and within apps on your behalf.

As an example, you can have Siri add an event to your calendar or open a link someone sent you in iMessage.

Apple claims that Apple Intelligence will be context-aware and tailored to the user. It will know what content has been shared with you by knowing personal information such as emails, calendar invitations, documents, etc.

It is well known that Apple has signed a deal with OpenAI for ChatGPT. This partnership is a big one, but perhaps not as involved as many think.

Within the tool or app, users can open Apple Intelligence and instruct it to send images, documents, and text prompts to ChatGPT without leaving the tool.

Similarly, Siri will be able to use GPT-4o to respond to inquiries and provide resources; GPT functionality is integrated into Siri's UI.

Apple emphasizes that Apple Intelligence respects user privacy. They claim that the new AI is "built with privacy at its core."

Much of this is due to on-device processing; by limiting the majority of AI resources to the iPhone, less data should leak outside. Nevertheless, some tools, such as ChatGPT, require cloud computing.

With that in mind, Apple has launched Private Cloud Compute. According to Apple, these new dedicated servers should be a black box for private data.

The Photos app will get a major upgrade with new layouts and customization options. iOS 18 and Apple Intelligence will allow Apple to add the ability to create custom categories and galleries within the app.

It will also add the ability to search using "natural language." Examples provided by the company include searching for "Katie with stickers on her face" or "Maya skateboarding in a tie-dye shirt."

Within the "Photos" app, there is a new tool called "Cleanup" that uses AI to remove distracting objects from images, such as other people or photobombs. Apparently, it works by simply tapping a button and carefully drawing a circle.

Memories is a new Apple Intelligence-powered feature that uses natural language processing to create memory movies based on specific requests. It can be requested by the user or created based on a theme identified by the iPhone from a photo.

Image Playground is Apple's version of AI image generators like DALL-E and Midjourney.

Image Playground's ability to generate images can be used across apps like Notes and Message. It will also offer its own app on the iPhone. There are three styles of images: animated, illustrated, and sketched.

Everything is generated locally, so nothing is sent from the device.

Genmoji is a new custom-made emoji that can be created by entering a description; Genmoji is generated with additional options.

Genmoji are part of the Image Playground and can be used in messages as regular emoji or as stickers or reactions. They can also be used to create animated greetings, such as sending someone a unique Happy Birthday image.

Apple Intelligence also offers a new writing tool, the big one is called Rewrite. with Rewrite, Apple Intelligence provides the user with different versions of a sentence, tailored to the reader and task tone.

All writing tools are system-wide and can be used with any Apple app or third-party app. As an example, they can be used in Mail, Messages, and Notes.

Proofread is Apple's version of Grammarly. It checks grammar, word choice, and sentence structure. This tool also suggests edits and explains those edits.

One major feature that almost all AI models seem to have to have is the ability to summarize sentences.

Apple Intelligence is no different. It has a summarization tool that allows users to select blocks of text and summarize them in the form of short paragraphs, bulleted lists, or tables.

One of the problems with our phones is the flood of notifications that can fill the screen if we do not disable them.

"Reduce interruptions" is a new feature that uses Apple Intelligence to understand the context of notifications and bring those that may be more important to the forefront.

This feature, for example, filters out email notifications but highlights notifications that remind you to pick up your child. It can also summarize long or overlapping notifications and show important details under "Priority Notifications."

Smart Reply is a feature of the email app that allows users to quickly create replies. The app uses AI to identify questions that need a reply and provides relevant choices to include in the reply. It also provides suggested responses.

As well as reducing interruptions, Mail has a new Priority Messages feature that uses Apple Intelligence to determine which emails to look at first. It then pushes those emails to the top of the inbox.

Priority Messages also summarizes long threads into appropriate detail without actually opening individual emails.

Apple Intelligence will be available for free as part of iOS 18, which will be released this fall. Presumably along with the iPhone 16 lineup. However, it will only be available in English.

For now, Apple AI is only available on the iPhone 15 Pro and iPhone 15 Pro Max. It is not known if all iPhone 16 will support Apple Intelligence or if it will be limited to the Pro and Pro Max versions of that iPhone.

.

Categories