Use screen actions to chat with Gemini

When you open Gemini over some apps, videos, or files, screen actions appear. These are shortcuts to start conversations and tasks using content, including screenshots, PDFs, and URLs from your screen. 

Get started

To get started, use gestures or say “Hey Google” to open Gemini over a webpage or app, like YouTube. Suggestion chips will appear above the Gemini chat box when they’re available. When you tap on a suggestion chip, Gemini may ask for more info before sending your chat.

When you tap on a suggestion, Gemini will upload the info on your screen or in the file for context. These suggestions include things like: 

• Talk Live: starts Gemini Live, using what’s on your screen as the starting point.
• Ask about… helps you ask about the screen, file, or a webpage you’re looking at.

Tip: To use suggestion chips, Gemini collects some info from what’s on your screen. For example, when you tap “Ask about this page” over a webpage, Gemini collects the URL and the info on that page, including info not visible. If you ask about a file like a PDF, Gemini accesses the full file to be able to respond about it.

Turn auto-submit on or off

All screen actions except “Ask about…” will automatically submit the content on your screen to Gemini when you tap them.

 To turn off auto-submit, you can press and hold on a suggestion chip then tap Turn off auto-submit. To enable auto-submit, you can press and hold on the “Ask about…” chip then tap Turn on auto-submit.