Can AI agents control smart home and IoT devices locally?

Most AI agents control devices the way a phone app does: through the maker’s cloud API or through a hub in the home that already controls them. The agent’s model decides what to do, and a tool layer turns that decision into an API call.

Four approaches are in use, and they differ mainly in whether the internet sits in the path. This table compares them as of October 2026.

The vendor cloud path is the easiest to start with and the only one that stops during an outage. The hub paths keep the command inside the home, and the last row covers devices that have no hub in common.

The Model Context Protocol (MCP) is an open protocol that connects an AI application to tools and data. A host application, such as a chat app or coding agent, creates an MCP client for each MCP server it uses. The server lists tools, and the model calls them with structured arguments.

MCP has two standard transports. With stdio, the client launches the server as a subprocess on the same machine. With Streamable HTTP, the client sends each message as an HTTP request to the server’s endpoint, which can be on the local network or the internet. MCP’s authorization specification is optional and, for HTTP transports, based on OAuth 2.1.

Home Assistant ships both sides as integrations, both added in Home Assistant 2025.2:

• The Model Context Protocol Server integration exposes Home Assistant to AI agents at its /api/mcp endpoint. Its documentation says the agent can control your devices and see the current state of your home, limited to the entities exposed to it. It supports MCP tools, prompts and a read-only context resource, but not sampling or notifications.
• The Model Context Protocol integration works the other way. It lets Home Assistant’s own conversation agents use tools from outside MCP servers, such as memory or web search. It supports only tools and connects to servers over SSE, so a server that speaks only stdio needs a proxy such as mcp-proxy.

To connect an agent to Home Assistant’s MCP server:

1. In Home Assistant, go to Settings, System, AI, enable MCP if it is off, and copy the MCP server URL.
2. Add the integration under Settings, Devices and services, and review its options: whether all LLM APIs are exposed, and whether only administrator accounts may use the endpoint.
3. Expose only the devices the agent should control, from the Expose tab under Settings, Voice assistants.
4. Add the URL to your MCP client. Home Assistant documents the steps for Claude, ChatGPT, Claude Code, Codex and OpenCode, and the client opens a browser to sign in to Home Assistant.

Where the client runs decides whether the internet is involved. Home Assistant says that if the AI agent runs in the cloud, the Home Assistant instance must be reachable from the internet, and that Claude connects to the MCP server from the cloud even when you use Claude Desktop. Its Codex example uses a local address, http://homeassistant.local:8123/api/mcp.

The model can run on your own hardware, with trade-offs in accuracy and hardware. Home Assistant’s Ollama integration adds a conversation agent powered by an Ollama server, which runs on macOS, Linux and Windows on your own network. When it is allowed to control Home Assistant, it acts only on exposed entities.

Home Assistant is frank about the limits of this experimental feature:

• Only models that support tools can control Home Assistant.
• Smaller models are more likely to make mistakes than larger ones, and may not reliably keep a conversation going while control is enabled.
• It recommends exposing fewer than 25 entities when experimenting with a local model.
• Its default context window is 8k tokens, and a larger window supports larger homes at the cost of more memory on the Ollama server.

Spoken commands can stay local too. Home Assistant’s local voice pipeline uses Speech-to-Phrase or Whisper to turn speech into text and Piper to speak the answer, all on your own hardware. It suggests Whisper, which is slower on small hardware, when you plan to pair the assistant with a language model. A fully local chain is therefore a microphone, a local speech engine, a local model, Home Assistant and the devices, with no step that needs the internet.

An agent rarely finds devices itself; it asks something that already knows them. A hub keeps the device list and exposes chosen devices as tools, so the agent’s discovery step is the MCP tools/list request.

Underneath, local discovery uses multicast DNS. Home Assistant’s Matter documentation says the Matter protocol relies on local IPv6 and mDNS traffic traveling freely in the network. Multicast DNS, RFC 6762, resolves names ending in .local with no DNS server, which is how homeassistant.local reaches the hub. DNS-Based Service Discovery, RFC 6763, lets a client list the named instances of a service type, such as every printer or every Matter device on the network.

Both stay inside one network segment with multicast allowed. VLANs and guest networks that block multicast break them, which is a common reason an agent or hub cannot see a device that a phone can. How do devices discover services without a server? covers the options in more depth.

Keeping an agent safe around devices means limiting what it can reach, keeping a person approving risky actions, and recording every call. The MCP specification says that for trust and safety there should always be a human in the loop with the ability to deny tool invocations.

The specification and Home Assistant’s documentation give concrete controls:

1. Expose the minimum. Home Assistant lets you choose which entities each assistant can control, and its MCP server and Ollama agent act only on exposed entities. Leave locks, garage doors and heating out until you trust the setup.
2. Require confirmation for actions that matter. MCP says clients should prompt for user confirmation on sensitive operations and show tool inputs before calling the server. Home Assistant notes that Claude asks for permission before calling any tools.
3. Restrict who can connect. Home Assistant’s MCP server requires an administrator account by default and uses OAuth 2.0 with PKCE. A remote client means the instance is reachable from the internet, so treat that exposure as part of the risk.
4. Do not trust tool descriptions blindly. MCP says clients must treat tool annotations as untrusted unless they come from trusted servers.
5. Bound and log every call. MCP says servers must validate inputs, enforce access controls and rate limit tool calls, and clients should set timeouts and log tool use for audit.
6. Make retries safe. A command with no reply may still have run. Give each action an ID so a retry is applied only once, and prefer “set to 20 degrees” over “raise by 2”.

When devices share only a radio link, the IP-based paths stop working, so discovery and calls have to travel over the radio itself. mDNS needs a shared IP network, MCP over Streamable HTTP needs an IP route, and stdio works only on one machine.

That is the case for agents on phones, handhelds, vehicles and robots that meet with no shared Wi-Fi. Two devices may share only Bluetooth LE, or a direct Wi-Fi link that one platform supports and the other does not. BLE vs Wi-Fi Direct vs Wi-Fi Aware compares those links. Software that works here has to carry three steps over the radio: advertise a capability, discover who offers it, and invoke it with a request and response. Each device also takes over the authorization a hub used to provide, deciding which peers may call it and which providers it will trust. How do AI agents talk to each other without a server? walks through that pattern.

Offline Protocol does not control smart home or IoT devices. It is not a hub, it does not speak Matter, Zigbee, Z-Wave or Thread, and its MCP server is for coding agents building apps with the SDK, not for operating devices.

It covers the last row of the table: agents and services on phones, computers and gateways that need to find and call each other without a central registry. In SDK v0.31.0, a device registers a service, other devices discover it with queries that spread across the mesh, and the caller sends a typed request and gets a response, with requests sealed in an end-to-end encrypted session once one exists. The discovery messages themselves are signed but readable by the devices that relay them, so a service description should hold nothing sensitive. On a local network the Python binding can also publish services as DNS-SD records. A host can run the SDK as a local service that programs reach over JSON-RPC on a WebSocket, and its HTTP front lets a program call an HTTP service on another device by that device’s address, with the request carried inside an end-to-end encrypted message. Device control stays with the hub or the device’s own protocol. For a site team, that means agents and handhelds keep handing work to each other on the spot instead of waiting on a cloud round trip for every handoff.