Apps exclude elders
Elderly parents living at home cannot operate smart-device apps — menus, pairing, updates. The devices end up controlled only by the children who installed them.
Agentic AI for Indian homes · Bengaluru
A fully autonomous AI brain for Indian homes
No app to learn — the family simply talks to the home on WhatsApp, by voice, in Hindi, English, Kannada or Marathi.
Built first for elderly parents living at home, on the affordable Tuya-based smart devices Indian families already own.
Languages at launch
- हिन्दी Hindi
- English
- ಕನ್ನಡ Kannada
- मराठी Marathi
SmartHomeAI · Home
Replies in Marathi
3:02 pm
0:03
“खूप गरम होत आहे”
“It is very hot.”
Checked before acting
- 34 °C
- 3 pm
- Her past preference
- Fan set to high
- Curtains closed
0:05
“खोलीत ३४ अंश तापमान आहे, म्हणून पंखा जोरात लावला आणि पडदे बंद केले.”
“It is 34 degrees in the room, so I have put the fan on high and closed the curtains.”
Voice notes on WhatsAppNothing new to install or read
Four languages at launchHindi, English, Kannada, Marathi
Brand-agnosticOne brain for any Tuya-based device
Offline-firstA local hub keeps routines running
The problem
Elders at home cannot use smart devices
Smart devices are cheap and everywhere in India. What is missing is the intelligence layer above them — one that is affordable, multilingual and trustworthy.
One app per brand
Wipro, Syska, GalMAx and Havells each ship their own app. The family becomes the integration layer, and every new device adds another app.
English-first voice
Mainstream voice assistants handle Hindi only partially, and Kannada or Marathi poorly. Elders who speak only their mother tongue are left out.
Reactive, not autonomous
Automations are still “if motion, then light”. No goals, no planning, no explanation. The home reacts; it does not look after anyone.
From smart to agentic
What changes when a device has goals
A smart device reacts to a trigger. An agentic device pursues a goal — it observes, reasons, plans, acts and reflects.
| Capability | Smart device (today) | Agentic device (SmartHomeAI) |
|---|---|---|
| Follows rules | Yes | Yes — and knows when to override them |
| Has goals | No | Yes: comfort, energy budget, safety |
| Handles ambiguity | No | Reasons through it in natural language |
| Multi-step planning | No | Plans a day, not a single action |
| Explains decisions | No | Every action carries a why |
| Self-corrects | No | Observes outcomes and updates memory |
| Coordinates devices | Limited scenes | Multi-agent negotiation across the home |
How it thinks
The agent loop: observe, reason, plan, act, reflect
Follow one request through the loop. A 72-year-old sends a WhatsApp voice note in Marathi: “खूप गरम होत आहे” — “it is very hot.” She expresses a feeling, not a command; the system carries the reasoning.
Step 1 of 5
Observe
The agent gathers context: sensor readings, who is at home, the electricity tariff, the weather — and any request from the family.
In the example
A Marathi voice note arrives. The room sensor reads 34 °C and it is 3 pm.
Step 2 of 5
Reason
A language model and planner interpret the goal behind the request in its context — even when the request is a feeling rather than a command.
In the example
“It is very hot” is a comfort goal, not an instruction. The agent weighs the time, the temperature and her past preference.
Step 3 of 5
Plan
The goal becomes an ordered list of tool calls, each with an expected outcome that can be checked afterwards.
In the example
- Fan to high
- Curtains closed
- Confirm in Marathi, with the reason
Step 4 of 5
Act
The plan is executed through the Tuya API and device actuators, with notifications to the family where needed.
In the example
The fan and curtains are switched through Tuya, and she receives a Marathi voice note confirming what was done and why.
Step 5 of 5
Reflect
Did it work? The agent checks the outcome against what it expected, then updates its memory and her preferences.
In the example
It confirms the devices responded and records the outcome, so the next hot afternoon starts from what she prefers.
No app to learn
Talk to the home on WhatsApp, by voice
Zero onboarding for the elder: WhatsApp is already on the phone. The rest of the family can message the home too.
-
Speak
She sends a WhatsApp voice note in her own language — the app she already uses every day.
-
Understand
Indian-language speech recognition and a language model extract intent, mood and context — including mixed Hindi–English.
-
Act
The agent plans and controls lights, fan, AC, plugs and door sensors through the Tuya platform.
-
Reply
Confirmation comes back as a voice note in the same language — no reading, no menus.
Fully autonomous, not just voice-controlled
The agent also acts without being asked, so routine comfort and safety never depend on someone remembering to ask.
- Lights on at dusk
- Fan off when the room is empty
- Geyser on before the usual bath time
- WhatsApp alert to the family if there is no movement by morning
System architecture
Five layers, one brand-agnostic brain
Any Tuya-based device — GalMAx, Wipro, Syska, Havells — plugs in through the perception and action layers without changing the brain.
-
Perception
WhatsApp voice notes, sensors, event streams
-
State & memory
Short-term context and long-term preferences
-
Reasoning
Goal-directed planning and tool selection
-
Action
Device control and user notifications
-
Reflection
Outcome checks and self-critique
Brand-agnostic by design
The reasoning layer never depends on a brand’s app. A new device only touches the perception and action layers.
Safety never waits on the cloud
A rule-based fallback runs locally, so safety-critical actions never wait on a language-model round trip.
Platforms and tools
Built on open and India-hosted components
Every component has an open or India-hosted option. AI4Bharat and Bhashini cover all 22 scheduled languages, so adding languages beyond the launch four is a data and testing task, not a platform change.
| Layer | Platform / tool | Role in SmartHomeAI |
|---|---|---|
| Messaging channel | WhatsApp Business Cloud API (Meta), via Gupshup or Twilio | Receive and send voice notes; family alerts |
| Speech-to-text | AI4Bharat IndicConformer · Bhashini ASR (MeitY) · Sarvam Saarika | Hindi, Kannada, Marathi, English and code-mixed speech |
| Text-to-speech | AI4Bharat Indic-TTS · Bhashini TTS · Sarvam Bulbul | Voice replies in the user’s language |
| Reasoning | Claude API (Anthropic) · Sarvam LLM for Indic text · local rule engine | Intent, planning, explanation; safe fallback |
| Device platform | Tuya IoT Platform (Cloud API, India DC) · Smart Life / DigiTap · tinytuya | Control of GalMAx, Wipro, Syska and Havells devices |
| Orchestration | Python (FastAPI) · n8n workflows | Agent loop, tool calls, schedules |
| Memory | SQLite / Redis · Chroma vector store | Preferences, history, context retrieval |
| Local hub | Raspberry Pi 5 · Zigbee gateway · Home Assistant (optional) | Offline routines, low latency, power-cut resilience |
| Hosting & monitoring | AWS Mumbai or Azure Central India · Grafana · audit log | Data residency, uptime, every action traceable |
Accounts and approvals
- Meta Business verification (WhatsApp)
- Tuya developer project — done
- Bhashini API access
- DPDP Act 2023 compliance
Where things stand
Work completed so far
SmartHomeAI is at proof-of-concept stage, with a real device under the agent’s control. The next step is a whole-home agent in pilot homes.
Done — Phase 0, proof of concept
- Cloud project “SmartHomeAI” created on the Tuya IoT platform (India data centre)
- Physical test device: GalMAx RGBW smart bulb paired and linked to the developer account
- API credentials provisioned; Python control stack built (tinytuya)
- Agent codebase structured: device control, Telegram interface, Claude-powered planner
- Reference architecture and teaching material on agentic devices authored
Next 6 months
- WhatsApp voice note → device action demo (Hindi, Marathi)
- Multi-device expansion: plug, fan controller, sensor kit
- Kannada and English; autonomous routines (dusk, empty room)
- First 5 pilot homes in Bengaluru
Why this is different — and why India
The intelligence and access layer, not more hardware
Global voice assistants are optimised for English-speaking markets and closed ecosystems. SmartHomeAI starts from the other end: Indian homes, Indian languages and the devices people already own.
Brand-agnostic by design
Built on the Tuya platform that already powers most affordable Indian smart devices. No new hardware to buy; the brain works with what people own.
Elders first, by voice, in their language
WhatsApp voice notes in Hindi, English, Kannada and Marathi — nothing to install, nothing to read. Built for the parent living at home, not the engineer who set it up.
Offline-first and resilient
Core routines run on a low-cost local hub; the cloud is an upgrade, not a dependency. Designed for power cuts and patchy networks.
Autonomous, explainable and safe
The agent looks after the home on its own and says why it acted, in a voice note. A family member can override anything; safety-critical devices use verified rules.
24-month roadmap
From one bulb to an open platform
Each phase ends with a demonstrable milestone: a working demo, a pilot report, a publication or an SDK release.
Phase 0 complete: proof of concept on the Tuya IoT platform
-
Phase 1Months 1–6
Whole-room agent
- WhatsApp voice interface: Hindi, English, Kannada, Marathi
- Multi-device control (light, plug, fan, sensors)
- First autonomous routines
- Local hub prototype
-
Phase 2Months 7–12
Whole-home agent
- Memory and preference learning
- Safety alerts to family (no movement, door left open)
- Guardrails and family override
- Pilot: 5–10 elder homes, Bengaluru
-
Phase 3Months 13–18
Multi-agent home
- Lighting, HVAC, security and energy agents
- Orchestrator with conflict resolution
- Explainability dashboard
- Pilot: 50 homes across 2 cities
-
Phase 4Months 19–24
Open platform
- Developer SDK and device onboarding
- Research publications and open datasets
- Course module and trained cohort
- Partner integrations, commercial pilot
Commitments
Deliverables and impact by month 24
60+pilot homes running the agent
4languages via WhatsApp voice
3+peer-reviewed publications
1open SDK and device-onboarding kit
Energy
Tariff-aware planning targets 10–15% lower household electricity use in pilots, measured against baseline months.
A target to be validated in the Phase 2 pilots, not a claim.
Inclusion
Elders living at home run the house by speaking naturally on WhatsApp in their own language, with no app to operate.
Education
A full course module and open teaching artifacts on agentic devices, grounded in a real deployed system.
Who is building this
Rigorous mathematics, production ML and years in the classroom
Dr. Milan Amrutkumar Joshi
Founder and Principal Investigator
- PhD in Mathematics; 15+ Scopus/SCI-indexed publications in topological data analysis, evolutionary optimisation and applied ML
- Former ML Lead, Climate Connect Digital (ReNew Power) — energy forecasting and optimisation in production
- Senior Faculty, Data Science & AI, Great Learning; adjunct at Northwestern University and Airtics Education
- 14 years teaching mathematics, ML and AI to engineers and working professionals
The team we are building
- Backend and IoT engineeringTuya integration, local hub, device onboarding
- Applied machine learningPlanner, memory, evaluation loop
- Speech and language researchIndian-language intent understanding
- Pilots and communityHome deployments, user studies, feedback
Interested in joining or advising? Get in touch.
Work with us
Help bring SmartHomeAI into Indian homes
SmartHomeAI is preparing its first pilot homes in Bengaluru. We would like to hear from:
Families in Bengaluru
With a parent at home who would rather speak to the house than learn an app. Join the first pilot homes.
Register interestDevice makers and installers
Tuya-based brands and installers who want an intelligence layer above their devices.
Discuss a partnershipResearchers and funders
Collaborate on agentic devices, Indian-language speech and open datasets.
Start a conversationDr. Milan Amrutkumar Joshi