CS34 omeAgent: A Sandbox for Simulating Smart Home

Project number: CS34 
Project Source: School of Computer Science 
Project Title: HomeAgent: A Sandbox for Simulating Smart Home Interaction 
and Automation Using Large Language Models 
Project Description and Scope: This project involves upgrading the HomeAgent 
sandbox framework, designed for simulating smart home environments and 
interactions with large language models (LLMs). HomeAgent currently 
integrates family agents and a HomeBot to emulate realistic human behaviours 
and smart home automation. The goal of this project is to enhance the existing 
framework with new features and improvements. 
Project Scope: 
• Upgrade to GPT-4o: 
o Objective: Update the underlying model of HomeAgent to GPT-4o. 
o Details: Integrate GPT-4o into the HomeAgent framework, 
replacing the current model to leverage the advanced reasoning, 
tool use, and human interaction capabilities of GPT-4o. 
• Add Family Agent Functionality: 
o Objective: Enhance the sandbox environment to support 
additional family agents. 
o Details: Modify the simulation environment to allow the inclusion 
of more family agents, enabling a more comprehensive simulation 
of household interactions and dynamics. 
• Extend Simulation Timeline: 
o Objective: Extend the simulation capabilities to cover a full week. 
o Details: Update the simulation timeline to allow scenarios and 
interactions to unfold over a week, providing a more detailed and 
extended observation of agent behaviours and smart home 
automation processes. 
• Improve User Interface: 
o Objective: Enhance the user interface to display environment data 
and agent actions more effectively. 
o Details: Redesign the user interface to provide real-time updates 
on the smart home environment, including temperature, 
humidity, and IoT device statuses, as well as detailed logs of agent 
actions and interactions. Expected outcomes/deliverables: Enhanced realism and complexity in smart 
home simulations. Improved interaction capabilities with more family agents. 
Comprehensive data collection and analysis over extended simulation periods. 
User-friendly interface providing clear and detailed insights into the smart 
home environment and agent behaviours 
Specific required knowledge, skills, and/or technology: Proficiency in Python 
and familiarity with the PyGame framework. Experience with deep learning, 
particularly with large language models such as GPT-4. UI/UX design skills for 
enhancing user interfaces. (Good to have but not essential). 
Fields that this project may involve: Software Development; Artificial 
Intelligence; NLP. 
Resources provided by the client: Provide working project code as the starting 
point. Provide detailed documentation as the current project description.